Skip to content

feat: add mev cli tests#300

Open
PoulavBhowmick03 wants to merge 2 commits intomainfrom
poulav/mev_test
Open

feat: add mev cli tests#300
PoulavBhowmick03 wants to merge 2 commits intomainfrom
poulav/mev_test

Conversation

@PoulavBhowmick03
Copy link
Copy Markdown
Collaborator

Closes #235

@iamquang95
Copy link
Copy Markdown
Collaborator

@claude

@claude
Copy link
Copy Markdown

claude bot commented Apr 2, 2026

Claude finished @iamquang95's task in 2m 58s —— View job


Code Review

  • Gather context and read changed files
  • Review crates/cli/src/commands/test/mev.rs
  • Review supporting changes
  • Post findings

Summary

This PR ports Charon's MEV relay test suite into the Pluto CLI, adding Ping, PingMeasure, and CreateBlock test cases plus two unit tests. The implementation is largely correct but has one functional parity gap (credential stripping), a concurrency correctness issue in timeout handling, and a structural refactor concern.


Findings

[High] mev_ping_measure_test ignores embedded URL credentials

  • mev.rs:319request_rtt is called with the raw target URL. But request_rtt (mod.rs:699) uses the URL as-is via reqwest, which does not handle user:pass@host in URLs in the same way a manual basic_auth call does.
  • mev_ping_test (line 287–294) correctly calls parse_endpoint_credentials and apply_basic_auth. mev_ping_measure_test skips this step entirely.
  • Impact: If a MEV relay endpoint contains embedded credentials (e.g. https://0x<pubkey>@relay.example.com), the PingMeasure test will either fail with a 401 or include credentials in the URL as plain text.
  • Fix this →

[Medium] Timeout in test_single_mev only emits one failure entry, not N

  • mev.rs:250–261 — On cancellation, only one TestResult::Fail is pushed for queued_tests[test_counter]. If multiple tests were still queued, the remaining ones (indices test_counter+1..) are silently dropped from the results.
  • Impact: Score calculation (via calculate_score) and result output will silently omit cancelled tests beyond the first, making the reported score look better than it is and breaking parity with the Go implementation which emits a failure for every remaining test.
  • Fix this →

[Medium] reqwest::Client is created per request, not reused

  • mev.rs:285, mev.rs:547, mev.rs:574, mev.rs:616 — A new reqwest::Client is instantiated for every HTTP call. The reqwest docs recommend creating a client once and reusing it, as it manages a connection pool internally.
  • Impact: Under a load test with --number-of-payloads > 1, connection pooling is never utilized; each round-trip re-establishes TCP connections. RTT measurements may be inflated by connection setup overhead, making results unreliable for CreateBlock.
  • Recommendation: Thread a shared reqwest::Client through TestMevArgs or pass it as a parameter to the helper functions.

[Low] wiremock removed from dev-dependencies without explanation

  • crates/cli/Cargo.toml diff — wiremock is removed from [dev-dependencies]. The PR description doesn't mention why.
  • This is fine if no existing tests in other modules depended on it; confirm no compile errors remain in the other test modules (beacon.rs, peers.rs, etc.).

[Low] create_mev_block bypasses epoch boundary refresh

  • mev.rs:387proposer_duties are fetched once per epoch. Inside create_mev_block, when MevError::StatusCodeNot200 triggers a slot advance (next_slot += 1, line 723), the epoch may roll over, but proposer_duties are not refreshed for the new epoch.
  • The fallback at line 681 (fetch_proposers_for_epoch) handles the case where the pubkey is not found, so this won't hard-fail — but it will make an extra network call per epoch boundary that crosses inside the retry loop.

[Low] Hardcoded signature in build_blinded_block_payload is intentional but undocumented

  • mev.rs:765 — The comment-less hardcoded sig_hex is a dummy BLS signature used so the relay returns 400 Bad Request (which request_rtt at line 755 treats as the expected status). This is correct behavior but should have a comment explaining why a dummy/invalid signature is intentional, to avoid future "cleanup" removing it.

Parity Matrix

Component Go Rust Match Notes
CLI flag --endpoints present present yes
CLI flag --beacon-node-endpoint present present yes
CLI flag --load-test present present yes
CLI flag --number-of-payloads present present yes
CLI flag --x-timeout-ms present present yes
Ping test present present yes
PingMeasure test present present partial credential stripping missing in Rust
CreateBlock test present present yes
format_mev_relay_name present present yes
Timeout/cancellation per test emits N failures emits 1 failure no remaining tests silently dropped
Client reuse shared client new per request no performance gap

Tests

Unit tests added:

  • test_format_mev_relay_name — covers standard, no-scheme, no-credential, and short-hash cases. Good coverage.
  • test_get_validator_pk_for_slot — covers found, adjacent, and not-found cases. Good coverage.

No integration tests (requiring live relay) — acceptable given the external dependency.

Clippy/fmt could not be verified (build tools not available in this environment).


Open Questions

  1. Was wiremock removed because beacon.rs/peers.rs tests that used it are also being refactored in a follow-up, or were those tests already removed?
  2. The request_rtt function (mod.rs:691) does not strip URL credentials — is this a known limitation of the shared helper, or should credential handling be added there?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement cli test mev

2 participants