You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR ports Charon's MEV relay test suite into the Pluto CLI, adding Ping, PingMeasure, and CreateBlock test cases plus two unit tests. The implementation is largely correct but has one functional parity gap (credential stripping), a concurrency correctness issue in timeout handling, and a structural refactor concern.
mev.rs:319 — request_rtt is called with the raw target URL. But request_rtt (mod.rs:699) uses the URL as-is via reqwest, which does not handle user:pass@host in URLs in the same way a manual basic_auth call does.
mev_ping_test (line 287–294) correctly calls parse_endpoint_credentials and apply_basic_auth. mev_ping_measure_test skips this step entirely.
Impact: If a MEV relay endpoint contains embedded credentials (e.g. https://0x<pubkey>@relay.example.com), the PingMeasure test will either fail with a 401 or include credentials in the URL as plain text.
[Medium] Timeout in test_single_mev only emits one failure entry, not N
mev.rs:250–261 — On cancellation, only one TestResult::Fail is pushed for queued_tests[test_counter]. If multiple tests were still queued, the remaining ones (indices test_counter+1..) are silently dropped from the results.
Impact: Score calculation (via calculate_score) and result output will silently omit cancelled tests beyond the first, making the reported score look better than it is and breaking parity with the Go implementation which emits a failure for every remaining test.
[Medium] reqwest::Client is created per request, not reused
mev.rs:285, mev.rs:547, mev.rs:574, mev.rs:616 — A new reqwest::Client is instantiated for every HTTP call. The reqwest docs recommend creating a client once and reusing it, as it manages a connection pool internally.
Impact: Under a load test with --number-of-payloads > 1, connection pooling is never utilized; each round-trip re-establishes TCP connections. RTT measurements may be inflated by connection setup overhead, making results unreliable for CreateBlock.
Recommendation: Thread a shared reqwest::Client through TestMevArgs or pass it as a parameter to the helper functions.
[Low] wiremock removed from dev-dependencies without explanation
crates/cli/Cargo.toml diff — wiremock is removed from [dev-dependencies]. The PR description doesn't mention why.
This is fine if no existing tests in other modules depended on it; confirm no compile errors remain in the other test modules (beacon.rs, peers.rs, etc.).
mev.rs:387 — proposer_duties are fetched once per epoch. Inside create_mev_block, when MevError::StatusCodeNot200 triggers a slot advance (next_slot += 1, line 723), the epoch may roll over, but proposer_duties are not refreshed for the new epoch.
The fallback at line 681 (fetch_proposers_for_epoch) handles the case where the pubkey is not found, so this won't hard-fail — but it will make an extra network call per epoch boundary that crosses inside the retry loop.
[Low] Hardcoded signature in build_blinded_block_payload is intentional but undocumented
mev.rs:765 — The comment-less hardcoded sig_hex is a dummy BLS signature used so the relay returns 400 Bad Request (which request_rtt at line 755 treats as the expected status). This is correct behavior but should have a comment explaining why a dummy/invalid signature is intentional, to avoid future "cleanup" removing it.
Parity Matrix
Component
Go
Rust
Match
Notes
CLI flag --endpoints
present
present
yes
CLI flag --beacon-node-endpoint
present
present
yes
CLI flag --load-test
present
present
yes
CLI flag --number-of-payloads
present
present
yes
CLI flag --x-timeout-ms
present
present
yes
Ping test
present
present
yes
PingMeasure test
present
present
partial
credential stripping missing in Rust
CreateBlock test
present
present
yes
format_mev_relay_name
present
present
yes
Timeout/cancellation per test
emits N failures
emits 1 failure
no
remaining tests silently dropped
Client reuse
shared client
new per request
no
performance gap
Tests
Unit tests added:
test_format_mev_relay_name — covers standard, no-scheme, no-credential, and short-hash cases. Good coverage.
test_get_validator_pk_for_slot — covers found, adjacent, and not-found cases. Good coverage.
No integration tests (requiring live relay) — acceptable given the external dependency.
Clippy/fmt could not be verified (build tools not available in this environment).
Open Questions
Was wiremock removed because beacon.rs/peers.rs tests that used it are also being refactored in a follow-up, or were those tests already removed?
The request_rtt function (mod.rs:691) does not strip URL credentials — is this a known limitation of the shared helper, or should credential handling be added there?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #235