iobuf/tests: scale as_scattered batch by input size#30427
Open
travisdowns wants to merge 1 commit intoredpanda-data:devfrom
Open
iobuf/tests: scale as_scattered batch by input size#30427travisdowns wants to merge 1 commit intoredpanda-data:devfrom
travisdowns wants to merge 1 commit intoredpanda-data:devfrom
Conversation
The as_scattered benchmark builds n deep copies of the source iobuf upfront before timing. With n=inner_iters (1000) and a 965 KiB source, the as_scattered_large case peaked at ~942 MiB which fit the 1 GiB benchmark budget alone but overflowed it once cumulative residual from earlier PERF_TESTs in the binary was accounted for, causing the iobuf_rpbench_test smoke test to crash with a seastar bad_alloc. Compute n inside as_scattered_bench so the upfront allocation is capped at ~100 MiB (10% of the 1 GiB budget) for any input size. Per-iteration cost is unchanged (inst column is per averaged iteration): test inst before inst after iobuf.as_scattered_small 5288.61 5288.61 iobuf.as_scattered_large 23247.02 23086.50 iobuf.as_scattered_many_frags 2214615.3 2214693.0
Contributor
There was a problem hiding this comment.
Pull request overview
Adjusts the as_scattered_* iobuf microbenchmarks to avoid large upfront allocations that can intermittently trigger Seastar bad_alloc in CI when multiple PERF_TESTs run in the same binary.
Changes:
- Compute the
as_scattered_benchbatch size (n) fromsource.size_bytes()to cap pre-timed deep-copy allocation to ~100 MiB. - Update
as_scattered_small/large/many_fragsPERF_TESTs to use the newas_scattered_bench(const iobuf&)signature. - Document the intent of the new sizing logic next to the benchmark helper.
Collaborator
CI test resultstest results on build#84242
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
as_scattered_*PERF_TESTs iniobuf_bench.ccbuildndeep copies oftheir source iobuf upfront before the timed region. With
n=inner_iters(1000) and the large-case source of 965 KiB, the upfront allocation peaked
at ~942 MiB — within the 1 GiB benchmark budget when the test ran in
isolation, but not after the cumulative residual from earlier PERF_TESTs in
the same binary. This caused
iobuf_rpbench_testto crash intermittentlyin the smoke test with a seastar
bad_alloc.Compute
ninsideas_scattered_benchfromsource.size_bytes()so thatupfront allocation is capped at ~100 MiB (10% of the 1 GiB budget) for any
input size. Per-iteration cost is unchanged (instructions per averaged
iteration are within measurement noise of before).
Backports Required
Release Notes