Skip to content

perf(pm): main-loop BFS resolver with rayon-offloaded simd_json parse #2938

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
elrrrrrrr wants to merge 17 commits into
base: next
Choose a base branch
from
Open

perf(pm): main-loop BFS resolver with rayon-offloaded simd_json parse #2938

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
3c2ee24
perf(pm): fold preload+BFS into concurrent futures::join! (method A)
elrrrrrrr May 12, 2026
f03ce5e
ci(bench): reduce bench_runs default 3→2 for faster PR feedback
elrrrrrrr May 12, 2026
4e6848d
perf(pm): experiment main-loop bfs resolver
elrrrrrrr May 12, 2026
1c2a02a
perf(pm): prioritize bfs manifest requests
elrrrrrrr May 12, 2026
7c4561e
perf(pm): rayon-offload simd_json in the cherry-picked main-loop reso…
elrrrrrrr May 12, 2026
5df4677
perf(pm): inline CoreVersionManifest parse + bump default concurrency…
elrrrrrrr May 13, 2026
e2a02e4
perf(pm): batch-drain completed fetches before re-pumping
elrrrrrrr May 13, 2026
cd38669
perf(pm): bump default manifest concurrency 64→128
elrrrrrrr May 13, 2026
f86667b
perf(pm): fill all idle slots with prefetch when demand queue is empty
elrrrrrrr May 13, 2026
163e656
perf(pm): deep transitive prefetch via spec-carrying FetchRequest::Full
elrrrrrrr May 13, 2026
93134a0
perf(pm): offload speculative version parse to rayon in deep prefetch
elrrrrrrr May 13, 2026
fb03b93
perf(pm): merge speculative version parse into FullManifest rayon task
elrrrrrrr May 13, 2026
6b2d7d2
perf(pm): use 4 independent HTTP client pools for connection diversity
elrrrrrrr May 13, 2026
fba11ee
perf(pm): increase client pool to 32 for bun-like connection rotation
elrrrrrrr May 13, 2026
b6f40bc
perf(pm): revert pool size 32→4 (32 causes TLS overhead regression)
elrrrrrrr May 13, 2026
70a4b86
perf(pm): share TLS session cache across client pools
elrrrrrrr May 13, 2026
b6c37ed
perf(pm): share TLS session cache across client pools on all platforms
elrrrrrrr May 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/pm-e2e-bench.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ on:
bench_runs:
description: '[phases] Runs per phase'
required: false
default: '3'
default: '2'
type: string

concurrency:
Expand Down Expand Up @@ -540,7 +540,7 @@ jobs:
env:
PROJECT: ${{ github.event.inputs.project || 'ant-design' }}
REGISTRY: 'https://registry.npmjs.org'
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '3' }}
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '2' }}
PM_LIST: 'utoo,utoo-next,utoo-npm,bun'
run: |
chmod +x bench/pm-bench-phases.sh
Expand All @@ -560,7 +560,7 @@ jobs:
env:
PROJECT: ${{ github.event.inputs.project || 'ant-design' }}
REGISTRY: 'https://registry.npmmirror.com'
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '3' }}
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '2' }}
PM_LIST: 'utoo,utoo-next,utoo-npm,bun'
run: |
mkdir -p /tmp/pm-bench-output
Expand Down Expand Up @@ -629,7 +629,7 @@ jobs:
env:
PROJECT: ${{ github.event.inputs.project || 'ant-design' }}
REGISTRY: 'https://registry.npmjs.org'
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '3' }}
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '2' }}
PM_LIST: 'utoo,utoo-npm,bun'
run: |
chmod +x bench/pm-bench-phases.sh
Expand All @@ -645,7 +645,7 @@ jobs:
env:
PROJECT: ${{ github.event.inputs.project || 'ant-design' }}
REGISTRY: 'https://registry.npmmirror.com'
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '3' }}
BENCH_RUNS: ${{ github.event.inputs.bench_runs || '2' }}
PM_LIST: 'utoo,utoo-npm,bun'
run: |
mkdir -p /tmp/pm-bench-output
Expand Down
6 changes: 5 additions & 1 deletion crates/ruborist/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,12 @@ workspace = true
tokio = { workspace = true, features = ["macros", "rt"] }

# Native (non-macOS) targets: reqwest's default rustls + ring.
# rustls + rustls-native-certs are direct deps so the multi-pool client
# can share a TLS session cache across pools (see service/http.rs).
[target.'cfg(not(any(target_arch = "wasm32", target_os = "macos")))'.dependencies]
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls-native-roots"] }
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls-native-roots"] }
rustls = { version = "0.23", default-features = false, features = ["ring", "logging", "std", "tls12"] }
rustls-native-certs = "0.8"

# Native-only dependencies (not compiled for WASM)
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
Expand Down
Loading
Loading