Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
366b4c1
bench: benchmark and optimize large-doc write and sync paths
marcus-pousette Apr 4, 2026
2823d1d
fix(playground): restore appended-children helper for hot-write bench
marcus-pousette Apr 6, 2026
ec5cd16
refactor(postgres-napi): move bench helpers behind testing export
marcus-pousette Apr 7, 2026
21bbc81
refactor(playground): centralize live-write bench hooks
marcus-pousette Apr 7, 2026
e3c28d0
refactor(postgres-napi): split native testing factory
marcus-pousette Apr 7, 2026
e96a020
refactor(playground): drop unrelated auth and row churn
marcus-pousette Apr 7, 2026
7ba4d31
refactor(playground): extract payload hydration helpers
marcus-pousette Apr 7, 2026
1332057
style(postgres-napi): format testing entrypoint
marcus-pousette Apr 7, 2026
eb552f4
refactor(bench): share fixture helper contract
marcus-pousette Apr 7, 2026
a6d334f
refactor(benchmark): unify shared helper utilities
marcus-pousette Apr 7, 2026
da8ecfa
refactor(benchmark): reuse shared live-write summary stats
marcus-pousette Apr 7, 2026
8707734
perf(sync-bench): support direct remote fixture priming
marcus-pousette Apr 7, 2026
3c9b21b
perf(sync-bench): skip eager remote fixture materialization
marcus-pousette Apr 8, 2026
453f250
perf(sync): stream empty-receiver uploads from backends
marcus-pousette Apr 8, 2026
50dc746
Revert "perf(sync): stream empty-receiver uploads from backends"
marcus-pousette Apr 8, 2026
80fb51e
test(sync): keep riblt flow-control case non-empty
marcus-pousette Apr 8, 2026
f890a7a
style(benchmark): format sync bench runner
marcus-pousette Apr 8, 2026
a9830cd
refactor(postgres-napi): keep native testing types separate
marcus-pousette Apr 14, 2026
98ac82e
refactor(postgres-napi): inline native testing bindings
marcus-pousette Apr 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
325 changes: 314 additions & 11 deletions .github/workflows/benchmarks.yml

Large diffs are not rendered by default.

50 changes: 50 additions & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,12 @@ Useful top-level entrypoints:

```sh
pnpm benchmark
pnpm benchmark:hot-write
pnpm benchmark:sqlite-node
pnpm benchmark:sqlite-node:hot-write
pnpm benchmark:sqlite-node:ops
pnpm benchmark:sqlite-node:note-paths
pnpm benchmark:postgres:hot-write
pnpm benchmark:sync
pnpm benchmark:sync:direct
pnpm benchmark:sync:local
Expand All @@ -40,12 +43,57 @@ pnpm benchmark:postgres
- One-time bootstrap/discovery tax before opening the regional websocket: `benchmark:sync:bootstrap`
- Local render cost after the data is already present: `benchmark:sqlite-node:note-paths -- --benches=read-children-payloads`
- Local mutation cost inside a large existing tree: `benchmark:sqlite-node:note-paths -- --benches=insert-into-large-tree`
- Hot write cost inside a large existing tree, standardized across local backends: `benchmark:hot-write`
- Protocol/storage baselines and worst-case stress: `sync-one-missing`, `sync-all`, `sync-children*`, `sync-root-children-fanout10`

That split is intentional:

- The sync benches answer "how long until the needed subtree data is in the local store?"
- The note-path benches answer "once the data is local, how quickly can the app render and mutate it?"
- The hot-write benches answer "how expensive is one new live edit after the large doc already exists?"

## Hot Write Benchmarks

Use the hot-write suite when you want an apples-to-apples answer for one edit against an existing large doc.

The current standardized cases are:

- `payload-edit`
- `insert-sibling`
- `move-leaf`
- `move-subtree`

`move-leaf` is the small-write move case.
`move-subtree` intentionally moves a top-level branch so its cost includes moving a larger subtree, not just a single node.

By default each case reseeds a balanced tree, performs exactly one live local mutation, and writes JSON under `benchmarks/hot-write/`.

You can also switch the suite into a warmed session mode so one already-open doc handles several writes before it closes. That is useful when you want to isolate ongoing live-edit cost from sample setup.

Top-level entrypoints:

```sh
pnpm benchmark:hot-write
pnpm benchmark:sqlite-node:hot-write -- --counts=10000,100000
TREECRDT_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:55432/postgres \
pnpm benchmark:postgres:hot-write -- --counts=10000,100000
```

Useful flags:

- `--benches=payload-edit,insert-sibling,move-leaf,move-subtree`
- `--counts=10000,100000`
- `--fanout=10`
- `--payload-bytes=512`
- `--writes-per-sample=10`
- `--warmup-writes=1`

Useful env vars:

- `TREECRDT_POSTGRES_URL=...` for the Postgres runner
- `HOT_WRITE_SKIP_SAMPLE_CLEANUP=1` if you want faster local iteration on very large Postgres samples and do not mind keeping temporary sample docs around

This suite is intentionally separate from the default `pnpm benchmark` run because reseeding large docs for each measured sample is expensive.

## Recommended Product-Facing Runs

Expand Down Expand Up @@ -140,6 +188,8 @@ pnpm benchmark:sync:remote -- \

For remote targets, `prime` now records the exact fixture doc ID locally under `tmp/sqlite-node-sync-bench/server-fixtures/`. That means a fresh endpoint can be primed once with `--server-fixture-cache=rebuild`, and later `--server-fixture-cache=reuse` runs on the same machine can reopen that exact remote fixture doc instead of relying on historical deterministic fixture residue.

If you also have direct Postgres access to that same deployment, add `--postgres-url=postgres://...` on the remote target. Then remote fixture priming can use the same direct balanced-fixture seed path as the local Postgres target instead of uploading the entire large tree over websocket. This is the practical path for `1m` remote/prod-like runs on dedicated environments.

By default, the local sync target runs the Postgres sync server in a spawned child process so local and remote measurements are closer to each other. When you add `--profile-backend`, the local target intentionally switches to the in-process server so per-backend timings are visible inside the benchmark process.

Local server benchmarks now seed the Postgres backend directly before the timer starts. That keeps the measured path honest, because the actual sync to the client still goes through the real websocket server, while avoiding huge protocol-seed setup costs that are not part of the benchmark question.
Expand Down
Loading