perf(worker-core): replace AsyncArrowWriter with parquet-ext#1852

Merged

JohnSwan1503 merged 13 commits intomainfrom

Swanson/worker-core/performance/replace-arrow-writer-with-parquet-ext

Feb 25, 2026

Commits on Feb 24, 2026

feat(worker-core): integrate parquet-ext where parquet::arrow::AsyncArrowWriter is used.
JohnSwan1503
committed
feat(parquet-ext): add in-tree parquet-ext crate with pipelined writer.
JohnSwan1503
committed
Merge branch main into Swanson/worker-core/performance/replace-arrow-writer-with-parquet-ext
JohnSwan1503
committed
formatting(paruet-ext): clippy lints and formatter run
JohnSwan1503
committed
refactor(pipeline): replace tokio::try_join with futures::try_join for async operations
JohnSwan1503
committed
refactor(executor): use ManuallyDrop for writer and encoder outboxes to prevent deadlock on shutdown
JohnSwan1503
committed
dep(parking_lot): add parking_lot as a workspace dependency across multiple crates
JohnSwan1503
committed
refactor(encoder): replace UnsafeCell with Mutex for thread-safe work item management
JohnSwan1503
committed
refactor(encoder): replace thread-local storage with BTreeMap for ordered leaf collection
JohnSwan1503
committed