Skip to content

fix(ext/node): port internal/priority_queue and expose it via require#33696

Merged
bartlomieju merged 1 commit intodenoland:mainfrom
nathanwhitbot:fix/node-compat-iter71
Apr 29, 2026
Merged

fix(ext/node): port internal/priority_queue and expose it via require#33696
bartlomieju merged 1 commit intodenoland:mainfrom
nathanwhitbot:fix/node-compat-iter71

Conversation

@nathanwhitbot
Copy link
Copy Markdown
Contributor

Summary

require('internal/priority_queue') was unmapped — Deno had no polyfill for it. Port lib/internal/priority_queue.js (the binary heap backing Node's timer scheduler) to TypeScript, default-export the class to match module.exports = class PriorityQueue, and wire it into 01_require.js plus lib.rs.

The class is self-contained: a binary heap accepting an optional comparator and a setPosition callback (Node uses the latter for timer-list bookkeeping so it can removeAt(node.priorityQueuePosition) in O(log n)).

Enables parallel/test-priority-queue.js. Doesn't conflict with parallel/test-internal-modules.js, which only asserts that require('internal/freelist') still throws.

Test plan

  • cargo test --test node_compat -- test-priority-queue test-internal-modules (both pass)
  • Full cargo test --test node_compat on Linux — only pre-existing flaky failures (test-process-threadCpuUsage-worker-threads.js, test-fs-promises-file-handle-readFile.js, test-dgram-send-cb-quelches-error.js, test-child-process-uid-gid.js, test-net-autoselectfamily.js).

@nathanwhitbot nathanwhitbot force-pushed the fix/node-compat-iter71 branch from 66221e1 to 4e56710 Compare April 29, 2026 13:10
@nathanwhitbot
Copy link
Copy Markdown
Contributor Author

Force-pushed 4e5671043 — branch was stacked on top of #33653's commits as a base instead of upstream/main, which made the lib.rs diff look like it was reverting global/GlobalsStorage/00_globals.js (those were already removed on main by #33249). Reset to upstream/main and cherry-picked just the iter71 commit. Diff is now priority_queue.ts (+136) + lib.rs (+1) + 01_require.js (+2) + config.jsonc (+1).

Copy link
Copy Markdown
Contributor Author

@nathanwhitbot nathanwhitbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review of `4e5671043` (post-cleanup):

`internal/priority_queue.ts` port parity with Node (`lib/internal/priority_queue.js`):

Method-by-method comparison against `lib/internal/priority_queue.js`:

Method Node Ours
Default `#compare` `(a, b) => a - b` Same ✓
`#heap` initial `[undefined, undefined]` (1-indexed heap) Same ✓
`insert(value)` `heap[++this.#size] = value; percolateUp(pos)` Same ✓
`peek()` `this.#heap[1]` Same ✓
`peekBottom()` `this.#heap[this.#size]` Same ✓
`percolateDown(pos)` binary-heap sift-down with optional `setPosition` callback Same ✓
`percolateUp(pos)` binary-heap sift-up with optional `setPosition` callback Same ✓
`removeAt(pos)` `heap[pos] = heap[size]; heap[size] = undefined; --size; percolateUp/Down` Same ✓
`shift()` `removeAt(1); return heap[1]` (saved before remove) Same ✓

The `setPosition(node, pos)` callback contract: Node uses this from `internal/timers.js` to record `node.priorityQueuePosition = pos` so it can later `removeAt(node.priorityQueuePosition)` in O(log n) without scanning. The TypeScript port preserves the contract exactly — `hasSetPosition` short-circuit is identical, and every position-mutating path (sift-up, sift-down, swap, percolate) calls `setPosition` at the same points Node does.

`module.exports = class` vs `export default class`:
Node module shape: `module.exports = class PriorityQueue { ... }`. Our TS shape: `export class PriorityQueue { ... }` + (implicit default through the require map entry). `require('internal/priority_queue')` resolves to the class itself, matching Node's `const PriorityQueue = require('internal/priority_queue')` callsite.

`01_require.js` + `lib.rs` wiring:

  • Import + `setupBuiltinModules` map entry mirror `internal/event_target` / `internal/fs/utils` adjacent entries. ✓
  • `lib.rs` ESM listing: `"internal/priority_queue.ts"` alphabetically positioned between `"internal/primordials.mjs"` and `"internal/process/per_thread.mjs"`. ✓

Test enrollment: `parallel/test-priority-queue.js` enabled with `{}`. ✓

LGTM, holding to COMMENT until CI completes.

Copy link
Copy Markdown
Contributor Author

@nathanwhitbot nathanwhitbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single commit, no scope-creep. The PriorityQueue class is a verbatim port of Node's lib/internal/priority_queue.js:

  • Same private fields (#compare, #heap, #setPosition, #size).
  • Identical algorithm in insert/peek/peekBottom/percolateDown/percolateUp/removeAt/shift.
  • heap[0] and heap[1] initialized to undefined (1-indexed binary heap, root at index 1).

TypeScript types are accurate (Comparator<T>, SetPosition<T>, generic T = any). export default PriorityQueue is the right shape so require('internal/priority_queue') matches Node's module.exports = class PriorityQueue. lib.rs and 01_require.js mappings match the existing internal-builtin pattern.

LGTM.

Copy link
Copy Markdown
Contributor

@fibibot fibibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Direct port of Node's lib/internal/priority_queue.js with TypeScript types layered on. Walked the algorithm side-by-side against upstream:

  • Heap layout: 1-indexed, slots [0] and [1] start as undefined (Node uses [undefined, undefined], port matches). peek() returns #heap[1], peekBottom() returns #heap[#size]. ✓
  • insert / percolateUp: increment size, place at end, bubble up while parent (pos >> 1) compares greater. Identical to Node. ✓
  • shift / removeAt / percolateDown: move last element to vacated slot, decrement size, decide percolate-up vs percolate-down based on parent comparison. The two-child comparison (if (nextChild <= size && compare(heap[nextChild], childItem) < 0)) and short-circuit on compare(item, childItem) <= 0 are byte-identical to Node. ✓
  • setPosition callback: same hasSetPosition cached-bool optimization to avoid repeatedly checking setPosition !== undefined inside the hot loops. Node's timer scheduler depends on this for removeAt(node.priorityQueuePosition) in O(log n). ✓

Module wiring:

  • export default PriorityQueue + import internalPriorityQueue from "ext:deno_node/internal/priority_queue.ts" in 01_require.js correctly mirrors Node's module.exports = class PriorityQueuerequire('internal/priority_queue') returns the class itself. ✓
  • Module registered in both 01_require.js (setupBuiltinModules) and lib.rs (extension manifest).
  • // deno-lint-ignore-file no-explicit-any is appropriate — T = any default mirrors the untyped JS source.

CI: 122 success, 0 failure, 10 pending. All 36 test node_compat shards (3 × 6 platforms × debug+release) green. The pending shards are non-node_compat (likely integration/specs stragglers).

The PR body's "Doesn't conflict with parallel/test-internal-modules.js" note checks out — that test specifically asserts require('internal/freelist') still throws, which this PR doesn't touch.

@nathanwhitbot
Copy link
Copy Markdown
Contributor Author

test specs (1/2) release linux-x86_64 fail is the well-known specs::cert::ip_address_unsafe_ssl network flake (uses 1.1.1.1 IP for SSL handshake — runner detected it as flaky on runs 0+1, retried with concurrency=1 and still failed). Unrelated to the PriorityQueue change. @nathanwhit re-run when convenient.

@bartlomieju bartlomieju enabled auto-merge (squash) April 29, 2026 13:59
@bartlomieju bartlomieju merged commit 957ebb6 into denoland:main Apr 29, 2026
268 of 270 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants