feat(plugin): render lanes — priority in the high bits of nextRenderTime

[harper-joseph]

Implements the decided design in #80. Closes #80 if it lands as-is.

The problem, and why the obvious fix doesn't work

claim is strictly nextRenderTime-ascending, so under any capacity deficit the queue serves whatever is oldest-due. #80 carries two production measurements saying that's the wrong order — ~46% of a 521,929-row overdue queue was bot-discovered rather than submitted, and absolute due time treats a 1 h homepage 3 h overdue exactly like a 48 h product page 3 h overdue (300% stale vs 6%).

I first built the wrong thing. I had a branch that ranked the drained claim window by relative lateness, (now − dueAt) / interval. #80 refutes that directly and it's worth restating because it's the trap:

relativeLateness(t) is linear in t with slope 1/interval, so two rows with different intervals cross exactly once. No stored key can express an order that changes with the clock.

And re-ranking inside the window doesn't rescue it: the window is anchored at the oldest due time and is already an EDF prefix, so a homepage 600 minutes down the queue is never read at all. Widening the window fails identically — under a deep backlog every row in it is ancient. That is exactly the reported symptom (a permanently stale homepage sitting behind the backlog), and it is not reachable by a comparator. Relative lateness is the rationale for lane assignment, never the comparator.

The encoding

nextRenderTime = lane × 2^42 + dueAt. One column, the existing index, no schema change, no second index, no migration — every pre-existing row is already a valid lane-0 row, and a lane change is an in-place numeric update. Lower value is claimed first, so lane order is priority order and urgent needs no encoding at all.

Measured in #80: three lanes interleaved in one index, each with its own watermark, 0.29–0.32 ms per lane; the same lane with its watermark reset, 3.46 ms. Interleaving is free — the watermark is the entire win, not the separation. A second index is not: the existing secondary index is already 39–48% of reschedule wall clock.

Where the review attention belongs

Per-lane watermarks live in their own shared buffer (util/laneFloor.js), not in the lease header. Widening renderLease.js's header changes the offset of every slot, and a named shared buffer is sized by its first allocation — so a worker generation that allocated the old layout hands the new code a correctly-sized-looking view with everything shifted, in the module that decides which URLs render. A second key cannot do that, and renderLease.js keeps every invariant and every test it has.

The floor rule is unchanged, per lane. runClaimPass derives the floor from the first due row that lane observed. There's a test asserting a lane's wedged floor cannot strand another lane — which is a strict improvement on today, where one permanently-failing product page pins the scan position for the homepage too.

Spill is dropped, not granted. The lane seek keeps exactly one condition (a two-sided range on this index measures 1,128–2,977 ms), so the upper bound stays in application code and a sparse lane's window runs on into the next lane's rows. Those are recognised, counted and dropped.

Reads decode unconditionally, including while the switch is off. Every reader treats this column as a timestamp — the invalidation accelerator does nextRenderTime − interval arithmetic on it, the console renders it as a date — and an encoded value used as one is a plausible-looking date 139 years per lane out. It's also what makes disabling a survivable rollback rather than a corpus-wide stranding.

Lanes and fairness

lane	derived from
urgent	admin re-render action; authenticated renderNow
submitted/b0…bN	has a sitemapUrl, banded by queue.lanes.ttlBands
discovered/b0…bN	crawler-found, banded the same way
cold	past the fast-retry lane, or a suppressed target's recheck

Banding is what makes "EDF within a lane ≈ relative lateness" true rather than aspirational: submitted here spans 1 h to 48 h, so an unbanded class reproduces TTL-blindness inside itself and the homepage just loses to submitted product pages instead of discovered ones. discovered is banded too — a deliberate deviation from #80's simulation, which modelled discovery as one lane; banding can only reduce within-lane lateness and doesn't touch the inter-lane floors the simulation measured.

Fairness is scheduler policy, not part of the key, so it's tunable live with no rewriting of stored rows. urgentMaxShare caps lane 0's drain share rather than its admission — a structural bound with no token bucket, so a bulk force-render can't take more than that fraction of any batch. minShare reserves floors: strict priority starves the tail at every capacity level including 100% (its lag numbers look good precisely because it drops work — starvation is invisible in a lag metric), while fixed shares summing to 1.0 leave nothing for the priority order to spend. An unclaimed floor is released back to lane order in the same pass.

Rollout — two steps, order matters

Lane 0 is urgent and lane 0 is the unencoded value. What makes the encoding migration-free is also the trap: on a fresh deploy the whole corpus reads as urgent and urgentMaxShare would ration the queue to a fifth of capacity. Separately, a row that is stuck never gets a next write, so a write-time rule alone cannot promote the very rows lanes exist to rescue.

curl -sk -X POST https://<node>:9926/prerender_admin/queue -u <super-user> -H 'content-type: application/json' -d '{"action":"restamp-lanes","limit":5000}'

Bounded, repeatable, cursor-free (a restamped row leaves the queried range, so there's no progress row to go stale), and it changes no due time — only the high bits move, so nothing renders sooner or later than it would have. Call until "done": true, then set queue.lanes.enabled: true. It refuses to run once lanes are on, because a lane-0 row can no longer be told from one an operator marked urgent.

It can't derive a stored changefreq interval, a demand-ladder rung, or cold — those live on the Target and reading it would be a point read per row on an 814k corpus — so a promoted catalog page is stamped one band slow and a suppressed URL one lane fast, both corrected on that row's next render.

Disabling is not an instant rollback: encoded rows stay encoded until each renders again and sort after every unencoded row meanwhile. Nothing is stranded; to get the old behaviour back at once, re-stamp with lanes disabled.

Observability

queue_health lane_granted, lane in the method slot. Lanes redistribute a fixed amount of work, so no total moves when they engage and this is the only series that shows the split. A lane at zero with a non-empty backlog is the failure to look for, and it's indistinguishable from health anywhere else.

Testing

746 pass / 0 fail. The 715 pre-existing tests are untouched — every lane parameter on runClaimPass defaults to the pre-lane behaviour, so an unlaned call is byte-for-byte the function that existed before.

test/renderLane.test.js pins the encoding round-trip at Long magnitudes; an absent due time surviving encoding (Number(null) is 0, 0 is finite, and a floor of 0 means no floor — one null row would unbound the whole claim scan); a wedged lane not stranding another; the per-lane floor rule with an in-flight row holding the floor; spill dropped and counted; floors as minimums with the top-up; the urgent cap holding when nothing else wants the capacity; unsorted/duplicated/empty band lists; and the restamp preserving due times exactly while batching one lane at a time.

The headline one is A DEEP BACKLOG IN A SLOW LANE DOES NOT HIDE THE FAST LANE'S ROWS — 60 product rows three days overdue plus one homepage two hours overdue, which un-laned never appears in the window at all.

Not in this PR

• Design: render priority as lanes encoded in the high bits of nextRenderTime (no schema change) #80 §5's POST /render_queue/prioritize. The two operator-intent paths that exist (admin re-render, renderNow) now write lane 0, which is the behaviour that endpoint would formalise.
• Per-lane backlog depth in the snapshot / console. lane_granted covers "is it working"; per-lane depth needs util/backlogSnapshot.js to become lane-aware and is better reviewed on its own.
• Reducing the volume of suppressed-page renders (the origin-miss head sniff and the browser's early bail). Independent of this.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces 'Render Lanes' to prioritize the prerender queue based on provenance and TTL bands, preventing short-TTL or sitemap-submitted pages from being starved during backlogs. The design encodes lanes within the existing nextRenderTime index and manages per-lane watermarks to avoid queue stranding. Feedback on these changes includes a critical bug fix in lowerFloorTo where an unbounded floor (0) could be overwritten, along with performance improvements such as caching configuration values on hot paths, breaking the claim loop early upon lane mismatch, and yielding the event loop during the restamp migration.

[gemini-code-assist]

When the floor is 0 (unbounded, meaning we scan from the absolute minimum of the lane), any call to lowerFloorTo with a minute greater than 0 will erroneously overwrite the 0 with the higher minute value. This effectively raises the floor from 0 to minute, restricting the scan range and potentially stranding any older overdue rows that are still waiting to be claimed. Since 0 is already the absolute minimum floor, lowerFloorTo should treat 0 as already lower than any target and return false immediately.

Suggested change

	// 0 already means unbounded, so nothing is lower.
	if (current !== 0 && current <= target) return false;
	// 0 already means unbounded, so nothing is lower.
	if (current === 0 || current <= target) return false;

[gemini-code-assist]

Calling bands() on every single row processed causes significant CPU overhead and allocation churn. We should cache the normalized configuration values that are accessed frequently on hot paths. When caching configuration values at the module level, pass the cached value as an explicit parameter to functions rather than closing over module-level variables to keep functions pure and testable. Use nullish coalescing assignment (??=) to handle cases where the module is imported before the configuration is first applied.

export const renderLane = (lane, rows, bands) => {
	// Use the explicitly passed bands array
};

References

1. Cache normalized configuration values or computed arrays that are accessed frequently on hot paths (e.g., multiple times per second) to avoid CPU overhead and allocation churn, recomputing them only when the configuration is applied.
2. When caching configuration values or computed sets at the module level (e.g., using config application hooks), pass the cached value as an explicit parameter to functions rather than closing over the module-level variable to keep functions pure and testable. Use nullish coalescing assignment (??=) to handle cases where the module is imported before the configuration is first applied, and add tests to verify that live configuration changes correctly update the cache.

[gemini-code-assist]

Since rows are sorted by their encoded value, all rows belonging to the current lane strictly precede any rows of subsequent lanes. Once a row with rowLane !== lane is encountered, it guarantees that all remaining rows in the rows array also belong to other lanes. Continuing the loop for these remaining rows is wasteful as they will all be skipped. Breaking the loop immediately (and calculating spilled as the remaining count of rows) avoids unnecessary iterations, saving CPU cycles.

Suggested change

	let leaseRefused = false;
	let skippedLeased = 0;
	let nonFinite = 0;
	let spilled = 0;
	for (const row of rows) {
	// `numberOf` because `Number(null)` is 0, which reads as "due since 1970" and would make an
	// absent due time the oldest due row in the corpus — pinning the floor at the epoch and naming
	// the wrong key as the row holding it. A missing due time is skipped and counted, not coerced.
	const at = numberOf(row.nextRenderTime);
	const { lane: rowLane, dueAt: at } = readRow(row.nextRenderTime);
	if (!Number.isFinite(at)) {
	nonFinite++;
	continue;
	}
	// SPILL. Rows arrive ascending by the ENCODED value, so every row of this lane precedes every
	// row of the next one — the first foreign row means this lane is exhausted. Counted separately
	// from `nonFinite` because it is not a defect: it is the read cost of keeping the query to one
	// condition, and it is the number that says whether a lane's scan limit is being wasted.
	if (rowLane !== lane) {
	spilled++;
	continue;
	}
	let leaseRefused = false;
	let skippedLeased = 0;
	let nonFinite = 0;
	let spilled = 0;
	let index = 0;
	for (const row of rows) {
	index++;
	const { lane: rowLane, dueAt: at } = readRow(row.nextRenderTime);
	if (!Number.isFinite(at)) {
	nonFinite++;
	continue;
	}
	if (rowLane !== lane) {
	spilled = rows.length - index + 1;
	break;
	}

[gemini-code-assist]

When iterating over large database cursors or registries in Node.js, yield the event loop unconditionally based on the number of rows scanned, rather than conditionally on writes or specific branches. This prevents event loop starvation, as awaiting on cursors only drains microtasks.

Suggested change

	export const restampPass = async ({ searchUnstamped, writeRow, limit = 5000 } = {}) => {
	const rows = [];
	// Drained before any write, exactly like the claim pass: Harper's long-transaction monitor aborts
	// a transaction that has pending writes when it fires, and a cursor left open across writes is
	// that shape. No `break` either — an abandoned iterator leaves its read transaction unreleased.
	for await (const row of searchUnstamped({ limit })) rows.push(row);
	export const restampPass = async ({ searchUnstamped, writeRow, limit = 5000 } = {}) => {
	const rows = [];
	let count = 0;
	for await (const row of searchUnstamped({ limit })) {
	rows.push(row);
	if (++count % 200 === 0) {
	await new Promise(resolve => globalThis.setImmediate(resolve));
	}
	}

References

1. When iterating over large database cursors or registries in Node.js, yield the event loop (e.g., using setImmediate) unconditionally based on the number of rows scanned, rather than conditionally on writes or specific branches. This prevents event loop starvation, as awaiting on cursors only drains microtasks. Use a consistent count-based trigger across the codebase.

[harper-joseph]

Closing in favour of the measured design. The benchmark in #119 refutes two of the figures this PR's encoding argument rests on: a second index costs ~13%, not the 39-48% that made a per-lane index look like a doubling of the hot write, and patch is 37% worse than put, so "a lane change is a cheap in-place numeric update" is not an advantage.

What actually decides it: reads are ~2.4 us/row and flat, so priority does not need to live in the stored key at all. Scoring in memory gives exact ordering instead of banded approximation, needs no encoding, no restamp migration, no decode-everywhere discipline and no rollback hazard — and changing the policy becomes a config change rather than a 1.6M-row rewrite.

The two constraints this PR was built around DO hold and carry into the replacement: the unfloored seek degrades 0.073 -> 5.60 ms over 40,000 reschedules while a floored one stays flat, and a two-sided range with an unfillable limit is 256x slower than one-sided. Branch feat/render-lanes can be deleted.