From 6948d9507ed9b9ccc79a0934d83324cbb1e55b5c Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 21 Aug 2026 14:20:19 -0400 Subject: [PATCH 1/5] bench(plugin): a reproducible harness for the render queue's storage costs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every scheduling decision in this package is justified by a number, and the two numbers the queue design rests on cannot be reproduced: #80 cites `20-lanesim.mjs` and `21-duerank.mjs`, neither of which is in this repository. Worse, the two figures that ARE in the tree disagree by 80x, and both describe a one-sided ascending range read over the same index on the same table: util/renderSchedule.js 20 keys in 0.43 ms ~21 us/row util/backlogSnapshot.js "~3.5s per 2,000 rows" ~1.75 ms/row Which one is right decides the architecture. At 21 us/row you can stream the whole due set through a bounded heap once a minute and order it exactly — the "buffer everything due now" design works. At 1.75 ms/row a 500k-row due set takes 15 minutes to walk and priority has to live in the index itself. The likely reconciliation is that backlogSnapshot yields every 200 rows beside bot traffic and seeks the absolute index minimum, while the claim scan seeks a floor and never yields — so this measures yielding as an explicit variable rather than assuming it away. Seven questions, each with a stated consequence for the design (see README): the claim-shaped read at four limits; yielding vs not; K per-lane seeks vs one large seek; single-attribute update vs whole-record put; one indexed attribute vs two; two-sided vs one-sided range with a limit that can and cannot fill; and whether the seek point degrades under churn while a floored seek stays immune. Q4 and Q5 decide the change worth making on correctness grounds regardless of the numbers: give the queue its own indexed column and leave the freshness deadline a plain unindexed timestamp, so nothing outside the funnel ever has to decode a due time. NEITHER RUN MODE HAS BEEN EXECUTED END TO END, and the README and run.sh both say so. Docker was unavailable where this was written; local mode reached Harper startup and failed in `checkForExistingInstall` against a freshly installed root — a Harper bootstrap problem rather than a problem with the measurements. The first run should be treated as calibration. Co-Authored-By: Claude Opus 5 --- bench/queue-index/README.md | 61 +++++++ bench/queue-index/bench.js | 283 +++++++++++++++++++++++++++++++ bench/queue-index/config.yaml | 11 ++ bench/queue-index/run.sh | 74 ++++++++ bench/queue-index/schema.graphql | 29 ++++ 5 files changed, 458 insertions(+) create mode 100644 bench/queue-index/README.md create mode 100644 bench/queue-index/bench.js create mode 100644 bench/queue-index/config.yaml create mode 100755 bench/queue-index/run.sh create mode 100644 bench/queue-index/schema.graphql diff --git a/bench/queue-index/README.md b/bench/queue-index/README.md new file mode 100644 index 0000000..485b51b --- /dev/null +++ b/bench/queue-index/README.md @@ -0,0 +1,61 @@ +# `bench/queue-index` — what the render queue's storage actually costs + +Every scheduling decision in this package is justified by a number, and the two numbers the queue +design rests on **cannot be reproduced**: prerender-plugin#80 cites `20-lanesim.mjs` and +`21-duerank.mjs`, neither of which is in this repository. This harness exists so the next schema +change is argued from figures anyone can regenerate. + +It also exists because the two figures currently in the tree **disagree by 80x**: + +| source | claim | implied | +| --- | --- | --- | +| `src/util/renderSchedule.js` | the claim scan returns 20 keys in 0.43 ms | ~21 µs/row | +| `src/util/backlogSnapshot.js` | "~3.5s per 2,000 rows" | ~1.75 ms/row | + +Both describe a one-sided ascending range read over the same index on the same table, and which one +is right decides the architecture: + +- At **21 µs/row**, streaming the whole due set through a bounded heap once a minute costs ~30 s of + background work on a 1.6M-row corpus. A "buffer everything due now and sort it there" design is + viable, and ordering can be exact. +- At **1.75 ms/row**, a 500k-row due set takes ~15 minutes to walk. Every design that reads the due + set is dead, and priority has to be expressed in the index itself. + +The likely reconciliation is that `backlogSnapshot` yields to the event loop every 200 rows *beside +bot traffic* and seeks the absolute index minimum, while the claim scan seeks a floor and never +yields — so the harness measures yielding as an explicit variable rather than assuming it. + +## Running it + +```bash +BENCH_MODE=docker ROWS=200000 ./run.sh +``` + +Docker mode mirrors how kohls-pr's CI stands Harper up. `BENCH_MODE=local` uses an installed root +under `$BENCH_ROOT` instead. Either way it must be an **isolated** Harper: the harness writes +hundreds of thousands of rows, and it refuses to run against `~/hdb`. + +`ROWS` defaults to 200,000 to match #80 so the numbers are comparable. Production is 1,619,000 keys +(814,200 targets × 2 device types) — per-row costs are what transfer, not totals. + +> **Neither mode has been run end to end yet.** Docker was unavailable where this was written, and +> local mode failed in Harper's `checkForExistingInstall` against a fresh root. Treat the first run +> as calibration. + +## What it answers, and what each answer decides + +| | question | if the answer is… | then | +| --- | --- | --- | --- | +| Q1 | per-row cost of the claim-shaped read at limits 20 → 20,000 | small (~tens of µs) | a background sweep of the due set is affordable; exact ordering is on the table | +| | | large (~ms) | priority must live in the index; no design may read the due set | +| Q2 | how much of Q1 is the yielding, not the engine | yielding dominates | the 80× is an artefact and Q1's plain number is the real one | +| Q3 | K per-lane seeks vs one large seek | per-lane ≈ free | the interleaved-lane design in #116 is sound | +| Q4 | single-attribute update vs whole-record `put` | update much cheaper | an in-place lane change is cheap; encoding is a good deal | +| Q5 | one indexed attribute vs two, on the write | ≈ equal | #80's "a second index doubles the hot write" is wrong, and a per-lane **table** becomes viable | +| | | two much slower | splitting `dueAt` out of the queue key must keep `dueAt` **unindexed** | +| Q6 | two-sided vs one-sided range, limit fillable and not | two-sided catastrophic when it cannot fill | keep the `<= now` half in application code, as `claimSchedules` does | +| Q7 | does the seek point degrade as rows churn away from it | yes, and a floored seek is immune | the claim floor stays load-bearing under any new design | + +Q4 and Q5 together decide the change I'd otherwise make on correctness grounds alone: giving the +queue its **own** indexed column and leaving the freshness deadline as a plain unindexed timestamp, +so nothing outside the funnel ever has to decode a due time. diff --git a/bench/queue-index/bench.js b/bench/queue-index/bench.js new file mode 100644 index 0000000..718b9ae --- /dev/null +++ b/bench/queue-index/bench.js @@ -0,0 +1,283 @@ +/** + * WHAT THE RENDER QUEUE'S STORAGE ACTUALLY COSTS. + * + * Every scheduling decision in this package is justified by a number, and the numbers that matter + * most were produced by throwaway scripts that are not in the repository — prerender-plugin#80 cites + * `20-lanesim.mjs` and `21-duerank.mjs`, neither of which exists here. So the two figures the queue + * design rests on cannot be reproduced, and they disagree with each other by 80x: + * + * `util/renderSchedule.js` — the claim scan returns 20 keys in 0.43 ms, i.e. ~21 us/row. + * `util/backlogSnapshot.js` — "Measured cost to calibrate against: ~3.5s per 2,000 rows", + * i.e. ~1.75 ms/row. + * + * Both are in the tree today, both describe a one-sided ascending range read over the same index on + * the same table, and which one is right decides the architecture. At 21 us/row you can afford to + * stream the whole due set through a bounded heap once a minute and order it exactly. At 1.75 ms/row + * a 500k-row due set takes 15 minutes to walk and any design that reads it is dead on arrival. + * + * The likely explanation is that they measure different things — `backlogSnapshot` yields to the + * event loop every 200 rows BESIDE BOT TRAFFIC and seeks the absolute index minimum, while the claim + * scan seeks a floor and never yields — but "likely" is not a basis for a schema change. So this + * harness measures both, separately, with the yielding as an explicit variable. + * + * ── WHAT IT ANSWERS, IN THE ORDER THE DESIGN NEEDS IT ─────────────────────────────────────────── + * + * Q1 Per-row cost of the claim-shaped read (one-sided, ascending, projected) at limits from 20 to + * 20,000. Decides whether ANY "buffer everything due now" design is viable. + * Q2 How much of Q1 is the yielding, not the storage engine. Isolates the 80x. + * Q3 K small per-lane seeks vs one large seek for the same number of granted rows — the cost of + * the interleaved-lane design in #116. + * Q4 `put` (whole record) vs a single-attribute update, on the reschedule path. Decides whether a + * lane change is cheap and whether splitting the column changes write cost. + * Q5 One indexed attribute vs two, on the write path. Tests #80's claim that a second index + * roughly doubles the hot write — which is the entire argument against a per-lane table. + * Q6 Two-sided range vs one-sided, with a limit that can fill and a limit that cannot. Tests the + * 1,128-2,977 ms figure in `claimSchedules` that keeps the `<= now` half in application code. + * Q7 Whether the seek point degrades as rows churn away from it (the 0.36 -> 6.25 ms finding that + * the claim floor exists to fix), and whether a floored seek is immune. + * + * ── HOW TO RUN IT ────────────────────────────────────────────────────────────────────────────── + * + * ROWS=200000 harper run bench/queue-index + * + * Against an ISOLATED Harper root, on its own ports — never a root that another instance is using, + * and never one holding real data. It writes hundreds of thousands of rows and drops its databases + * on the way in. `bench/queue-index/run.sh` sets that up. + * + * ROWS defaults to 200,000, matching #80 so the numbers are comparable. The production corpus is + * 1,619,000 keys (814,200 targets x 2 device types); per-row costs are what transfer, not totals. + */ + +const ROWS = Math.max(1_000, Number(process.env.ROWS) || 200_000); +const REPEATS = Math.max(1, Number(process.env.REPEATS) || 5); +const MINUTE = 60_000; + +// Realistic key shape and length: the index stores these, so a short synthetic key would understate +// every read. Mirrors the production cache key (`url|deviceType`) on the route that dominates the +// corpus. +const keyFor = (i) => `https://www.kohls.com/product/prd-${i}/some-reasonably-long-product-slug|desktop`; + +// The lane encoding under test — 2^42 ms per lane, lane 0 identity. Kept local so this harness has +// no dependency on the branch that introduces it. +const LANE_STRIDE = 2 ** 42; + +const now = Date.now(); +// Due times spread over a 24h window ending now, so most rows are overdue and the shape matches a +// node that is behind — the state ordering actually matters in. +const dueFor = (i) => now - Math.floor((i / ROWS) * 24 * 60) * MINUTE; + +const pct = (sorted, p) => sorted[Math.min(sorted.length - 1, Math.floor((sorted.length - 1) * p))]; + +/** Run `fn` REPEATS times and report min/median — min is the least noisy comparison. */ +const time = async (label, fn) => { + const samples = []; + let result; + for (let r = 0; r < REPEATS; r++) { + const started = performance.now(); + result = await fn(); + samples.push(performance.now() - started); + } + samples.sort((a, b) => a - b); + return { label, minMs: samples[0], medianMs: pct(samples, 0.5), rows: result ?? null }; +}; + +const drain = async (iterable) => { + let n = 0; + let last = -Infinity; + let outOfOrder = 0; + for await (const row of iterable) { + // ASSERTED, not assumed. A read that silently stops being index-ordered would make every + // number here meaningless while still looking fast — the exact failure the query-shape comment + // in `claimSchedules` warns about. + const v = Number(row.queueKey ?? row.nextRenderTime); + if (v < last) outOfOrder++; + last = v; + n++; + } + return { n, outOfOrder }; +}; + +/** The same drain, yielding to the event loop every `every` rows — backlogSnapshot's shape. */ +const drainYielding = async (iterable, every = 200) => { + let n = 0; + for await (const _row of iterable) { + if (++n % every === 0) await new Promise((resolve) => setImmediate(resolve)); + } + return { n, outOfOrder: 0 }; +}; + +const oneSided = (table, from, limit, attr = 'nextRenderTime') => + table.search( + { + conditions: [{ attribute: attr, comparator: 'greater_than_equal', value: from }], + sort: { attribute: attr }, + select: ['cacheKey', attr, 'fromSitemap'], + limit, + }, + { replicateFrom: false } + ); + +const twoSided = (table, from, to, limit, attr = 'nextRenderTime') => + table.search( + { + conditions: [ + { attribute: attr, comparator: 'greater_than_equal', value: from }, + { attribute: attr, comparator: 'less_than_equal', value: to }, + ], + sort: { attribute: attr }, + select: ['cacheKey', attr, 'fromSitemap'], + limit, + }, + { replicateFrom: false } + ); + +const seed = async (table, shape) => { + const started = performance.now(); + for (let i = 0; i < ROWS; i++) { + const dueAt = dueFor(i); + if (shape === 'A') { + await table.put(keyFor(i), { nextRenderTime: dueAt, fromSitemap: i % 2 === 0 }); + } else { + // Lanes spread across the corpus the way the taxonomy would: a small fast band, a large + // slow one, and a discovered tail. The distribution matters for the per-lane seek test. + const lane = i % 100 === 0 ? 1 : i % 3 === 0 ? 2 : 4; + await table.put(keyFor(i), { + queueKey: lane * LANE_STRIDE + dueAt, + dueAt, + fromSitemap: i % 2 === 0, + }); + } + // Yield periodically or the seed monopolizes the loop and the storage engine never gets to + // flush; this is the seed, not a measurement, so the yield cost is not being attributed to + // anything. + if (i % 2_000 === 0) await new Promise((resolve) => setImmediate(resolve)); + } + return performance.now() - started; +}; + +export async function handleApplication() { + const { BenchA, BenchB, BenchC } = databases.bench; + const out = { rows: ROWS, repeats: REPEATS, harper: server?.version ?? 'unknown', phases: {} }; + const log = (...args) => console.log(...args); + + log(`[bench] seeding ${ROWS.toLocaleString()} rows x 3 shapes...`); + + // ---- Q5 / write cost: seeding IS the write benchmark ------------------------------------------ + const seedA = await seed(BenchA, 'A'); + const seedB = await seed(BenchB, 'B'); + const seedC = await seed(BenchC, 'C'); + out.phases.write = { + note: 'Q5 — per-row put cost. A and B have ONE indexed attribute, C has two. If C is materially ' + + 'slower than B, a second index really does cost what #80 says and a per-lane table is expensive; ' + + 'if A and B match, splitting the deadline out of the queue key is free.', + A_oneIndex_usPerRow: (seedA * 1000) / ROWS, + B_oneIndex_split_usPerRow: (seedB * 1000) / ROWS, + C_twoIndexes_usPerRow: (seedC * 1000) / ROWS, + }; + log('[bench] write:', JSON.stringify(out.phases.write, null, 2)); + + // ---- Q1 / Q2: the claim-shaped read, and how much of it is the yielding ----------------------- + const floor = now - 24 * 60 * MINUTE; + const reads = []; + for (const limit of [20, 200, 2_000, 20_000]) { + const plain = await time(`oneSided limit=${limit}`, () => drain(oneSided(BenchA, floor, limit))); + const yielded = await time(`oneSided limit=${limit} yielding/200`, () => + drainYielding(oneSided(BenchA, floor, limit)) + ); + reads.push({ + limit, + rowsRead: plain.rows.n, + outOfOrder: plain.rows.outOfOrder, + plain_usPerRow: (plain.minMs * 1000) / Math.max(1, plain.rows.n), + plain_totalMs: plain.minMs, + yielding_usPerRow: (yielded.minMs * 1000) / Math.max(1, yielded.rows.n), + yielding_totalMs: yielded.minMs, + }); + } + out.phases.read = { + note: 'Q1/Q2 — per-row cost of the claim-shaped read, plain vs yielding every 200 rows. The tree ' + + 'holds two figures for this that differ 80x (21us/row in renderSchedule.js, 1.75ms/row in ' + + 'backlogSnapshot.js). If the plain number is the small one and yielding explains the rest, then ' + + 'a background sweep of the due set is affordable and a ready-set design is on the table.', + samples: reads, + }; + log('[bench] read:', JSON.stringify(out.phases.read, null, 2)); + + // ---- Q3: K per-lane seeks vs one large seek --------------------------------------------------- + const lanes = [1, 2, 4]; + const perLane = await time('3 lane seeks, limit 20 each', async () => { + let total = 0; + for (const lane of lanes) { + const from = lane * LANE_STRIDE + floor; + total += (await drain(oneSided(BenchB, from, 20, 'queueKey'))).n; + } + return { n: total, outOfOrder: 0 }; + }); + const oneBig = await time('1 seek, limit 60', () => drain(oneSided(BenchB, floor, 60, 'queueKey'))); + out.phases.lanes = { + note: 'Q3 — the interleaved-lane design pays one seek per lane. #80 measured 0.29-0.32ms per lane ' + + 'and claimed interleaving is free; this is that claim against one seek for the same row count.', + threeLaneSeeks_ms: perLane.minMs, + oneSeekSameRows_ms: oneBig.minMs, + }; + log('[bench] lanes:', JSON.stringify(out.phases.lanes, null, 2)); + + // ---- Q6: two-sided range, fillable and not --------------------------------------------------- + const fillable = await time('twoSided, limit fills', () => drain(twoSided(BenchA, floor, now, 20))); + // A window with almost nothing in it, so the limit can never fill — the "nothing is due" steady + // state, which is where the 480x regression was measured. + const empty = await time('twoSided, limit cannot fill', () => + drain(twoSided(BenchA, now + 10 * MINUTE, now + 11 * MINUTE, 20)) + ); + const emptyOneSided = await time('oneSided, same empty window', () => + drain(oneSided(BenchA, now + 10 * MINUTE, 20)) + ); + out.phases.twoSided = { + note: 'Q6 — `claimSchedules` keeps the `<= now` half in application code because a two-sided range ' + + 'measured 1,128-2,977ms when the limit cannot fill (only the first condition becomes the index ' + + 'range; the second is a post-filter). This is that comparison.', + fillable_ms: fillable.minMs, + cannotFill_ms: empty.minMs, + cannotFill_oneSided_ms: emptyOneSided.minMs, + }; + log('[bench] twoSided:', JSON.stringify(out.phases.twoSided, null, 2)); + + // ---- Q4 / Q7: reschedule churn, and whether the seek point degrades -------------------------- + // The reschedule pattern: read the head, move those rows into the future, repeat. This is what + // leaves dead index entries AT the seek point, and it is the measurement the claim floor exists + // to answer (0.36 -> 6.25ms over 40,000 reschedules, permanent). + const churn = Math.min(ROWS, Number(process.env.CHURN) || 40_000); + const headSeekBefore = await time('unfloored head seek, before churn', () => + drain(oneSided(BenchA, 0, 20)) + ); + let moved = 0; + const churnStarted = performance.now(); + for (let i = 0; i < churn; i++) { + // Single-attribute update, not a whole-record put: Q4. If this is materially cheaper than the + // seed's per-row put, then an in-place lane change is cheap and the encoding is a good deal. + await BenchA.patch(keyFor(i), { nextRenderTime: now + (i % (24 * 60)) * MINUTE }); + moved++; + if (i % 2_000 === 0) await new Promise((resolve) => setImmediate(resolve)); + } + const churnMs = performance.now() - churnStarted; + const headSeekAfter = await time('unfloored head seek, after churn', () => drain(oneSided(BenchA, 0, 20))); + const flooredSeekAfter = await time('FLOORED seek, after churn', () => + drain(oneSided(BenchA, now - 60 * MINUTE, 20)) + ); + out.phases.churn = { + note: 'Q4/Q7 — per-row single-attribute patch cost (vs the whole-record put above), and whether the ' + + 'seek point degrades as rows churn away from it. A floored seek should be immune; an unfloored ' + + 'one should not.', + reschedules: moved, + patch_usPerRow: (churnMs * 1000) / Math.max(1, moved), + unflooredSeekBefore_ms: headSeekBefore.minMs, + unflooredSeekAfter_ms: headSeekAfter.minMs, + flooredSeekAfter_ms: flooredSeekAfter.minMs, + }; + log('[bench] churn:', JSON.stringify(out.phases.churn, null, 2)); + + log('\n[bench] RESULT\n' + JSON.stringify(out, null, 2)); + // Exit rather than leave a server up: this is a one-shot measurement, and the numbers above are + // only valid while nothing else is touching the tables. + process.exit(0); +} diff --git a/bench/queue-index/config.yaml b/bench/queue-index/config.yaml new file mode 100644 index 0000000..181389c --- /dev/null +++ b/bench/queue-index/config.yaml @@ -0,0 +1,11 @@ +# Benchmark component for the RenderSchedule queue index. +# +# NOT SHIPPED. This is a dev harness, declared as its own Harper component so it runs against a real +# Harper storage engine rather than against a mock — the whole point is to price the storage layer, +# which is the one thing a unit test cannot tell you. +rest: true + +graphqlSchema: + files: 'schema.graphql' + +pluginModule: 'bench.js' diff --git a/bench/queue-index/run.sh b/bench/queue-index/run.sh new file mode 100755 index 0000000..949c572 --- /dev/null +++ b/bench/queue-index/run.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +# Run the queue-index benchmark against an ISOLATED Harper. +# +# Isolation is not a nicety. The harness writes hundreds of thousands of rows and measures a storage +# engine, so it must not share a root with a running instance (two processes on one LMDB root), must +# not share a root with real data, and must not collide on ports with a Harper you already have up. +# +# BENCH_MODE=docker (default) a throwaway container, the way kohls-pr's CI stands Harper up. +# BENCH_MODE=local an installed local root under $BENCH_ROOT. +# +# VERIFICATION STATUS, stated because an unrun benchmark runner is worse than none: neither mode has +# been executed end to end. Docker was unavailable on the machine this was written on (daemon not +# running), and the local mode reached Harper's startup and then failed in `checkForExistingInstall` +# ("database 'system' does not exist") against a freshly installed root — a Harper bootstrap problem, +# not a problem with the measurements. Expect to debug the harness once before trusting a number +# from it, and treat the first run as a calibration run. +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +MODE="${BENCH_MODE:-docker}" +PORT="${BENCH_PORT:-9977}" +ROWS="${ROWS:-200000}" +REPEATS="${REPEATS:-5}" +CHURN="${CHURN:-40000}" +HDB_VERSION="${HDB_VERSION:-latest}" + +if [[ "$MODE" == "docker" ]]; then + CONTAINER="${BENCH_CONTAINER:-prerender-queue-bench}" + docker rm -f "$CONTAINER" >/dev/null 2>&1 || true + echo "run.sh: starting harperfast/harper:$HDB_VERSION as $CONTAINER on $PORT" + # The component is MOUNTED rather than deployed: `harper deploy` packs and installs, which is the + # right thing for a real component and pointless overhead for a harness that only needs its two + # files visible. If the image's component root differs, override BENCH_COMPONENT_DIR. + docker run --rm \ + --name "$CONTAINER" \ + -e HDB_ADMIN_USERNAME=bench_admin \ + -e HDB_ADMIN_PASSWORD="bench_only_$RANDOM" \ + -e OPERATIONSAPI_NETWORK_PORT="$((PORT + 4))" \ + -e THREADS_COUNT=1 \ + -e ROWS="$ROWS" -e REPEATS="$REPEATS" -e CHURN="$CHURN" \ + -p "$PORT:9926" \ + -v "$HERE:${BENCH_COMPONENT_DIR:-/hdb/components/queue-index}:ro" \ + "harperfast/harper:$HDB_VERSION" + exit $? +fi + +ROOT="${BENCH_ROOT:-${TMPDIR:-/tmp}/prerender-bench-root}" +case "$ROOT" in + "$HOME"/hdb|"$HOME"/hdb/*) echo "run.sh: refusing to use your real Harper root ($ROOT)" >&2; exit 1 ;; +esac + +if [[ -n "${KEEP_ROOT:-}" && -d "$ROOT/database/system" ]]; then + echo "run.sh: reusing $ROOT (KEEP_ROOT set)" +else + rm -rf "$ROOT" + echo "run.sh: installing a fresh Harper root at $ROOT" + # Non-interactive: the installer takes the uppercased prompt names as env vars, so no TTY needed. + ROOTPATH="$ROOT" \ + HDB_ADMIN_USERNAME=bench_admin \ + HDB_ADMIN_PASSWORD="bench_only_$RANDOM" \ + HTTP_PORT="$PORT" \ + OPERATIONSAPI_NETWORK_PORT="$((PORT + 4))" \ + REPLICATION_SECUREPORT="$((PORT + 1))" \ + harper install +fi + +CFG="$ROOT/harperdb-config.yaml" +# One worker: these are per-operation costs, and several workers racing the same tables would measure +# contention instead. +[[ -f "$CFG" ]] && { sed -i.bak -E '/^threads:/,/^[a-zA-Z]/ s/^ count: .*/ count: 1/' "$CFG"; rm -f "$CFG.bak"; } + +echo "run.sh: root=$ROOT port=$PORT rows=$ROWS" +cd "$HERE" +ROOTPATH="$ROOT" ROWS="$ROWS" REPEATS="$REPEATS" CHURN="$CHURN" harper run . diff --git a/bench/queue-index/schema.graphql b/bench/queue-index/schema.graphql new file mode 100644 index 0000000..fac6332 --- /dev/null +++ b/bench/queue-index/schema.graphql @@ -0,0 +1,29 @@ +# Three candidate shapes for the render queue's schedule table. Same row count, same key shape, same +# access patterns — the only variable is how the scheduling order is stored and indexed. + +# A — TODAY. One indexed Long that is BOTH the freshness deadline and the queue position. +type BenchA @table(database: "bench") @export { + cacheKey: String @primaryKey + nextRenderTime: Long @indexed + fromSitemap: Boolean +} + +# B — THE SPLIT. The queue position is its own indexed column; the deadline is a plain unindexed +# timestamp that nothing has to decode. Same NUMBER of indexed attributes as A, so if A and B write +# at the same rate then separating the two concepts is free. +type BenchB @table(database: "bench") @export { + cacheKey: String @primaryKey + queueKey: Long @indexed + dueAt: Long + fromSitemap: Boolean +} + +# C — BOTH INDEXED. Prices the second index on the hot write path, which is the claim in #80 that +# "the existing secondary index is 39-48% of reschedule wall clock" and therefore that a per-lane +# index or table roughly doubles it. +type BenchC @table(database: "bench") @export { + cacheKey: String @primaryKey + queueKey: Long @indexed + dueAt: Long @indexed + fromSitemap: Boolean +} From 30612f92ddef530e595c39708e202da2da30d5dd Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 21 Aug 2026 14:52:34 -0400 Subject: [PATCH 2/5] bench(plugin): make the two scale-dependent measurements actually measure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both were wrong in the first run, and one of them looked like a refutation. Q6 (two-sided range) used a window ABOVE every seeded row, so there were no rows above the lower bound at all — and the failure mode being tested is precisely "O(rows above the lower bound)". It measured an empty seek and reported it as cheap. The window now sits LOW and NARROW with a limit larger than the number of matches, so the post-filter has to walk. Q7 (seek degradation) patched keys in key order. Production repeatedly CLAIMS THE HEAD and writes those rows forward, so dead index entries pile up at the seek point rather than spreading over a key range — which is the whole effect. It now reads the head, moves what it read, repeats, and samples both seeks as it goes so a trend is visible instead of a before/after pair one cold page can dominate. The floored seek was also pinned at `now - 60min`, inside the region the churn had just rewritten, which is why it measured SLOWER than the unfloored seek; the floor now follows the first row observed, as production's does. Also records the three things that made the harness silently succeed while running nothing: a read-only mount fails Harper's component loader with EROFS on the node_modules symlink; `handleApplication` is never invoked on a root component (it is the hook for a component used AS an extension, so it needs `jsResource`); and Harper intercepts `process.exit`. Co-Authored-By: Claude Opus 5 --- bench/queue-index/README.md | 14 +++- bench/queue-index/bench.js | 144 +++++++++++++++++++++++++--------- bench/queue-index/config.yaml | 7 +- bench/queue-index/run.sh | 13 ++- 4 files changed, 132 insertions(+), 46 deletions(-) diff --git a/bench/queue-index/README.md b/bench/queue-index/README.md index 485b51b..4e45c36 100644 --- a/bench/queue-index/README.md +++ b/bench/queue-index/README.md @@ -38,9 +38,17 @@ hundreds of thousands of rows, and it refuses to run against `~/hdb`. `ROWS` defaults to 200,000 to match #80 so the numbers are comparable. Production is 1,619,000 keys (814,200 targets × 2 device types) — per-row costs are what transfer, not totals. -> **Neither mode has been run end to end yet.** Docker was unavailable where this was written, and -> local mode failed in Harper's `checkForExistingInstall` against a fresh root. Treat the first run -> as calibration. +Docker mode is the verified path (Harper 5.2.4, `ROWS=200000`). Two things about it are worth +knowing before you edit the harness: + +- The component is **staged into a temp dir and mounted read-write.** Harper's component loader + creates `node_modules` inside the component directory to symlink the `harper` module, so a + read-only mount fails the component with `EROFS` — and the server then comes up perfectly happy + having loaded nothing. +- The entry point is **`jsResource`, not `pluginModule`.** `handleApplication` is the hook Harper + calls on a component another component *uses* as an extension; a root component declaring it is + simply never invoked. Same silent-success failure. +- Harper **intercepts `process.exit`**, so the harness signals itself instead. ## What it answers, and what each answer decides diff --git a/bench/queue-index/bench.js b/bench/queue-index/bench.js index 718b9ae..cce376c 100644 --- a/bench/queue-index/bench.js +++ b/bench/queue-index/bench.js @@ -155,7 +155,22 @@ const seed = async (table, shape) => { return performance.now() - started; }; -export async function handleApplication() { +// Runs on load. `jsResource` modules are imported after the schema is applied, so `databases.bench` +// is populated by the time this executes; the retry below exists only so a load-order change in +// Harper reports itself as a wait rather than as a TypeError on `databases.bench`. +const tablesReady = async () => { + for (let i = 0; i < 50; i++) { + if (databases?.bench?.BenchA) return true; + await new Promise((resolve) => setTimeout(resolve, 200)); + } + return false; +}; + +async function main() { + if (!(await tablesReady())) { + console.error('[bench] databases.bench never appeared — the schema did not load'); + process.exit(1); + } const { BenchA, BenchB, BenchC } = databases.bench; const out = { rows: ROWS, repeats: REPEATS, harper: server?.version ?? 'unknown', phases: {} }; const log = (...args) => console.log(...args); @@ -222,23 +237,41 @@ export async function handleApplication() { }; log('[bench] lanes:', JSON.stringify(out.phases.lanes, null, 2)); - // ---- Q6: two-sided range, fillable and not --------------------------------------------------- - const fillable = await time('twoSided, limit fills', () => drain(twoSided(BenchA, floor, now, 20))); - // A window with almost nothing in it, so the limit can never fill — the "nothing is due" steady - // state, which is where the 480x regression was measured. - const empty = await time('twoSided, limit cannot fill', () => - drain(twoSided(BenchA, now + 10 * MINUTE, now + 11 * MINUTE, 20)) + // ---- Q6: two-sided range, and it has to be set up correctly to mean anything ----------------- + // + // THE FIRST VERSION OF THIS TEST WAS WORTHLESS and it is worth saying why, because it looked like + // a refutation. It used a window ABOVE every seeded row (`now+10min .. now+11min`), so there were + // no rows above the lower bound at all — and the described failure mode is precisely + // "O(rows above the lower bound)". It measured an empty seek and reported it as cheap. + // + // To exercise the post-filter the window must be LOW (so almost the whole table sits above the + // lower bound) and NARROW (so the matching rows run out), with a limit LARGER than the number of + // matches — then the engine keeps walking past the window hunting for matches that do not exist. + // Due times here are minute-floored across 1,440 distinct minutes, so one minute holds about + // ROWS/1440 rows; a limit well above that cannot fill. + const oldest = dueFor(ROWS - 1); + const perMinute = Math.ceil(ROWS / 1440); + const cannotFillLimit = perMinute * 10; + const narrowTwoSided = await time(`twoSided, low+narrow window, limit ${cannotFillLimit} cannot fill`, () => + drain(twoSided(BenchA, oldest, oldest + MINUTE, cannotFillLimit)) ); - const emptyOneSided = await time('oneSided, same empty window', () => - drain(oneSided(BenchA, now + 10 * MINUTE, 20)) + const sameOneSided = await time(`oneSided, same lower bound, limit ${cannotFillLimit}`, () => + drain(oneSided(BenchA, oldest, cannotFillLimit)) ); + const fillable = await time('twoSided, wide window, limit fills', () => drain(twoSided(BenchA, floor, now, 20))); out.phases.twoSided = { note: 'Q6 — `claimSchedules` keeps the `<= now` half in application code because a two-sided range ' + 'measured 1,128-2,977ms when the limit cannot fill (only the first condition becomes the index ' - + 'range; the second is a post-filter). This is that comparison.', - fillable_ms: fillable.minMs, - cannotFill_ms: empty.minMs, - cannotFill_oneSided_ms: emptyOneSided.minMs, + + 'range, the second is a post-filter, so the cost is O(rows above the lower bound)). The window ' + + 'must be low and narrow with an unfillable limit or the test does not touch that path.', + rowsAboveLowerBound: ROWS, + matchesInWindow: perMinute, + limitThatCannotFill: cannotFillLimit, + twoSided_cannotFill_ms: narrowTwoSided.minMs, + twoSided_cannotFill_rowsReturned: narrowTwoSided.rows.n, + oneSided_sameLowerBound_ms: sameOneSided.minMs, + oneSided_rowsReturned: sameOneSided.rows.n, + twoSided_fillable_ms: fillable.minMs, }; log('[bench] twoSided:', JSON.stringify(out.phases.twoSided, null, 2)); @@ -246,38 +279,73 @@ export async function handleApplication() { // The reschedule pattern: read the head, move those rows into the future, repeat. This is what // leaves dead index entries AT the seek point, and it is the measurement the claim floor exists // to answer (0.36 -> 6.25ms over 40,000 reschedules, permanent). + // PRODUCTION-SHAPED CHURN, which the first version of this was not. It patched keys 0..N in key + // order; production repeatedly CLAIMS THE HEAD of the index and writes those rows into the future, + // so the dead entries accumulate at the seek point rather than being spread over a key range. The + // difference matters exactly for the effect being measured, so this reads the head, moves what it + // read, and repeats — and samples the seek cost as it goes, so a trend is visible rather than only + // a before/after pair that one cold page can dominate. + // + // The floored seek is also fixed. Pinning it at `now - 60min` put it in the region the churn had + // just rewritten, which is why it measured SLOWER than the unfloored seek; production's floor + // tracks the oldest row still due, so that is what is tracked here. const churn = Math.min(ROWS, Number(process.env.CHURN) || 40_000); - const headSeekBefore = await time('unfloored head seek, before churn', () => - drain(oneSided(BenchA, 0, 20)) - ); + const batch = 20; + const trend = []; let moved = 0; - const churnStarted = performance.now(); - for (let i = 0; i < churn; i++) { - // Single-attribute update, not a whole-record put: Q4. If this is materially cheaper than the - // seed's per-row put, then an in-place lane change is cheap and the encoding is a good deal. - await BenchA.patch(keyFor(i), { nextRenderTime: now + (i % (24 * 60)) * MINUTE }); - moved++; - if (i % 2_000 === 0) await new Promise((resolve) => setImmediate(resolve)); + let churnMs = 0; + let floorValue = 0; + + const sampleSeek = async () => ({ + unfloored: (await time('unfloored', () => drain(oneSided(BenchA, 0, batch)))).minMs, + floored: (await time('floored', () => drain(oneSided(BenchA, floorValue, batch)))).minMs, + }); + trend.push({ reschedules: 0, ...(await sampleSeek()) }); + + while (moved < churn) { + // Read the head the way `claim` does... + const head = []; + for await (const row of oneSided(BenchA, floorValue, batch)) head.push(row); + if (head.length === 0) break; + // ...and let the floor follow the first row observed, which is the production rule. + floorValue = Number(head[0].nextRenderTime); + + const started = performance.now(); + for (const row of head) { + // Whole-record put, matching `processJobResult`: one write per completed render, moving the + // row a full interval into the future. + await BenchA.put(row.cacheKey, { + nextRenderTime: now + 24 * 60 * MINUTE + moved * MINUTE, + fromSitemap: row.fromSitemap, + }); + moved++; + } + churnMs += performance.now() - started; + if (moved % 2_000 < batch) { + await new Promise((resolve) => setImmediate(resolve)); + trend.push({ reschedules: moved, ...(await sampleSeek()) }); + } } - const churnMs = performance.now() - churnStarted; - const headSeekAfter = await time('unfloored head seek, after churn', () => drain(oneSided(BenchA, 0, 20))); - const flooredSeekAfter = await time('FLOORED seek, after churn', () => - drain(oneSided(BenchA, now - 60 * MINUTE, 20)) - ); + trend.push({ reschedules: moved, ...(await sampleSeek()) }); + out.phases.churn = { - note: 'Q4/Q7 — per-row single-attribute patch cost (vs the whole-record put above), and whether the ' - + 'seek point degrades as rows churn away from it. A floored seek should be immune; an unfloored ' - + 'one should not.', + note: 'Q7 — does the seek point degrade as rows churn away from it (the 0.36 -> 6.25ms finding the ' + + 'claim floor exists to fix), and is a floored seek immune? Head-claim-then-reschedule, the ' + + 'production shape, sampling both seeks as it goes. Also Q4: per-row put on the reschedule path.', reschedules: moved, - patch_usPerRow: (churnMs * 1000) / Math.max(1, moved), - unflooredSeekBefore_ms: headSeekBefore.minMs, - unflooredSeekAfter_ms: headSeekAfter.minMs, - flooredSeekAfter_ms: flooredSeekAfter.minMs, + put_usPerRow: (churnMs * 1000) / Math.max(1, moved), + trend, }; log('[bench] churn:', JSON.stringify(out.phases.churn, null, 2)); log('\n[bench] RESULT\n' + JSON.stringify(out, null, 2)); - // Exit rather than leave a server up: this is a one-shot measurement, and the numbers above are - // only valid while nothing else is touching the tables. - process.exit(0); + // Harper INTERCEPTS `process.exit`, so a one-shot harness cannot end itself that way — it printed + // its results and then sat there as a running server. Signal the process instead. + console.log('[bench] done'); + process.kill(process.pid, 'SIGTERM'); } + +main().catch((e) => { + console.error('[bench] failed', e); + process.exit(1); +}); diff --git a/bench/queue-index/config.yaml b/bench/queue-index/config.yaml index 181389c..bfe99e6 100644 --- a/bench/queue-index/config.yaml +++ b/bench/queue-index/config.yaml @@ -8,4 +8,9 @@ rest: true graphqlSchema: files: 'schema.graphql' -pluginModule: 'bench.js' +# `jsResource`, NOT `pluginModule`. `handleApplication` is the hook Harper calls on a component that +# another component USES as an extension — a root component declaring it is simply never invoked, and +# the server comes up perfectly healthy having run nothing. A jsResource module runs on load, which is +# what a one-shot harness wants. +jsResource: + files: 'bench.js' diff --git a/bench/queue-index/run.sh b/bench/queue-index/run.sh index 949c572..22c1003 100755 --- a/bench/queue-index/run.sh +++ b/bench/queue-index/run.sh @@ -28,9 +28,14 @@ if [[ "$MODE" == "docker" ]]; then CONTAINER="${BENCH_CONTAINER:-prerender-queue-bench}" docker rm -f "$CONTAINER" >/dev/null 2>&1 || true echo "run.sh: starting harperfast/harper:$HDB_VERSION as $CONTAINER on $PORT" - # The component is MOUNTED rather than deployed: `harper deploy` packs and installs, which is the - # right thing for a real component and pointless overhead for a harness that only needs its two - # files visible. If the image's component root differs, override BENCH_COMPONENT_DIR. + # STAGED INTO A TEMP DIR AND MOUNTED READ-WRITE, not mounted read-only from the repo. Harper's + # component loader creates `node_modules` inside the component directory to symlink the `harper` + # module, so a read-only mount fails the whole component with EROFS and the server comes up + # perfectly happy with nothing loaded. Staging keeps that write out of the working tree. + STAGE="$(mktemp -d)" + cp "$HERE"/config.yaml "$HERE"/schema.graphql "$HERE"/bench.js "$STAGE/" + trap 'rm -rf "$STAGE"' EXIT + docker run --rm \ --name "$CONTAINER" \ -e HDB_ADMIN_USERNAME=bench_admin \ @@ -39,7 +44,7 @@ if [[ "$MODE" == "docker" ]]; then -e THREADS_COUNT=1 \ -e ROWS="$ROWS" -e REPEATS="$REPEATS" -e CHURN="$CHURN" \ -p "$PORT:9926" \ - -v "$HERE:${BENCH_COMPONENT_DIR:-/hdb/components/queue-index}:ro" \ + -v "$STAGE:${BENCH_COMPONENT_DIR:-/home/harperdb/harper/components/queue-index}" \ "harperfast/harper:$HDB_VERSION" exit $? fi From 1ab6fe4bdb359e9775da22ba434cf5455ffa704f Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 21 Aug 2026 16:06:12 -0400 Subject: [PATCH 3/5] fix(bench): make the failure paths actually exit, and the staged mount readable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two findings from @gemini-code-assist on #119, both applied. PROCESS.EXIT ON THE ERROR PATHS. I fixed the success path to signal itself because Harper intercepts `process.exit`, and then left both failure paths on `process.exit(1)` — so a schema that never loaded, or any unhandled rejection, printed its message and then hung as a running server. A failed run looked exactly like a slow one. Both now signal SIGTERM. MKTEMP PERMISSIONS. `mktemp -d` is 0700 owned by the host user, and the container runs as `harperdb` and has to CREATE `node_modules` inside the mount to symlink itself. Without a chmod the component fails with EACCES and the server comes up perfectly healthy having loaded nothing — the same silent-success failure this harness already hit once with a read-only mount. Docker Desktop on macOS remaps ownership and hid it; on Linux and in CI it would not. Safe to widen: a throwaway directory holding three harness files, removed on exit. --- bench/queue-index/bench.js | 9 +++++++-- bench/queue-index/run.sh | 7 +++++++ 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/bench/queue-index/bench.js b/bench/queue-index/bench.js index cce376c..5aac2a4 100644 --- a/bench/queue-index/bench.js +++ b/bench/queue-index/bench.js @@ -169,7 +169,10 @@ const tablesReady = async () => { async function main() { if (!(await tablesReady())) { console.error('[bench] databases.bench never appeared — the schema did not load'); - process.exit(1); + // SIGTERM, not `process.exit`: Harper intercepts exit, so this path would print and then hang as + // a running server. Same reason as the success path at the end of `main`. + process.kill(process.pid, 'SIGTERM'); + return; } const { BenchA, BenchB, BenchC } = databases.bench; const out = { rows: ROWS, repeats: REPEATS, harper: server?.version ?? 'unknown', phases: {} }; @@ -347,5 +350,7 @@ async function main() { main().catch((e) => { console.error('[bench] failed', e); - process.exit(1); + // SIGTERM for the same reason: an intercepted `process.exit` leaves a failed run hanging, which + // looks exactly like a slow one. + process.kill(process.pid, 'SIGTERM'); }); diff --git a/bench/queue-index/run.sh b/bench/queue-index/run.sh index 22c1003..8a8e26d 100755 --- a/bench/queue-index/run.sh +++ b/bench/queue-index/run.sh @@ -33,6 +33,13 @@ if [[ "$MODE" == "docker" ]]; then # module, so a read-only mount fails the whole component with EROFS and the server comes up # perfectly happy with nothing loaded. Staging keeps that write out of the working tree. STAGE="$(mktemp -d)" + # WORLD-WRITABLE ON PURPOSE. `mktemp -d` is 0700 owned by the host user; the container runs as + # `harperdb`, and Harper's component loader has to CREATE `node_modules` inside the mount to symlink + # itself. Without this the component fails with EROFS/EACCES and the server comes up perfectly + # healthy having loaded nothing — the silent-success failure this harness has already hit once. + # (Docker Desktop on macOS remaps ownership and hides it; on Linux and in CI it does not.) Safe + # here: a throwaway directory holding three harness files, removed on exit. + chmod 777 "$STAGE" cp "$HERE"/config.yaml "$HERE"/schema.graphql "$HERE"/bench.js "$STAGE/" trap 'rm -rf "$STAGE"' EXIT From f4616386abb806c8aa047d077e8b103bd1914ca6 Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 21 Aug 2026 16:07:09 -0400 Subject: [PATCH 4/5] docs(bench): record the third silent-success failure, and name the pattern MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The chmod fix in 1ab6fe4 was the third time this harness came up healthy having loaded nothing — after a read-only mount and after `handleApplication` on a root component. Two of those were already listed; this adds the third and states the common shape, which is the actually useful warning: every one of them fails by SUCCEEDING. Harper starts, reports healthy, and runs none of the harness. So the operational rule is that a run printing no `[bench]` lines means the component did not load, not that the measurement is slow. --- bench/queue-index/README.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/bench/queue-index/README.md b/bench/queue-index/README.md index 4e45c36..b901970 100644 --- a/bench/queue-index/README.md +++ b/bench/queue-index/README.md @@ -48,7 +48,16 @@ knowing before you edit the harness: - The entry point is **`jsResource`, not `pluginModule`.** `handleApplication` is the hook Harper calls on a component another component *uses* as an extension; a root component declaring it is simply never invoked. Same silent-success failure. -- Harper **intercepts `process.exit`**, so the harness signals itself instead. +- The staged dir is **`chmod 777`.** `mktemp -d` is 0700 owned by the host user, and the container runs + as `harperdb` and has to *create* `node_modules` inside the mount — so without it the component + fails with `EACCES` and, again, the server comes up healthy having loaded nothing. Docker Desktop on + macOS remaps ownership and hides this; Linux and CI do not. +- Harper **intercepts `process.exit`**, so the harness signals itself instead — on the failure paths as + well as the success one, or a failed run hangs and looks exactly like a slow one. + +The pattern is the thing to watch, not any individual cause: **every one of these fails by succeeding.** +Harper starts, reports healthy, and runs none of the harness. If a run prints no `[bench]` lines, assume +the component did not load rather than that the measurement is slow. ## What it answers, and what each answer decides From 269104ec5dbbcc251dcf14b66f0eaba74f0959f6 Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 21 Aug 2026 16:07:36 -0400 Subject: [PATCH 5/5] style(bench): prettier The prior two commits were written in a worktree with no node_modules, so prettier could not run on them. --- bench/queue-index/README.md | 38 ++++++++++++++++++------------------ bench/queue-index/bench.js | 39 +++++++++++++++++++++---------------- 2 files changed, 41 insertions(+), 36 deletions(-) diff --git a/bench/queue-index/README.md b/bench/queue-index/README.md index b901970..269e54c 100644 --- a/bench/queue-index/README.md +++ b/bench/queue-index/README.md @@ -7,10 +7,10 @@ change is argued from figures anyone can regenerate. It also exists because the two figures currently in the tree **disagree by 80x**: -| source | claim | implied | -| --- | --- | --- | -| `src/util/renderSchedule.js` | the claim scan returns 20 keys in 0.43 ms | ~21 µs/row | -| `src/util/backlogSnapshot.js` | "~3.5s per 2,000 rows" | ~1.75 ms/row | +| source | claim | implied | +| ----------------------------- | ----------------------------------------- | ------------ | +| `src/util/renderSchedule.js` | the claim scan returns 20 keys in 0.43 ms | ~21 µs/row | +| `src/util/backlogSnapshot.js` | "~3.5s per 2,000 rows" | ~1.75 ms/row | Both describe a one-sided ascending range read over the same index on the same table, and which one is right decides the architecture: @@ -21,8 +21,8 @@ is right decides the architecture: - At **1.75 ms/row**, a 500k-row due set takes ~15 minutes to walk. Every design that reads the due set is dead, and priority has to be expressed in the index itself. -The likely reconciliation is that `backlogSnapshot` yields to the event loop every 200 rows *beside -bot traffic* and seeks the absolute index minimum, while the claim scan seeks a floor and never +The likely reconciliation is that `backlogSnapshot` yields to the event loop every 200 rows _beside +bot traffic_ and seeks the absolute index minimum, while the claim scan seeks a floor and never yields — so the harness measures yielding as an explicit variable rather than assuming it. ## Running it @@ -46,10 +46,10 @@ knowing before you edit the harness: read-only mount fails the component with `EROFS` — and the server then comes up perfectly happy having loaded nothing. - The entry point is **`jsResource`, not `pluginModule`.** `handleApplication` is the hook Harper - calls on a component another component *uses* as an extension; a root component declaring it is + calls on a component another component _uses_ as an extension; a root component declaring it is simply never invoked. Same silent-success failure. - The staged dir is **`chmod 777`.** `mktemp -d` is 0700 owned by the host user, and the container runs - as `harperdb` and has to *create* `node_modules` inside the mount — so without it the component + as `harperdb` and has to _create_ `node_modules` inside the mount — so without it the component fails with `EACCES` and, again, the server comes up healthy having loaded nothing. Docker Desktop on macOS remaps ownership and hides this; Linux and CI do not. - Harper **intercepts `process.exit`**, so the harness signals itself instead — on the failure paths as @@ -61,17 +61,17 @@ the component did not load rather than that the measurement is slow. ## What it answers, and what each answer decides -| | question | if the answer is… | then | -| --- | --- | --- | --- | -| Q1 | per-row cost of the claim-shaped read at limits 20 → 20,000 | small (~tens of µs) | a background sweep of the due set is affordable; exact ordering is on the table | -| | | large (~ms) | priority must live in the index; no design may read the due set | -| Q2 | how much of Q1 is the yielding, not the engine | yielding dominates | the 80× is an artefact and Q1's plain number is the real one | -| Q3 | K per-lane seeks vs one large seek | per-lane ≈ free | the interleaved-lane design in #116 is sound | -| Q4 | single-attribute update vs whole-record `put` | update much cheaper | an in-place lane change is cheap; encoding is a good deal | -| Q5 | one indexed attribute vs two, on the write | ≈ equal | #80's "a second index doubles the hot write" is wrong, and a per-lane **table** becomes viable | -| | | two much slower | splitting `dueAt` out of the queue key must keep `dueAt` **unindexed** | -| Q6 | two-sided vs one-sided range, limit fillable and not | two-sided catastrophic when it cannot fill | keep the `<= now` half in application code, as `claimSchedules` does | -| Q7 | does the seek point degrade as rows churn away from it | yes, and a floored seek is immune | the claim floor stays load-bearing under any new design | +| | question | if the answer is… | then | +| --- | ----------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------- | +| Q1 | per-row cost of the claim-shaped read at limits 20 → 20,000 | small (~tens of µs) | a background sweep of the due set is affordable; exact ordering is on the table | +| | | large (~ms) | priority must live in the index; no design may read the due set | +| Q2 | how much of Q1 is the yielding, not the engine | yielding dominates | the 80× is an artefact and Q1's plain number is the real one | +| Q3 | K per-lane seeks vs one large seek | per-lane ≈ free | the interleaved-lane design in #116 is sound | +| Q4 | single-attribute update vs whole-record `put` | update much cheaper | an in-place lane change is cheap; encoding is a good deal | +| Q5 | one indexed attribute vs two, on the write | ≈ equal | #80's "a second index doubles the hot write" is wrong, and a per-lane **table** becomes viable | +| | | two much slower | splitting `dueAt` out of the queue key must keep `dueAt` **unindexed** | +| Q6 | two-sided vs one-sided range, limit fillable and not | two-sided catastrophic when it cannot fill | keep the `<= now` half in application code, as `claimSchedules` does | +| Q7 | does the seek point degrade as rows churn away from it | yes, and a floored seek is immune | the claim floor stays load-bearing under any new design | Q4 and Q5 together decide the change I'd otherwise make on correctness grounds alone: giving the queue its **own** indexed column and leaving the freshness deadline as a plain unindexed timestamp, diff --git a/bench/queue-index/bench.js b/bench/queue-index/bench.js index 5aac2a4..1521bbf 100644 --- a/bench/queue-index/bench.js +++ b/bench/queue-index/bench.js @@ -185,9 +185,10 @@ async function main() { const seedB = await seed(BenchB, 'B'); const seedC = await seed(BenchC, 'C'); out.phases.write = { - note: 'Q5 — per-row put cost. A and B have ONE indexed attribute, C has two. If C is materially ' - + 'slower than B, a second index really does cost what #80 says and a per-lane table is expensive; ' - + 'if A and B match, splitting the deadline out of the queue key is free.', + note: + 'Q5 — per-row put cost. A and B have ONE indexed attribute, C has two. If C is materially ' + + 'slower than B, a second index really does cost what #80 says and a per-lane table is expensive; ' + + 'if A and B match, splitting the deadline out of the queue key is free.', A_oneIndex_usPerRow: (seedA * 1000) / ROWS, B_oneIndex_split_usPerRow: (seedB * 1000) / ROWS, C_twoIndexes_usPerRow: (seedC * 1000) / ROWS, @@ -213,10 +214,11 @@ async function main() { }); } out.phases.read = { - note: 'Q1/Q2 — per-row cost of the claim-shaped read, plain vs yielding every 200 rows. The tree ' - + 'holds two figures for this that differ 80x (21us/row in renderSchedule.js, 1.75ms/row in ' - + 'backlogSnapshot.js). If the plain number is the small one and yielding explains the rest, then ' - + 'a background sweep of the due set is affordable and a ready-set design is on the table.', + note: + 'Q1/Q2 — per-row cost of the claim-shaped read, plain vs yielding every 200 rows. The tree ' + + 'holds two figures for this that differ 80x (21us/row in renderSchedule.js, 1.75ms/row in ' + + 'backlogSnapshot.js). If the plain number is the small one and yielding explains the rest, then ' + + 'a background sweep of the due set is affordable and a ready-set design is on the table.', samples: reads, }; log('[bench] read:', JSON.stringify(out.phases.read, null, 2)); @@ -233,8 +235,9 @@ async function main() { }); const oneBig = await time('1 seek, limit 60', () => drain(oneSided(BenchB, floor, 60, 'queueKey'))); out.phases.lanes = { - note: 'Q3 — the interleaved-lane design pays one seek per lane. #80 measured 0.29-0.32ms per lane ' - + 'and claimed interleaving is free; this is that claim against one seek for the same row count.', + note: + 'Q3 — the interleaved-lane design pays one seek per lane. #80 measured 0.29-0.32ms per lane ' + + 'and claimed interleaving is free; this is that claim against one seek for the same row count.', threeLaneSeeks_ms: perLane.minMs, oneSeekSameRows_ms: oneBig.minMs, }; @@ -263,10 +266,11 @@ async function main() { ); const fillable = await time('twoSided, wide window, limit fills', () => drain(twoSided(BenchA, floor, now, 20))); out.phases.twoSided = { - note: 'Q6 — `claimSchedules` keeps the `<= now` half in application code because a two-sided range ' - + 'measured 1,128-2,977ms when the limit cannot fill (only the first condition becomes the index ' - + 'range, the second is a post-filter, so the cost is O(rows above the lower bound)). The window ' - + 'must be low and narrow with an unfillable limit or the test does not touch that path.', + note: + 'Q6 — `claimSchedules` keeps the `<= now` half in application code because a two-sided range ' + + 'measured 1,128-2,977ms when the limit cannot fill (only the first condition becomes the index ' + + 'range, the second is a post-filter, so the cost is O(rows above the lower bound)). The window ' + + 'must be low and narrow with an unfillable limit or the test does not touch that path.', rowsAboveLowerBound: ROWS, matchesInWindow: perMinute, limitThatCannotFill: cannotFillLimit, @@ -281,7 +285,7 @@ async function main() { // ---- Q4 / Q7: reschedule churn, and whether the seek point degrades -------------------------- // The reschedule pattern: read the head, move those rows into the future, repeat. This is what // leaves dead index entries AT the seek point, and it is the measurement the claim floor exists - // to answer (0.36 -> 6.25ms over 40,000 reschedules, permanent). + // to answer (0.36 -> 6.25ms over 40,000 reschedules, permanent). // PRODUCTION-SHAPED CHURN, which the first version of this was not. It patched keys 0..N in key // order; production repeatedly CLAIMS THE HEAD of the index and writes those rows into the future, // so the dead entries accumulate at the seek point rather than being spread over a key range. The @@ -332,9 +336,10 @@ async function main() { trend.push({ reschedules: moved, ...(await sampleSeek()) }); out.phases.churn = { - note: 'Q7 — does the seek point degrade as rows churn away from it (the 0.36 -> 6.25ms finding the ' - + 'claim floor exists to fix), and is a floored seek immune? Head-claim-then-reschedule, the ' - + 'production shape, sampling both seeks as it goes. Also Q4: per-row put on the reschedule path.', + note: + 'Q7 — does the seek point degrade as rows churn away from it (the 0.36 -> 6.25ms finding the ' + + 'claim floor exists to fix), and is a floored seek immune? Head-claim-then-reschedule, the ' + + 'production shape, sampling both seeks as it goes. Also Q4: per-row put on the reschedule path.', reschedules: moved, put_usPerRow: (churnMs * 1000) / Math.max(1, moved), trend,