Known Gaps

Known gaps

Defects and unfinished seams named here rather than left to be discovered. As of 2026-10-01.

A reference manual that hides these is lying to the reader.

Read the section headings first. Open is what is wrong with the code you can install today. Open by decision will not be fixed and says why. Not built yet is unfinished work, not a defect. Deferred by plan 101 is ranked work a later plan owns. Closed is history — every row there carries the release that fixed it and the workaround for anyone still pinned below it.

Only the footer stamps the installable version. Resolve it yourself rather than reading it here:

npm view @ultimat3/core version

A row that says fixed on main is fixed in the repository and in no published release; if you are on an earlier version, treat those rows as open and take the workaround. [Unreleased] in CHANGELOG.md is the source of truth for what the next release carries — As of 2026-10 it is 23.0.0 in flight, and most of what it fixes was never a row here. Read that section rather than this sentence: grep -n '^## ' CHANGELOG.md finds its bounds, and it changes on any commit.

Publication is not a gap. All 31 workspaces are on the registry As of 2026-08-20, checked by scripts/registry-audit.ts daily; @ultimat3/scraping was the last never-published package and was bootstrapped by hand at 2.0.0 (PUBLISHING.md step 1). bun add @ultimat3/scraping resolves — check it with npm view @ultimat3/scraping version.

Open

Gap	Symptom	Work around it by
On an s3 disk a grant's maxBytes bounds nothing at the upload	As of 2026-10. S3 has no request header for a size and Bun.S3Client's presign covers method, expiry and content type, so a browser holding a grantUpload URL can PUT an object of any size into the bucket. It is measured afterwards: promoteAttachment reads stat().size against the policy and refuses (X_STORAGE_TOO_LARGE), and get() refuses to buffer an object over the disk's maxGetBytes. Still open: a grant minted with a target lands straight on the row's key and is never promoted, so nothing measures it; and the oversized bytes sit in the bucket until sweepOrphans or a lifecycle rule removes them	grant with no target and call promoteAttachment({ …, policy }); for a targeted grant, check (await disk.stat(key)).size before writing the key to the row; cap the bucket itself with a provider policy → Storage and uploads
A queued offline write is not bound to its principal on the server	As of 2026-10. The fence is in the page: a principal change abandons the running replay, refuses a write in flight (X_OFFLINE_QUEUE_ABANDONED) and wipes the queue. A second TAB whose replay is already past its store read when another tab changes the session cookie still sends its remaining entry under the new session — one entry, one in-flight pass, on a shared browser	sign out by full navigation (signOutHeaders() clears site data, and every boot wipes other principals' queues); do not share a browser profile between accounts with offline writes queued
Presence leaves past the lease roster cap	the sweep leader ships at most 4,096 member ids on its lease (PRESENCE_LEASE_ROSTER_LIMIT). When the leader's node dies, members beyond that whose TTL expired before the successor's first leading pass are never announced as left. A successor cannot diff against the shared store: the expired members are already gone from it	nothing to do: each client's next beat is answered with the whole roster, so it heals within one beat (10 s by default)
A deployed fresh scaffold has no session: every policy-protected route is anonymous	x new writes apps/web/app/auth/dev-actor.ts, a cookie-named viewer installed in development only, so x dev boots with no warning and /dashboard opens. A container deploy still logs X_CONFIG_INVALID: 7 route(s) declare auth: 'required' and no authenticator is configured until the app issues real sessions — by design: a dev actor in production would be an open door. This was true only when something told the process it was production — As of 2026-09-17, installDevAuthenticator reads its environment with fallback: 'production' rather than trusting tryResolveEnvironment's own default (development), so a process naming NEITHER ULTIMATE_ENV nor NODE_ENV — which used to read as development, indistinguishable from a real x dev — now reads as production and installs nothing. x dev declares ULTIMATE_ENV=development for exactly the process this closes, so a bare x dev is unaffected. As of 2026-08-23	run the call the warning prints, at module scope in a file under apps/*/: configureAuthenticator((request) => viewerFor(request.header('cookie'))), resolving to the Actor in apps/web/shared/actor.ts. The sync half is createSyncNode({ authenticate }) → Policies and authz
An offline scrape reads /robots.txt over the real network	every fakeBrowser / fixtureBrowser run takes the default robots: 'obey', and the gate reads https://<host>/robots.txt with the platform fetch before the first navigation. Measured As of 2026-08-19 on runScrape against fakeBrowser: one fetch to https://shop.test/robots.txt leaves the process, and the run still reports refused: 0. Under bun test the sealed network does refuse it — and robotsFetcher's catch { return undefined } swallows the refusal, which the gate reads as "no robots.txt", which is allow-everything. So the suite is green either way and neither the egress nor the seal is visible in it. Contradicts SECURITY.md's egress row and packages/scraping/CLAUDE.md's "an offline driver that fell through to the network would make a green suite secretly live"	declare the policy in the definition under test — robots: { ignore: 'fixture host, no live origin' } — which returns a gate that reads nothing at all. There is no boolean and no ambient off switch, by design — see Scraping
bun run error-render cannot see a catch (error) binding	the gate matches a parameter annotated unknown/any (UNKNOWN_BINDING in scripts/error-render.ts:158). A catch (error) binding is unknown by inference under useUnknownInCatchVariables and carries no annotation, so it matches nothing — which is how String(error) echoing a caller's own request body into a 422 cause: and into the log store stayed green in packages/http/src/request.ts through 3.0.0. That instance is fixed; the blind spot is not	review every catch by hand: a caught value reaches a cause:/fix: only through renderThrowable(error) from @ultimat3/core. bun run error-render --json is a floor, never a proof — its own file header lists what it cannot see
x g writes hand-wrapped source instead of formatting what it writes	every template is a string literal with its line breaks typed by hand, and no generator runs a formatter. A template edited past 100 columns lands in an app whose lint step is biome check .. This is about x g output, not the scaffold: a fresh x new app lints clean, zero diagnostics across every source file it writes, measured As of 2026-09-11. The import-order half is closed As of 2026-08-23: sortedImports in packages/cli/src/templates/imports.ts orders every emitted import block the way Biome's organizeImports wants, and generate-format.test.ts runs the real Biome over a scaffold named zebra-demo (sorts after ultimat3) so the order cannot regress. Open since #127 for the remaining line-width half	run bunx biome check --write over the files the generator just listed. Feasible to close with no new dependency — ExecOptions.stdin already exists on packages/cli/src/exec.ts:25
The cache fill fence crosses nodes for tag busts through Redis only	As of 2026-10. A read-through fill re-checks a fence before it writes. Inside one process that covers everything. Across nodes: the Redis tier keeps leased generation keys a bust writes and a fill samples before load() and re-reads after its SET (packages/cache/src/redis-fence.ts), so a tag bust on another pod withdraws the fill even when the broadcast is lost — at one extra round trip before each load() and one after the SET. Three things still do not cross. Key busts: CacheStack.write and .drop mark { key } locally and the wire form carries tags only. A deployment with no Redis tier: there is no shared store to hold a generation, so the window is the broadcast's. Another pod's LRU: a value it cached before the bust stays until its TTL when the broadcast is lost	give a cache: query a TTL you can afford to be wrong for on one pod, keep redis in cache.tiers on a multi-pod deploy, and prefer a tag bust over cache.drop(key): only the first one is broadcast and fenced
A replicated table with no replica identity is warned about, never refused	As of 2026-10-02. A channel declared with params does need REPLICA IDENTITY FULL on its records tables — a DELETE is routed by the old row's params, which a key-only image does not carry, so members keep the deleted record; x db gen grants it since 24.0.0 (one migration for an app that already declares such a channel), and the replicator warns (replication.channel_identity_partial) about a table that still lacks it. Live queries do not need REPLICA IDENTITY FULL on a keyed table: the shared window holds the whole row, so a row leaving the result set and a delete are decided from the window — proved on real WAL by pg-identity-window.live.test.ts (out-of-filter update, into-filter update, delete with a key-only before-image, updates to rows never held). What still breaks is a table with no identity at all — NOTHING, or DEFAULT with no primary key — whose UPDATE/DELETE Postgres refuses once published. X_LIVE_REPLICA_IDENTITY names exactly those at preflight, and warns rather than throws; no x verify step reads it. The REPLICA IDENTITY FULL the app's declarations need is checked at the gate: since 24.0.0 the drift step reports a params-channel or subscribes: table whose FULL no migration recorded (X_DB_SCHEMA_UNMIGRATED) — against the migrations, never the live database	give the table a primary key, or ALTER TABLE <t> REPLICA IDENTITY FULL; — the warning names every table that needs it
Nothing checks .env.example against the schema	assertEnvExample(schema, text) ships and throws X_ENV_EXAMPLE_DRIFT for a declared key the file is missing — and has no shipped caller: no x verify step, no CLI command and no boot hook runs it, As of 2026-08-19	call it from a test of your own — three lines, and it is then a gate step: assertEnvExample(schema, await Bun.file(ENV_EXAMPLE_PATH).text()) → Configuration
HPAs read <unknown> without a metrics adapter	the chart half is done — values.yaml declares metricsPort: 9090, every role but migrate emits the container port, service.yaml publishes it and templates/servicemonitor.yaml ships the scrape target. Two things still stand and neither is the chart's to fix: serviceMonitor.enabled defaults false, because a cluster without the Prometheus operator has no such CRD and helm install fails on an unknown kind; and turning scraped series into the Pods metrics an HPA reads needs a custom-metrics adapter, which is the cluster's	set serviceMonitor.enabled: true and install a metrics adapter; until then disable the HPAs and pin replicas. Do not hand-add a metrics container port — the chart emits one and a duplicate is rejected by the API server → Observability
--target binary has never been served from a bare VM	the target compiles and boots — x build --target binary passes --define ULTIMATE_FRAMEWORK_VERSION, docker/Dockerfile passes it too, and the image build ends in /out/app --version so a binary that cannot answer fails the build. What is unmeasured is the rest: no scaffolded app has been compiled, copied to a VM with no Bun on it, and served under systemd	prove it for your own app before you depend on it — build, scp, run ./app --version, then ROLE=web PORT=3000 ./app and hit /readyz. Report what breaks → Deployment
A local typecheck step can report green on a tree CI fails, on the same commit — one sighting, unreproduced	a scaffolded app's tsc -b is incremental (tsconfig.tsbuildinfo, gitignored). Observed once (2026-09-17, TypeScript 7.0.2): after a git stash -u → branch switch → x verify on the other branch → switch back → stash pop sequence, a type error that surfaces in an UNCHANGED file (a widened index signature in a file it imports) passed locally and failed CI; rm tsconfig.tsbuildinfo then reproduced CI's red locally. Five faithful reproduction attempts against 7.0.2 all stayed correctly red (#450); a second tsc racing the same buildinfo is the one suspect not excluded. x verify deliberately takes two flags and no --cold, so nothing ships for a cause nobody has shown	after switching branches in a working tree, rm tsconfig.tsbuildinfo before trusting a local typecheck; CI is always cold and is the judge of record. If it recurs, capture tsconfig.tsbuildinfo BEFORE deleting it and attach it to #450
X_ROUTE_MODE_INVALID's fix line offers two edits and one does not work	it reads "wrap the data-dependent part of <file> in <Suspense fallback={…}> or change render to 'ssr'" (packages/render/src/modes.ts:193). Only the second half works: Solid's <Suspense> throws getContextId cannot be used under non-hydrating context under this renderer at any Solid version — the server JSX factory is inert by design and is not a Solid renderer	take the second branch, render: 'ssr'. Async data needs no boundary at all: renderToHtml awaits async components and promise children
A pod evaluates the framework from source, one module at a time — ~28 Mi of resident memory per pod that a bundle would not hold	As of 2026-10-01, measured on the framework serve graph alone: 1,071 modules from source settle at 80.4 Mi; the same graph as one bun build --target=bun file is 5 modules and settles at 51.8 Mi — ~34 kB of RSS per source module, the runtime's cost and not the code's. It is the largest saving left after the per-role load and the prebuilt store (a web pod of the demo app settles at 97–102 Mi, a worker at 80–85 Mi). Not a patch: a registry is a module instance, so the framework bundled apart from the app is two @ultimat3/jobs, two @ultimat3/entity — the app registers into one and the boot reads the other. The unit has to be the app and the framework as one graph, per role, built inside the image build from a generated entry that imports what scanAppModules would, with the .tsx/.scss loaders as build plugins. Containers only either way (axiom 7). Design note: docs/architecture/13-topology-runtime.md	nothing to do in an app. Size a pod's memory request from process_resident_memory_bytes on /metrics, not from a guess
Every role evaluates all of @ultimat3/jobs — 96 modules (As of 2026-10-02, by file) — whatever it runs	As of 2026-10-01, read off the serve graph by the names each role's code uses: a web pod needs 43 of them (enqueue, the outbox relay, the purge task), a scheduler 46, a worker 62. The rest ride the barrel: worker.ts and its eighteen dependencies in a web pod, the backfill() / exportJob() / webhook() factories (17 modules, 123 kB of source) in an app that declares none, and the memory driver (3 modules) outside a test. ~1.6 Mi per web pod at the measured 34 kB a module. Closing it means subpath entries (@ultimat3/jobs/worker, one per factory) and every importer rewritten — a breaking move for a saving the per-role bundle above makes moot, so it waits on that decision. The operator surface is not in this list: JobIntrospection is a member of the driver, and the worker announces itself and counts settles through it	nothing to do in an app
A job's runAt is stamped from the enqueuer's clock	As of 2026-10. An enqueuer running ahead of the database delays its own "immediate" jobs; up to 1 s ahead still wakes a worker, beyond that the job starts on the poll. A delayed job enqueued by another process starts within the idle ceiling after its runAt — 5 s with a proven wake, 2 s without; only a retry or a sleep the worker itself handed back gets a due-time timer	NTP on every pod; for a deadline tighter than the ceiling, enqueue at the moment the work is due rather than with runAt
onSettled runs at most once across a crash	As of 2026-10, by decision of the first cut: the hook runs after the settle, inline, by the worker whose settle landed. A worker killed between the settle and the hook does not run it, and nothing re-drives it	record what must be recorded for certain inside the body, in a step.run, and use onSettled for what may be lost
The CDP driver's allowHosts judges NAMES, and a page's WebSocket is not screened at all	As of 2026-10. Measured on Chrome 150 with localBrowser(): interception (page-level and the browser-level Fetch that now covers popups) never pauses a WebSocket handshake, so new WebSocket('ws://off-list/…') from a scraped page connects. And the browser resolves its own names, so a host a wildcard admitted that resolves to a private address is reached from the browser leg — the HTTP leg and the robots read resolve and pin (packages/scraping/src/pinned-host.ts), the browser cannot be pinned through this port	list exact hosts rather than '*' on a scrape that runs where private addresses are reachable, and dial a scrape through an egress proxy outside that network
ScrapeReport.usage.bytesIn counts the HTTP leg only	As of 2026-10. The CDP port subscribes to requests, not to their sizes, so the browser leg's bytes are not in the figure (packages/scraping/src/usage.ts)	treat bytesIn as a lower bound; a provider's own byte count, returned through the resolver's cost, is the billable one
An org-scoped operator's runs list reads x_jobs with no index on tenant_id	As of 2026-10. The jobs dashboard keeps an actor with an orgId to its own org's runs through JobFilter.tenantId, and x_jobs.tenant_id is a plain column: the only index naming it is the idempotency one, led by name (packages/jobs/src/driver-pg-ddl.ts). A platform operator's list is unaffected	on a large queue, add create index on x_jobs (tenant_id, created_at desc, id desc) in an app migration, or operate the dashboard as a platform operator (an actor with no orgId)
Retry-from-step on the jobs dashboard is a free-text field	As of 2026-10. job.retry-from-step takes { step: t.string.min(1) } (packages/admin/src/jobs/job-actions.ts); the run's detail page shows its steps, but the form does not offer them as a choice.	copy the step name from the run's steps section on the same page
The admin MCP batch tool takes ids only	As of 2026-10. A batch action is one MCP tool, admin.action.<name>, and it takes the rows by id; "all matching" — the list's own filter — is a screen control only	list the rows with admin.<entity>.list and its where, then pass their ids
The manifest does not record a resource's permission noun or an action's matching	As of 2026-10. x.manifest.json's admin section records each action's permission, input, when and batch, and each route's permissions — so job:read is visible on the jobs routes — but not AdminResourceOptions.permission or whether an action declares matching (packages/manifest/src/sources-admin.ts). x manifest diff therefore cannot class a change to either	read them from the declaration in apps/admin/app/admin/admin.ts
No counts on the jobs dashboard's state tabs	As of 2026-10. stats() counts per queue across every tenant, so a count on a tab would show an org-scoped operator other orgs' numbers; the tabs carry none and the overview's tiles carry the counts, for a platform operator only. A per-tenant count is a second query nothing issues yet	a platform operator reads the overview tiles; an org-scoped operator reads a tab's own page

Open by decision

Each of these is a defect somebody has already argued about, and the reasoning is why it stays.

Gap	Why it stays
A local write that died after its last rename is listed with no content type until the key is written again	As of 2026-10. localDriver commits in four steps (marker, sidecar, bytes, clear marker) and a marker left behind means "re-check this pair". get() and stat() do re-check against the bytes and answer correctly; list() never reads bytes, so it reports that object with no contentType and etag: '' — honest, and it stays that way until the next put() of that key. stream() returns bytes only and checks nothing. Two files cannot be committed in one step without changing the on-disk layout
localDriver cannot hold a key and a key beneath it (a and a/b)	As of 2026-10. An object is a file at its key, and a POSIX path is a file or a directory. The second put() is refused by code (X_STORAGE_KEY_CONFLICT) where it used to be a bare ENOTDIR; s3Driver and memoryDriver hold both. Storing objects under suffixed names would lift it, at the price of a layout migration for every existing root — and the keys grantUpload and scopedKey build never nest an object under another, so the limit is met only by hand-built keys
insert({ ...row }) of a row whose nullable .sealed() column was read from the repository stores NULL, silently	a sealed property on a repository row is not enumerable, so a spread leaves it behind (As of 2026-10). A required one is refused by name (X_INVARIANT_VIOLATED, fix { ...row, <col>: row.<col> }); an absent nullable column is a legal insert, and nothing tells a spread from an omission without making every omission an error. Copy a sealed column by name
A drift fix: for a table in public does not name its schema	As of 2026-10-02. A fix for a table read from any other schema carries set search_path = "<schema>";; one for public is left unqualified, so it relies on the pasting session resolving public first. Postgres' default path is "$user", public, so a schema named after the connecting role that holds a same-named table shadows it. Check show search_path in that session, or prefix the pasted statement with set search_path = public;
Drift never compares an index's where text	the catalog answers its own rewriting of the expression ((deleted_at IS NULL)) where the snapshot holds the author's spelling (deleted_at is null), so comparing the strings reports drift on a database that is exactly right. Normalising them means shipping an expression parser to compete with the server's. Three of the four parts of an index are compared now — columns, uniqueness, direction, and the predicate's presence — so a partial index recreated as a total one and a desc index rebuilt ascending are both caught (packages/db/src/drift.ts, compareIndexes). x db gen compares the text, where both sides are generated. A predicate rewritten by hand into a different predicate of the same presence is the residue, and it is invisible: regenerate and read the diff
A long backfill() retains one step name per batch	createStepRunner's claimed Set is the authority for X_STEP_DUPLICATE and is deliberately unbounded: a 5M-row sweep at batch: 250 holds 20,000 short strings (~1–2 MB) for the life of the attempt, released when it ends. MAX_TRACE_NAMES bounds the reported trace, not the membership set. Bounding the set would let a duplicate step name through after 200 batches, and silently replaying a step is the worse failure. Raise batch if the attempt is long enough for the retention to matter
The admin has no one-click select-all	As of 2026-10, by decision: the admin ships zero JavaScript (hydrate: 'never'), so every control is a link or a native form. A row checkbox joins the batch form through its form attribute, "all matching" is a radio whose meaning is the list's URL, and an action's input is its own page. Selecting every row on one page is one click per row. An island would buy that click and cost the admin its zero-script rule, and "all matching" already covers the whole list

Not built yet

Open	Where it stands
Two-platform deploy proof	milestone 11, still 🚧. All three build targets, both compose files and the Helm chart ship — and x new now writes the chart too. What is not demonstrated is the demo app on Compose and on Kubernetes from one image with a rolling restart invisible to connected clients
Multi-node realtime	the forced-restart benchmarks are measured, on one sync node over InProcessTransport — neither run crossed NATS and neither subscribes to a live query, so no cursor, snapshot or gap-repair path is under test. Fanout, throughput and per-node socket capacity across nodes remain targets → Realtime
x db studio	planned. X_DB_STUDIO_FAILED is reserved and not thrown; the command exits X_NOT_IMPLEMENTED pointing at x dev → the /_x db panel. Keeping a bunx drizzle-kit studio shell-out would have meant a second schema engine for one subcommand
Deferred, As of 2026-08	the plugin API, multi-region replication, and the Redis/NATS job drivers (realtime tier 3, local-first, shipped in 21.0.0). Each sits behind an interface that ships today and throws X_NOT_IMPLEMENTED with a runnable fix: rather than pretending to work

Deferred by plan 101

As of 2026-10. Gaps the platform-readiness work ranked and did not take, with the evidence that ranked them — a later plan's, not this release's. Evidence is from read-only surveys of a downstream app on Ultimate and of systems built without it; no row is a defect in what ships.

Gap	Evidence	Precedent
defineError({ code, status, cause, fix })	565 hand-written error classes and 51 status registrations in one app	Rails' rescue_from and one status table
a native form's refusal answer — setRedirect covers success (packages/http/src/redirect.ts)	234 wrapper calls in 142 files	Rails' form convention: 303 on success, 422 re-rendering the form with its errors; flash
route-group defaults	111 routes × ~25 repeated lines	layouts, before_action, controller inheritance
a catalog subset for t() inside an island	94 label types, 3,092 lines of label files	lazy lookup t('.title') scoped to the view
a server-first dialog, menu and combobox with a small enhancer	~2,200 app-side lines; 13 catalog components at 0 imports	HTML first, a small controller second
QueryRef derived from the declaration	Queries and live queries "Not derived" row	—
a task that runs without a companion job; a boot hook	35 task/job pairs, 2 boot stand-ins	recurring tasks that name a command; initializers
a failure bundle an agent can act on	built by hand in two surveyed systems	the error page: trace, request, a console
batches with completion callbacks, named fleet-wide rate limiters, debounce, priority, job expiry, argument encryption	each present in a surveyed job system; none asked for by the proving case	batches and limiters in established job systems; queue priorities
proxy inventory, captcha, OTP relay	product features of a scraping service — the app's, by axiom 8	—

The test-kit row of that plan's table shipped in 23.0.0 work (renderView, runJobs({ actor }), mountIslandState) and is not repeated here.

Closed

Historical. Every row is a defect that shipped; the third column is for readers still pinned below the release that fixed it. Upgrading is the fix, and the breaking entries are the cost (Upgrading).

Rows marked main are fixed in the repository and in no published release yet.

Gap	Fixed in	If you are pinned below that
A tag-only ISR page never went stale	main	No boot attached its ISR controller, so invalidateTags reached no revalidator and a revalidate: { tags } page with no ttl served its first render for the life of the process — on every pod, while report.isr listed it as revalidated. x dev, the container's web role and appRoutes() now attach the controller they build (packages/cli/src/runtime-isr.ts) and detach it on stop; isr-tag-bust.e2e.test.ts proves x-ultimate-isr: stale then the new body. Below it, give every isr route a ttl as well as its tags
The ISR cache key was the pathname, so a query string was ignored	4.0.0	the key is isrKey(url, locale) — pathname, the negotiated locale, then the sorted query — so ?q=bob and ?q=alice are two documents. Below it, put the varying part in the path (/search/[q]) or take render: 'ssr'
invoke()'s cache bust was not transaction-aware	main	An action invoked inside withTransaction busted cache.invalidates when its handler returned — before the commit — so a concurrent read refilled the entry from the pre-commit row and the stale value survived to its TTL, and a rollback busted for a write nobody made. The bust is now handed to the root transaction's onCommit (packages/action/src/cache-gate.ts): it fires once the root COMMIT is answered, never for a rollback or X_DB_TRANSACTION_ABORTED. Below it: do the transactional work inside the handler, or drop cache.invalidates from the declaration and call invalidateTags yourself after the transaction returns
Nothing pinned the documented generator file counts	4.0.0	scripts/generator-counts.ts reads every page that states how many files x new or x g resource writes and answers X_DOC_FILE_COUNT_STALE on the gate's manifest step. Below it, derive rather than quote: x new myapp --dry-run --json | jq '.data.files | length'
SessionInit.proxy was declared on the scrape seam and read by nothing	23.0.0	scrape({ egress: (input) => proxyUrl }) now reaches the driver as SessionInit.proxy and wins over the driver's own proxy: the launched browser, the HTTP leg and the robots read all dial it, an offline session reports it, and a driver that cannot dial it refuses with X_SCRAPE_EGRESS_UNSUPPORTED. On 22.x the exit is a driver option — localBrowser({ proxy }) / remoteBrowser({ proxy }) — one for every session → Scraping
No test file was typechecked	main	All 30 package tsconfig.jsons carry "exclude": ["src/**/*.test.ts"] — they still do — so bun run typecheck, a tsc -b, reads none of the 984 test files under packages/*/src and the gate's typecheck step was green over every one. Re-derive both: grep -l 'src/\*\*/\*.test.ts' packages/*/tsconfig.json | wc -l and find packages -path '*/src/*' -name '*.test.ts*' -not -path '*/dist/*' | wc -l. Closed by a second program, not by editing the 30: tsconfig.tests.json compiles the tests with noEmit, scripts/test-typecheck-gate.ts runs it on a per-package ratchet that may only fall, and it rides the gate's manifest step (scripts/verify.ts:247) rather than typecheck, which takes no host findings. It landed at 446 errors over 161 files in 27 packages (measured 2026-08-19) and scripts/lib/test-typecheck-pins.ts now pins 2, with 29 of 30 packages at zero — so a new test compiles the day it is written. The residue is one named defect, both errors in packages/entity/src/pg-driver.test.ts: Repo's full-row write members take the ROW type where money's WRITE type belongs, and two attempted fixes were reverted with evidence rather than silenced — that pins file carries the argument. On any published release: nothing to work around at runtime, the tests run either way. Typecheck one package's tests yourself with bunx tsc --noEmit over a copy of its config with the exclude dropped
A live query re-delivered the raw table row, and mis-ordered a projected window	5.0.1	Two defects with one cause: a ChangeEvent carries the whole TABLE row, and a live query's result set is whatever its sql returned. The leak — every patch forwarded the change row unnarrowed, so a column the projection dropped went out on the socket the moment it CHANGED; examples/dummy's feed projects ten columns and one publish delivered updatedAt, and a column like a salary or a private note would have gone the same way. The per-subscriber gate could not help: it decides whether a ROW is delivered, never which of its columns. The mis-ordering — match() decided position by comparing the change row against the rows the WINDOW holds, so an orderBy on a column the projection omits measured a real value against nothing: every update read as a move, and an arriving row landed wherever undefined sorted. Now a patch row is narrowed to the columns the query actually returned, and a position the window cannot answer for is a refill — one re-read and a re-snapshot — rather than a guess. A DELETE still patches incrementally, because it decides no position. #230. On 5.0.0 and below: give a live query's rows the key they are ordered by, and do not rely on a projection to withhold a column from a live subscriber
jobs.driver selected no driver	5.0.0	JobsConfig.driver accepted 'postgres' | 'redis' | 'nats' and had no reader anywhere — boot always built createPgDriver, and packages/jobs/src/driver.ts's own header already said so. jobs: { driver: 'redis' } therefore did not boot-and-then-throw as this wiki once claimed: it changed nothing and you silently got Postgres — the same shape as realtime.heartbeatMs, and worse, because it failed silently in the dangerous direction. Five shipped fix: lines named it as the repair for X_NOT_IMPLEMENTED, which is a fix: that is a no-op; those were corrected in 4.1.0 and the field itself is deleted in 5.0.0, along with the JobsDriver type nothing else used. On 4.1.0 and below the field still typechecks and still does nothing — swap the driver with setJobDriver(createPgDriver({ executor })), or setJobDriver(createMemoryDriver()) in a test, and never through app.config.ts
A caller-controlled string could add a line to the 3-line error format	5.0.0	bun run error-render refuses a parameter typed unknown/any; a value already typed string renders without throwing, so nothing objected — while a newline in one writes a second line an operator, a CI log or the dev overlay's <pre> reads as a genuine framework message. Three holes shipped in @ultimat3/auth under a green check, the worst reachable by an unauthenticated stranger with one crafted OIDC token. The first fix escaped at each of the six RENDERERS, which could not hold: six is a number that only goes up, and it covered none of the renderers an app writes. Now UltimateError's and SchemaError's constructors escape code, title, cause, fix and docs — so .message, .cause, format(), toJSON() and any renderer anyone writes are one line by construction, and singleLine() is idempotent so a call site that already escaped is unharmed. On 4.1.0 and below, pass error.cause through singleLine() from @ultimat3/core before you render it yourself
on delete was declared and reached no SQL	4.0.0	references(() => orgs.id, { onDelete: 'cascade' }) type-checked and the rule was dropped one layer below the declaration, so no generated add constraint ever spelled one and a drift check had nothing truthful to compare. Now: ColumnDescription/ReferenceDescription carry onDelete, addForeignKey writes the clause, and a rule changed on either side is changed-foreign-key drift whose fix: is the drop/add pair. On 3.0.0, add the clause to the add constraint the generator emitted, before applying the migration: … references "orgs" ("id") on delete cascade;. Editing it after it applies moves the checksum → X_MIGRATION_CONFLICT
A references() removed from a column emitted nothing	4.0.0	the key stayed on the database, the snapshot beside it recorded foreignKeys: [] — actively denying a constraint the catalog held — and compareForeignKeys judges the declared side, so no check could see it. Now foreignKeyPlan emits the drop constraint in up and the add in down, naming the constraint the previous snapshot recorded rather than the name the generator would have picked. On 3.0.0, write the drop into the next generated migration by hand before applying it: alter table "posts" drop constraint "posts_org_id_fkey";
Drift ignored an index's direction and its predicate's presence	4.0.0	a desc index rebuilt ascending served a feed's newest page off the wrong end, and a partial index recreated as a total one silently widened the constraint — both read ok: true. Both are compared now; asc is normalised to null first, since Postgres stores an ascending index as not-descending. The predicate text is still uncompared and always will be — see Open by decision. On 3.0.0, x db gen compares all five fields, so regenerate and inspect the diff
An unknown flag was refused before a planned command could answer honestly	4.0.0	the parser read flags against the spec first, so x logs tail --follow reported X_CLI_BAD_FLAG instead of X_NOT_IMPLEMENTED. A planned command now answers with its own status whatever flags it was given; a shipped command still refuses an unknown flag, which is correct — x env is shipped and declares --json, --help, --cwd and --verbose, so anything else is X_CLI_BAD_FLAG for the ordinary reason that no such flag exists. On 3.0.0, run the flagless form of a planned command to see the real message
x deploy --method helm threw X_NOT_IMPLEMENTED in a scaffolded app	4.0.0	x new wrote no docker/helm, and the command carried a "does this build implement helm?" branch over a build that implemented it completely. x new now writes the chart — 8 files, Chart.yaml + values.yaml + 6 templates — and the lying branch is deleted, so a missing chart is helm's own error. The framework repo's own docker/helm carries two more templates the scaffold does not (pdb.yaml, servicemonitor.yaml). On 3.0.0, copy docker/helm from the framework repo, or use --method compose
.env.development and .env.production shipped in the image	4.0.0	the scaffold's .dockerignore excluded .env and .env.*.local — neither pattern matches .env.production, which is the file docker-compose.prod.yml's env_file: tells the operator to create. This page said the leak was "harmless as generated"; that was false, and a real docker build proved both files land in a layer. Now **/.env + **/.env.* + !**/.env.example, in the framework's file, both tracked apps' and the one x new writes. On 3.0.0, add those three lines to docker/Dockerfile.dockerignore yourself — and rebuild, because an image already built still carries them
realtime.heartbeatMs was read by nothing	4.0.0, and BREAKING	the key sat in RealtimeConfig with a default of 15 000 and no reader anywhere. It is deleted: RealtimeConfig is { enabled, tier, transport, urlEnv }, the socket beat is the client's new LiveClient({ heartbeatMs }) and the presence beat is derived (PresenceRegistry.heartbeatMs is max(1000, floor(ttlMs / 3))). There is no runtime refusal — section() copies every own key of the patch and validate() checks only named fields, so an app that keeps the key keeps it silently. The failure is at typecheck: TS2353, excess property on Input<RealtimeConfig> — and an app that builds its config object into a variable first loses excess-property checking and gets no error at all. On 3.0.0, delete the key; setting it changes no behaviour either way
@ultimat3/action's stableStringify folded -0, NaN and Infinity together	3.0.0	this page asked for work that was already done. The hash canonicalizer was split out and moved down: canonicalJson and fingerprint live in @ultimat3/core (tier 0), are injective, and give NaN, ±Infinity and -0 bare tokens of their own plus tagged forms for Date, Map and Set. requestHash is fingerprint(input), the job dedupe key is action:<name>:<fingerprint>, and @ultimat3/query's copy was deleted rather than fixed. stableStringify survives as the OpenAPI document serializer alone, where emitting valid JSON is the requirement
.job() produced a handle nothing could enqueue	3.0.0	agentJob() in @ultimat3/ai is the shipped bridge — an agent as durable, resumable, budgeted background work, with the queue accepting the handle. Pinned by packages/ai/src/agent-job.test.ts, whose test is named for this gap: "agentJob() produces a handle the queue accepts, where .job() never could"
A branch reaper could drop another app's databases	4.0.0	listBranches() walks pg_database for the whole server, so two Ultimate apps on one Postgres plus one nightly sweep was the other app's branches dropped by a DROP DATABASE nobody asked for. The marker is now <base>:<iso> — BranchInfo carries base — and reapBranches skips any branch whose base is not this database. A pre-3.x marker records no base, so it is skipped too, never dropped: self-healing with no migration, because the next createBranch writes the base down. On 3.0.0, run the reaper only against a server this app owns
x db branch ls could not see a branch made before the psql shell-out was removed	2.0.0	the old path issued CREATE DATABASE … TEMPLATE through psql and wrote no comment, so a branch made by a 1.2.0 or earlier CLI is absent from ls — and drop may only remove what ls shows. Nothing back-fills the comment. Drop it by hand from the string the refusal names: psql "$DATABASE_URL" -c 'DROP DATABASE "<source>_branch_<slug>"', where <slug> is the branch name with every character outside [A-Za-z0-9_] replaced by _
MCP tool names were published in snake_case and served verbatim	2.0.0, and BREAKING	openapi.json carried "mcpTool": "publish_post" while the only name tools/call accepts is the export name, publishPost — 15 of the 17 mcpTool values in the two tracked apps' committed specs named no served tool. toToolName is deleted from @ultimat3/action and @ultimat3/query, all three publishers spell the export name, and packages/mcp/src/cross-surface.test.ts drives a tools/call with the name OpenAPI published. On 1.2.0, ignore the published value and call the export name — x actions list --json prints it as name → MCP and AI
Shared cache tier invalidation DELed keys it never declared in KEYS	2.0.0	it failed on Dragonfly and on Redis Cluster — a cluster cannot route a key it was not told about. The script returns the member list and the tier deletes value keys client-side, one key per DEL, so every delete is slot-local. On 1.2.0, single-node Redis, or a cache tier that is not the shared one → Caching and invalidation
resolveEnvironment existed twice, with different return types	2.0.0, and BREAKING	@ultimat3/core and @ultimat3/seo both exported it. seo's is deleted along with SeoEnvironment; RobotsConfig.environment and isIndexable() take core's Environment, 'preview' is spelled 'staging', and core gained tryResolveEnvironment() for callers that must answer rather than throw — ULTIMATE_ENV is in no env schema, so a robots.txt render is routinely its first reader. On 1.2.0, import one with an alias → Configuration
Two head serializers, one of them weaker	2.0.0, and BREAKING	@ultimat3/seo's renderHeadTags escaped </ only — not <!--<script>, which moves the tokenizer into script-data-escaped state — and applied that code rule to a JSON body; nothing called it. @ultimat3/render's renderHead, the path every x dev and every build takes, emitted <script> content entirely raw. renderHeadTags is gone and render escapes raw-text content and JSON-LD by each element's own rule. On 1.2.0, render the head through renderHead(headFromMeta(meta, seoRenderers())) and treat any string interpolated into meta.ld as untrusted
docker-compose.prod.yml paired a published host port with replicas: 3	2.0.0	one host port has exactly one binder, so the second container died on Bind for 0.0.0.0:3000 failed: port is already allocated. web and sync declare replicas: 1 in all four files. sync's PORT was wrong in the same file and fixed with it — the role binds PORT + 1, so PORT: 3001 opened 3002 while publishing 3001. On 1.2.0, set replicas: 1; to scale anyway, drop ports: and put your own reverse proxy on the compose network, or climb to the chart's per-role HPA → docs/idea/17-scale-ladder.md
X_MIGRATE_CONCURRENT was reserved and never thrown	2.0.0	the lock was a blocking pg_advisory_lock, so an overlapping migrator waited forever — no timeout, no exit code, and a wedged predecessor held helm upgrade --wait inside one statement with nothing in the logs. acquireLock now polls pg_try_advisory_lock once per 500ms against a 60s budget (MIGRATION_LOCK_WAIT_MS). On 1.2.0, kill the wedged migrator by hand
x db gen / x db migrate shelled out to bunx drizzle-kit	2.0.0	which x new neither installed nor configured, so bin/setup — the scaffold's own documented first command — failed on "drizzle.config.json file does not exist". gen calls generateMigration() and migrate/reset call migrate(), the engine ROLE=migrate already ran. On 1.1.0, generate with generateMigration + writeSchemaHash from @ultimat3/db directly, then apply with ROLE=migrate bun apps/web/server.ts
x new wrote 0000_initial.sql with no snapshot sidecar	2.0.0	so a scaffolded app started on the one state x db gen refuses: x db migrate applied it and then answered X_DB_DRIFT naming x db gen, whose own X_MIGRATION_SNAPSHOT_MISSING said "restore from version control" for a file version control never had. x new writes no migration at all now — x db gen is the one writer of packages/db/migrations. On 1.2.0, before the first migrate: rm packages/db/migrations/0000_initial.sql packages/db/migrations/0000_initial.hash && x db gen "initial"
A generated migration's foreign keys could not apply	2.0.0	every key was a references clause inside create table, walked in the app's import order — which says nothing about which table a key points at. Measured against PGlite: create table "comments" (… references "posts" …) ran before create table "posts". Every key is its own alter table … add constraint after all tables now, and down drops constraints before tables. On 1.2.0, move each clause out by hand before the migration is applied
The generated .snapshot.json failed the app's own lint	2.0.0	JSON.stringify(value, null, 2) never collapses a one-element array and Biome always does, so x verify answered X_LINT_FAILED on a file no author typed. The serialiser is a fixed point of Biome 2.5.5 at lineWidth: 100, and x new's biome.json excludes **/migrations. On 1.2.0, add "!**/migrations" to files.includes
Composite indexes emitted one mangled column name	2.0.0	indexes: [{ on: ['orgId','createdAt'] }] emitted on "todos" ("org_id_created_at"), which will not apply. The description carries the column list, the predicate and the direction, and the generator spells all three. On 1.2.0, write the index by hand
A migration with two statements failed to apply	2.0.0	cannot insert multiple commands into a prepared statement on the embedded database, and against a server the moment the text carried a bound value. migrate() and rollback() split with statementsOf() and send one statement at a time inside the same transaction. On 1.2.0, one statement per migration file
Every generator but x g resource imported files only x g resource wrote	2.0.0	the templates opened with import * as repo from '../repo', so the generated file did not load: X_CLI_UNEXPECTED from every registry command, --feature <slice> included. Each generator composes the slice modules its own source imports now, and a module the slice already has is skipped rather than overwritten, --force included. On 1.2.0: x g resource <slice> first, then the narrower generator with --feature <slice>
Jobs and tasks registered anonymously	4.0.0	a fresh scaffold had no apps/web/api/index.ts, so they registered as anonymous-job-2 / anonymous-task-1. x new writes it. On 3.0.0, add defineApi yourself → 4 · Jobs and realtime
Generated tests landed in the wrong x verify step	4.0.0	contractTest / liveTest / jobTest inside a plain *.test.ts ran under unit, and x test contract answered X_TEST_NO_FILES. The generators emit the typed filename now — <name>.contract.test.ts for x g action/x g mutator, <name>.live.test.ts for x g query --live, <name>.job.test.ts for x g job, x g task and x g backfill. On 3.0.0, rename by hand: the filename is the type
A disk registered under a name that is not its driver's 404'd its own signed URLs	4.0.0	localDriver minted /_storage/<driver>/<key> while the mounted route resolves the segment through the registry, so a disk registered as uploads signed a URL nothing served. StorageDriver.registerAs(diskName) is called by defineStorage at boot and the driver hangs its URLs off that; signedUrlBase is read off the driver by the verifying half, so the minter and the verifier cannot state it twice. On 3.0.0, register the local disk under the driver's own name (local)
acceptSignedUpload was reachable from no route	22.2.0	the framework mounted only GET /_storage/:disk/*key, so every grantUpload URL answered 405 (#145, #523). storageRoutes() now mounts the PUT beside it in x dev and runRole. It answers 201 { key } and validates against the grant's own signed constraints. On 22.1.0, mount the PUT in your app around acceptSignedUpload, or upload through a multipart action
/_storage served only the host's disk	22.2.0	disks an app declared with defineStorage signed URLs that 404'd, and an app with no storage:read got a runtime 500 X_PERMISSION_UNKNOWN (#524). The routes now read the process's one registry, which is the app's when it declares one, and x verify's policy step reports the missing permission. On 22.1.0, return the bytes from an action
The robots.txt read left from the worker's IP while every page load left through the proxy	4.0.0	two client identities presented to one origin — and an origin reachable only through the proxy answered nothing, which the gate reads as "no restrictions". ScrapeSession.proxy reports the exit the driver resolved, and createRobotsGate takes it as a resolver (proxy: () => sessionProxy) because the gate is an argument to driver.open() while the exit is decided inside it. The read also carries the run's deadline, its cancellation and a 500 KiB cap. On 3.0.0, use robots: { ignore: '<reason>' } on a proxied scrape, or accept that an unreadable robots.txt reads as allow
A malformed request body echoed itself into the 422	4.0.0	the parser's own message quotes the bytes it choked on, and it reached the cause: and the log store through String(error). The caller-facing half now names the format alone — could not parse the body as JSON — and the parser's message rides in meta through renderThrowable. On 3.0.0, do not log a 422's cause verbatim into anything a third party reads
createRateLimiter({ now })	4.0.0, and BREAKING	renamed to ({ clock }), the same Clock shape createRequestContext's init.clock takes — a second spelling of "what time is it" is a second way to set one number. The edit: createRateLimiter({ config, now: () => t }) becomes createRateLimiter({ config, clock: { now: () => new Date(t) } })

Reserved-but-unthrown error codes are listed in full under Error codes → Reserved codes. Symptom-first triage is Troubleshooting.