fix(edge): stop agent-edge 404ing every real /api/* route on GET

[sarthakagrawal927]

The bug

agent-edge.mjs ran a subtractive catch-all before OpenNext ever saw the request:

// JSON errors for unknown /api/* paths.
if (path.startsWith('/api/')) {
  return jsonError(404, 'not_found', `Unknown API path: ${path}`, path);
}

handleAgentEdge only engages for GET/HEAD, so every GET to a real /api/* route was 404'd at the edge and never reached its handler. POST was unaffected — that asymmetry is the tell.

Live reproduction against https://karte.cc (before this PR)

/api/welcome         GET=404  POST=401   ← real handler answering on POST
/api/pages           GET=404  POST=401   ← this route EXPORTS GET; broken
/api/demo-chat       GET=404  POST=400
/api/agent-waitlist  GET=404  POST=400
/api/v1/agents       GET=404  POST=401
/api/ai              GET=200             ← edge-owned path, allow-listed

$ curl -s https://karte.cc/api/pages
{"error":{"code":"not_found","message":"Unknown API path: /api/pages","path":"/api/pages"}}

Scope

21 of 58 route handlers under src/app/api export GET and were therefore shadowed in production:

• /api/ai/markdown
• /api/auth/[...all] — the better-auth catch-all, so auth flows that rely on GET are affected
• /api/chat/[slug]/messages
• /api/og
• /api/pages
• /api/pages/[pageId]/conversations
• /api/pages/[pageId]/conversations/[conversationId]
• /api/pages/[pageId]/domains
• /api/pages/[pageId]/emails
• /api/pages/[pageId]/emails/[emailId]
• /api/pages/[pageId]/generated-status
• /api/pages/[pageId]/info
• /api/pages/[pageId]/links
• /api/pages/[pageId]/opportunities
• /api/pages/[pageId]/projects
• /api/pages/[pageId]/sections
• /api/pages/[pageId]/timeline
• /api/settings/ai-key
• /api/settings/knowledgebase
• /api/v1/agents
• /api/v1/agents/[slug]

The remaining 37 handlers export only POST/PATCH/DELETE and were unaffected — which is why the outage was invisible to the app's own write paths.

The fix

Mirrors the shape just landed for the identical defect in Codevetter/starboard#102, adapted to karte's worker.mjs wiring.

1. The edge becomes a strict pre-handler. handleAgentEdge answers only for paths its EXACT_ROUTES map grants it (/llms.txt, /llms-full.txt, /index.md, /robots.txt, /openapi.json, /openapi.yaml, /api/ai) and returns null for everything else. It can no longer conclude that a path does not exist — only Next.js knows the route table.
2. The JSON-404 envelope moves to a post-handler. New withApiJsonNotFound(request, response) is applied in worker.mjs to the response coming back from OpenNext. Only when Next.js itself 404s an /api/* path does the HTML error page become {"error":{"code":"not_found",…}}. A route handler's own JSON 404 body is left untouched.
3. /api/* is excluded from the markdown/Accept branch, so an Accept: text/markdown header can never divert a real API request to the agent markdown 404.

worker.mjs has exactly two call sites an /api/* request can reach (the non-GET branch — which also carries HEAD — and the non-cacheable-document branch; API paths are never in CACHEABLE_EXACT), and both are wrapped.

Why this shape cannot rot

The old allow-list was subtractive: it denied everything under /api/ except a hand-maintained set, so any route added later was born broken and nothing in the repo noticed. The new arrangement is additive — the edge is granted specific paths and defers on all others — and the 404 decision is delegated to the only component that actually knows the route table. Adding a handler under src/app/api/ makes it reachable with zero edge changes. On top of that, the regression test enumerates the route list off the filesystem (it.each(API_ROUTE_PATHS) over a readdirSync walk of src/app/api), so a newly added handler is covered the moment it lands — no second hand-maintained list to rot.

Regression test

tests/agent-edge-api-routing.unit.test.mjs drives the real worker.mjs entrypoint (not a reimplementation) against a stubbed OpenNext handler. vitest.config.ts aliases ./.open-next/worker.js (a gitignored build artifact) and cloudflare:workers to fixtures under tests/fixtures/.

It asserts every discovered API route is reachable through the edge on GET, that HEAD works, that an Accept header cannot divert one, that an unknown /api/* path still yields the JSON 404, that a handler's own JSON 404 survives, that non-API 404s still render HTML, and that all edge-owned surfaces (/api/ai, llms.txt, robots.txt, homepage markdown negotiation, markdown 404) still work.

Non-vacuousness proof

With the test in place and only the agent-edge.mjs + worker.mjs changes stashed:

$ git stash push -- agent-edge.mjs worker.mjs
$ pnpm vitest run tests/agent-edge-api-routing.unit.test.mjs
⎯⎯⎯⎯⎯⎯ Failed Tests 63 ⎯⎯⎯⎯⎯⎯⎯
AssertionError: expected [] to deeply equal [ '/api/pages' ]
AssertionError: expected [] to deeply equal [ '/api/agent-waitlist' ]
AssertionError: expected [] to deeply equal [ '/api/ai/markdown' ]
AssertionError: expected [] to deeply equal [ '/api/ai/models' ]

 Test Files  1 failed (1)
      Tests  63 failed | 6 passed (69)

With the fix restored:

$ pnpm vitest run tests/agent-edge-api-routing.unit.test.mjs
 Test Files  1 passed (1)
      Tests  69 passed (69)

(expected [] to deeply equal [...] = the stub OpenNext handler was never called, i.e. the edge swallowed the request.)

No Playwright test was added: karte's e2e suite runs against next dev (playwright.config.ts webServer), which does not put worker.mjs in the request path at all, so an e2e assertion there would have passed against the broken code and proved nothing.

Verification

Command	Result
pnpm typecheck	clean
pnpm test (vitest)	22 files, 163 tests passed (was 21/94)
pnpm docs:check	0 errors, 8 warnings (all pre-existing, in docs/archive/)
pnpm lint (biome check .)	1 error — pre-existing on main, in .fleet/evidence/landing-audit/scorecard.json (formatting), committed by 00683c0. Confirmed failing on a clean main checkout; not touched here. All files in this PR are Biome-clean.

Generated-file provenance

agent-edge.mjs carries the same fleet provenance as starboard's — its header says "Portable agent-edge handler — copy or generate into each product. Spec: fleet-ops/docs/agent-indexing-standard.md", and the payload is marked // biome-ignore format: generated payload from apply-agent-surfaces. Regenerating from an unfixed upstream template would silently reintroduce this outage, so docs/architecture/edge-worker.md gains a carry-forward note (and docs/development/conventions.md a "don't reintroduce" entry) recording the defect and pointing at the test that fails loudly if the catch-all comes back. No generator outside this repo was modified.

⚠️ Production stays broken until someone deploys

Karte deploys are manual (workflow_dispatch / pnpm deploy:cf). Merging this PR does not fix https://karte.cc — every GET /api/* will keep returning {"error":{"code":"not_found",…}} until a deploy ships this Worker. Nothing was deployed as part of this change.

Post-deploy check:

curl -s -o /dev/null -w '%{http_code}\n' https://karte.cc/api/pages          # expect 401, not 404
curl -s -o /dev/null -w '%{http_code}\n' https://karte.cc/api/auth/session   # expect not-404
curl -s https://karte.cc/api/ai | head -c 80                                 # edge surface still 200
curl -s https://karte.cc/api/definitely-not-a-real-path                      # JSON not_found envelope

⚠️ CI is red for a pre-existing reason, not this change

pnpm quality starts with pnpm check (biome check .) and aborts on the
.fleet/evidence/landing-audit/scorecard.json formatting error introduced by
00683c0 on main — so CI never reaches the test step here. The same run is red
on main itself (run 33262947516, sha 00683c0, before this branch existed).
Deliberately not fixed here: it is an unrelated file. Once it is formatted, this
branch's pnpm test passes 163/163 locally, as above.

🤖 Generated with Claude Code