fix(catalogs): close resource-consumption gaps in agent catalog access

[bradhe]

This PR addresses the resource-consumption feedback on the DuckDB agent query path (follow-up to #328). The security review is settled; these are the remaining ceilings that did not hold.

Nested JSON walked under the 1 MiB response cap. The byte counter approximated serialized size and ignored quotes, escapes, and separators, so an array of empty strings counted as roughly zero bytes. SELECT list_transform(range(10000000), x -> '') AS xs came back with truncated: false while serializing to 120 MB, which defeats the point of having a response cap at all. The counter now computes the exact compact-JSON size (asserted against serde_json in tests), the budget check runs per value so an oversized row stops converting early, and the MCP layer serializes catalog results compactly so pretty-print whitespace cannot outgrow the ceiling the rows were admitted under.

Errors were an unbounded output channel. A DuckDB error can echo the offending value: SELECT CAST(repeat('x', 2000000) AS INTEGER) produced a 2 MB error containing the whole string, so an aggregate over a catalog column could exfiltrate past the result caps through the error message. Agent-facing errors are now bounded to 4 KiB, redacted before bounding so truncation cannot leave the token intact, with the full redacted message kept in the debug log.

Parallel calls multiplied the per-query limits. Every MCP request opened its own session, each entitled to 1 GiB of engine memory and 2 GiB of spill, and cancelling a request left the query running. Sessions now run inside a process-wide two-slot semaphore, so the worst case is bounded at twice the per-session ceilings instead of scaling with request count. I considered fully serializing sessions, but one slow analytical scan would then block every other call, and two slots keeps that from happening while staying bounded. A dropped request future now interrupts its running query via a cancel handle, and a request cancelled while queued never starts.

Discovery was unbounded. tower_catalogs_show listed every table with no limits into model context. The MCP path now runs the listing under the agent ceilings and reports tables_truncated; the CLI show stays unbounded because a person asked for the listing.

On the CI question: the Rust tests (including the adversarial suite and the new regression tests here) do run in CI, cargo test --all-features on every PR covers the whole workspace. What does not run in CI is the behave e2e file mcp_catalogs.feature, which is gated on TOWER_TEST_CATALOG plus a real TOWER_URL because it exercises a live attach against a real storage catalog, and CI only has the mock server. That gating is intentional, but it does mean the e2e scenarios only run when someone points them at a real environment.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2360125a-5170-4501-8647-80d34ee612f3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch duckdb-resource-limits-3136aa

---

_{Comment @coderabbitai help to get the list of available commands.}

[bradhe]

Closed by the branch rename; superseded by #333 (same commits, from the renamed branch fix/catalog-agent-resource-limits).