Skip to content

fix(mcp): report validation failures to agents instead of a generic error - #211

Merged
chenmingtang830 merged 2 commits into
mainfrom
fix/mcp-tool-error-messages
Sep 30, 2026
Merged

chenmingtang830 merged 2 commits into
mainfrom
fix/mcp-tool-error-messages

Conversation

@Thru-Echoes

Copy link
Copy Markdown
Collaborator

Summary

Agents calling the local stdio MCP server never saw why a call was rejected. Every validation failure came back as the single line Error executing tool <name>; the reason went to the server's stderr. This PR makes the reason reach the agent, publishes the shape of the applicability card in the tool schema, and tests both through the real server.

What changed:

  • Every registered tool goes through one wrapper that turns the MCP layer's own request errors and any ProofpressError into the SDK's ToolError, so the reason the client reads is code: message. A ProofpressError keeps the operation envelope's code, message and details (for example operation_rejected: applicability must contain at least one discovery field) and notes when a retry may help. A check in the MCP layer (a new McpRequestError) gets invalid_tool_request, the code the hosted transport already uses. A failure of the server's own environment (a transport error, an I/O error on the workspace) forwards only its code and a fixed phrase, so the agent can tell a broken server from a bad request while the message, which can name a file or an upstream service, goes to the server log. Any other exception, a plain ValueError included, is left to the SDK and still comes back as the generic line, so genuine crashes stay hidden as before.
  • applicability on proofpress_propose_claim is now a typed card. The published input schema lists title and description as non-empty strings and when_relevant, keywords and validity_conditions as nullable arrays of non-empty strings, and requires at least one field. A single sentence, an empty item or an unknown key is rejected at argument validation with a message that names the field. The hosted HTTP tool list publishes the same shape, and the kernel now exports the card's field names so both transports are checked against it.
  • The SDK client reports a non-JSON HTTP response (a proxy or maintenance page answering 200 with HTML) as the transport error invalid_transport_response instead of letting a JSON decode error escape.
  • docs/REMOTE_MCP.md states the field types and the error behaviour of both transports.
  • CI installs the mcp extra so the server-layer tests run on Python 3.11, 3.13 and 3.14.

Why

The official SDK's tool wrapper forwards only the text of its own ToolError. Proofpress raises ValueError from gateway checks and ProofpressError from the kernel's operation envelope, so the SDK treated every contract violation as a crash. In the measurement harness, an agent that sent when_relevant as one sentence got no usable feedback, retried blind (6 of 12 and then 10 of 19 propose calls in one session failed this way), and finally proposed claims without any discovery card. The hosted HTTP transport already returned the message; only the stdio server hid it. The existing test for actionable errors called the gateway directly, so it could not catch this.

Not done on purpose: accepting a single string for the list fields as a one-item list. The kernel and the hosted transport would still reject a string, and the two transports would then disagree on the contract. With the schema and the error text now naming the field, an agent can correct the call on the first retry.

Verification

  • New tests in tests/test_mcp_adapter.py run through the SDK server, not the gateway: a string for any of the three list fields, an empty item and an unknown field name the field; gateway and kernel messages arrive with their code; a null list field is treated as absent, as the kernel does; a full five-field card round-trips; an I/O or transport failure arrives as its code and the fixed phrase while the file name goes to the log; a retryable error says so; a decode failure, a plain ValueError and an injected crash still return only the generic line; each of the 19 registered tools forwards a failure raised by its own gateway method (arguments synthesised from each tool's published schema); both transports publish the same card shape and the same field set as the kernel; a real stdio subprocess client sees the reason for argument, gateway and kernel failures; and the module still imports with the mcp extra and its dependencies blocked.
  • tests/test_python_sdk.py starts a local HTTP server that answers 200 with HTML and checks the client raises invalid_transport_response without echoing the body.
  • python -m unittest discover -s tests with the mcp and oaff extras passes on Python 3.11, 3.12, 3.13 and 3.14 (435 tests, 1 unrelated skip).
  • ruff check src tests and python -m compileall src tests pass; pyright reports no errors on the transport, hosted schema, client and test files touched here.

Risks / rollback

Request-level error text that used to stay in the server log now reaches the agent; it is the same text the Python SDK client already hands to every caller with the same credentials. Failures of the server's environment forward only their code. The kernel contract is unchanged. Revert the single commit to restore the previous behaviour.

References

  • Kernel rules for the card: _applicability and _text_list in src/proofpress/kernel/operations.py.
  • SDK behaviour: Tool.run in the official SDK's mcp/server/mcpserver/tools/base.py (mcp 2.x).

…rror

The stdio MCP server registers every tool on the official SDK, whose tool
wrapper forwards only the text of its own ToolError to the client.
Proofpress reports contract violations as ValueError (gateway checks) and
ProofpressError (the kernel's operation envelope, raised by the client), so
every rejected call reached the agent as "Error executing tool <name>" with
the reason left in the server log. An agent that sent
applicability.when_relevant as one sentence could not learn what to change,
retried blind, and then dropped the discovery card altogether.

- Route every registered tool through one wrapper that re-raises the MCP
  layer's own request errors (a new McpRequestError raised by the gateway
  checks) and ProofpressError as ToolError. The reason always reads
  "code: message": the operation envelope's code, message and details for a
  ProofpressError (with a retryable note), and invalid_tool_request for a
  gateway check, the code the hosted MCP transport already uses. A failure
  of the server's own environment (a transport error, the kernel's
  operation_io_error) forwards its code and a fixed phrase only, and the
  message, which can name a file or an upstream service, is logged on the
  server. Any other exception, a plain ValueError included, is left to the
  SDK, so a genuine crash still reaches the client as the generic line only.
- Type the applicability parameter as a TypedDict so the published input
  schema lists title and description as non-empty strings and when_relevant,
  keywords and validity_conditions as nullable arrays of non-empty strings,
  requires at least one field, and rejects a string, an empty item or an
  unknown key at argument validation with a message naming the field. The
  hosted HTTP tool list publishes the same shape. The kernel now exports the
  card's field names so both transports are checked against it.
- Report a non-JSON HTTP response as the transport error
  invalid_transport_response instead of letting json.JSONDecodeError escape
  the SDK client.
- Test at the layer agents use: through MCPServer.call_tool and through a
  real stdio client for argument, gateway and kernel failures. Guards: every
  registered tool forwards a failure raised by its own gateway method,
  request-level and server-side envelope errors get the intended text, a
  crash stays generic, both transports publish the same card shape as the
  kernel's field list, and the module still imports with the mcp extra and
  its dependencies blocked.
- Install the mcp extra in CI so those tests run there.

Tests: python -m unittest discover -s tests on Python 3.11, 3.12, 3.13 and
3.14 with the mcp and oaff extras.
@vercel

vercel Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
proofpress Ready Ready Preview Sep 30, 2026 4:12pm UTC

Request Review

@chenmingtang830
chenmingtang830 merged commit 1bfdd5d into main Sep 30, 2026
10 checks passed
@chenmingtang830
chenmingtang830 deleted the fix/mcp-tool-error-messages branch September 30, 2026 16:17

This branch was successfully deployed

1 active deployment
Preview — 0910b9ea Deployed Sep 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants