fix(mcp): report validation failures to agents instead of a generic error - #211
Merged
Merged
Conversation
…rror The stdio MCP server registers every tool on the official SDK, whose tool wrapper forwards only the text of its own ToolError to the client. Proofpress reports contract violations as ValueError (gateway checks) and ProofpressError (the kernel's operation envelope, raised by the client), so every rejected call reached the agent as "Error executing tool <name>" with the reason left in the server log. An agent that sent applicability.when_relevant as one sentence could not learn what to change, retried blind, and then dropped the discovery card altogether. - Route every registered tool through one wrapper that re-raises the MCP layer's own request errors (a new McpRequestError raised by the gateway checks) and ProofpressError as ToolError. The reason always reads "code: message": the operation envelope's code, message and details for a ProofpressError (with a retryable note), and invalid_tool_request for a gateway check, the code the hosted MCP transport already uses. A failure of the server's own environment (a transport error, the kernel's operation_io_error) forwards its code and a fixed phrase only, and the message, which can name a file or an upstream service, is logged on the server. Any other exception, a plain ValueError included, is left to the SDK, so a genuine crash still reaches the client as the generic line only. - Type the applicability parameter as a TypedDict so the published input schema lists title and description as non-empty strings and when_relevant, keywords and validity_conditions as nullable arrays of non-empty strings, requires at least one field, and rejects a string, an empty item or an unknown key at argument validation with a message naming the field. The hosted HTTP tool list publishes the same shape. The kernel now exports the card's field names so both transports are checked against it. - Report a non-JSON HTTP response as the transport error invalid_transport_response instead of letting json.JSONDecodeError escape the SDK client. - Test at the layer agents use: through MCPServer.call_tool and through a real stdio client for argument, gateway and kernel failures. Guards: every registered tool forwards a failure raised by its own gateway method, request-level and server-side envelope errors get the intended text, a crash stays generic, both transports publish the same card shape as the kernel's field list, and the module still imports with the mcp extra and its dependencies blocked. - Install the mcp extra in CI so those tests run there. Tests: python -m unittest discover -s tests on Python 3.11, 3.12, 3.13 and 3.14 with the mcp and oaff extras.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Agents calling the local stdio MCP server never saw why a call was rejected. Every validation failure came back as the single line
Error executing tool <name>; the reason went to the server's stderr. This PR makes the reason reach the agent, publishes the shape of theapplicabilitycard in the tool schema, and tests both through the real server.What changed:
ProofpressErrorinto the SDK'sToolError, so the reason the client reads iscode: message. AProofpressErrorkeeps the operation envelope's code, message and details (for exampleoperation_rejected: applicability must contain at least one discovery field) and notes when a retry may help. A check in the MCP layer (a newMcpRequestError) getsinvalid_tool_request, the code the hosted transport already uses. A failure of the server's own environment (a transport error, an I/O error on the workspace) forwards only its code and a fixed phrase, so the agent can tell a broken server from a bad request while the message, which can name a file or an upstream service, goes to the server log. Any other exception, a plainValueErrorincluded, is left to the SDK and still comes back as the generic line, so genuine crashes stay hidden as before.applicabilityonproofpress_propose_claimis now a typed card. The published input schema liststitleanddescriptionas non-empty strings andwhen_relevant,keywordsandvalidity_conditionsas nullable arrays of non-empty strings, and requires at least one field. A single sentence, an empty item or an unknown key is rejected at argument validation with a message that names the field. The hosted HTTP tool list publishes the same shape, and the kernel now exports the card's field names so both transports are checked against it.invalid_transport_responseinstead of letting a JSON decode error escape.mcpextra so the server-layer tests run on Python 3.11, 3.13 and 3.14.Why
The official SDK's tool wrapper forwards only the text of its own
ToolError. Proofpress raisesValueErrorfrom gateway checks andProofpressErrorfrom the kernel's operation envelope, so the SDK treated every contract violation as a crash. In the measurement harness, an agent that sentwhen_relevantas one sentence got no usable feedback, retried blind (6 of 12 and then 10 of 19 propose calls in one session failed this way), and finally proposed claims without any discovery card. The hosted HTTP transport already returned the message; only the stdio server hid it. The existing test for actionable errors called the gateway directly, so it could not catch this.Not done on purpose: accepting a single string for the list fields as a one-item list. The kernel and the hosted transport would still reject a string, and the two transports would then disagree on the contract. With the schema and the error text now naming the field, an agent can correct the call on the first retry.
Verification
ValueErrorand an injected crash still return only the generic line; each of the 19 registered tools forwards a failure raised by its own gateway method (arguments synthesised from each tool's published schema); both transports publish the same card shape and the same field set as the kernel; a real stdio subprocess client sees the reason for argument, gateway and kernel failures; and the module still imports with themcpextra and its dependencies blocked.invalid_transport_responsewithout echoing the body.python -m unittest discover -s testswith themcpandoaffextras passes on Python 3.11, 3.12, 3.13 and 3.14 (435 tests, 1 unrelated skip).ruff check src testsandpython -m compileall src testspass; pyright reports no errors on the transport, hosted schema, client and test files touched here.Risks / rollback
Request-level error text that used to stay in the server log now reaches the agent; it is the same text the Python SDK client already hands to every caller with the same credentials. Failures of the server's environment forward only their code. The kernel contract is unchanged. Revert the single commit to restore the previous behaviour.
References
_applicabilityand_text_listin src/proofpress/kernel/operations.py.Tool.runin the official SDK'smcp/server/mcpserver/tools/base.py(mcp 2.x).