@hasmcp/mcp-spec-test

A black-box conformance suite for any MCP server. It speaks the protocol over the wire and nothing else — no plugin, no instrumentation, no knowledge of the implementation — so the same suite runs against a local process, a staging deployment or somebody else's hosted endpoint without modification.

It supports the two most recent spec revisions only — currently 2026-07-28 and 2025-11-25 — and always tracks the newest one. Anything older is reported as out of scope rather than judged. See Supported revisions.

npx @hasmcp/mcp-spec-test@latest -u https://mcp.example.com/mcp -t <token>
npx @hasmcp/mcp-spec-test@latest -c "npx -y @modelcontextprotocol/server-everything"

It reports what the server does, where it deviates from the spec, and — as its own section — what could not be checked:

MCP 2026-07-28 conformance report
target    https://mcp.example.com/mcp
transport streamable-http
supported 2026-07-28, 2025-11-25

FAILED (2) — the server deviates from the spec here

  Result envelope
    ✗ cacheable list results carry the schema-required cache hints
        tools/list: ttlMs must be non-negative
    ✗ results identify the server in _meta
        expected io.modelcontextprotocol/serverInfo in result _meta, got undefined

NOT VERIFIED (4) — skipped; a skip is not a pass

  Capability methods
    – prompts/list returns schema-conformant prompts
    – prompts/get returns messages with a role and content
        target does not advertise the "prompts" capability: ["tools","resources"]

PASSED (29)
  ...

Summary
  29 passed  2 failed  4 not verified

Verdict: not conformant — 2 requirements violated.

Exit code is 0 when nothing failed.

Requirements this revision does not have are not reported

A case that does not run means one of two things, and only one of them is worth a reader's attention:

NOT VERIFIED — the requirement applies to this target, but the run could not establish it: a capability was not advertised, a call needed arguments the suite will not invent, a stream never opened. This is the list to work through before claiming conformance, which is why it is printed above the passes.
Not applicable — the revision under test does not define the thing, or the transport has no such requirement. initialize is not a gap in a 2026-07-28 run; that revision replaced the handshake with server/discover. These are omitted entirely — not listed, not counted. A requirement the spec does not define is not a result, and showing it only invites the question of why the suite is testing something the spec does not have.

So the totals describe what actually applied, and a clean run against a server on an older revision reads as a plain answer:

PASSED (20)
  ...

Summary
  20 passed  0 failed  0 not verified
  20 cases applied, in 292ms

Verdict: conformant to 2025-11-25 — every requirement that applies to this
revision and transport was checked and passed.

The same suite reports 36 applied cases against a 2026-07-28 server. The difference is not something withheld; those revisions have different requirements. Which cases belong to which is listed in full under Every case, so the numbers are always explainable.

Supported revisions

The suite reasons about the two most recent revisions — currently 2026-07-28 and 2025-11-25 — and the newest of those is the one whose conformance it asserts.

That window is not a hardcoded list. It is read from the schemas vendored under spec/, so the newest revision on disk is always the revision under test, and adding support for a new spec is dropping its published schema.json at spec/<date>/schema.json — no code change, and no way for a list and the shipped schemas to disagree about what is supported.

A CI job checks that promise instead of trusting it: it asks the spec repository what revisions exist and fails when one is published that is not vendored here. "Always tests the latest spec" is the kind of claim that decays silently, so it is verified on every push.

Both revisions in the window get a full conformance pass — point --spec-version at either. Which cases run is decided by the schema, not by a date: lib/schema.mjs derives a feature set from the vendored $defs, so a revision that has no server/discover skips those cases because it does not define the request, and gets its handshake checked instead.

2026-07-28: discover ✓  handshake ✗  perRequestVersion ✓  resultEnvelope ✓  listenSubscriptions ✓
2025-11-25: discover ✗  handshake ✓  perRequestVersion ✗  resultEnvelope ✗  resourceSubscribe ✓

That is why a skip reads "2025-11-25 does not define resultEnvelope; nothing to assert" rather than naming a version the suite was told about. It also means a future revision that moves features around selects the right cases with no code change — the point of vendoring schemas rather than branching on dates.

Revisions outside the window are reported as out of scope, not judged. A server advertising 2024-11-05 as its only older revision gets the backward-compatibility cases skipped with that reason, rather than checked against assumptions the suite cannot back with a schema. The handshake is checked against the window too: a server that answers a stock client with an ancient revision is reported, not passed over.

The revision is chosen from what the server serves

Point the suite at a server and it asks what revisions that server offers, then tests the newest one inside its window. A server on 2025-11-25 gets a 2025-11-25 verdict; you do not have to know which revision it speaks before you can test it. The report says where the answer came from:

$ npx @hasmcp/mcp-spec-test@latest -u "https://example.com/mcp?token=..."
testing 2025-11-25 — the newest supported revision this server offers, per the handshake

server/discover is asked first, since it lists every servable revision. A server that answers it with an error predates it, and the handshake is asked instead. But a target that cannot be reached — rate-limited, or a transport failure — resolves to neither, and the run says so rather than guessing:

could not establish which revisions this server offers; testing the newest
supported one — pass --spec-version to be explicit

That distinction is load-bearing. The handshake reports one negotiated version, usually an older one, so treating a throttled discover as "no discover" would quietly test a different revision than the server offers and publish a confident verdict for it.

This matters because the capability cases assert against the schema of the revision under test, and those requirements differ — ListToolsResult requires resultType, ttlMs and cacheScope in 2026-07-28 and only tools before it. Running them against a server on another revision would fail it for not implementing something it never claimed, so they skip instead. Correct, but "28 not verified" is a useless answer to "is my server conformant", which is what the auto-selection fixes.

An explicit --spec-version always wins. "Check my server against 2026-07-28" is a legitimate question, and the answer to it must not silently become the answer to a different one.

Rate limits are waited out, not reported as failures

A 429 says nothing about conformance, so the suite neither fails the case nor hammers the target. It opens a breaker — every request holds off — waits for the interval the target asked for in Retry-After, or backs off from a second, and carries on when the limit clears. --retry-budget-ms bounds the total wait (default 60s); past that the case reports itself as not verified, naming the rate limit, because that is what happened.

For a target whose limit a full pass would exceed anyway, pace the run instead:

# the server allows 60 requests per 60s
npx @hasmcp/mcp-spec-test@latest -u "$URL" --rate-limit 60

--rate-limit is in requests per minute so the number can be copied straight from the server's configuration. It also runs the test files one at a time — node:test gives each file its own process, so without that the limit would quietly mean that many requests per file.

Every case, one by one

43 cases in total, of which a given run reports only those that apply to the target. Three independent gates decide that, and each case below states its own:

Revision — derived from the vendored schema. server/discover cases cannot run against a revision whose schema has no DiscoverRequest.
Capability — read from what the server advertises. A server with no prompts capability skips the prompt cases instead of failing them.
Transport — a few requirements exist only on Streamable HTTP (status codes, response headers). They skip on stdio because they are absent there, not because they were met.

Cases ruled out by the revision or the transport are omitted from the report rather than listed — see above. Cases that applied but could not be checked are always printed, with the reason.

server/discover — 2026-07-28 only

The revision made discover mandatory: a client must be able to pick a version with no handshake and no session. A revision without it negotiates at initialize instead, and these skip.

server/discover is answered without a session or handshake
Calls discover with no version declared at all and checks the result carries every field the schema marks required. This is the case that fails first against a server built for an older revision, which is the correct outcome — it does not implement a MUST of the revision under test.
server/discover advertises the versions the server can serve
supportedVersions must be a non-empty array of ISO dates, and the version the server negotiates for a client that declares none must itself appear in it. A server promising a default it cannot serve leaves version-less clients stuck.
server/discover is a CacheableResult with usable cache hints
ttlMs must be a non-negative number and cacheScope must be public or private. Discover exists to be cached; hints a client cannot act on defeat it.
server/discover reports server identity and capabilities
capabilities must be an object, and the server must name itself. serverInfo is not schema-required here, but a server that cannot say what it is makes every diagnostic downstream guesswork.
server/discover is stable across calls within its own TTL
Calls discover twice and compares supportedVersions. A server that varies its answer has made its own ttlMs a lie, and a client that cached it is wrong.
server/discover advertises a revision this suite supports
Checks the advertised list overlaps the two-revision window. Without an overlap this tool cannot make a conformance statement at all, and says so rather than asserting the current spec against a server built for an older one.

The handshake — 2025-11-25 only

What discover replaced. Every revision before 2026-07-28 negotiates once, at initialize, and expects notifications/initialized before ordinary traffic.

initialize returns the schema-required fields
InitializeResult requires capabilities, protocolVersion and serverInfo. A client cannot proceed without all three, so they are checked against the schema's own list rather than a hand-copied one.
the handshake settles on the revision under test
The suite asks for the revision it is testing and checks that is what it got. Carrying on after a silent downgrade would assert one revision's requirements against a server that agreed to a different one.
an unsupported version offered at the handshake is refused or downgraded, not echoed
Offers 1999-01-01. Refusing is conformant and answering with a real version is conformant; echoing back a version the server cannot speak is not, because the client will then go on to speak it.

Version negotiation — 2026-07-28 only

The revision declares the protocol version on every request, in _meta, rather than once per session.

a version declared in _meta is accepted
The baseline: a correctly versioned request is answered. Runs against a real capability-backed method where the server has one, falling back to discover.
the negotiated version is echoed in the response header (Streamable HTTP only)
The server must echo MCP-Protocol-Version, which is how a client confirms what it is actually talking to rather than what it asked for.
header and _meta version disagreement is a HeaderMismatch error (Streamable HTTP only)
Sends deliberately disagreeing values and expects -32020 with a 400. The spec requires the disagreement to be rejected rather than silently resolved in favour of one side, because either choice would be invisible to the client.
an unsupported version is rejected with the supported list
Expects -32022 carrying data.supported, and checks that list matches what discover advertises. A client that retries from the error and a client that reads discover must not be told different things.
an unsupported version is rejected with a 400 on Streamable HTTP (Streamable HTTP only)
The status-code half of the requirement above, which only exists over HTTP.
clientInfo is optional (SHOULD, not MUST)
Sends a request with clientCapabilities and protocolVersion but no clientInfo, which must still succeed. A server that requires it has turned a SHOULD into a MUST and will reject conformant clients.

Version negotiation — both revisions

a request with no version at all is served on the default
Backward compatibility from the client's side: a client that declares nothing must still be served, on whatever version the server defaults to. Checked on both revisions because both make the promise.

Result envelope — 2026-07-28 only

resultType on every result, ttlMs/cacheScope on the cacheable ones, and serverInfo moved into result _meta when the handshake went away.

every result carries the required resultType
Checks each advertised cacheable list method returns resultType: "complete". Methods the server does not advertise are skipped, not failed.
cacheable list results carry the schema-required cache hints
Every field the schema marks required on each list result, plus ttlMs non-negative and cacheScope one of public/private. The assertion list comes from the schema, so tightening the schema tightens this case.
results identify the server in _meta
_meta must carry io.modelcontextprotocol/serverInfo with a name and version. This is where a client learns what it is talking to now that there is no initialize result to read it from.
a client on an older version receives no newer-revision fields
The compatibility guarantee in the other direction: a request made on an older revision must come back with no resultType, ttlMs, cacheScope or _meta. This is the case that catches a server implementing the envelope by simply adding the fields everywhere. It needs a second in-window revision the server also serves, and skips saying so if there is not one.
schema sanity: the envelope fields match the features selected
Guards the vendored schema from being swapped underneath the suite, which would silently weaken every case above. Needs no server.

subscriptions/listen — 2026-07-28 only

One long-lived POST-response stream replacing the HTTP GET stream and resources/subscribe.

subscriptions/listen acknowledges only the opted-in notification types
Opts into one type and checks the acknowledgment names it and nothing else. The acknowledgment is what a client relies on to know what it will receive, so a server adding types to it is promising traffic the client is not prepared for.
the acknowledgment carries the subscription id for correlation
The ack's _meta must carry the listen request id. Without it a client running several subscriptions cannot tell which one was acknowledged.
a listen requesting no notification types is not a subscription to everything
Asks for nothing. Rejecting that is conformant; answering it by subscribing to everything is not, since the client has no way to interpret what arrives.
a cancelled subscription ends with a conformant teardown result, if it sends one
Opens a subscription, cancels it with notifications/cancelled, and checks the teardown result carries resultType and names the subscription it ended. Closing the stream quietly is also conformant, so that outcome skips — what this will not do is let a malformed teardown pass unnoticed.
schema sanity: SubscriptionsListenResult requires _meta and resultType
The schema side of the same contract, which holds even against a server that never tears down. Needs no server.

Capability methods — both revisions, gated on what the server advertises

These are the methods a server exists to serve. Each is gated on the capability in question, so the same suite covers a tools-only server and a full one.

tools/list returns schema-conformant tools (needs tools)
Every tool checked against the schema's required fields, names unique, and inputSchema an object of type object. A duplicate name makes a tool unaddressable, since tools/call selects by it.
tools/call on an unknown tool is an error, not a crash (needs tools)
Either a JSON-RPC error or isError: true is conformant. Silently succeeding is not — a client would treat a call that never happened as done.
tools/call returns a schema-conformant CallToolResult (needs tools)
Calls a tool and checks the result's required fields and content blocks. A tool declaring an outputSchema must return structuredContent, since that declaration is a promise clients build against. The suite will not invent arguments for a tool whose side effects it cannot know — pass --tool-args to opt in, otherwise it skips and says so.
prompts/list returns schema-conformant prompts (needs prompts)
Schema-required fields on every prompt, and unique names for the same reason tools need them.
prompts/get returns messages with a role and content (needs prompts)
Required fields on the result and on each message, with role one of user/assistant. Use --prompt-args for prompts with required arguments.
resources/list returns schema-conformant resources (needs resources)
Required fields, plus every uri actually parsing as a URI — one that does not cannot be handed back to resources/read.
resources/templates/list returns schema-conformant templates (needs resources)
Same treatment for templates. Skips if the server does not answer the method.
resources/read returns contents for every sampled resource (needs resources)
Reads several listed resources rather than the first — a server that special-cases its first entry passes a one-resource check and fails a client on the second click. A listed resource that cannot be read is a failure, not a skip: the server put it in the list. Bounded by --resource-sample (default 5), and it says how many of how many it read.
resources/read on an unknown uri is an error (needs resources)
The error path, so a client can distinguish a missing resource from an empty one.

Pagination — both revisions

following nextCursor terminates and does not repeat a page (needs tools)
Walks nextCursor to the end, failing if a tool appears twice or a cursor is reissued. The bug this catches is a cursor that yields the same page forever: a paginating client loops and a single-page check never sees it. Bounded by --page-limit.
an invalid pagination cursor is rejected (SHOULD) (needs tools)
Sends a cursor the server never issued. The spec recommends -32602 here rather than requiring it, so this is recorded under RECOMMENDED rather than failed — but silently returning page one is worth knowing, because a client cannot tell a rejected cursor from a reset one.

Official SDK interop — both revisions

These cases drive the target with @modelcontextprotocol/sdk rather than this suite's own client. That is the point of them: an independent implementation checks a server's backward-compatibility claim against something that is not this suite's assumptions. What the SDK does or does not support is read from the SDK at runtime, never asserted here from memory.

the official SDK does not yet implement the newest revision
The suite drives the newest revision over raw JSON-RPC because the official client cannot yet. Rather than assume that stays true, this reads the SDK's own supported-version list at runtime and fails on purpose once it covers the newest revision — the signal to migrate these cases onto the official client. Pinned to the newest revision, not the one under test, so --spec-version set to the older one does not trip it. Needs no server.
a stock official-SDK client completes the handshake
A real third-party client connects and gets usable serverInfo and capabilities. Skips if the server serves nothing the SDK speaks.
the handshake settles on a revision inside the supported window
A stock client will accept revisions considerably older than this suite reasons about. This checks what the server actually answers one with, so a fallback past the supported window is reported rather than passed over.
a stock official-SDK client can list tools (needs tools)
listTools() throws if the payload fails the SDK's own schema validation, so a clean round-trip is independent evidence that no newer-revision field is leaking to a client that negotiated an older one.
a completely unconfigured SDK client works (Streamable HTTP only)
A transport built with no requestInit, no custom headers and no server-specific configuration at all. If it passes, a stock MCP client can be pointed at the server and simply work. Skips when credentials are header-bound, since a client must then be configured for them.

Suite integrity — both revisions

the suite is reading a schema that matches the features it selected
Cross-checks the vendored schema against the feature set derived from it, so a schema swapped underneath the suite surfaces here rather than as a quietly weaker pass everywhere else. Needs no server.

Assertions come from the schema, not from us

lib/schema.mjs reads the published schema.json for the revision under test — vendored at spec/<revision>/schema.json — and exposes its own required lists and method name constants. Tests assert against those rather than hand-copied field names, so tightening the schema tightens the suite. Guard cases check that the schema being read really is the revision claimed, so the assertions cannot be silently weakened by pointing at an older one.

It asks the server what to test

Nothing has to be configured to describe the target. The suite performs one version-less server/discover and derives the rest from the answer — which is exactly what that method exists for:

a server that does not advertise the revision under test skips the revision-specific cases, quoting the list it did advertise
capabilities absent from discover skip their methods rather than failing on something the server never claimed
the backward-compatibility cases pick a real older revision out of supportedVersions — the newest one that is also inside the supported window — instead of trusting a configured constant

Usage

Both transports the spec defines are supported. Pass one target, not both — they select different transports.

# Streamable HTTP
npx @hasmcp/mcp-spec-test@latest -u https://mcp.example.com/mcp -t <token>

# stdio
npx @hasmcp/mcp-spec-test@latest -c "node ./build/index.js"
npx @hasmcp/mcp-spec-test@latest -c "uvx mcp-server-git --repository ."

Every flag has an environment-variable twin and flags win, so a one-off check reads naturally on the command line while CI can keep a token out of the process list:

MCP_URL=https://mcp.example.com/mcp MCP_TOKEN="$TOKEN" npx @hasmcp/mcp-spec-test@latest

flag	env	meaning
`-u`, `--url`	`MCP_URL`	Streamable HTTP endpoint
`-c`, `--command`	`MCP_COMMAND`	command to spawn and speak stdio to
`-t`, `--token`	`MCP_TOKEN`	bearer token
`--auth-mode`	`MCP_AUTH_MODE`	`header` (default), `query`, `none`
`--auth-header`	`MCP_AUTH_HEADER`	header carrying the token (default `authorization`)
`--auth-scheme`	`MCP_AUTH_SCHEME`	token prefix (default `Bearer`; empty for a bare token)
`--auth-query-param`	`MCP_AUTH_QUERY_PARAM`	query parameter in query mode (default `token`)
`-H`, `--header`	`MCP_EXTRA_HEADERS`	extra headers, `"x-tenant: acme, x-api-key: abc"`
`--spec-version`	`MCP_SPEC_VERSION`	revision to test; must be inside the supported window (default: the newest vendored)
`--spec-path`	`MCP_SPEC_PATH`	a `schema.json` to assert against instead of the vendored copy
`--default-version`	`MCP_SERVER_DEFAULT_VERSION`	the version the server negotiates for a client declaring none; probed if omitted
`--stream-budget-ms`	`MCP_STREAM_BUDGET_MS`	how long streaming cases wait (default 4000)
`--tool-args`	`MCP_TOOL_ARGS`	arguments per tool, inline JSON or `@file` — see below
`--prompt-args`	`MCP_PROMPT_ARGS`	arguments per prompt, same shape
`--resource-sample`	`MCP_RESOURCE_SAMPLE`	how many listed resources to read (default 5)
`--page-limit`	`MCP_PAGE_LIMIT`	pages to follow before calling pagination broken (default 10)
`--rate-limit`	`MCP_RATE_LIMIT`	pace requests to at most n per minute
—	`MCP_RETRY_BUDGET_MS`	how long to wait out a rate limit (default 60000; 0 disables)
`--verbose`	`MCP_VERBOSE`	also print the target's own stderr
`--only <pattern>`	—	run only test files matching a substring
`--tap`	—	raw TAP instead of the report, for CI parsing

Docker

The image bundles Node, Python/uv, Go, Rust and Ruby, since a -c command that launches a real-world MCP server is as likely to be uvx or cargo run as npx:

docker run --rm <account>/mcp-spec-test -u https://mcp.example.com/mcp -t <token>
docker run --rm <account>/mcp-spec-test -c "uvx mcp-server-git --repository ."

A stdio target that lives on the host (a local checkout, a Unix socket, a repository the command needs to read) has to be reachable inside the container too — mount it and point the command at the mounted path:

docker run --rm -v "$PWD:/repo" <account>/mcp-spec-test -c "uvx mcp-server-git --repository /repo"

Non-standard credentials

The spec expects an OAuth 2.1 bearer token in the standard Authorization header, which is the default. Servers that deviate can still be tested:

# a custom header
npx @hasmcp/mcp-spec-test@latest -u "$URL" -t "$TOKEN" --auth-header x-api-key

# a bare token with no scheme
npx @hasmcp/mcp-spec-test@latest -u "$URL" -t "$TOKEN" --auth-header x-api-key --auth-scheme ''

# a query parameter
npx @hasmcp/mcp-spec-test@latest -u "$URL" -t "$TOKEN" --auth-mode query

Query mode is worth understanding rather than reaching for: a token in a URL lands in access logs, proxy logs, browser history and Referer headers. What it buys is interop. A credential in a non-standard header obliges every integrator to plumb that header through their client before anything works; a query parameter lets a stock client connect untouched. The suite has a case for exactly that (a completely unconfigured SDK client works), which skips with the reason when credentials are header-bound — so rather than take the claim on trust, point the suite at your endpoint and see whether that case runs.

Calls the suite will not guess at

A tool with required arguments is not called. The suite has no way to know that delete_everything wants {"confirm": true} and search wants {"query": "x"}, and inventing values for the first to exercise the second is not a trade a test tool gets to make on your behalf. So those cases skip and say so.

You know your own server, so you can opt in:

npx @hasmcp/mcp-spec-test@latest -u "$URL" \
  --tool-args '{"search":{"query":"conformance"},"echo":{"message":"hi"}}' \
  --prompt-args '{"review":{"file":"README.md"}}'

Named tools are preferred over argument-free ones, since calling them exercises argument handling as well. Anything not named keeps the old behaviour.

An argument set stops being something you want to quote on a command line as soon as it has a nested object or an entry per tool, so both flags also take @path to a file holding the same JSON — the curl convention:

npx @hasmcp/mcp-spec-test@latest -u "$URL" --tool-args @tool-args.json

{
  "webSearch":   { "queryArgs": { "q": "mcp spec conformance" } },
  "imageSearch": { "queryArgs": { "q": "model context protocol" } }
}

Relative paths resolve against your shell's working directory, and the file is read and validated before the run, so a wrong path or malformed JSON is one clear error rather than the same failure repeated inside every case that needed it. The @ form works through the environment variables too (MCP_TOOL_ARGS=@/etc/mcp/tool-args.json), which keeps a large argument set out of a CI command line.

How credentials are chosen

Whatever you already have is used, and OAuth is a last resort rather than a first move:

what you pass	what happens
nothing, and the endpoint is public	it just runs — no credential, no probe result to act on
`?token=…` already in the URL	used as-is; the endpoint answers, so nothing is negotiated
`-t <token>`	sent per `--auth-mode` (header by default)
`-H "authorization: Bearer …"` or any custom header	sent as given
nothing, and the endpoint answers `401`	the OAuth flow below — `client_credentials` where the authorization server offers it, a browser sign-in where it does not

The check is one request: whatever credentials you configured are sent, and if the endpoint answers, there is nothing to negotiate. That matters — probing without them would manufacture a 401 and go off registering a client for someone who had already supplied a credential.

OAuth 2.1

An endpoint that answers 401 with no credential supplied is handled without being told to. The suite walks the discovery chain the spec defines, registers a client dynamically if you have none, and then takes whichever grant the authorization server actually offers — client_credentials unattended, or a browser sign-in with PKCE where that is the only way in:

# nothing but the URL — a client is registered dynamically
npx @hasmcp/mcp-spec-test@latest -u https://mcp.example.com/mcp

# with credentials you already hold
npx @hasmcp/mcp-spec-test@latest -u https://mcp.example.com/mcp \
  --client-id "$CLIENT_ID" --client-secret "$CLIENT_SECRET" --scope mcp:read

Every step is printed, because registering a client creates state on somebody else's authorization server and that should never happen quietly:

endpoint returned 401; discovering OAuth configuration
  found protected-resource metadata at https://mcp.example.com/.well-known/oauth-protected-resource/mcp
  found authorization server metadata at https://auth.example.com/.well-known/oauth-authorization-server
  no client credentials given; registering a client with https://auth.example.com/register
  registered client_id dcr-1
  obtained an access token for resource https://mcp.example.com/mcp

flag	env	meaning
`--client-id`	`MCP_CLIENT_ID`	OAuth client id
`--client-secret`	`MCP_CLIENT_SECRET`	OAuth client secret
`--scope`	`MCP_SCOPE`	scope to request
`--no-register`	`MCP_NO_REGISTER`	never register dynamically, even where offered
`--interactive`	`MCP_OAUTH_INTERACTIVE=always`	sign in through a browser even where a machine grant exists
`--no-interactive`	`MCP_OAUTH_INTERACTIVE=never`	never open a browser; fail with a reason instead
`--no-browser`	`MCP_NO_BROWSER`	print the authorization URL rather than launching a browser
`--redirect-host`	`MCP_REDIRECT_HOST`	loopback host for the redirect (default `127.0.0.1`)
`--redirect-port`	`MCP_REDIRECT_PORT`	pin the redirect port instead of taking a free one
`--auth-timeout`	`MCP_AUTH_TIMEOUT_MS`	how long to wait for the redirect (default `180000`)

What it implements, and why each part matters:

Both discovery entry points. resource_metadata from the WWW-Authenticate header takes precedence; without it, the well-known URIs are tried in the specified order — path-scoped first, then root. Endpoints do it both ways.
Resource Indicators (RFC 8707). The token request carries resource=<canonical MCP server URI>, so the token is bound to this server as its audience. Query and fragment are stripped: credentials often ride in the query string, and a token audience must not be keyed to a secret.
The issuer check. Authorization server metadata is refused unless its issuer matches the issuer used to construct the URL. Skipping it is how a client is talked into sending credentials to an attacker, and the spec names that attack explicitly. A mismatch is reported, not worked around:
```
refused: declares issuer "https://somewhere-else.example", expected "https://auth.example.com"
```
Dynamic Client Registration as the fallback the spec prescribes, with application_type: "native" — a conformance runner is not a browser and has no redirect to come back to.
Tolerance for two deviations seen in the wild, reported rather than hidden. RFC 9728 defines authorization_servers as an array of issuer identifiers; some servers send authorization_server as a string, and some put the metadata document's URL there instead of the issuer. Both are usable once noticed, and discovery continues while printing what it found:
```
note: metadata uses "authorization_server" (a string); RFC 9728 defines
      "authorization_servers" as an array of issuer identifiers
```
Where the value is a metadata URL it is fetched directly, and the document's issuer must share its origin — so the anti-spoofing property survives the tolerance.

Which specifications this follows

document	what is implemented
RFC 9728 Protected Resource Metadata	both discovery entry points; `authorization_servers`, plus tolerance for the singular and metadata-URL forms with a note
RFC 8414 Authorization Server Metadata	well-known URI inserted between host and path (§3), the `openid-configuration` fallbacks (§5), and the §3.3 issuer-identity check
RFC 7591 Dynamic Client Registration	JSON POST to `registration_endpoint` (§3.1), `client_id` required in the response (§3.2.1), `error`/`error_description` reported (§3.2.2)
RFC 8707 Resource Indicators	the `resource` parameter on the token request, from the canonical MCP server URI
RFC 6749 §4.4	the `client_credentials` grant
RFC 6749 §4.1	the `authorization_code` grant, with `state` verified on the redirect and `redirect_uri` repeated on the exchange
RFC 7636 PKCE	`S256` only; the challenge is the SHA-256 of the ASCII of the verifier (§4.2), and the verifier accompanies the exchange (§4.5)
RFC 8252 OAuth for Native Apps	loopback redirect on an ephemeral port (§7.3), `127.0.0.1` in preference to `localhost`, public client with no secret (§8.5)

Two details that matter in practice:

The registered auth method is the one used. RFC 7591 makes token_endpoint_auth_method a statement about what the client will do, so registering client_secret_post and then sending HTTP Basic is a contradiction an authorization server may reject. One function picks the method from token_endpoint_auth_methods_supported, and both registration and the token request go through it. If the server registers a different method than the one asked for, its answer wins.
response_types is always stated, never defaulted. RFC 7591 defaults an omitted response_types to ["code"], which declares a client that visits the authorization endpoint. So it is ["code"] for the interactive flow, which does, and [] for client_credentials, which never goes near it — rather than leaving a default to say something the client did not mean.

Client ID Metadata Documents — the option MCP ranks above dynamic registration — are not used here, and the reason is a constraint in the draft rather than an oversight. A metadata document cannot carry a shared secret:

the token_endpoint_auth_method property MUST NOT include client_secret_post, client_secret_basic, client_secret_jwt, or any other method based around a shared symmetric secret […] the client_secret and client_secret_expires_at properties MUST NOT be used

The client_credentials grant is authenticated only by the client credential, so under that restriction the sole way to use it is private_key_jwt with a published jwks_uri — which needs a private key. A tool published to npm cannot ship one; a secret shared with everybody is not a secret.

That restriction is no obstacle to the interactive flow, which is a public client with no secret by design — a metadata document would suit it. What is missing is somewhere to publish one: the document has to be fetchable at the client_id URL, and a package that runs on your machine has no such URL to offer. Dynamic registration needs nothing hosted, so that is what both flows use. You can pass a URL-form client_id and it is sent as given, in which case the document is yours to host.

RFC 8414 §3 requires the metadata path to use https. Loopback targets are treated as fixtures; any other non-https endpoint gets a warning, because the token being negotiated would cross the network in clear text.

The token is never stored. It is obtained, held in memory, and passed to the test processes; nothing in the package writes to the filesystem, and no token, secret or client secret is ever printed. Each run negotiates its own.

Signing in through a browser

Plenty of authorization servers issue no machine grant at all. Asked for client_credentials they answer with their grant list and nothing else, and for a long time that was where this tool stopped — which meant the servers most likely to need conformance testing were the ones it could not reach.

So it does what an MCP client does: authorization_code with PKCE and a loopback redirect, per RFC 8252. A short-lived HTTP server binds an ephemeral port on 127.0.0.1, that address becomes the redirect URI, your browser opens, and the authorization code arrives on the one request that comes back. The server is closed on every path out — including a failure three steps before it would have been used.

# nothing but the URL. The grant is chosen from what the server offers
npx @hasmcp/mcp-spec-test@latest -u https://mcp.example.com/mcp

endpoint returned 401; discovering OAuth configuration
  found protected-resource metadata at https://mcp.example.com/.well-known/oauth-protected-resource/mcp
  found authorization server metadata at https://mcp.example.com/.well-known/oauth-authorization-server
  using the authorization_code grant with a loopback redirect and PKCE
  no client credentials given; registering a client with https://mcp.example.com/oauth2/register for redirect http://127.0.0.1:56029/callback
  registered client_id dcr-1 using none
  waiting for you to authorize this run in a browser. If one did not open, visit:
    https://mcp.example.com/oauth2/authorize?response_type=code&client_id=…
  authorization code received; exchanging it for a token
  obtained an access token for resource https://mcp.example.com/mcp

Which grant runs, and who decides. client_credentials is preferred wherever it is advertised, because it needs nobody present and a CI run that could have got a token unattended should not stop to ask for one. The browser flow is what happens when there is no machine grant — but only where there is somebody to notice it:


a terminal on stderr, `CI` unset	the browser flow is available
output piped, or `CI` set	it is not; the run fails with a reason instead of waiting for a redirect that can never arrive
`--interactive`	the browser flow, even where `client_credentials` exists — "check that the sign-in works" is a fair thing to ask of a conformance tool
`--no-interactive`	never, whatever the terminal says

Neither heuristic is always right, which is why both switches exist: a developer piping output to a file still has a browser, and a self-hosted runner with CI set might be driven by hand.

Why PKCE, specifically. The loopback redirect is the weak point of the native-app flow: any process on the machine can race for the port, and the code arrives over plain http. PKCE (RFC 7636) binds the code to a secret this process generated and never transmitted, so an intercepted code is inert. Two consequences worth stating:

Only S256. A server advertising just plain is refused, not accommodated. plain puts the verifier itself in the authorization request — through the browser, the address bar, and the server's logs — leaving the flow no safer than one with no PKCE while looking like it had some.
No code_challenge_methods_supported is not a blocker. RFC 8414 makes the field optional and OAuth 2.1 requires S256 support of every authorization server, so S256 is attempted anyway — with a note, because a server omitting it is worth knowing about:
```
note: the authorization server advertises no code_challenge_methods_supported;
      attempting S256, which OAuth 2.1 requires of it
```

A public client, registered as one. The interactive flow has no secret to keep — this package is on npm, and a secret shared with everybody is not a secret — so it registers with token_endpoint_auth_method: none where the server offers it, grant_types: ["authorization_code"], response_types: ["code"], and the exact redirect_uris it will come back to. Registering the client_credentials shape and then visiting the authorization endpoint is the contradiction that produces invalid_client several steps later, where the cause is no longer visible.

The order matters too: the receiver binds before the client registers, so the port in redirect_uris is the port that is actually listening. RFC 8252 §7.3 requires an authorization server to allow a varying port on a loopback redirect; --redirect-port pins one for the servers that do not, and --redirect-host localhost covers the ones that only accept that spelling.

What the browser sees, and what it keeps. The callback page says one line and is served no-store with referrer-policy: no-referrer. It never contains the authorization code — a page that echoed it would put it in the browser's history and its cache. A request to any other path is a 404, so the /favicon.ico a browser asks for unprompted cannot be mistaken for the redirect and abandon a flow still in progress.

What can go wrong, and what it says. A redirect carrying the wrong state is refused as not belonging to this request, which is the CSRF the parameter exists to prevent. A closed tab becomes no redirect to http://127.0.0.1:56029/callback within 180s rather than a hang. An error=access_denied in the redirect is reported as the authorization server's own answer. None of these is fatal to the run: cases report themselves not verified, as with any refused credential.

One inherent limit: the redirect goes to your browser's loopback, so the browser has to be on the same machine as the runner. Over SSH, use --no-browser and forward the port, or pass a token you already hold.

A refused credential is treated like a rate limit, not like a finding: cases report themselves not verified with the endpoint's own rejection. An unauthenticated run proves nothing about conformance, so it must not read as thirty-six violations.

In CI

- run: npx @hasmcp/mcp-spec-test@1 --tap
  env:
    MCP_URL: ${{ vars.MCP_URL }}
    MCP_TOKEN: ${{ secrets.MCP_TOKEN }}

--tap emits raw TAP for a parser; without it you get the human report, which is usually what you want in a log anyway.

Note the @1 rather than @latest. Every other example here uses @latest, which is right for a command someone types — they should get the current suite. In a pipeline it means a release of this package can change a build nobody touched, and a conformance suite is the worst place for that: a new case would read as the server having regressed. Pinning the major keeps bug fixes flowing while a new requirement lands only when you bump it.

Output formats

By default the report is printed. --output writes it to a file instead:


`--output stdio`	print the report (the default; identical to omitting the flag)
`--output md`	write Markdown, for a pull request or a wiki
`--output html`	write a single self-contained HTML file
`--output json`	write JSON, for something that will read it

npx @hasmcp/mcp-spec-test@latest -u "$URL" --output html
report written to /tmp/mcpspectest-260823071819.html

npx @hasmcp/mcp-spec-test@latest -u "$URL" --output json --output-folder ./reports
report written to /home/you/reports/mcpspectest-260823072650.json

Files are named mcpspectest-<YYMMDDHHMMSS>.<ext>, timestamped in UTC to match the generatedAt inside. Nothing is overwritten, so repeated runs leave a history; .gitignore here already covers them, and you may want the same line in yours.

Without --output-folder, a report lands in the OS temp directory — /tmp (or $TMPDIR) on Linux and macOS, the user's Temp folder on Windows — rather than wherever the command happened to be run from, so a one-off run does not litter a project checkout. --output-folder overrides that, resolved relative to where you ran from, and the folder is created if it does not exist. It is checked before the suite runs — a mistyped path should not cost a full conformance run and then throw the results away. If the write fails anyway, the report is printed rather than lost.

Only the one line naming the file goes to the terminal — the report itself does not, so --output json leaves stdout free.

The HTML is deliberately self-contained: no stylesheet, script or font is fetched, because an artifact viewer is often offline or behind a strict CSP, and a report that renders differently depending on the network is not a record of anything.

The JSON carries a schema field (hasmcp.mcp-spec-test.report/1) and the same verdict, counts and per-case detail as the printed report, plus the cases that did not apply to this revision — which the human formats leave out, since they are not results that were withheld.

--output cannot be combined with --tap, which replaces the report wholesale.

One thing to know about credentials

A token in the target URL is redacted from the report — the parameter name is kept, since knowing a token was there explains a lot, but the value is replaced. That applies to the printed report too, not only to files.

A secret passed inside --command is not redacted. It is an arbitrary string, and guessing at secrets inside it would be false confidence rather than safety, so if you spawn a server with a credential on its command line, treat the report as sensitive.

What a skip means

A skip is not a pass. subscriptions/listen needs a transport that really streams, so those cases skip against an endpoint that answers the POST with a plain JSON body. Capability cases skip when the capability is not advertised. HTTP-specific cases skip on stdio, because those requirements do not exist there — not because they were satisfied.

Read the reasons before claiming conformance. The suite prints every one.

Testing a HasMCP server

HasMCP is one target among any others. Its MCP endpoint ignores Authorization, so the credential goes in the URL:

npx @hasmcp/mcp-spec-test@latest \
  -u "https://app.hasmcp.com/mcp/<serverId>?token=<server access token>"

Quote the URL — an unquoted ? and & are shell metacharacters. No -t is needed: the token is already in the URL, so the suite sends no credential of its own.

The endpoint also accepts the token in an x-hasmcp-key header (-t <token> --auth-header x-hasmcp-key), and the middleware reads that header first, falling back to the query parameter. The query form is the one worth documenting because it needs no client-side plumbing at all, so a stock MCP client can be pointed at a HasMCP endpoint and simply work. It is a real trade rather than a free win: a bearer token in a URL lands in access logs, proxy logs, browser history and Referer headers, so prefer the header where you control the client.

Two other things that will bite:

The default rate limit (HASMCP_APP_MCP_RATELIMIT_MAX_PER_IP, 60 per 60s) is lower than a full pass, and surfaces as an opaque transport error rather than anything mentioning rate limits. Raise it on the instance under test.
The management API uses Authorization while the MCP endpoint does not. Two different schemes on the same deployment.

Development

npm install
npm test -- -u http://localhost:8080/mcp/<serverId> -t <token>
node --test "tests/**/*.test.mjs"     # raw node:test output, no report

The suite is plain node:test; bin/mcp-spec-test.mjs only turns arguments into the environment the tests read and runs them under the reporter in lib/reporter.mjs. Node 20 or newer.

file	role
`lib/env.mjs`	target and credential resolution
`lib/transport.mjs`	Streamable HTTP and stdio behind one interface
`lib/rpc.mjs`	a hand-written client for the revision under test
`lib/schema.mjs`	assertions derived from the published schema
`lib/probe.mjs`	capability discovery, and the guards tests skip on
`lib/session.mjs`	the `initialize` handshake, for revisions that have one — opened once, and never re-entered: `initialize` is sent outside any session, because a conformant server refuses a second handshake that reuses a session id
`lib/level.mjs`	the MUST/SHOULD distinction
`lib/oauth.mjs`	OAuth discovery, dynamic registration, both grants
`lib/loopback.mjs`	PKCE, the loopback redirect receiver, opening a browser
`lib/reporter.mjs`	the conformance report
`spec/<revision>/schema.json`	the vendored published schemas; the newest is the revision under test
`tests/*.test.mjs`	the conformance cases — what runs against your server
`tests/unit/*.test.mjs`	this package's own tests, for the parts that cannot be checked by eye

The two kinds of test are kept apart deliberately, and the runner's glob is non-recursive so they stay apart. tests/unit/ verifies this tool — that the PKCE challenge matches the RFC's test vector, that the loopback receiver refuses a redirect with the wrong state, that a mock authorization server rejecting a bad verifier is noticed — and none of that belongs in a report answering "is my server conformant". They are excluded from the published package for the same reason. Run them with npm run test:unit.

Adding a revision

Drop the published schema.json at spec/<date>/schema.json. That is the whole change: the window is derived from the directory, so a newer date becomes the revision under test automatically and the oldest falls out of the window. Keeping older schemas on disk is harmless — only the newest REVISION_WINDOW (2) are supported, and lib/env.mjs is where that number lives if it ever needs to change. Cases that assert schema-required fields follow the new schema without being edited; cases that name a method use the schema's own const, so a renamed method surfaces as a missing definition rather than a silent pass.

Both revisions in the window get a full pass, including the handshake-based one — which cases run follows from the schema's feature set rather than from the date. A revision whose schema is not vendored is refused outright rather than half-tested:

$ MCP_SPEC_VERSION=2024-11-05 npx @hasmcp/mcp-spec-test@latest -u "$URL"
MCP_SPEC_VERSION=2024-11-05 is not supported: it is not vendored.
Supported: 2026-07-28, 2025-11-25.

Telemetry

When a run finishes, the suite sends an anonymous count of how it went: the spec revision tested, the transport, and which case numbers passed, failed, or could not be verified. It tells us which revisions people test against and which cases fail most often in the wild.

Nothing identifying is sent. Not the target URL or command, not tool or resource names, not tokens, not error messages. The server's announced name is sent only as a salted 64-bit hash, computed locally — the name itself never leaves your machine — so repeat runs can be recognised as the same server without us knowing which server it is.

To turn it off:

npx @hasmcp/mcp-spec-test@latest -u "$URL" --disable-telemetry=1
# or
MCP_DISABLE_TELEMETRY=1 npx @hasmcp/mcp-spec-test@latest -u "$URL"

The call is fire-and-forget with a two-second timeout, and every failure is silent: it cannot change a verdict, an exit code, or anything printed. No test traffic is added either — the name is read from the single pre-flight request the suite already makes to see which revisions the server offers.

Credits

Built by HasMCP.

License

Apache License 2.0 — see LICENSE.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
bin		bin
lib		lib
scripts		scripts
spec		spec
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json

Latest commit

History

Folders and files

Repository files navigation