docs, ci: correct the drifted documentation and make CI reproducible by yesdevnull · Pull Request #42 · yesdevnull/trenchcoat

yesdevnull · 2026-08-17T23:14:12Z

An audit of the documentation, the Claude Code instructions, and CI, with the fixes. Every claim below was checked against the code or the running binary rather than taken from an existing doc.

The documentation was wrong in four places

The README contradicted the matcher on glob semantics. It said header, query and body globs use path.Match semantics where * does not match /, and advised using contains or regex for bodies containing paths. The matcher compiles those patterns to (?s)^...$ with * as .*, so * spans / and newlines and [ is a literal. CLAUDE.md had it right. Anyone following the README was working around a limitation that does not exist.

help.txt is go:embed-ed, so its drift shipped in the binary. --tls-server-name had been missing from trenchcoat --help since the flag was added five months ago, and the proxy section omitted --verbose and --log-format while the examples used --verbose with proxy.

docs/test-coverage-analysis.md had drifted five months. It claimed 184 tests against the 314 that exist, gave 90.5% total against the real 88.7%, omitted internal/httputil and examples/go-tests from its package table, and proposed WithTLS and WithSelfSignedTLS as future work when both had shipped.

goreleaser check was already failing. archives.format and archives.format_overrides.format are deprecated in v2.

The root cause, and the fix

The coat specification lived in both README.md and CLAUDE.md. Two copies of a spec is two chances to be wrong, and they had already diverged on a fact users act on.

README.md is now the specification and CLAUDE.md says so. The coat format, URI and value matching, precedence, validation rules, the programmatic API and proxy capture behaviour are documented once, in the file a user actually reads. CLAUDE.md keeps what a user does not need — repository structure, size limits, matcher internals, the invalid-coat-from-the-programmatic-API path, and how to work on the repo — and drops from 505 lines to 331.

CLAUDE.md gains a "Documentation surfaces" table naming which of the four docs must move for a given kind of change. All four had drifted at least once and nothing had written down that they travel together.

Documented for the first time

Both have shipped for months and appeared in neither the README nor CLAUDE.md:

Response body templating — the available fields, that body_file contents are templated too, and that a parse failure is silent while an execution failure warns.
${VAR} and ${VAR:-default} substitution — that it is textual and runs over the whole file before parsing, that :- treats an empty value as unset, and that an unset variable with no default survives as literal ${VAR} rather than failing the load. Verified against the binary.

Also new to the README: which request headers proxy capture drops and why every one it kept would become a match constraint on replay; the content-negotiation trap that follows from dropping Accept; glob-metacharacter escaping in captured paths; the decoded-path collapse; --tls-server-name and x509negativeserial; strict parsing and the anchor-on-a-real-field idiom; the full validation rule list; and how to wire coatfile.schema.json into an editor, which nothing had ever explained.

CI

Correctness. govulncheck ran as govulncheck ./... || echo "::warning::", so a vulnerable dependency never failed a build — it now fails the job. govulncheck and goimports were installed with @latest inside a workflow that pins every action to a SHA; both are now pinned behind # renovate: annotations that a new customManager reads, so they stay current without going unpinned. The Build job stamped main.version from git describe --tags against a shallow clone that fetches no tags, so every binary claimed a bare commit hash as its version. A concurrency group stops a superseded pull request run instead of finishing a six-way build matrix nothing will read.

New. .golangci.yml, so make lint and the Lint job run the same linters rather than whatever each installed binary calls standard. misspell is set to the British locale the conventions ask for, which caught four Americanised comments. bodyclose is deliberately excluded — its three hits are false positives, since the test helper closes the response from t.Cleanup and the linter cannot see through that.

A Hooks job runs .claude/hooks/test_hooks.py on Python 3.9, the floor the hooks are written against because macOS ships 3.9.6. A GoReleaser Config job runs goreleaser check on every build rather than leaving it to be discovered by a failed release. And .github/workflows/release.yaml runs GoReleaser on a v* tag — the config existed but nothing had ever invoked it, and no tag has ever been cut.

Coverage report

docs/test-coverage-analysis.md is deleted. scripts/coverage-report.sh produces the same information on demand, so there is nothing left to go stale: the per-package table and total by default, --functions or --min N to list what is under-covered, --html for the browsable report. Test output goes to a log so the coverage numbers are not buried; on failure it prints the tail and names the log, because coverage numbers from a failed suite are meaningless. make coverage calls it rather than reimplementing it.

That file's "Remaining Gaps" section held forward-looking items that were not coverage data, so each was checked before the file went. Two were already implemented. One — the proxy having no TLS listener — is real and has been moved to the roadmap. One — enforcing a TLS 1.2 floor on the mock server — turned out to be obsolete and is recorded as declined, with the reasoning below.

Roadmap

Reviewed against the code as it now stands, and two of the five open proposals were dead:

Conditional responses proposed a when: condition on individual responses so one coat could serve both a normal and a retry case. Declined — the motivating example already works, using two coats and body_match: contains, because a body constraint counts towards specificity and so outranks the unconstrained fallthrough. The working recipe is recorded in its place. Stateful mock behaviour is a state machine expressed in YAML, with no debugger, no types and no way to assert on a transition that did not fire; the programmatic API has all three. The original proposal doubted itself on this point.

Sharing response fragments is rescoped rather than dropped: duplication within one YAML file is already solved by anchors and merge keys, so what remains is what anchors cannot reach — across files, and JSON coat files, which have no anchors at all. OpenAPI import keeps its place with smaller scope, since OpenAPI carries example and examples fields and a spec that has them needs no heuristics.

Eight proposals are added, six of them found by reading the implementation during this audit rather than by wanting a feature, so each names the code that makes it a gap. The two at the top: call reporting (Server.CallCount already records match counts but only the Go API can reach it, so no other test suite can assert on calls at all) and response body redaction (--strip-headers shows the threat model is understood, but only for headers — response bodies are captured verbatim, so capturing an OAuth flow writes access_token into a committed file). Then trenchcoat explain, unreachable-coat warnings, capture filters beyond the URI, capture replay verification, and — verified absent — response throttling and SSE/chunked streaming, neither of which the server can do, since nothing uses http.Flusher and delay_ms is applied entirely before the body.

Declined explicitly rather than carried over: enforcing a TLS 1.2 floor on the mock server. Probed directly, the server already answers a TLS 1.0 or 1.1 ClientHello with protocol version not supported — Go's own default server minimum is 1.2. Recording it as work would have been inventing it.

demo.md

Regenerated against a build of this branch, not hand-edited: showboat extract, an edit to the emitted command list, then a rebuild, so every output block is still the real output of the command above it.

Two blocks had stopped matching the tool. The captured coat recorded Accept, User-Agent and a response Content-Length, all three deliberately dropped by dfccb4a — so the demo's centrepiece was showing a shape the proxy no longer writes. The root help block was cobra's generated command listing, from before help.txt became the command's Long.

Regenerating that second one verbatim would have pasted the whole 200-line manual into the top of a walkthrough, making this a fourth copy of the coat specification to drift out of sync with the other three. The command is now trenchcoat --help | head -6 — the COMMANDS block and nothing else — with the surrounding note saying where the rest lives.

Correcting an earlier claim in this description: the trenchcoat proxy --help block was not stale. It already carried --tls-server-name at line 83. That flag was missing from cmd/trenchcoat/help.txt, which is fixed here, and the two were conflated. Two blocks were stale, not three.

Verification

go test -race ./..., golangci-lint run ./..., go vet ./..., gofmt -l, goimports -l, actionlint, goreleaser check, and .claude/hooks/test_hooks.py on Python 3.9 all pass locally, and all 12 CI jobs pass on the branch.

The glob dialect table was checked against doublestar directly — * stops at /, ? does not match /, [abc] is a character class, \[abc\] is a literal. The ${VAR} semantics, the conditional-responses recipe, the TLS floor and the captured-coat shape were all checked against the running binary rather than read off the source.

Left undone, deliberately

No LICENSE. The repo has none, and the README tells people to go install it. Until one exists, nobody technically has the right to use it. Left out at the author's request.

paths-ignore is left in place with a comment explaining the trap. It skips the workflow only for a pull request whose entire diff is docs — for a pull request it is evaluated against the whole base..head diff rather than the latest push. That is safe only while no job is a required status check; main is not currently protected. If protection is enabled, drop the filters rather than trying to make a skipped job report success.

🤖 Generated with Claude Code

https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

@latest

govulncheck ran as `govulncheck ./... || echo "::warning::"`, so a vulnerable dependency never failed a build and the only signal was a warning line in a log nobody reads. It now fails the job. govulncheck and goimports were installed with @latest inside a workflow that pins every action to a SHA, so an upstream release could break an unrelated pull request and no two runs were guaranteed to use the same tool. Both are pinned in an env block, each behind a renovate annotation that a new customManager reads, so they stay current without going unpinned. The Build job stamped main.version from `git describe --tags` against a shallow clone that fetches no tags, so every binary claimed a bare commit hash as its version. It now checks out full history. Also adds what was missing rather than wrong: - .golangci.yml, so `make lint` and the Lint job run the same linters instead of whatever each installed golangci-lint calls standard. misspell is set to the British locale the prose conventions ask for, which caught four Americanised comments; errorlint and unconvert were already clean. bodyclose is deliberately left out -- every HTTP call here is in a test whose helper closes the response from t.Cleanup, which it cannot see through. - A Hooks job running .claude/hooks/test_hooks.py on Python 3.9, the floor the hooks are written against because macOS ships 3.9.6. - A GoReleaser Config job. `goreleaser check` failed on this repo: archives.format and format_overrides.format are deprecated in v2. Fixed, and now checked on every run rather than discovered by a failed release. - A concurrency group, so a superseded pull request run stops instead of finishing a six-way build matrix nothing will read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

docs/test-coverage-analysis.md was a point-in-time report checked into the repository, and it had drifted five months. It claimed 184 tests against the 314 that exist, gave a total of 90.5% against the real 88.7%, omitted internal/httputil and examples/go-tests from its package table entirely, and its "Remaining Gaps" section proposed WithTLS and WithSelfSignedTLS as future work when both had already shipped. A reader trusting it would have drawn the wrong conclusion about every one of those. scripts/coverage-report.sh produces the same information on demand, so there is nothing left to go stale. It prints the per-package table and the total by default, takes --functions or --min N to list what is under-covered, and --html for the browsable report. Test output goes to a log so the coverage numbers are not buried; on failure it prints the tail and names the log, because coverage numbers from a failed suite are meaningless. make coverage now calls the script rather than reimplementing it. .gitignore picks up coverage-test.log, and the editor and OS droppings that were relying on each contributor having a global gitignore. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The coat specification lived in both README.md and CLAUDE.md, and the two had diverged on a fact users act on: the README said header, query and body globs use path.Match semantics where * does not match /, while the matcher compiles them to (?s)^...$ with * as .*, so * spans / and newlines and [ is a literal. CLAUDE.md was right. Anyone following the README's advice to "use contains or regex for bodies containing paths" was working around a limitation that does not exist. README.md is now the specification and CLAUDE.md says so. The coat format, URI and value matching, precedence, validation rules, the programmatic API and proxy capture behaviour are documented once, in the file a user actually reads. CLAUDE.md keeps what a user does not need: repository structure, size limits, matcher internals, the invalid-coat-from-the-programmatic-API path, and how to work on the repo. It drops from 505 lines to 331. The two glob dialects are now a table showing *, **, ? and [ side by side in both, checked against doublestar rather than asserted. Documented in the README for the first time, though both have shipped for months: - Response body templating -- the fields, that body_file contents are templated too, and that a parse failure is silent while an execution failure warns. - ${VAR} and ${VAR:-default} substitution -- that it is textual and runs over the whole file before parsing, that :- treats empty as unset, and that an unset variable with no default is left as literal text rather than failing the load. Verified against the binary, not read off the source. Also added to the README: which request headers proxy capture drops and why every one it kept would become a match constraint on replay; the content negotiation trap that follows from dropping Accept; glob-metacharacter escaping in captured paths; the decoded-path collapse; --tls-server-name and x509negativeserial; strict parsing and the anchor-on-a-real-field idiom; validation rules; and how to wire coatfile.schema.json into an editor, which nothing had ever explained. help.txt is go:embed-ed, so its drift shipped in the binary: --tls-server-name had been missing since the flag was added five months ago, and the proxy section omitted --verbose and --log-format while the examples used --verbose with proxy. All three added, along with the two glob dialects and the captured header rules. coatfile.schema.json gains the delay maxima and the body_match-requires-body dependency that validate.go has always enforced, and stops describing URI globs as just */?. CLAUDE.md gains a "Documentation surfaces" table naming which of the four docs must move for a given kind of change. Every one of them had drifted at least once, and nothing had ever written down that they travel together. ROADMAP.md stops estimating complexity in days. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

Two of the five open proposals were dead. Conditional responses proposed a 'when:' condition on individual responses so one coat could serve both a normal and a retry case -- but the motivating example already works today with two coats and body_match: contains, because a body constraint counts towards specificity and so outranks the unconstrained fallthrough. The working recipe is recorded in its place. Stateful mock behaviour is a state machine expressed in YAML, with no debugger, no types and no way to assert on a transition that did not fire; the programmatic API has all three. The original proposal doubted itself on this point. Sharing response fragments is rescoped rather than dropped. Duplication within one YAML file is already solved by anchors and merge keys, so what is left is the part anchors cannot reach: across files, and JSON coat files, which have no anchors at all. OpenAPI import keeps its place with a smaller scope. The objection was that mapping a schema to a realistic body needs heuristics, but OpenAPI carries example and examples fields, and a spec that has them needs none. Passthrough mode is promoted to the top. It is the incremental-mocking workflow the tool is named for, and both halves already exist. Two items are added, both surfaced by auditing the docs rather than by wanting a feature: - The proxy has no TLS listener. Proxy has Start but no StartTLS, so its client-facing side is plain HTTP only, while connections to the upstream already do TLS. This was recorded only in the coverage report that has since been deleted. - Windows has no test coverage at all. CI is ubuntu-only while the Build job cross-compiles for Windows and the README offers it as supported. One item from that same deleted report is explicitly declined rather than carried over: enforcing a TLS 1.2 floor on the mock server. Probed directly, the server already answers a TLS 1.0 or 1.1 ClientHello with 'protocol version not supported' -- Go's own default server minimum is 1.2. Recording it as work would have been inventing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

… code Every item here came from reading the implementation during the documentation audit rather than from wanting a feature, so each names the code that makes it a gap. Call reporting is the widest asymmetry in the tool. Server.CallCount already records match counts, but only the Go API can reach it -- serve has no flag. A Python or shell suite can point at a trenchcoat mock and get responses, yet can never assert that the code under test called an endpoint twice, which is half the reason to prefer a mock over a static fixture server. --report and --strict are presentation over data the server already keeps. Response body redaction is the one security gap on the list. --strip-headers defaults to Authorization, Cookie and Set-Cookie, so the threat model is understood -- but only for headers. Response bodies are captured verbatim, so capturing an OAuth flow writes access_token and refresh_token into a file people commit. The rest, in rank order: trenchcoat explain, because five precedence tie-breakers currently need a running server and verbose logs to reason about; unreachable-coat warnings, because a coat shadowed by an earlier one is invisible until someone wonders why editing a mock changes nothing, and matchScore already computes what it takes to prove it; capture filters beyond the URI, because shouldCapture takes only the path so "capture only 2xx" cannot be expressed; and capture replay verification, aimed at the footguns the README now documents, where a dropped Accept or a collapsed encoded path yields a capture that looks right and silently serves the wrong thing. Two shipped as proposals rather than as recorded limitations. Nothing in the server uses http.Flusher and the body is written in one call, so SSE and chunked streaming cannot be mocked at all -- a whole category of client, including most current LLM APIs. And delay_ms is applied entirely before the response, so a coat models a slow server but never a slow transfer, leaving client read timeouts untestable. Both need the same flushing work. Passthrough mode keeps its place but drops to third, behind two items that are cheaper, or in the redaction case more urgent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The comment said the filters skip the workflow for a docs-only change, which reads as though a docs-only push to a mixed pull request would skip. It does not: for a pull request, paths-ignore is evaluated against the whole base..head diff rather than the latest push, so a pull request touching any non-docs file runs every time. Only a pull request whose entire diff is docs is skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

Copilot

Pull request overview

This PR audits and corrects drift across Trenchcoat’s user-facing documentation and embedded help, and tightens CI/release automation to be reproducible and fail correctly (notably for vuln scanning and GoReleaser config validation).

Changes:

Consolidates the coat/CLI specification into README.md, updates embedded CLI help, and trims CLAUDE.md to internal invariants/workflow guidance.
Makes CI more reproducible and correctness-oriented (pinned tool versions, govulncheck now fails the job, adds hook tests + GoReleaser config check, cancels superseded PR runs).
Replaces a stale committed coverage report with an on-demand coverage script and updates build tooling to use it.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scripts/coverage-report.sh	Adds an on-demand coverage reporting script used by `make coverage`.
renovate.json	Adds a custom Renovate regex manager for pinned workflow tool versions.
README.md	Updates/expands the user-facing spec (glob dialects, capture behaviour, templating, env substitution, validation rules, releases).
Makefile	Switches `make coverage` to use the new coverage script; cleans up new log artifact.
internal/server/verbose_test.go	British spelling correction in test comments.
internal/proxy/proxy.go	British spelling correction in a comment.
docs/test-coverage-analysis.md	Deletes drift-prone committed coverage analysis doc.
docs/ROADMAP.md	Reworks roadmap structure and adds rationale/declined items.
coatfile.schema.json	Updates schema descriptions and enforces `body_match` ⇒ `body`; adds delay max constraints.
cmd/trenchcoat/help.txt	Updates embedded help text (glob semantics, proxy flags, header capture notes).
cmd/trenchcoat/commands_test.go	British spelling correction in a comment.
CLAUDE.md	Refocuses on internal invariants/workflow; adds doc-surfaces table; removes duplicated spec content.
.goreleaser.yaml	Updates deprecated archive format keys for GoReleaser v2 compatibility.
.golangci.yml	Adds a pinned linter set and config (incl. UK misspell locale).
.gitignore	Ignores coverage log and common editor/OS files.
.github/workflows/release.yaml	Adds a tag-triggered GoReleaser release workflow.
.github/workflows/ci.yaml	Pins tool versions, fixes govulncheck behaviour, adds hook + goreleaser check jobs, adds concurrency, fetches tags for version stamping.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Two ways the report could mislead without saying anything, both from PR review. A non-numeric --min was accepted and handed to awk, which coerces it to 0. Nothing is under 0%, so a typo printed an empty 'Functions below abc%' section -- indistinguishable from a codebase where every function is fully covered. It is now rejected, along with anything outside 0-100. Decimals stay valid because go tool cover reports to one decimal place, so --min 88.5 is a reasonable thing to ask for. The per-package table anchored on a leading 'ok', which drops any package that has no test files. Under -coverprofile such a package does not print the '? pkg [no test files]' line it prints otherwise -- it prints a tab-indented line with no prefix at all, '\tpkg\t\tcoverage: 0.0%% of statements'. Matching on 'coverage:' catches both shapes. A coverage report that silently omits the untested packages is worse than no report. The '---' half of the old pattern went with it: it only ever matched '--- FAIL' lines, and the script exits before printing this section when the suite fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

Two output blocks had stopped matching what the tool does. The captured coat recorded Accept, User-Agent and a response Content-Length. All three were deliberately dropped by dfccb4a, which stopped capturing headers that tie a coat to one client, so the demo's centrepiece -- the thing it points at to show what capture produces -- was showing a shape the proxy no longer writes. The root help block was cobra's generated command listing, from before help.txt became the command's Long description. Regenerating it verbatim would have pasted the whole 200-line manual into the top of a walkthrough, and made this a fourth copy of the coat file specification to drift out of sync with the other three. The command is now `trenchcoat --help | head -6`, which is the COMMANDS block and nothing else, with the surrounding note saying where the rest lives. Regenerated with `showboat extract`, an edit to the emitted command list, and a rebuild -- not by hand -- so every output block is still the real output of the command above it. Run against a build of this branch with TZ=UTC, matching the timestamp convention the document already used. The showboat id changed because the document was rebuilt rather than patched. Nothing references the old one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

docs/demo.md serves five coat files and `cat`s each one, but none of them were in the repository. Regenerating the document meant reconstructing all five from the `cat` blocks inside the document itself -- which worked only because those blocks happen to print their own contents. That is luck, not a design, and it would have failed for any fixture the demo used without displaying. The five are committed under docs/demo-fixtures/, byte-identical to what the document records, so a regeneration reproduces the `cat` output exactly instead of merely closely. scripts/regenerate-demo.sh makes the regeneration reproducible. Doing it by hand means getting four things right, each of which silently records wrong output if missed: build the binary from the working tree rather than trusting whatever `trenchcoat` is on PATH, copy the fixtures into a throwaway directory because the run also writes captured coats, force TZ=UTC to match the timestamp convention the document has always used, and never hand-edit the result. --check reports drift without touching the file. It filters out everything that necessarily differs between runs -- slog timestamps, the document's own generation stamp and id, and the HTTP Date header inside captured coats -- so a clean run says "no behavioural drift" and exits 0 rather than crying wolf on every invocation. Verified both ways: clean against the current document, and exit 1 with a readable diff when a recorded body is altered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

invalid.yaml exists so the demo can show what trenchcoat validate reports, which means 'trenchcoat validate docs/demo-fixtures/' exits non-zero by design. Without a note saying so, the obvious readings are that the file is broken and wants fixing, or that a validation check belongs on this directory. Both would be wrong. The README also records that these files are reproduced verbatim in demo.md, so editing one here without regenerating leaves the document disagreeing with the repository. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The script required showboat already on PATH and failed with 'install it, then re-run', which meant a go install before anyone could regenerate the demo. It now invokes 'uvx showboat@latest', so uv is the only prerequisite and there is nothing to install first. Verified end to end with no showboat binary reachable at all, and the missing-uv path verified to fail with a message naming what to get. Also corrects a comment that had gone stale within its own file: it claimed --check always reports a difference because of the timestamps, which stopped being true once the filter learned to ignore log timestamps, the document id and the HTTP Date header inside captured coats. A clean run now says so and exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

docs/demo.md records the real output of every command it shows, and nothing checked that it still did. It went five months advertising a captured coat shape the proxy had stopped producing. The Demo Drift job runs scripts/regenerate-demo.sh --check, so that can now only happen for as long as it takes CI to run. The demo had to stop sleeping first. Every block started a server and waited a flat second, which is both dead time and a flake: a sleep long enough to be reliable on a loaded runner is wasted on every other run, and one short enough to feel fast eventually loses. Each block now waits for the listener to accept a connection, and the proxy block waits for the capture files to appear rather than guessing at how long an asynchronous write takes -- captures are written atomically, so a file existing means it is complete. The wait uses bash's /dev/tcp rather than curl or nc, for two reasons that both show up as corrupted output otherwise. A curl probe against the *proxy* would be forwarded upstream and captured, adding a coat file to the listing the document is in the middle of recording. And macOS nc -z announces "Connection to localhost port 9100 succeeded!" on stderr, which Showboat would capture into the output block. /dev/tcp is silent, sends nothing, and needs no external binary on either platform. Rebuilt through showboat extract and a rebuild rather than by hand. The output blocks are byte-identical to before: 22 changed lines, all of them inside the ```bash blocks. The readiness probes do not perturb what the demo records. The job is capped at 10 minutes. It finishes in seconds, so the cap exists only so that a listener which never comes up fails rather than holding a runner. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

…time The paths-ignore description was imprecise in the way that had already misled someone: on a pull request the filters are evaluated against the whole base..head diff, not the latest push, so a pull request touching any non-docs file runs every time. ci.yaml was corrected when this was found; CLAUDE.md kept the loose wording. Two traps worth writing down, both hit repeatedly while auditing the docs. A stale ~/go/bin/trenchcoat shadows a build of the working tree and there is no way to tell them apart -- both report 'version dev' -- so behaviour gets checked against the wrong binary unless PATH is pinned. And the README has been wrong where the code was not, which makes running the binary against a scratch coat file the cheaper way to settle a question than reading the source. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

filter() dropped every line starting with `time=` outright, so the whole of each slog line was exempt from the drift check -- roughly twenty recorded lines, including `msg="coats loaded" count=N`, `msg="server started" address=...` and the proxy startup line carrying `write_dir=`, `filter=` and `dedupe=`. A `--verbose` log line is a documented CLI surface, so the job's own claim that a behaviour change cannot leave the demo advertising the old behaviour was false for exactly the output most likely to change. Mask the volatile token instead of deleting the line. The timestamp becomes `<TS>`, the generation stamp `*<GENERATED>*`, the document id `<ID>` and the recorded HTTP Date header `<DATE>`; everything else on those lines now participates in the diff. Proven against a throwaway copy of the tree with `"coats loaded"` renamed: before, `--check` printed "No behavioural drift" and exited 0; after, it prints the five changed log lines and exits 1. A clean tree still reports no drift. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

ci.yaml ignores docs/** and **.md, and paths-ignore skips the whole workflow rather than a job. docs/demo.md and docs/demo-fixtures/ both live under docs/, so a pull request that hand-edited the demo or changed a fixture ran no CI at all -- the drift check was off for precisely those changes, while docs/demo-fixtures/README.md tells contributors --check is the guard. paths-ignore cannot be negated, so move the job to .github/workflows/demo.yaml and trigger it on an allow-list of paths a rerun of the demo depends on: the document, the fixtures, cmd/, internal/, the script itself, go.mod, go.sum and the workflow file. Actions has no cross-workflow `needs`, so Build no longer waits on it and Demo Drift gates as its own check; both the `needs` list and CLAUDE.md say so. Also drop examples/** from paths-ignore. examples/go-tests is a real test package that `go test ./...` compiles and runs, so a pull request touching only it was skipping the Test job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

@latest

… policy ci.yaml pins tool versions so an upstream release cannot break an unrelated pull request, and CLAUDE.md repeats that as a rule -- while the demo check installs Showboat at @latest on every run. Keeping @latest is the decision; leaving it as an unnoticed contradiction was not. Say why it is exempt: Showboat renders a document rather than gating the code, and a contributor running scripts/regenerate-demo.sh locally gets whatever uvx resolves regardless, so pinning it in CI would only make CI disagree with them. Say what the cost is: a Showboat release that changes block ordering or whitespace shows up as drift pointing at code nobody touched, and the answer is to regenerate or to pin. The generation stamp and document id are masked by the script's filter, so a version bump alone does not trip it, and Demo Drift no longer gates Build, so it costs one check rather than the pull request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The table claimed `{base}_{unix_timestamp}.yaml`. The proxy writes `{base}.yaml` -- internal/proxy/proxy.go's `case "skip"` returns the bare base, with a comment explaining that a timestamp made the name unpredictable and let two concurrent captures write the two files skip exists to prevent. Confirmed by running the proxy with --dedupe skip: two requests for /hello produced one file, GET_hello_200.yaml. Say why the name is stable, and that the difference from overwrite is which capture survives: overwrite replaces the earlier one, skip keeps it. The overwrite and append rows were already right. No other file in the repository repeats the claim. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The six /dev/tcp listener probes and the capture-file probe all spun forever. The likeliest drift in the proxy -- a change to the capture naming, which is the historical failure the drift check exists to catch -- means the file the block waits for is never written, so the loop spun until timeout-minutes killed the job ten minutes later with no diff and no diagnostic, having burnt a runner. Each wait is now capped at roughly ten seconds and, on expiry, names what never appeared and exits non-zero: the port for a listener, the contents of captured/ for the capture probe. The probes stay TCP-only -- a curl probe against the proxy is forwarded upstream and captured, which would change the output the document records. Regenerated via `showboat extract`, not hand-edited. Only the ```bash blocks changed; every ```output block is byte-identical bar the timestamps, document id and recorded HTTP Date, and `--check` is clean against the result. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

…k tests Roughly 280 lines of argument parsing, validation and diff logic had no tests, in a repository whose stated convention is test-driven development and which already sets the opposite precedent for its own tooling. A test asserting "a changed log message is reported as drift" would have caught the drift filter deleting every log line it was supposed to compare. scripts/test_scripts.py follows .claude/hooks/test_hooks.py: stdlib only, real tools, throwaway directories, nothing mocked, runnable on Python 3.9. Each script does `cd "$(dirname "$0")/.."`, so a copy dropped into <tmp>/scripts/ treats <tmp> as the repository -- the fixtures are the smallest tree that shape accepts. coverage-report.sh: --min rejecting a non-numeric percentage, one above 100 and a missing value; an unknown argument; and against a real one-package Go module, --min taking a decimal and listing only the uncovered function, and a seeded failing test making the script exit non-zero without printing coverage numbers. regenerate-demo.sh: a throwaway module whose ./cmd/trenchcoat prints one slog line, recorded as a one-block Showboat document. Rerunning it must report no drift, because only the timestamp differs -- and renaming the log message must report drift and exit 1. That second test is the regression test for the filter fix: reinstating the old `grep -v` filter fails it with "0 != 1". The whole suite runs in under five seconds. Wired into the CI job, renamed Claude Code Hooks & Scripts, with uv installed so the Showboat test runs rather than skips. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

… log `|| true` threw the status away, and the only guard left was that the output file was non-empty -- so a Showboat that half-ran surfaced as a large spurious diff, and on the regenerate path a partial document was copied over docs/demo.md. verify.log lived in the work directory the EXIT trap removes, so the diagnostics went with it. Capture the status. Measured against showboat 0.6.1: `verify` exits 1 whenever the rerun differs from the recording, which the log timestamps guarantee it always does, so 0 and 1 are both normal and the verdict still comes from the filtered diff. Anything else is Showboat or uv failing: print the tail, name the log, and exit without touching the document. Keep the log at demo-verify.log, gitignored alongside coverage-test.log, whenever the answer is anything but "no drift" -- including the drift path, where a spurious diff is the symptom of exactly this class of failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The demo's blocks pipe through jq and drive curl throughout. Run without them, the script faithfully recorded "jq: command not found" into docs/demo.md as though trenchcoat had printed it -- a document asserting output no command produced, which is the one failure it exists to prevent. Only uv was guarded. CI has all three; this is the local path. The guard sits before the build, so a PATH with neither reaches it. Covered by a test that runs the script with a PATH holding only bash and uv. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

…thing Two ways coverage-report.sh could tell you less than it knows. The suite runs without -v, so the log is mostly test-emitted slog output. On a real forced failure the "--- FAIL" line survived a 20-line tail by two lines; a noisier one buries it entirely. Grep the failure lines out first and keep the tail as context. The regression test seeds a package whose failing test is followed by forty lines of chatter -- before this, stderr held nothing but chatter. And the coverage table's grep exits 1 when it matches nothing, which under `set -o pipefail` ended the script on the spot: no error, no total, no profile path. Unlikely, since a passing `go test ./...` always emits coverage lines, but it is the exact failure mode the rest of this file argues against. Warn instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

With no -coverpkg, each package is measured only by its own tests, so a helper exercised from elsewhere reads as uncovered. Measured on a clean checkout: `--min 60` reports `0.0% StringPtr internal/coat/types.go:44`, which internal/matcher's tests and the public API's both call; CallCount and ResetCalls in internal/server the same. The script's stated use is deciding where the next test should go, so it was sending you to write a redundant one. -coverpkg=./... was tried and rejected on the evidence. Runtime is fine (13.1s to 14.2s), but it redefines each package's number as its share of the whole module: internal/httputil drops from 100.0% to 0.1%, internal/coat from 97.2% to 15.7%, and "Coverage by package" stops meaning what it says. Trading a correct headline table for a correct function list is not an improvement, so the caveat is written down instead -- in the script header, under the "Functions below N%" heading where the misleading figures appear, and in CLAUDE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The size-limits table merged them and claimed both are marked. Only one is: recordCall reads maxRecordBodySize+1 and appends "...(truncated)" (internal/server/server.go:422-428), while renderTemplate takes a plain io.LimitReader(origBody, maxRecordBodySize) -- no +1, no marker, no way to tell from inside the template that anything was dropped (:513-517). README is right in both its statements; only CLAUDE.md ran them together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

release.yaml triggered on a v* tag with no gate, and ci.yaml does not trigger on tags at all. GoReleaser compiles but does not test, so tagging a broken commit, a commit on a branch other than main, or a main commit whose CI was skipped by the path filters published binaries and checksums nobody had verified. Run the suite with the race detector in release.yaml and make goreleaser need it. The gate has to live in this workflow because Actions has no cross-workflow `needs`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

.golangci.yml and CLAUDE.md both claimed a local run matches CI. The config pins the linter set, not the linter: `make lint` runs whatever is on PATH, and a different golangci-lint carries a different staticcheck vintage -- exactly the skew the comment claimed to have solved. The Makefile only checked that some golangci-lint existed. `make lint` now reads the pinned version out of .github/workflows/ci.yaml, so Renovate's bump there remains the only copy, compares it against `golangci-lint version --short`, and warns on a mismatch before running. Verified both ways: silent when they agree, and `make lint CI_GOLANGCI_LINT_VERSION=9.9.9` prints the warning. Both comments now say the version is pinned in CI only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The matchString ended `\s*"(?<currentValue>[^"]+)"`, so the quotes were mandatory. `GOLANGCI_LINT_VERSION: v2.12.2` is valid YAML and behaves identically, but would not match -- and Renovate reports nothing for a customManager that matches nothing, so the pin would quietly stop being maintained. That is the rot the manager exists to prevent. Make the quotes optional. Verified against the real .github/workflows/ci.yaml: all three pins still match, and the unquoted form now matches too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

…n express `maximum: 60000` on each of delay_ms and delay_jitter_ms is right, but the validator caps their sum (internal/coat/validate.go), and the old wording -- "Combined with delay_jitter_ms, must not exceed 60000" -- read as though the schema enforced it. Verified both ways: a coat with 60000 in each field passes the schema and `trenchcoat validate` rejects it with "combined 'delay_ms' and 'delay_jitter_ms' must not exceed 60000". JSON Schema cannot express the sum cleanly, so the description now says so rather than leaving the gap implied away: editor tooling will green-light a coat the server refuses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The workflow list still described a single CI workflow doing everything and a release that only published. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The minimal-PATH test symlinked shutil.which(tool) without checking it. On a machine without bash or uv on PATH that passes None to symlink_to, which raises a TypeError reading like a bug in the test rather than the missing prerequisite it is -- and the file already has require() for exactly this, used by three other tests. Also annotates run()'s env parameter as dict | None rather than a bare dict defaulted to None; the file carries 'from __future__ import annotations', so PEP 604 is available and it still compiles on 3.9. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

The Demo Drift job failed on its first real run, and the diff was not drift: time=<TS> ... msg="context canceled, shutting down" ... -time=<TS> ... msg="context canceled, shutting down" ... time=<TS> ... msg="proxy stopped" +time=<TS> ... msg="context canceled, shutting down" ... time=<TS> ... msg="server stopped" The proxy capture block starts two servers and killed both with a single `kill %2 %1`. They then shut down concurrently, logging to the same stdout, so whether the two identical "context canceled" lines land before "proxy stopped" or straddle it is a race between two processes. The recorded document holds one interleaving; the runner produced the other. Neither is more correct, and the check would have failed intermittently forever on a document nobody had touched. It passed locally only because this machine happened to reproduce the recorded order -- which is the worst kind of green, and precisely why the earlier five-month drift went unnoticed. Killing the proxy, waiting for it, then killing the upstream makes the ordering a consequence of the script rather than of scheduling. Verified by running --check three times against the rebuilt document: clean each time. Rebuilt through showboat extract and a rebuild, not by hand. The only output change is the shutdown ordering; every other block is byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136bDtWBAdKtufTHcXoSwQK

yesdevnull and others added 5 commits August 18, 2026 08:50

yesdevnull marked this pull request as ready for review August 17, 2026 23:27

Copilot AI lite review requested due to automatic review settings August 17, 2026 23:27

Copilot started reviewing on behalf of yesdevnull August 17, 2026 23:28 View session

Copilot AI reviewed Aug 17, 2026

View reviewed changes

Comment thread scripts/coverage-report.sh

Comment thread scripts/coverage-report.sh Outdated

yesdevnull and others added 20 commits August 18, 2026 09:42

yesdevnull and others added 5 commits August 18, 2026 11:20

yesdevnull merged commit 82754e9 into main Aug 18, 2026
13 checks passed

yesdevnull deleted the docs/audit-fixes branch August 18, 2026 01:39

Navigation Menu

docs, ci: correct the drifted documentation and make CI reproducible - #42

docs, ci: correct the drifted documentation and make CI reproducible#42
yesdevnull merged 31 commits into
mainfrom
docs/audit-fixes

yesdevnull commented Aug 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

yesdevnull commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The documentation was wrong in four places

The root cause, and the fix

Documented for the first time

CI

Coverage report

Roadmap

demo.md

Verification

Left undone, deliberately

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yesdevnull commented Aug 17, 2026 •

edited

Loading