Skip to content

docs: sync reference docs with the code - #114

Merged
Masber merged 6 commits into
mainfrom
chore/docs-sync-and-drift-gate
Aug 26, 2026
Merged

Masber merged 6 commits into
mainfrom
chore/docs-sync-and-drift-gate

Conversation

@rjanalik

@rjanalik rjanalik commented Aug 21, 2026 •

Copy link
Copy Markdown
Collaborator

Why

CLAUDE.md had uncommitted edits, so I reviewed it and the docs it points at for accuracy. The drift was worse than expected, and it had one root cause: generated artifacts are CI-gated, prose is not.

The clearest case: f9160d26 removed POST /sat-file/validate — route, handler, service function, and backend trait method — and touched zero .md files. The endpoint stayed documented as live in five documents. GET /summary survived two renames the same way.

This PR fixes the drift. It no longer tries to close the gap that caused it — see the note at the bottom.

The audience is deliberately agents first: CLAUDE.md and ARCHITECTURE.md are what Claude reads before touching this repo, and a confidently wrong sentence there costs more than a missing one. Human-facing docs were brought along where they stated the same wrong fact.

What's here (6 commits, reviewable in order)

Commit What
9b39eb9d docs: sync CLAUDE/ARCHITECTURE/README/GUIDE/SECURITY with current code
086aaa51 docs(cli,shared): name the real cli.toml key in group-default docstrings
97c2ce6d ci: correct the doctest comment on the doctest step
bddea2d8 docs: purge the removed POST /sat-file/validate and GET /summary
fe2573e5 docs: state the real scope of the request-timeout layer
54a731b0 docs: finish killing the "manta-shared holds the backend dispatcher" myth

Start with bddea2d8 — it's the largest and touches five files.

Highlights

  • CLAUDE.md documented unmerged work as current fact. The MANTA_* error-code catalog, ERRORS.md, and the ErrorResponse::new chokepoint are on FEAT: custom error handling with stable MANTA_* error codes #110, not main. Moved to a "Work in flight" table where each row carries a one-command check proving it isn't merged.
  • ARCHITECTURE.md had seven factual errors, several describing layers that were deliberately removed — so the doc sent readers off to rebuild something torn out on purpose. The worst: infra_backend/ described as a wrapper layer whose per-domain wrappers were explicitly deleted to cut an abstraction. Its "adding a command" recipe also pointed at the wrong file for a new dispatch arm (backend_dispatcher/mod.rs holds only the macro and the mod lines; the impls are in sibling files).
  • Three rustdoc comments named a cli.toml key that doesn't exist. main.rs reads hsm_group; an operator following the docs got no error, just a silently-ignored key. Two are in published manta-shared, so they ship to docs.rs. See the discussion in the comments — this is a correction toward the key the code actually reads, not a new use of the HSM name.
  • The removed SAT pre-flight left an undocumented consequence: an apply can now fail partway through with earlier elements already applied. That's stated now.
  • README.md and manta-server/Cargo.toml both still called manta-shared the home of the backend dispatcher — the same error fixed in ARCHITECTURE.md's crate tree, left standing in the two places a reader lands first.
  • CLAUDE.md overstated the CLI's independence from the backend crates. No CLI code calls a backend trait, but cargo tree -p manta-cli shows manta-cli → manta-shared → manta-backend-dispatcher at depth 2 — it is still compiled, so a bad sibling beta can still break the CLI build. csm-rs / ochami-rs really are absent.

Verification

Re-run against the current head (54a731b0):

  • cargo test --workspace — 684 passed, 0 failed, 3 ignored (11 test binaries incl. doctests)
  • cargo fmt -p manta-shared -p manta-cli -p manta-server -- --check — clean
  • cargo doc --workspace --no-deps with the CI RUSTDOCFLAGS — exit 0
  • anyhow boundary grep — clean
  • openapi.json — byte-identical to --emit-openapi output
  • man/ + autocomplete_shell_scripts/ — byte-identical after MANTA_REGENERATE_DOCS=1 cargo build -p manta-cli

cargo clippy reports one pre-existing error in sat_file/exec.rs, in untouched code — it's the newer-local-toolchain false positive from CLIPPY-NOTE.md, not flagged by the pinned 1.92.0 that CI runs. (This branch's CLAUDE.md now documents that trap.)

Explicitly out of scope

The prose-docs CI gate was dropped. An earlier revision added scripts/check-docs.py plus an allowlist to keep this drift from recurring. Removed on review feedback — the machinery cost more in reader confusion than it bought. The docs: commits that fix the drift stay; the gate does not. It was run once locally before deletion as a review aid, and the docs pass clean (8 documents, every endpoint / anchor / repo-path reference resolves).

API.md and CLI.md are unaudited below endpoint level — request/response shapes, status codes, query params, field tables. A file that had two entirely dead sections likely has subtler drift. DOCS-1, DOCS-2, DOCS-4, DOCS-5, DOCS-6 in the internal review are all still open; that's a content audit for another pass.

GET /runtime-configuration is served but undocumented in API.md. Found while reviewing, left open — it's a docs gap, not a false statement, and filling it belongs with the content audit above.

Dropping the HSM name (per review feedback) is a code change — rename the cli.toml key with a back-compat alias, rename config set|unset hsm, then follow with the docstrings. Deliberately not ridden along on a docs-sync PR.

BUG-1 (no request timeout on /v2/auth/*) has its documentation half corrected in fe2573e5 — ARCHITECTURE.md claimed the layer covered "every API route" while its own diagram showed otherwise. The code gap is untouched and that finding stays open. Making the prose honest makes the gap more visible, not less.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TwtnVriwmTHHRZD8eYi2i6

rjanalik and others added 4 commits August 21, 2026 10:45
The reference docs had drifted from the code in ways that cost time
rather than merely being untidy -- several described layers that were
deliberately removed, so they sent a reader off to rebuild something
that had been torn out on purpose.

CLAUDE.md:
- The "Error codes" section documented the MANTA_* catalog, ERRORS.md,
  and the ErrorResponse::new chokepoint as current fact. None of it is
  on main -- it lives on feature/custom-error-codes (PR #110, draft).
  Moved to a new "Work in flight" table, together with manta-cache
  (PR #111), each row carrying a one-command check that proves it is
  not merged yet.
- Sibling pins said mbd beta.13 / csm-rs beta.19 / ochami-rs beta.13;
  the manifest says beta.15 / beta.23 / beta.17. Rather than re-pin
  numbers that go stale, point at [workspace.dependencies] as the
  source of truth and say so explicitly.
- "doctests -- NOT run by plain cargo test" is false. `cargo test
  -p manta-shared` prints "Doc-tests manta_shared" and runs 8 of them.
- Added the toolchain caveat: rust-toolchain.toml's 1.92.0 pin only
  applies when cargo resolves through rustup's shim, so a Homebrew
  cargo silently yields a newer clippy than CI.
- Added that openapi.json is a build input, not documentation.

ARCHITECTURE.md -- seven factual corrections:
- manta-shared no longer holds the backend dispatcher (this
  contradicted line 29 of the same file).
- The CLI has no handlers/ directory; dispatch/process.rs routes.
- http_client/ has no query.rs, no QueryBuilder, and no per-resource
  sub-modules -- endpoints are reached via the progenitor client.
- infra_backend/ is not a wrapper layer. The per-domain wrappers were
  deliberately deleted; service code calls infra.backend.<method>(...)
  directly. The old text invited reintroducing the removed layer.
- AppContext is 13 fields, not 10 (read_only, token, session missing).
- Entry point is dispatch::process::process_cli.
- service/ and handlers/ module lists were missing 5 and 3 modules.

...and four things that were absent entirely: a "Generated code"
section (with the regeneration step added to the add-a-command
recipe), the CLI read-only gate, jwt_ops moving to manta-shared, and
the fact that JWT signatures are never verified locally -- which
imposes a real rule on future code and was only in a rustdoc comment.

README.md: applied the same SOCKS fix already made on
manta-cache-stage-1-2 (1265d19) -- one paragraph said the server has
no per-site SOCKS knob and the next said server.toml holds per-site
SOCKS proxies. Also added the cli.toml keys the example omitted
(hsm_group, read_only, poll knobs) so it matches examples/cli.toml.

GUIDE.md: the `manta config show` sample printed a "SOCKS5 proxy" row.
push_local_section emits exactly four rows and never that one --
fabricated output in a ```text block reads as literally observed.

SECURITY.md: the no-local-signature-verification note pointed at
server::common::jwt_ops "documented inline at the top of that module",
but that module is now an 8-line re-export. Repointed at the canonical
manta_shared::common::jwt_ops where the caveat actually lives.

Cargo.toml (comment only): dropped the claim that building manta
requires the sibling checkouts because image_pipeline.rs needs
unreleased SatTrait additions. Verified false on all counts -- no
.cargo/config.toml, no adjacent checkouts, image_pipeline.rs
references neither SatTrait nor manta_backend_dispatcher, and
`cargo check --workspace` is clean from crates.io.

Verified: cargo test --workspace (684 passed), fmt, rustdoc with the
CI RUSTDOCFLAGS, anyhow boundary, openapi.json sync, and man/shell
completion sync all clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ldf1GrsA11XsLDJ188arZ7
Three docstrings for the "operator default group" field named a
cli.toml key that does not exist. main.rs reads
`settings.get_string("hsm_group")`, so an operator following any of
these and writing the documented key into cli.toml gets no error and
no warning -- get_string(...).ok() swallows the miss and the default
is silently absent. A wrong key name fails more quietly than a wrong
type does.

  app_context.rs:59         `parent_group`       -> `hsm_group`
  kernel_parameters.rs:117  `parent_group_group` -> `hsm_group`
  group.rs:70               `group_name`         -> `hsm_group`

The correct value is not a guess: six sibling *Params structs
document the same field and already say `hsm_group`
(configuration.rs, boot_parameters.rs, hardware.rs, cluster.rs,
template.rs). All three of these trace back to the parent_hsm_group
removal in MIGRATING.md 5.7, where the key was deleted and these
docstrings were half-renamed to plausible-looking names instead of to
hsm_group. The group.rs one was the most misleading: it named
`group_name`, which is the sibling field on the same struct, so it
read as if the config key and the wire field were the same thing.

Two of the three are in manta-shared, which is published, so these
ship to docs.rs and to the GitHub Pages site that docs.yml builds
from target/doc.

No OpenAPI regeneration needed: both edited structs derive only Debug,
not ToSchema, and neither string appears in openapi.json. Confirmed by
regenerating the spec and diffing -- byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ldf1GrsA11XsLDJ188arZ7
The comment claimed "Plain `cargo test --workspace` does NOT run
these." It does. `cargo test -p manta-shared` prints "Doc-tests
manta_shared" and runs 8 of them, and a full `cargo test --workspace`
prints Doc-tests sections for both manta_shared and manta_server.

The claim is load-bearing beyond this file: it was copied into
CLAUDE.md, where it told future readers they needed an extra command
they already had.

The step itself is unchanged and still worth keeping -- it reports a
doctest failure as a doctest failure rather than burying it in the
combined run, and it stays correct if the test step ever gains
--all-targets, which does suppress doctests. The comment now says
that instead of asserting something false.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ldf1GrsA11XsLDJ188arZ7
…oints

Found by diffing every `METHOD /path` mentioned in API.md against the
paths in the CI-gated openapi.json. Two endpoints were documented as
live across five files while returning 404.

POST /sat-file/validate was removed in f9160d2 ("feat(sat)!: remove
POST /sat-file/validate endpoint chain") along with the backend's
SatTrait::validate_sat_file. That commit touched zero .md files and
deferred the follow-ups, so the endpoint's ghost survived in:

  API.md         full endpoint section, apply-flow bullet, sequence
                 diagram round-trip, 2 server-config requirement rows
  CLI.md         "Pre-flight server-side validation" callout
  GUIDE.md       2 prose paragraphs + a mermaid flowchart node
  MIGRATING.md   section 5.10, which announces the feature as new
  ARCHITECTURE.md  the apply-flow description

Nothing replaced it, so the docs now say so plainly: validation is
client-side only (dangling image_ref, cycles), nothing checks the file
against live backend state, and an apply can therefore fail partway
through with earlier elements already applied. That last consequence
was not stated anywhere and is the part an operator needs.

MIGRATING.md 5.10 is rewritten rather than deleted -- section 5 is a
chronological within-v2 delta list, and operators who upgraded through
the intervening betas did see the pre-flight behaviour.

GET /summary was renamed twice (summary -> cache -> analysis;
fd5e801, a182763). BackendSummary still exists in
types/api/analysis.rs, so the endpoint lives on as GET
/analysis/images -- which API.md already documents in its own
section. The stale "## Summary" section was a duplicate of it under
the dead path.

Also fixes two pre-existing broken anchor links found while checking
that the deleted sections left no dangling references:

  GUIDE.md     -> MIGRATING.md 5.11 (slug was missing two hyphens
                  that the backticked `--dry-run` contributes)
  MIGRATING.md -> #2-server-setup, a heading that does not exist
                  (it is "2. Site operators (deploying the stack)")

Every internal anchor link across the 13 tracked docs now resolves.

Verified: cargo test --workspace (684 passed), fmt, openapi.json still
byte-identical to the emitted spec.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ldf1GrsA11XsLDJ188arZ7

@Masber Masber left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the impression that this PR would only focus on CLAUDE.md file but I see more changes on source file and documentation. I added some comments to the code to review

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to get rid off HSM words in manta because I think OpenCHAMI may not use it in the future

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to get rid off HSM words in manta because I think OpenCHAMI may not use it in the future

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to get rid off HSM words in manta because I think OpenCHAMI may not use it in the future

rjanalik and others added 2 commits August 21, 2026 16:44
ARCHITECTURE.md said the TimeoutLayer is "applied to every API route".
It is not: build_router applies it to the `api` sub-router only, and
`/v2/auth/*` is a separate sub-router carrying the rate limiter and body
redaction with no timeout. The file already contradicted itself -- the
middleware diagram twelve lines above shows the split correctly.

Consequence worth naming: a hung upstream Keycloak on POST /v2/auth/token
is not bounded by [server].request_timeout_secs.

This is the documentation half of REVIEW_FINDINGS.md BUG-1. The code gap
is deliberately untouched -- that finding stays open. Making the prose
honest makes the gap more visible, not less.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ldf1GrsA11XsLDJ188arZ7
…myth

The sync commit corrected ARCHITECTURE.md's crate tree but left the
identical wrong line in README.md and in manta-server's own Cargo.toml
comment -- the two places a reader most likely lands first. Both now
say what manta-shared actually carries; StaticBackendDispatcher and the
csm-rs/ochami-rs impls are server-only.

Two more accuracy fixes in the same vein:

ARCHITECTURE.md's "adding a command" recipe, step 5, sent the reader to
backend_dispatcher/mod.rs to add a dispatch arm. mod.rs holds only the
shared imports, the dispatch! macro and the `mod` lines; every
`impl <Trait> for StaticBackendDispatcher` block lives in that trait's
sibling file. The same document says so correctly two sections earlier.
Following the recipe meant editing the wrong file.

CLAUDE.md claimed the CLI "no longer links the backend traits at all".
No CLI code calls one, which was the point being made -- but
manta-backend-dispatcher is still compiled for `cargo build -p
manta-cli`, because manta-shared depends on it for the type re-exports
in types::dto. `cargo tree -p manta-cli` shows the edge at depth 2. The
distinction matters when a bad sibling beta breaks the CLI build and
the docs say that cannot happen. csm-rs and ochami-rs really are absent
from that tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TwtnVriwmTHHRZD8eYi2i6
@rjanalik
rjanalik force-pushed the chore/docs-sync-and-drift-gate branch from cb7d268 to 54a731b Compare August 21, 2026 15:02
@rjanalik

Copy link
Copy Markdown
Collaborator Author

Force-pushed. Three changes since @Masber's review: the CI docs gate is gone, the hsm_group question is answered below, and four accuracy bugs found while re-reviewing are fixed.

1. Dropped the prose-docs CI gate (f1ca52eb)

Removed on review feedback — the check added a layer of machinery that is confusing to read for what it buys. scripts/check-docs.py, .github/doc-check-allow.txt, the CI step and the CLAUDE.md bullet are all gone; nothing in the tree references them any more. The docs: commits that fixed the drift stay.

I did run the script once locally as a review aid before deleting it, and the docs pass clean: 8 documents, every endpoint / anchor / repo-path reference resolves. It also surfaced one gap that is out of scope here and left open: GET /runtime-configuration is served by the API but never documented in API.md.

Note

The PR description above is now stale — it still describes the gate as a headline feature. Ignore the "## The new gate" section and the f1ca52eb row.

2. hsm_group — @Masber, this is a correction, not a reintroduction

Full agreement on the direction, but these three docstrings are the wrong place to act on it. All three named a cli.toml key that does not exist:

File Was Is
crates/manta-cli/src/common/app_context.rs:59 parent_group hsm_group
crates/manta-shared/src/types/api/kernel_parameters.rs:117 parent_group_group hsm_group
crates/manta-shared/src/types/api/group.rs:70 group_name hsm_group

hsm_group is what the code actually reads and writes today:

  • crates/manta-cli/src/main.rs:123 — settings.get_string("hsm_group"), the only read of the default group
  • crates/manta-cli/src/dispatch/config/set_hsm.rs:61 — doc["hsm_group"] = value(new_hsm)
  • crates/manta-cli/src/dispatch/config/unset_hsm.rs:25 — doc.remove("hsm_group")
  • README.md lines 33, 154 and 264 already document hsm_group on main — untouched by this PR
  • Five sibling *Params docstrings (configuration.rs, boot_parameters.rs, hardware.rs, cluster.rs, template.rs) already said hsm_group before this PR

Net effect on the HSM name across crates/: 1172 occurrences on main → 1175 on this branch. Three words, all of them corrections. No user-facing surface changed — manta config set hsm and the HSM_GROUP_NAME positional are as they were.

The cost of reverting them is real: get_string(...).ok() swallows a missing key, so an operator who follows the old docstrings and writes parent_group into cli.toml gets no error, no warning, and silently no default group. A wrong key name fails more quietly than a wrong type does.

Dropping HSM properly is a code change — rename the cli.toml key (with a back-compat alias so existing configs keep working), rename config set|unset hsm, then update these docstrings to follow. Happy to open that as a separate PR if you want it; it shouldn't ride along on a docs-sync change, and until it lands the docstring has to name the key that actually works.

3. Four accuracy fixes found on re-review (54a731b0)

Verified the sync commits against the code rather than reading for plausibility — the CLI dispatch/ layout, the 13-field AppContext, the http_client/ file set, infra_backend.rs's two methods, the service/handler module lists, MUTATING_VERBS, the poll defaults, the 3.1→3.0 down-convert in build.rs, the MIGRATING §5.11 anchor slug, and both work-in-flight rows all check out. Four things did not:

  1. README.md:93 still described manta-shared as holding the "backend dispatcher" — the exact myth this PR set out to kill, fixed in ARCHITECTURE.md's crate tree but missed in the identical README line.
  2. crates/manta-server/Cargo.toml:20 — same stale claim, in the comment sitting right where anyone adding a dependency will read it.
  3. ARCHITECTURE.md, "adding a command" step 5 sent the reader to backend_dispatcher/mod.rs to add a dispatch arm. mod.rs holds only the shared imports, the dispatch! macro and the mod lines; every impl <Trait> for StaticBackendDispatcher block lives in that trait's sibling file. The same document says so correctly two sections earlier — following the recipe meant editing the wrong file.
  4. CLAUDE.md claimed the CLI "no longer links the backend traits at all". No CLI code calls one, which was the point being made, but cargo tree -p manta-cli shows manta-cli → manta-shared → manta-backend-dispatcher v1.0.0-beta.15 at depth 2 — it is still compiled, so a bad sibling beta can still break the CLI build. That is the opposite of what someone debugging exactly that failure would conclude. Reworded; csm-rs and ochami-rs really are absent from that tree.

Docs re-verified clean after the edits.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TwtnVriwmTHHRZD8eYi2i6

@rjanalik rjanalik changed the title docs: sync reference docs with the code, and gate them in CI docs: sync reference docs with the code Aug 21, 2026
@Masber
Masber merged commit b17efdf into main Aug 26, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants