docs: professionalise the documentation set and gate it - #8
Merged
Merged
Conversation
Retires six superseded documents, adds the repository-health layer, records five decisions that were in force and unrecorded, reorganises docs/ around a reference section, stamps provenance on every page, and adds a documentation gate so none of this can silently rot again. Retired (never deleted, via retire.py, ledger rows appended): - INSTALL.md and packages/auto_apply/docs/INSTALL.md - two of three disagreeing install guides, both documenting a PyPI package that does not exist. - packages/auto_apply/CONTRIBUTING.md and CODE_OF_CONDUCT.md - moved to the repository root where GitHub's community profile looks for them. - packages/auto_apply/CHANGELOG.md - claimed an 'Initial public release' on 2025-10-20 that never happened. - packages/auto_apply/docs/deployment/enterprise_admin_policy.md - a heading-for-heading duplicate of user_guide/admin_policy.md. Added at the repository root: SECURITY.md, SUPPORT.md, GOVERNANCE.md, DISCLAIMER.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, an honest CHANGELOG.md, three issue forms, a pull-request template, CODEOWNERS and dependabot.yml. GitHub community profile: 2 of 8 files present before, 8 of 8 now. Added ADRs for decisions already in force: - 013 static path retirement (supersedes 006). The zero-browser mode was removed on 2026-09-08 because no discovery provider could run without a browser, so the perception adapter was constructed and never reached. The capability is written as deferred, with its seams named and a restoration trigger, not cancelled. - 014 typed UI port. 015 polled UI state. 016 retirement over deletion. - 017 the documentation gate. Added docs/STATUS.md as the single readiness authority - previously asserted in four places that disagreed - and docs/DOCUMENTATION_STANDARDS.md. Added tests/infrastructure/test_docs_gate.py: 12 pins, every one mutation-tested. Front matter, link resolution, retired-module references, install extras, the ADR register, site navigation reachability, citation version parity, conversational residue, and single-source readiness. Repointed test_install_commands_exist.py at the relocated CONTRIBUTING.md. It skips files that do not exist, so the retirement had silently emptied it of the eight 'uv sync --extra' commands the contributor guide documents. Measured before and after: provenance on 3 of 65 pages -> 64 of 64; broken relative links 11 -> 0; docs pages unreachable from the built site 20 of 55 -> 0 of 64; documents describing retired modules as live 8 -> 0; undocumented CI gates 2 of 4 -> 0; ADRs 12 -> 17; one page ended with a chat-transcript sign-off, committed verbatim -> removed. Also: mkdocs no longer publishes the seven retired documents to the public site, and packages/auto_apply/site/ is gitignored - mkdocs build writes 113 generated files there. Suite 1384 -> 1396 passed, 2 skipped. mypy clean on 260 source and 147 test files. ruff F821 clean. mkdocs build --strict passes.
Reviewer's GuideThis PR professionalizes the repository’s documentation and community surface, consolidates status and reference material, preserves superseded content through retirement ledgers, and adds a blocking documentation gate plus MkDocs navigation checks to prevent future drift. Flow diagram for documentation retirement and canonical replacementflowchart LR
Obsolete[Superseded document] --> Retire[retire.py]
Retire --> Ledger[old_retired_files ledger]
Retire --> Archive[Retained with origin stamp]
Obsolete --> Canonical[Canonical replacement]
Canonical --> Nav[MkDocs navigation]
Archive -. excluded from public site .-> Site[Published documentation]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 4 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="CONTRIBUTING.md" line_range="54-61" />
<code_context>
+
+Contributors and CI therefore use **uv**, which is itself pip-installable:
+
+```bash
+pip install uv
+uv sync
+```
+
+`uv sync`, run from the repository root, creates `.venv`, installs the workspace
</code_context>
<issue_to_address>
**issue (bug_risk):** These commands are documented as runnable from the repository root, but the root `pyproject.toml` is a virtual workspace with no `[project]` table or optional extras. `uv sync --extra nlp` therefore does not select the `auto_apply` workspace member's extra; contributors must use a package-qualified command such as `uv sync --package auto_apply --extra nlp`, so the documented optional-install path fails or targets no project.
**Triggers:** When a contributor follows the optional-feature instructions from the repository root.
**Suggested fix:** Qualify the commands with `--package auto_apply`, or document that they must be run from the package project with the correct uv invocation.
```suggestion
uv sync --package auto_apply --extra nlp # SpaCy — smarter vetting
uv sync --package auto_apply --extra browser # Playwright
uv sync --package auto_apply --extra ai # GPT4All — local LLM answers
uv sync --package auto_apply --extra semantic # sentence-transformers — semantic role alignment
uv sync --package auto_apply --extra captcha # offline audio transcription
uv sync --package auto_apply --extra stealth # undetected-chromedriver
uv sync --package auto_apply --extra research # pyarrow + pandas — Parquet export
uv sync --package auto_apply --extra all # all of the above
```
</issue_to_address>
### Comment 2
<location path="packages/auto_apply/tests/infrastructure/test_docs_gate.py" line_range="210-224" />
<code_context>
+# --------------------------------------------------------------------------
+
+def test_every_relative_link_resolves() -> None:
+ """11 links were broken before this pin, including the Architecture Bible
+ linked from both READMEs at a path it has never occupied.
+ """
+ broken: list[str] = []
+
+ for path in _all_markdown():
+ rel = path.relative_to(_REPO_ROOT).as_posix()
+ for target in _LINK.findall(_read(path)):
+ if target.startswith(("http://", "https://", "mailto:", "#", "tel:")):
+ continue
+ bare = target.split("#", 1)[0].strip()
+ if not bare:
+ continue
+ resolved = (path.parent / bare).resolve()
+ if not resolved.exists():
+ broken.append(f"{rel} -> {target}")
+
</code_context>
<issue_to_address>
**issue (testing):** The link gate strips every URL fragment before checking existence, so a link such as `page.md#missing-heading` passes whenever `page.md` exists even though the rendered documentation contains a broken in-page link.
**Triggers:** When a Markdown link has a nonexistent heading anchor.
**Suggested fix:** Resolve and validate the fragment against the target Markdown heading or generated anchor IDs instead of discarding it.
</issue_to_address>
### Comment 3
<location path="packages/auto_apply/tests/infrastructure/test_docs_gate.py" line_range="254-265" />
<code_context>
+def test_no_document_references_a_retired_module() -> None:
</code_context>
<issue_to_address>
**issue (testing):** The retired-module check skips an entire document if it contains any occurrence of `retired`, `retirement`, `superseded`, or `old_retired_files`, regardless of whether the retired module reference is itself described as retired. A document can therefore mention a retired module as live and evade the gate by adding an unrelated retirement sentence.
**Triggers:** When a document contains both an unrelated retirement term and a reference to a retired module.
**Suggested fix:** Check the context around each retired filename, or require the specific filename to be accompanied by an explicit retired/superseded statement.
```suggestion
offenders: list[str] = []
for path in _all_markdown():
text = _read(path)
lowered = text.lower()
rel = path.relative_to(_REPO_ROOT).as_posix()
for name in sorted(retired_names):
escaped_name = re.escape(name.lower())
acknowledged = re.search(
rf"(?:\b(?:retired|superseded)\b[^.!?]*{escaped_name}"
rf"|{escaped_name}[^.!?]*\b(?:retired|superseded)\b)",
lowered,
)
if name.lower() in lowered and not acknowledged:
offenders.append(f"{rel} names '{name}' without saying it is retired")
```
</issue_to_address>
### Comment 4
<location path="packages/auto_apply/docs/DOCUMENTATION_STANDARDS.md" line_range="47-48" />
<code_context>
+
+## 2. Status markers
+
+Every significant claim carries one. **A claim with no marker is a defect in the
+document.**
+
+| Marker | Meaning | Test for using it |
+| --- | --- | --- |
+| `[LIVE]` | On a real execution path, exercised by a live run | Can you name the run? |
+| `[WIRED]` | Connected and tested; not proven in a live run | Does a test exercise it end to end? |
+| `[PARTIAL]` | Works within a stated boundary | Have you stated the boundary? |
+| `[ORPHAN]` | Built; has no consumer | Can you show the one-line proof — no caller, not constructed? |
+| `[PLANNED]` | Decided, not built | Is the decision recorded in an ADR? |
+| `[GOAL]` | Intended direction, no committed design | Are you sure it is not `[PLANNED]`? |
+
+An `[ORPHAN]` claim must carry its **one-line proof of death** next to it. A
</code_context>
<issue_to_address>
**issue (broader_impact):** The standards state that every significant claim must carry a status marker and that an unmarked claim is a defect, but `test_docs_gate.py` has no check for status markers. New or modified documentation can therefore contain unmarked claims while all documentation gates pass, recreating the exact intent-as-achievement drift this change claims to prevent.
**Triggers:** When a contributor adds or changes a significant claim without `[LIVE]`, `[WIRED]`, `[PARTIAL]`, `[ORPHAN]`, `[PLANNED]`, or `[GOAL]`.
**Suggested fix:** Add a gate that checks significant prose or explicitly mark the status-marker rule as review-only rather than claiming it is enforced by the documentation gate.
```suggestion
Every significant claim carries one. **A claim with no marker is a defect in the
document. The status-marker rule is review-only; the documentation gate does not
enforce it.**
```
</issue_to_address>Sourcery assessment
Approval pending. 4 findings to address first.
Blocking findings: CONTRIBUTING.md:61, packages/auto_apply/tests/infrastructure/test_docs_gate.py:224, packages/auto_apply/tests/infrastructure/test_docs_gate.py:265, packages/auto_apply/docs/DOCUMENTATION_STANDARDS.md:48
Comment on lines
+54
to
+61
| uv sync --extra nlp # SpaCy — smarter vetting | ||
| uv sync --extra browser # Playwright | ||
| uv sync --extra ai # GPT4All — local LLM answers | ||
| uv sync --extra semantic # sentence-transformers — semantic role alignment | ||
| uv sync --extra captcha # offline audio transcription | ||
| uv sync --extra stealth # undetected-chromedriver | ||
| uv sync --extra research # pyarrow + pandas — Parquet export | ||
| uv sync --extra all # all of the above |
There was a problem hiding this comment.
issue (bug_risk): These commands are documented as runnable from the repository root, but the root pyproject.toml is a virtual workspace with no [project] table or optional extras. uv sync --extra nlp therefore does not select the auto_apply workspace member's extra; contributors must use a package-qualified command such as uv sync --package auto_apply --extra nlp, so the documented optional-install path fails or targets no project.
Triggers: When a contributor follows the optional-feature instructions from the repository root.
Suggested fix: Qualify the commands with --package auto_apply, or document that they must be run from the package project with the correct uv invocation.
Suggested change
| uv sync --extra nlp # SpaCy — smarter vetting | |
| uv sync --extra browser # Playwright | |
| uv sync --extra ai # GPT4All — local LLM answers | |
| uv sync --extra semantic # sentence-transformers — semantic role alignment | |
| uv sync --extra captcha # offline audio transcription | |
| uv sync --extra stealth # undetected-chromedriver | |
| uv sync --extra research # pyarrow + pandas — Parquet export | |
| uv sync --extra all # all of the above | |
| uv sync --package auto_apply --extra nlp # SpaCy — smarter vetting | |
| uv sync --package auto_apply --extra browser # Playwright | |
| uv sync --package auto_apply --extra ai # GPT4All — local LLM answers | |
| uv sync --package auto_apply --extra semantic # sentence-transformers — semantic role alignment | |
| uv sync --package auto_apply --extra captcha # offline audio transcription | |
| uv sync --package auto_apply --extra stealth # undetected-chromedriver | |
| uv sync --package auto_apply --extra research # pyarrow + pandas — Parquet export | |
| uv sync --package auto_apply --extra all # all of the above |
Comment on lines
+210
to
+224
| """11 links were broken before this pin, including the Architecture Bible | ||
| linked from both READMEs at a path it has never occupied. | ||
| """ | ||
| broken: list[str] = [] | ||
|
|
||
| for path in _all_markdown(): | ||
| rel = path.relative_to(_REPO_ROOT).as_posix() | ||
| for target in _LINK.findall(_read(path)): | ||
| if target.startswith(("http://", "https://", "mailto:", "#", "tel:")): | ||
| continue | ||
| bare = target.split("#", 1)[0].strip() | ||
| if not bare: | ||
| continue | ||
| resolved = (path.parent / bare).resolve() | ||
| if not resolved.exists(): |
There was a problem hiding this comment.
issue (testing): The link gate strips every URL fragment before checking existence, so a link such as page.md#missing-heading passes whenever page.md exists even though the rendered documentation contains a broken in-page link.
Triggers: When a Markdown link has a nonexistent heading anchor.
Suggested fix: Resolve and validate the fragment against the target Markdown heading or generated anchor IDs instead of discarding it.
Comment on lines
+254
to
+265
| acknowledging = ("retired", "retirement", "superseded", "old_retired_files") | ||
| offenders: list[str] = [] | ||
|
|
||
| for path in _all_markdown(): | ||
| text = _read(path) | ||
| lowered = text.lower() | ||
| if any(word in lowered for word in acknowledging): | ||
| continue | ||
| rel = path.relative_to(_REPO_ROOT).as_posix() | ||
| for name in sorted(retired_names): | ||
| if name in text: | ||
| offenders.append(f"{rel} names '{name}' without saying it is retired") |
There was a problem hiding this comment.
issue (testing): The retired-module check skips an entire document if it contains any occurrence of retired, retirement, superseded, or old_retired_files, regardless of whether the retired module reference is itself described as retired. A document can therefore mention a retired module as live and evade the gate by adding an unrelated retirement sentence.
Triggers: When a document contains both an unrelated retirement term and a reference to a retired module.
Suggested fix: Check the context around each retired filename, or require the specific filename to be accompanied by an explicit retired/superseded statement.
Suggested change
| acknowledging = ("retired", "retirement", "superseded", "old_retired_files") | |
| offenders: list[str] = [] | |
| for path in _all_markdown(): | |
| text = _read(path) | |
| lowered = text.lower() | |
| if any(word in lowered for word in acknowledging): | |
| continue | |
| rel = path.relative_to(_REPO_ROOT).as_posix() | |
| for name in sorted(retired_names): | |
| if name in text: | |
| offenders.append(f"{rel} names '{name}' without saying it is retired") | |
| offenders: list[str] = [] | |
| for path in _all_markdown(): | |
| text = _read(path) | |
| lowered = text.lower() | |
| rel = path.relative_to(_REPO_ROOT).as_posix() | |
| for name in sorted(retired_names): | |
| escaped_name = re.escape(name.lower()) | |
| acknowledged = re.search( | |
| rf"(?:\b(?:retired|superseded)\b[^.!?]*{escaped_name}" | |
| rf"|{escaped_name}[^.!?]*\b(?:retired|superseded)\b)", | |
| lowered, | |
| ) | |
| if name.lower() in lowered and not acknowledged: | |
| offenders.append(f"{rel} names '{name}' without saying it is retired") |
- Retired-module detection was too permissive: the check skipped an entire document if the word 'retired' appeared anywhere in it, so one unrelated sentence was a blanket exemption. The acknowledgement is now the literal 'old_retired_files'. Measured: all six documents that legitimately name a retired module already carry it, so this is strictly tighter with no false positives. A same-line rule was tried first and produced 20+ false positives in ADR-006, ADR-010, ADR-016 and the Architecture Bible. - CONTRIBUTING.md claimed six documentation rules were enforced by the gate. Two of them - status markers, and correcting documents in the same change - are not and cannot be machine-checked. That is a document describing intent as achievement, which is the defect this gate exists to remove. The list is now split into gate-enforced and review-enforced. - The gate does not validate link fragments, deliberately, because the Architecture Bible's table of contents slugifies differently on GitHub and on the built site. The reason was in mkdocs.yml but not in the standards. Now stated in DOCUMENTATION_STANDARDS.md section 12. Not changed: the review also reported that 'uv sync --extra' needs --package qualification. Measured with uv 0.8.17 against this workspace - 'uv sync --extra nlp' and '--extra all' both exit 0, and '--extra turbo' errors with 'Extra turbo is not defined in any project's optional-dependencies table'. uv resolves extras across workspace members from the root. The documented commands are correct as written. Suite 1396 passed, 2 skipped. mkdocs build --strict passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retires six superseded documents, adds the repository-health layer, records five decisions that were in force and unrecorded, reorganises docs/ around a reference section, stamps provenance on every page, and adds a documentation gate so none of this can silently rot again.
Retired (never deleted, via retire.py, ledger rows appended):
Added at the repository root: SECURITY.md, SUPPORT.md, GOVERNANCE.md, DISCLAIMER.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, an honest CHANGELOG.md, three issue forms, a pull-request template, CODEOWNERS and dependabot.yml. GitHub community profile: 2 of 8 files present before, 8 of 8 now.
Added ADRs for decisions already in force:
Added docs/STATUS.md as the single readiness authority - previously asserted in four places that disagreed - and docs/DOCUMENTATION_STANDARDS.md.
Added tests/infrastructure/test_docs_gate.py: 12 pins, every one mutation-tested. Front matter, link resolution, retired-module references, install extras, the ADR register, site navigation reachability, citation version parity, conversational residue, and single-source readiness.
Repointed test_install_commands_exist.py at the relocated CONTRIBUTING.md. It skips files that do not exist, so the retirement had silently emptied it of the eight 'uv sync --extra' commands the contributor guide documents.
Measured before and after: provenance on 3 of 65 pages -> 64 of 64; broken relative links 11 -> 0; docs pages unreachable from the built site 20 of 55 -> 0 of 64; documents describing retired modules as live 8 -> 0; undocumented CI gates 2 of 4 -> 0; ADRs 12 -> 17; one page ended with a chat-transcript sign-off, committed verbatim -> removed.
Also: mkdocs no longer publishes the seven retired documents to the public site, and packages/auto_apply/site/ is gitignored - mkdocs build writes 113 generated files there.
Suite 1384 -> 1396 passed, 2 skipped. mypy clean on 260 source and 147 test files. ruff F821 clean. mkdocs build --strict passes.
Summary by Sourcery
Professionalise the repository documentation and enforce its consistency, provenance, and published-site reachability through automated checks.
New Features:
Bug Fixes:
Enhancements:
Build:
Documentation:
Tests:
Chores: