Self-Healing Docs 📚🔧

A GitHub Action that monitors your codebase, detects when code changes make documentation inaccurate, and either auto-generates a PR with corrected docs or flags the discrepancies for human review.

Every team's docs are perpetually stale. This Action fixes that.

How it works

PR opened ──> git diff ──> meaningful-change filter ──> code-to-docs graph lookup
                                                              │
                     suspect doc sections <───────────────────┘
                                │
                LLM staleness verification (filters false positives)
                                │
              ┌─────────────────┴──────────────────┐
        high confidence,                     low confidence /
        simple change                        complex change
              │                                    │
     auto-fix PR opened                  PR comment flags sections
              └────────────► summary comment ◄─────┘
        "3 verified accurate, 1 auto-fixed, 2 flagged for review"

1. Index — parses code (AST) into chunks and markdown docs into sections, links them via name-matching heuristics plus optional text-embedding-3-small cosine similarity (cached in ChromaDB), and persists the graph to .selfhealing/docmap.json.
2. Detect — parses the PR's git diff, maps hunks to code chunks, and skips comment-only / whitespace / test-file changes.
3. Verify — for each linked doc section, an LLM (Claude Sonnet or GPT-4o) compares old code, new code, and doc content to confirm real staleness.
4. Repair — a targeted rewrite preserves style and untouched prose; a second LLM pass validates the fix before anything is committed.
5. Report — high-confidence simple fixes become an auto-fix PR; everything else is flagged in a PR comment with links.

Usage

# .github/workflows/doc-check.yml
name: Doc Check
on:
  pull_request:
    paths: ["**/*.py"]
permissions:
  contents: write
  pull-requests: write
jobs:
  doc-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: adeebshafi521/self-healing-docs@v1
        with:
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
          github-token: ${{ secrets.GITHUB_TOKEN }}
          confidence-threshold: "0.8"

Inputs

Input	Default	Description
anthropic-api-key	—	Claude for verification & repair
openai-api-key	—	GPT-4o and/or embeddings
confidence-threshold	0.8	Min confidence to auto-fix
base-branch	main	Branch doc-fix PRs target
fail-on-stale	false	Fail the check when sections are flagged
github-token	required	Opens PRs, posts comments

Outputs

stale-sections, auto-fixed-count, flagged-count, fix-pr-url

Local usage

pip install -r requirements.txt
export PYTHONPATH=src

# Build the code-to-docs index (embeddings used if OPENAI_API_KEY is set)
python -m selfhealing.cli index --repo /path/to/repo

# Check a branch against main (dry run: no PRs/comments, no file edits)
export ANTHROPIC_API_KEY=sk-ant-...
python -m selfhealing.cli check --repo /path/to/repo --base origin/main --dry-run --report report.json

Development

python -m pytest tests -q   # 7 tests, no API keys needed (LLM calls stubbed)

Project layout:

src/selfhealing/
  code_parser.py    # AST → code chunks
  doc_parser.py     # markdown → sections + code refs
  embeddings.py     # OpenAI embeddings + ChromaDB cache
  linker.py         # link graph build/persist (docmap.json)
  diff_parser.py    # git diff → changed chunks, noise filtering
  verifier.py       # LLM staleness verification
  repair.py         # correction generation + validation + apply
  github_action.py  # fix-PR creation, summary comments, outputs
  cli.py            # `index` / `check` commands (Action entrypoint)

Accuracy measurement (Phase 5)

Fork a well-documented project, install the Action, and make deliberate doc-invalidating changes. Track and report in this README:

Metric	Value
True positives	TBD
False positives	TBD
False negatives	TBD
Correction quality (human-rated)	TBD