Code Covenant

Status: v0.1.0-dev. Publication gate passed — see docs/PUBLICATION_CRITERIA.md.

Code Covenant is a governance layer for agentic coding. Before an AI agent changes code, the code should have a contract. After the agent changes code, the system should prove whether the contract survived.

What's built (Phases 0–11)

• CPC schema: The canonical Code Purpose Contract data model.
• CPC parser: Extract CPC headers from Python and TypeScript source.
• CPC formatter: Render a CPC dict back into a comment block.
• CPC validator: Enforce required fields, allowed enum values, no placeholders.
• validate_cpc_format tool: Scan a directory and fail if any file's CPC is missing or malformed.
• inventory + report CLIs (Phase 4): list and aggregate CPCs across a project without mutating anything — table, JSON, single-CPC dump, markdown/JSON aggregate report.
• Contractizer: AST-based scanner that drafts CPCs for every function, class, method, async function, and file in a target project. Two modes:
  • inventory — emits cpc_inventory.json, manifest.json, report.md.
  • annotate — also writes an annotated copy of the source tree with CPC comments inserted above each definition (decorator-safe, AST-preserving).
• AST-equivalence + compile preservation: the annotator refuses to write any file whose AST changes, and skips sources that fail to compile.
• Proposal Gate: evaluates a proposed change to a target file against compile, test, and benchmark gates; grades each before-constraint (PASS/PARTIAL/FAIL) against the after-CPC; grades risk; decides merge / review / reject. Every run produces a timestamped proposal folder (before.py, after.py, diff.patch, metrics.json, constraint_grade.json, decision.json, rationale.md) and appends to ledger.jsonl. Auto-rollback after any runtime-gate failure; explicit rollback restores any merged change.
• Single-Target Iterative Runner (Phase 6): loops propose → gate against one file with either a Python callable (ScriptedEngine) or a shell command. Bounded by --max-attempts and --duration; stops on the first merge by default. Rollback uses the existing propose rollback CLI against any merged attempt's before.py.
• Portfolio Runner: timeboxed loop across a contractized project. Scans for file-level CPCs, applies a scope filter, picks targets longest-idle-first (with revisits after every target is attempted), hands each one to a pluggable engine (ShellEngine with output_mode = file or stdout; CodexEngine and ClaudeCodeEngine as real thin wrappers defaulting to stdout; ScriptedEngine for library use), runs the result through the proposal gate, classifies failures by dimension, and emits morning_brief.md, failure_heatmap.json, and prompt_analytics.json when the budget is spent.
• AGENTS.md + Codex skill: Instructions so AI agents cannot complete work without valid CPCs.
• Templates and examples: Good/bad CPC fixtures for agents to learn from.
• scripts/check.sh: One-command gate: validator + pytest + compileall.

End-to-end tutorial

docs/TUTORIAL.md walks through every CLI in order against a real public codebase (click), from "fresh install" to "morning brief and analytics." Includes an ASCII architecture diagram of how the pieces fit together.

What's done

The canonical Phase list (0–11), the publication-gate public demo, productionization (CI / publish workflows / pre-commit / logging / install smoke / mypy), the LLM-driven layer (CPC refinement + semantic constraint grading + prediction → attribution loop), and the narrative tutorial are all shipped. Remaining work is project-specific: e.g., wiring up a real benchmark for metric_before/metric_delta capture, or composing the existing pieces into a nightly run on your own codebase.

See docs/ROADMAP.md and docs/ANALYTICS_LAYER.md for the per-phase status and the analytics spec.

Try the inventory and report CLIs

# Table listing of every file-level CPC with filters.
python -m code_covenant.cli inventory code_covenant --cohort gate

# Full CPC for a single module.
python -m code_covenant.cli inventory code_covenant --show code_covenant.contracts.model

# JSON export of the scan.
python -m code_covenant.cli inventory code_covenant --json > inventory.json

# Aggregate markdown report to stdout or a file.
python -m code_covenant.cli report code_covenant --title "Self-Audit"
python -m code_covenant.cli report code_covenant --output .code_covenant/report.md

Try the contractizer

# Dry: emit inventory/manifest/report only.
python -m code_covenant.cli contractize tests/fixtures/sample_project \
    --output /tmp/cc_out \
    --super-cohort sample_suite

# Annotate: also write copies with CPC headers inserted.
python -m code_covenant.cli contractize tests/fixtures/sample_project \
    --output /tmp/cc_out \
    --super-cohort sample_suite \
    --mode annotate

Try the proposal gate

# Evaluate a proposed change to target.py against tests and auto-merge if clean.
python -m code_covenant.cli propose evaluate path/to/target.py \
    --after path/to/after.py \
    --proposals-dir proposals \
    --ledger ledger.jsonl \
    --test-command "pytest -q" \
    --merge-mode auto

# Roll back a merged proposal.
python -m code_covenant.cli propose rollback <module> <proposal_id> \
    path/to/target.py \
    --proposals-dir proposals

Try the single-target iterator

python -m code_covenant.cli iterate path/to/target.py \
    --engine shell \
    --engine-command "python optimizer.py --before {before} --after {after}" \
    --proposals-dir proposals \
    --ledger ledger.jsonl \
    --max-attempts 10 \
    --duration 300 \
    --merge-mode auto

Each attempt writes a full proposal folder (so propose rollback works on any merged attempt). Use --no-stop-on-merge to keep going after the first merge.

Public-codebase demo

See examples/public_demo/click/ for a full end-to-end run against click 8.3.1: contractize generated 543 draft CPCs across 17 files (AST preserved on every file), and DECANTER classified 555 sections across all six hook levels.

Try the prediction → attribution loop

# iterate with both an engine and a predictor LLM
python -m code_covenant.cli iterate path/to/target.py \
    --engine shell \
    --engine-command "python optimizer.py --before {before} --after {after}" \
    --predictor-llm-command "claude -p" \
    --proposals-dir proposals \
    --ledger ledger.jsonl \
    --max-attempts 5 \
    --merge-mode auto

Before each attempt, the predictor LLM receives the CPC and the before/after sources and returns a JSON forecast: expected decision, expected metric delta, expected failure mode, and a one-sentence rationale. After the gate runs, the attribution loop compares the forecast to the actual outcome and records the result. Every ledger row ends up with prediction_summary and attribution_summary populated:

{
  "prediction_summary": {
    "expected_decision": "merge",
    "rationale": "semantically equivalent",
    ...
  },
  "attribution_summary": {
    "prediction_was_correct": true,
    "predicted_decision": "merge",
    "actual_decision": "merge",
    "delta_assessment": "no_data",
    "notes": "decision matched (merge)"
  }
}

LLM errors degrade to a captured-error record; the gate keeps running.

Try LLM-driven CPC refinement

# Ask Claude Code to refine the CPC of a single file.
python -m code_covenant.cli refine path/to/target.py \
    --llm-command "claude -p" \
    --output refined.json

# Show only the proposed changes.
python -m code_covenant.cli refine path/to/target.py \
    --llm-command "claude -p" \
    --changes-only

The LLM receives the draft CPC plus the source and returns a JSON object of just the editable fields it wants to change. Locked fields (module, language, authority, source_basis, cohort, super_cohort) are never touched. Output is review-only — refine never writes back into the source.

Opting into semantic constraint grading

from code_covenant.gate.pipeline import ProposalConfig, evaluate_proposal
from code_covenant.llm import ShellLLMAdapter

config = ProposalConfig(
    target_path=...,
    after_source=...,
    proposals_dir=...,
    ledger_path=...,
    semantic_grader=ShellLLMAdapter(command_template="claude -p"),
)
result = evaluate_proposal(config)

When semantic_grader is set, the gate replaces the literal text-match constraint grader with an LLM that decides PASS/PARTIAL/FAIL per constraint by reading the before- and after-source. LLM failures degrade to PARTIAL; the gate never crashes.

Try the analytics research loop

# After a portfolio or iterate run, turn the ledger into analytics.
python -m code_covenant.cli analytics ledger.jsonl \
    --output .code_covenant/analytics \
    --minimize latency_ms

Writes engine_comparison.json, pareto_frontier.json, noise_floor_report.json, and recommended_next_run.yaml. Benchmarks that print METRIC: <name> <value> lines feed the Pareto frontier and the noise-floor analysis.

Try DECANTER

python -m code_covenant.cli decanter path/to/project \
    --output .code_covenant/decanter

Writes agent_native_map.json and decanter_report.md: every section classified into hook levels 0–6, prohibited zones flagged, passive observation hooks proposed for the safe levels.

Try Piranha Lab (sandbox-only)

python -m code_covenant.cli piranha path/to/project \
    --sandbox-root /tmp/piranha-sandbox \
    --i-understand-piranha-is-destructive \
    --engine utility \
    --test-command "python -m pytest -q -p no:cacheprovider" \
    --rounds 5 \
    --duration 300 \
    --max-attempts-per-round 10

The default utility engine proposes conservative AST-derived rewrites (Boolean and guard-return collapse, any() reductions, and filtered comprehensions). Promotion is fail-closed: exact CPC bytes and public signatures must survive, capabilities cannot expand, immutable tests must pass in an OS-confined process, and every declared trusted metric must be present and strictly Pareto-improve over the current champion. The original source remains byte-identical.

Piranha refuses existing unowned sandbox roots, source symlinks, unsafe CPC module paths, missing behavior tests, and unsupported confinement hosts. The built-in host backend is macOS sandbox-exec; use a dedicated VM/container on other hosts. Run the adversarial check-of-the-check with:

python -m code_covenant.tools.piranha_utility_verify

See docs/PIRANHA_UTILITY_CAMPAIGNS.md for model-engine and campaign guidance.

Try the portfolio runner

# Scan a contractized project, let a shell-based engine propose changes for up
# to 30 minutes, and route every proposal through the gate.
python -m code_covenant.cli portfolio path/to/project \
    --output .code_covenant/portfolio \
    --engine shell \
    --engine-command "python optimizer.py --before {before} --after {after}" \
    --duration 1800 \
    --max-attempts 50 \
    --merge-mode review \
    --max-risk medium

When the run is over, read .code_covenant/portfolio/morning_brief.md for the summary, failure_heatmap.json for per-target/per-dimension counts, and prompt_analytics.json for per-prompt attempt/merge/reject stats.

Running the gate

bash scripts/check.sh

This runs the CPC validator, pytest, and compileall. All three must pass.

CI runs the same gate on every push and pull request — .github/workflows/check.yml. A PyPI publish workflow triggered on release tags lives at .github/workflows/publish.yml (no-op until a PYPI_API_TOKEN repo secret is configured).

Installing

pip install -e ".[dev]"
bash scripts/install_smoke.sh   # verifies every CLI entry point resolves

Pre-commit hook

Add to your project's .pre-commit-config.yaml:

repos:
  - repo: https://github.com/marlenehoover/code-covenant
    rev: v0.1.0
    hooks:
      - id: validate-cpc

The hook runs code-covenant-validate-cpc on the full repository before every commit and fails if any CPC is missing or malformed.

Global flags

Every subcommand accepts top-level --verbose (DEBUG logging to stderr) and --quiet (WARNING and above). Example: code-covenant --verbose portfolio src/ --engine shell ....

The dogfood test

Code Covenant's own source files must pass Code Covenant's own validator. If they don't, the tool is lying about what it can enforce.