diff --git a/RELEASE.md b/RELEASE.md index 954cfa2..991a4c8 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -25,26 +25,16 @@ See [TEST_ENVIRONMENT.md](TEST_ENVIRONMENT.md) for prerequisites and troubleshoo ## Release contract -A version is releasable only when: - -1. `npm run verify` passes (automated in CI on every PR). -2. Every host declared in `scripts/skill-resources.json` has a current - full-milestone receipt under - `docs/trust-validation/evidence/releases//`. -3. `npm run verify:release` proves that only trust evidence or its summaries - changed after the tested candidate SHA. - -The package's `prepublishOnly` script runs `verify:release`, so a local -`npm publish` cannot bypass the gate. - -Details: [docs/trust-validation/TRUST-VALIDATION-SPEC.md](docs/trust-validation/TRUST-VALIDATION-SPEC.md). +The [Trust Validation Spec](docs/trust-validation/TRUST-VALIDATION-SPEC.md#release-contract) +owns the release gate, live-check scope, and receipt requirements. ## Recording trust evidence Set the intended package name and version in `package.json` and `package-lock.json` before freezing the candidate. Changes to either invalidate existing candidate proof. -Before publishing: +When the [live-check scope](docs/trust-validation/TRUST-VALIDATION-SPEC.md#live-check-scope) +requires host receipts, complete these steps before publishing: 1. Freeze a clean candidate on `main`: @@ -52,10 +42,10 @@ Before publishing: bash scripts/prepare_release_evidence.sh --candidate . ``` -2. Run each host harness from the prepared directories (see +2. Run each required host harness from the prepared directories (see [HOST-MATRIX.md](docs/trust-validation/HOST-MATRIX.md)). -3. Validate the full set locally: +3. Validate the required set locally: ```bash npm run verify:release @@ -69,8 +59,8 @@ Ordinary PRs and `main` pushes run automated verification, including the trust validator tests, without requiring refreshed release receipts. The `verify-release-evidence` check name remains active on every PR. Product and host contract changes can therefore merge before the next candidate is frozen. -Passing PR checks does not establish release trust: all eleven hosts must have -current candidate proof before either publication path can pass `verify:release`. +Passing PR checks does not establish release trust. Both publication paths +must pass the [release contract](#release-contract). ## Publishing @@ -191,8 +181,8 @@ the helper stays silent (see [UPDATE.md](UPDATE.md)). ## What CI does not automate -- Full live milestone runs on every host — recorded manually per release; - validated by `verify:release`. +- Full live milestone runs — record required receipts manually according to + the [release contract](#release-contract). - npm publish on every merge — automated publishing uses the Release workflow after the trust gate passes; see [First npm publication](#first-npm-publication) for the initial manual publish. diff --git a/docs/trust-validation/CODEX-COMPARISON.md b/docs/trust-validation/CODEX-COMPARISON.md index 3818a4d..49ab1b3 100644 --- a/docs/trust-validation/CODEX-COMPARISON.md +++ b/docs/trust-validation/CODEX-COMPARISON.md @@ -2,7 +2,8 @@ This evaluation compares GSD Path with a direct Codex implementation of the same widget-counter requirements. It measures one fixture, not global -optimality. It is separate from the all-host release gate. +optimality. It is separate from the +[release gate](TRUST-VALIDATION-SPEC.md#release-contract). Commit the candidate first. Use a new evaluation directory outside the source repository. Preparation makes separate local bare remotes and checkouts from @@ -57,8 +58,8 @@ Product correctness, measurement completeness, and pipeline trust are separate. A zero CLI exit is not proof of a milestone or child dispatch. To validate a Codex milestone receipt, pass `report --receipt /absolute/codex.md`; the runner uses the existing receipt validator against the pinned candidate and manifest. -That is fixture-level proof only. Release trust still requires current tracked -receipts for every declared host through `npm run verify:release`. +That is fixture-level proof only. See the +[release contract](TRUST-VALIDATION-SPEC.md#release-contract) for release trust. ## Verification changes in this candidate diff --git a/docs/trust-validation/HOST-MATRIX.md b/docs/trust-validation/HOST-MATRIX.md index 504e708..4f41eff 100644 --- a/docs/trust-validation/HOST-MATRIX.md +++ b/docs/trust-validation/HOST-MATRIX.md @@ -1,8 +1,8 @@ # Host trust matrix -`supported` means the installer and static dispatch contract exist. A release -is end-to-end trusted only when `npm run verify:release` finds a current passing -full-milestone receipt for every host below. +`supported` means the installer and static dispatch contract exist. See the +[release contract](TRUST-VALIDATION-SPEC.md#release-contract) for release trust +and required live-check scope. Release 1.1.0 uses frozen candidate `af0b082964510c471798826d7e2e05617d8d6dc3` from main on 2026-09-18, after PRs #124 and #125. Every host below has a @@ -72,21 +72,13 @@ plus an instruction to propose doc-vs-code rulings without writing them. ## Release procedure (from the 2026-09-06 Codex receipt run) -`npm run verify:release` accepts only host receipts as `.md` files at the root -of `evidence/releases//`; other notes live under `notes/`. Every -receipt must name one frozen candidate, and only evidence files plus this -matrix and TRUST-EVIDENCE.md may change after that candidate. So the order -is: merge every code and documentation change first, freeze the candidate, -run all eleven host receipts against it, then publish only with owner authorization. For receipt assembly -options, see the [assembly tool](evidence/releases/1.0.0/codex/release_receipt.py) +Follow [Recording trust evidence](../../RELEASE.md#recording-trust-evidence) +and the [receipt format](LIVE-EVIDENCE-TEMPLATE.md). Keep non-receipt notes +under `evidence/releases//notes/`. For receipt assembly options, +see the [assembly tool](evidence/releases/1.0.0/codex/release_receipt.py) usage docstring. -Ordinary PR and `main` CI run automated tests, including the trust validator -tests; they do not require refreshed release receipts. The manual **Release -trust evidence** workflow validates a frozen candidate and its receipts. Both -npm `prepublishOnly` and the Release workflow still require `verify:release` -with current proof from all eleven hosts. Existing receipts remain evidence -for their named candidate only. +See [CI tiers](../../RELEASE.md#ci-tiers) for development and release checks. ### 1.0.0 receipts (2026-09-15) diff --git a/docs/trust-validation/README.md b/docs/trust-validation/README.md index adf87c2..29a2873 100644 --- a/docs/trust-validation/README.md +++ b/docs/trust-validation/README.md @@ -1,8 +1,8 @@ # Trust validation -Validation workstream for every host declared in the installer manifest. A -host remains release-blocking until its current versioned live milestone -receipt passes the shared evidence validator. +Validation workstream for every host declared in the installer manifest. +The [release contract](TRUST-VALIDATION-SPEC.md#release-contract) defines +automated checks and when live host receipts are required. **Start here:** [TRUST-VALIDATION-SPEC.md](TRUST-VALIDATION-SPEC.md) **Manual runs:** [TRUST-EVIDENCE.md](TRUST-EVIDENCE.md) diff --git a/docs/trust-validation/TRUST-VALIDATION-SPEC.md b/docs/trust-validation/TRUST-VALIDATION-SPEC.md index 2b6cd6b..153ba42 100644 --- a/docs/trust-validation/TRUST-VALIDATION-SPEC.md +++ b/docs/trust-validation/TRUST-VALIDATION-SPEC.md @@ -1,28 +1,52 @@ # GSD Path — Trust Validation Spec **Status:** Active release contract -**Updated:** 2026-08-26 +**Updated:** 2026-09-20 **Scope:** Every host declared in `scripts/skill-resources.json` **Install path:** Node or Python installer from a clone; Node is the npm and interactive entry point. ## Release contract -A release that changes pipeline contracts is trusted only when: +Every release is trusted only when: 1. `npm run verify` passes. -2. Every declared host has one current full-milestone receipt and structured, +2. Every affected host has one current full-milestone receipt and structured, tracked per-step evidence under `evidence/releases//`. Each receipt must identify a distinct host run, landing commit, ship commit, and integration commit. -3. `npm run verify:release` accepts those receipts and proves that only trust - evidence or its summaries changed after the tested candidate SHA. +3. `npm run verify:release` determines the affected hosts, accepts required + receipts, and proves that only trust evidence or its summaries changed + after their tested candidate SHA. With no affected hosts, automated checks + suffice; the validator reports an empty host list, not a live-test pass. + +### Live-check scope + +`scripts/check_trust_evidence.py` compares HEAD with the nearest reachable +`v` tag, excluding the current package version's tag. This prevents a +new release tag from hiding its own changes. Without a prior tag or with +shallow history, every host requires evidence. + +| Changed files | Required live checks | +|---|---| +| `platforms//` | That host; multiple host changes combine | +| Shared adapters, skills, runtime scripts, `AGENTS.md`, `WORKFLOW.md`, or unclassified paths | Every host | +| `docs/`, other root Markdown, `tests/`, `.github/`, `daemon/` | None | +| `scripts/bump_version.mjs`, `scripts/update_release_docs.mjs`, `scripts/prepare_release_evidence.sh`, `scripts/check_trust_evidence.py` | None; automated verification still applies | +| `package.json`, `package-lock.json` | None only when changes are limited to the top-level version and lockfile root-package version; otherwise every host | + +Added and deleted paths count, including both sides of renames. No old receipt +is relabeled as current evidence. Required receipts retain all existing +candidate, native-child, task, review, archive, integration, and guard checks. + +Release 1.2.0 includes shared pipeline changes, so every declared host still +requires current live evidence. `npm publish` runs this gate through the package's `prepublishOnly` lifecycle. The release-trust workflow provides the same check on demand before publishing. -Missing credentials, an unavailable host, or a manual result marked partial or -unverifiable blocks release. It never becomes an implicit pass. +Missing credentials, an unavailable required host, or required evidence marked +partial or unverifiable blocks release. It never becomes an implicit pass. ## Evidence layers @@ -31,7 +55,7 @@ unverifiable blocks release. It never becomes an implicit pass. | Unit and integration tests | Install, state routing, handoff validation, isolation, recovery, guards, archive, integration | `npm run verify` | | Lightweight live smoke | Real host invocation and a bounded artifact | `.github/workflows/dogfood.yml` | | Full live milestone | Real child dispatch, build, review, archive, merge, and tag on one host | release receipt | -| Release reconciliation | Every host passed at one candidate and later changes are evidence-only | `npm run verify:release` | +| Release reconciliation | Affected hosts passed at one candidate and later changes are evidence-only | `npm run verify:release` | Simulated full-cycle tests are strong evidence for the deterministic disk and Git contract. They do not replace real child-agent execution. @@ -57,9 +81,9 @@ APIs that can prove real dispatch for each host. Current status is recorded in | Gap | Required closure | |---|---| -| Full live milestone evidence is missing for advertised hosts | Record one passing current receipt per host | +| Full live milestone evidence is missing for affected hosts | Record one passing current receipt per affected host | | Native guards are installed automatically only for Claude, Codex, and Cursor; Codex requires manual project and hook trust | Activate and validate Codex through `/hooks`; add host-native adapters where official APIs support them, otherwise retain an explicit lower tier | -| Not every host is available to the maintainer today | Keep the release gate blocked until access or maintainer-reviewed evidence exists | +| A required host is unavailable to the maintainer | Keep the release gate blocked until access or maintainer-reviewed evidence exists | | Live host APIs can change independently of this repo | Keep lightweight dogfood scheduled where credentials exist and re-record full evidence for pipeline-contract releases | ## Historical evidence diff --git a/scripts/check_trust_evidence.py b/scripts/check_trust_evidence.py index d90dc85..0db1db9 100644 --- a/scripts/check_trust_evidence.py +++ b/scripts/check_trust_evidence.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Validate release evidence for every supported GSD Path host.""" +"""Validate release evidence for affected GSD Path hosts.""" import argparse import json @@ -851,6 +851,53 @@ def _require_current_tracked_evidence( ) +def release_scope(repo: Path, version: str, hosts: Sequence[str]) -> Mapping: + """Select live checks from the previous reachable release; unknowns fail closed.""" + tags = [tag for tag in _git(repo, "tag", "--merged", "HEAD", "--list", "v*").splitlines() + if tag != f"v{version}" and re.fullmatch(r"v\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?", tag)] + if not tags or _git(repo, "rev-parse", "--is-shallow-repository") == "true": + return {"baseline": None, "hosts": list(hosts), "live_changes": ["no complete release baseline"]} + matches = [argument for tag in tags for argument in ("--match", tag)] + baseline = _git(repo, "describe", "--tags", "--abbrev=0", *matches, "HEAD") + # Disable rename detection so moving a contract into docs cannot hide deletion. + paths = _git(repo, "diff", "--no-renames", "--name-only", "-z", baseline, "HEAD").split("\0") + required = set() + live_changes = [] + release_tools = { + "scripts/bump_version.mjs", "scripts/update_release_docs.mjs", + "scripts/prepare_release_evidence.sh", "scripts/check_trust_evidence.py", + } + for path in filter(None, paths): + if path in {"package.json", "package-lock.json"}: + # Only version fields are exempt; dependency and packaging changes are shared. + if not (repo / path).is_file() or not _git(repo, "ls-tree", "--name-only", baseline, "--", path): + required.update(hosts) + live_changes.append(path) + continue + previous = json.loads(_git(repo, "show", f"{baseline}:{path}")) + current = _read_json(repo / path) + for document in (previous, current): + document.pop("version", None) + root_package = document.get("packages", {}).get("") if path == "package-lock.json" else None + if isinstance(root_package, dict): + root_package.pop("version", None) + if previous == current: + continue + elif (path.startswith(("docs/", "tests/", ".github/", "daemon/")) + or path in release_tools + or ("/" not in path and path.endswith(".md") + and path not in {"AGENTS.md", "WORKFLOW.md"})): + continue + parts = PurePosixPath(path).parts + if len(parts) > 2 and parts[0] == "platforms" and parts[1] in hosts: + required.add(parts[1]) + else: + required.update(hosts) + live_changes.append(path) + return {"baseline": baseline, "hosts": [host for host in hosts if host in required], + "live_changes": live_changes} + + def validate_repository(repo: Path) -> Mapping: repo = repo.resolve() if _git(repo, "status", "--porcelain", "--untracked-files=all"): @@ -866,13 +913,17 @@ def validate_repository(repo: Path) -> Mapping: raise EvidenceError("host manifest is empty") if not isinstance(version, str) or not re.fullmatch(r"[0-9A-Za-z.+-]+", version): raise EvidenceError("package version is missing or invalid") + scope = release_scope(repo, version, hosts) + hosts = scope["hosts"] + if not hosts: + return {**scope, "candidate": None, "version": version} evidence_root = ( repo / "docs" / "trust-validation" / "evidence" / "releases" / version ) missing = [host for host in hosts if not (evidence_root / f"{host}.md").is_file()] if missing: raise EvidenceError(f"missing host evidence: {', '.join(missing)}") - extra = sorted(path.stem for path in evidence_root.glob("*.md") if path.stem not in hosts) + extra = sorted(path.stem for path in evidence_root.glob("*.md") if path.stem not in host_contracts) if extra: raise EvidenceError(f"unexpected host evidence: {', '.join(extra)}") @@ -936,7 +987,7 @@ def validate_repository(repo: Path) -> Mapping: raise EvidenceError( "non-evidence changes follow the tested candidate: " + ", ".join(disallowed) ) - return {"candidate": candidate, "hosts": hosts, "version": version} + return {**scope, "candidate": candidate, "version": version} def parser() -> argparse.ArgumentParser: diff --git a/tests/test_trust_evidence.py b/tests/test_trust_evidence.py index 43c0eae..f283c48 100644 --- a/tests/test_trust_evidence.py +++ b/tests/test_trust_evidence.py @@ -539,6 +539,87 @@ def commit_receipts(self): self.git("add", "-A") self.git("commit", "-qm", "trust evidence") + def release_change(self, path, content="changed\n"): + target = self.repo / path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(content, encoding="utf-8") + self.commit_receipts() + self.candidate = self.git("rev-parse", "HEAD").stdout.strip() + + def test_release_scope_skips_live_runs_for_non_host_changes(self): + self.git("tag", "v1.2.2") + for path in ("README.md", "docs/guide.md", "tests/test_example.py", + ".github/workflows/release.yml", "daemon/gsd_daemon/serve.py", + "scripts/update_release_docs.mjs", "scripts/check_trust_evidence.py"): + with self.subTest(path=path): + self.release_change(path) + result = check_trust_evidence.validate_repository(self.repo) + self.assertEqual([], result["hosts"]) + self.assertEqual("v1.2.2", result["baseline"]) + + def test_release_scope_accepts_version_only_package_change(self): + self.git("tag", "v1.2.2") + self.release_change("package.json", json.dumps({"version": "1.2.4"})) + self.assertEqual([], check_trust_evidence.validate_repository(self.repo)["hosts"]) + self.release_change("package.json", json.dumps({"version": "1.2.4", "scripts": {"test": "false"}})) + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + + def test_release_scope_requires_only_changed_host(self): + self.git("tag", "v1.2.2") + self.release_change("platforms/alpha/dispatch.md") + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing host evidence: alpha$"): + check_trust_evidence.validate_repository(self.repo) + self.receipt("alpha") + self.commit_receipts() + self.assertEqual(["alpha"], check_trust_evidence.validate_repository(self.repo)["hosts"]) + + def test_release_scope_lockfile_exempts_only_package_version(self): + lock = {"version": "1.2.2", "packages": {"": {"version": "1.2.2"}, + "node_modules/example": {"version": "1.0.0"}}} + self.release_change("package-lock.json", json.dumps(lock)) + self.git("tag", "v1.2.2") + lock["version"] = lock["packages"][""]["version"] = "1.2.3" + self.release_change("package-lock.json", json.dumps(lock)) + self.assertEqual([], check_trust_evidence.validate_repository(self.repo)["hosts"]) + lock["packages"]["node_modules/example"]["version"] = "2.0.0" + self.release_change("package-lock.json", json.dumps(lock)) + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + + def test_release_scope_shared_adapter_requires_all_hosts(self): + self.git("tag", "v1.2.2") + self.release_change("platforms/shared-agents/dispatch.md") + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + + def test_release_scope_new_lockfile_requires_all_hosts(self): + self.git("tag", "v1.2.2") + self.release_change("package-lock.json", json.dumps({"version": "1.2.3"})) + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + + def test_release_scope_current_tag_does_not_hide_shared_changes(self): + self.git("tag", "v1.2.2") + self.release_change("skills/gsd-path/SKILL.md") + self.git("tag", "v1.2.3") + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + + def test_release_scope_unknown_changes_require_all_hosts(self): + self.git("tag", "v1.2.2") + self.release_change("new-runtime.py") + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + + def test_release_scope_rename_cannot_hide_shared_contract_deletion(self): + self.release_change("WORKFLOW.md") + self.git("tag", "v1.2.2") + self.git("mv", "WORKFLOW.md", "README.md") + self.commit_receipts() + with self.assertRaisesRegex(check_trust_evidence.EvidenceError, "missing.*alpha, beta"): + check_trust_evidence.validate_repository(self.repo) + @mock.patch.dict( "os.environ", {