Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,16 @@ All notable changes are documented here. RealDone follows semantic versioning wh

### Added

- Fail-closed per-trace retention for automatic scans and contract/MCP verification: bounded ZIP inspection now checks generated sensitive fields, contract `secretEnv`, opaque Playwright auth-state cookies/tokens and generic credential patterns before report linkage, with additive value-free suppression metadata for deleted unsafe or unscannable traces.
- Fresh fingerprint-bound Phase J qualification across TodoMVC, Actual Budget, Conduit SQLite/PostgreSQL 17/Supabase Data API and Pocket Ledger, including PostgreSQL CRUD cleanup, Level 7 roles and a new Codex MCP RD901 regression/repair cycle.
- Coverage-balanced quick-scan scheduling that represents more routes, action kinds, intents, and keyboard/submit activation paths within the same finite action budget, while placing known policy/environment denials after runnable actions.
- Additive action-selection telemetry in JSON and HTML reports, including eligible/selected/omitted counts, route coverage, semantic coverage, denial counts, the deterministic strategy version, and an explicit partial-scan explanation.
- Fresh fingerprint-bound qualification across TodoMVC, Actual Budget, Conduit/SQLite, Conduit/PostgreSQL 17 + Supabase Data API, and Codex-generated Pocket Ledger, including PostgreSQL create/update/delete cleanup, Level 7 roles, and a real Codex MCP baseline → RD901 regression → repair cycle.
- Hosted main run `30195945498` passed and signed all 15 normative gates across Linux, Windows and macOS for the Phase I fingerprint and installed-package path.

### Fixed

- Playwright traces containing known sensitive field values, generic credential/token material, invalid ZIP data, or bounded-inspection limit failures are removed immediately instead of surviving until the aggregate release artifact gate.
- Report timelines and environment summaries now render arrows and separators as UTF-8 instead of mojibake.

## [1.3.3] - 2026-07-26
Expand Down
4 changes: 4 additions & 0 deletions docs/COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

Unreleased main fingerprint `f3f65840…` preserved the same matrix and package path in hosted run [`30195945498`](https://github.com/datzle123/RealDone/actions/runs/30195945498). All 15 gates passed on Linux, Windows and macOS with Node 20/22, and GitHub signed the aggregate evidence and gate report for merge commit `2572aba57edf418f87750352ed3b3d36868015ac`.

The subsequent Phase I status merge `42006c4` independently repeated the full matrix and signed 15/15 in hosted main run [`30196808460`](https://github.com/datzle123/RealDone/actions/runs/30196808460).

| Surface | Release gate |
| --- | --- |
| Node.js | 20 and 22 |
Expand All @@ -26,3 +28,5 @@ Codex and Claude Code integrations are command presets, not embedded SDKs. RealD
## Unreleased report compatibility

Coverage-balanced action selection adds an optional `completeness.selection` object to `scan.json`. Existing `schemaVersion: "1.0"` fields and meanings are unchanged, old reports remain readable, and validators use passthrough/additive compatibility. Consumers may ignore the new telemetry or use it to distinguish discovered, selected, omitted, and route-represented action coverage.

Secret-safe trace retention adds optional `evidence.traceSuppression` on scan findings and optional `artifacts.traceSuppressions` on contract verification. Existing trace paths are unchanged when a ZIP passes inspection; rejected traces are absent by design. Consumers that ignore the new metadata continue to read schema `1.0`, while Windows, macOS and Linux use the same bounded `fflate` ZIP inspection path.
2 changes: 1 addition & 1 deletion docs/PERFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Quick scan remains intentionally light: one Chromium worker, no provider/databas

`--deep` opens one additional browser context per executed mutation to confirm persistence scope. Keep it opt-in for important flows or scheduled audits rather than paying that cost in every quick scan.

`--trace` records Playwright snapshots/screenshots and `--video` records the browser viewport. Both add I/O, storage, and post-processing work; use them for diagnosis or release evidence rather than routine scans. `--trace-on-failure` starts bounded tracing but deletes passing traces, retaining portable ZIP evidence only for findings or failed contract verification.
`--trace` records Playwright snapshots/screenshots and `--video` records the browser viewport. Both add I/O, storage, and post-processing work; use them for diagnosis or release evidence rather than routine scans. Every newly closed trace receives one bounded ZIP secret inspection before it can be linked; unsafe, invalid, oversized, or over-expanded traces are deleted fail-closed. `--trace-on-failure` then deletes safe passing traces, retaining portable ZIP evidence only for findings or failed contract verification.

`scan --full` raises the default safe budgets to 100 pages, 500 actions, and 30 minutes, enables deep persistence and trace-on-failure, but never enables destructive or external effects. Explicit budget flags or policy values still win, and exhausted budgets set `truncated`.

Expand Down
7 changes: 6 additions & 1 deletion docs/PRODUCT_SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -658,6 +658,8 @@ Replay
Baseline
```

Mỗi Playwright trace chỉ được liên kết vào evidence sau khi ZIP vừa tạo đã qua kiểm tra secret có giới hạn. Phép kiểm tra phải gồm cả sensitive field được điền, `secretEnv`, opaque cookie/token trong Playwright `storageState` đã được cấp, và các pattern credential tổng quát. Nếu phát hiện secret, ZIP không đọc được, hoặc vượt giới hạn kiểm tra, RealDone phải xóa trace theo hướng fail-closed, không công bố đường dẫn đã bị loại, và chỉ ghi metadata lý do không chứa giá trị hay fingerprint của secret. Quy tắc này áp dụng giống nhau cho scan tự động, contract verification, browser matrix, CI và MCP.

---

# 13. State Snapshot Engine
Expand Down Expand Up @@ -1109,6 +1111,7 @@ Report phải phân biệt:
* unverified action;
* regression;
* expected change.
* trace đã bị loại bởi kiểm tra secret, tách biệt với việc không bật trace hoặc xóa passing trace theo `--trace-on-failure`.

---

Expand Down Expand Up @@ -1213,6 +1216,8 @@ RD_TEST_<timestamp>_<random>

Cleanup ledger phải ghi mọi resource được tạo.

Trace là artifact nhạy cảm: redaction trong JSON/DOM không đủ để chứng minh ZIP Playwright an toàn. Mọi trace được giữ lại phải qua kiểm tra bounded ngay sau khi đóng trace; trace không kiểm tra được hoặc có secret phải bị xóa trước khi report được ghi.

---

# 26. Benchmark System
Expand Down Expand Up @@ -1354,7 +1359,7 @@ Một release chỉ được phát hành khi:
11. Environment health gate pass.
12. Cross-platform smoke pass.
13. Report schema backward-compatible.
14. Không có secret trong artifact.
14. Không có secret trong artifact; trace bị phát hiện không an toàn hoặc không kiểm tra được phải bị xóa và không được liên kết trong report.
15. Case study bên ngoài không regression nghiêm trọng.

Không được release chỉ vì:
Expand Down
10 changes: 6 additions & 4 deletions docs/PRODUCT_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
**Snapshot:** 2026-07-26
**Released full product:** **IMPLEMENTED in `v1.3.3`**
**Latest release qualification:** hosted run `30189340006` passed and signed all 15 gates for the versioned source commit and installed-package path.
**Latest main qualification:** hosted run `30195945498` passed and signed all 15 gates for Phase I fingerprint `f3f65840…`.
**Latest main qualification:** hosted run `30196808460` passed and signed all 15 gates for Phase I status merge `42006c4` and fingerprint `f3f65840…`.
**Area coverage:** **22/22 `IMPLEMENTED`**, **0/22 `PARTIAL`**, **0/22 `PLANNED`**. Detector catalog: **58/58 production-classified and gated**.

This file reports current evidence against the normative [`PRODUCT_SPECIFICATION.md`](PRODUCT_SPECIFICATION.md). It does not reduce that specification. `IMPLEMENTED` requires executable evidence; `PARTIAL` means useful code exists but the full normative behavior or release gate does not.
Expand All @@ -12,6 +12,8 @@ Release `v1.3.3` additionally ran ten pinned MIT projects and detected 10/10 inj

Phase I hardening is complete on `main`: coverage-balanced quick-scan scheduling and additive selection telemetry are bound to fingerprint `f3f65840…`. Local type/unit/build (140 tests with 2 expected service-dependent skips), audit, full Chromium smoke, pack and installed-tarball smoke passed. Fresh repository-bound runs covered all five normative external cases: TodoMVC exercised both Enter paths; Actual Budget preserved its safe history-dependent `UNCERTAIN`; Conduit passed untruncated SQLite and PostgreSQL 17 + Supabase Data API scans plus real create/update/delete cleanup; Pocket Ledger passed six visible actions, Level 7 roles, and a new Codex MCP RD901 regression/repair cycle. Hosted run [`30195945498`](https://github.com/datzle123/RealDone/actions/runs/30195945498) passed 15/15 on Linux, macOS and Windows; `gh attestation verify` accepted the signed aggregate evidence and gate report. This main-branch qualification is not a new npm version by itself; `v1.3.3` remains the latest published release.

Phase J is an unreleased hardening candidate: scan and contract/MCP traces are inspected immediately after capture, and unsafe or unscannable ZIPs are deleted before report linkage. Candidate fingerprint `0b39d542…` has fresh SHA-256-bound TodoMVC, Actual Budget, Conduit SQLite, Conduit PostgreSQL 17 + Supabase Data API, PostgreSQL CRUD cleanup, Pocket Ledger Level 7 and Codex MCP regression/repair evidence; those inputs merged with the six hosted candidate attestations to pass 15/15 locally, while the signed hosted aggregate rerun remains pending. The existing full-product status remains valid because this strengthens §12–13/§21/§25 without narrowing behavior; Phase J itself remains a release candidate until its branch and hosted gates pass.

| Specification area | Status | Evidence currently in the repository | Remaining normative gap |
| --- | --- | --- | --- |
| §4 Quick scan | IMPLEMENTED | `scan`, browser execution, report, reload checks, fixture smoke, and deterministic coverage-balanced selection with explicit eligible/selected/omitted/route telemetry | Broader action and environment coverage is tracked below |
Expand All @@ -22,16 +24,16 @@ Phase I hardening is complete on `main`: coverage-balanced quick-scan scheduling
| §6 Project Discovery/Runtime Manager | IMPLEMENTED | `init` profiles single-package/monorepo projects; npm fallback without lock metadata; project-local Python virtual environments; zero-config static HTML plus conventional Node, Django/FastAPI/Flask, Laravel/PHP, Rails, ASP.NET Core, Spring Boot, Deno, Go and Rust managed runtimes; occupied-port rejection, development/production/Docker health-checks, logs, bounded restarts and process cleanup; installed-tarball scans start, verify and stop metadata-free static and npm projects in hosted run `29977292441` | Custom runtimes use an explicit HTTP URL; new ecosystem hints must add cross-platform broken/control and package evidence |
| §7 Environment Health Gate | IMPLEMENTED | main/route document, critical asset/content-type, bootstrap/render, health-endpoint and auth-state checks; separate `environment.json`; broken/control fixtures and TodoMVC defect copy | Additional ecosystem-specific diagnostics may be added without changing the implemented fail-closed contract |
| §8–11 Discovery, classification, data, executor | IMPLEMENTED | forms/links/buttons/native controls, Enter, hover/context, lazy-scroll, popup/download, opt-in same-origin iframe, complex-action recording boundaries, constraint-aware canaries, network-idle/stale/retry safety, and budget selection that avoids route/semantic starvation and known-denial budget waste | Framework-specific discovery adapters may broaden coverage without changing the implemented safe-execution contract |
| §12–13 Evidence and snapshots | IMPLEMENTED | URL, redacted semantic DOM/control hashes, cookie hashes, local/session storage, bounded IndexedDB metadata, request/response, WebSocket frames, console/page error, upload/download digests, screenshots, trace/video/timeline, value-free source diffs, and redacted automatic provider evidence/artifacts; reproductions retain value-free provider requirements and replay requires fresh exact causal confirmation | New evidence adapters must preserve the implemented redaction, linkage and replay contract |
| §12–13 Evidence and snapshots | IMPLEMENTED | URL, redacted semantic DOM/control hashes, cookie hashes, local/session storage, bounded IndexedDB metadata, request/response, WebSocket frames, console/page error, upload/download digests, screenshots, trace/video/timeline, value-free source diffs, and redacted automatic provider evidence/artifacts; every retained browser/contract trace passes bounded ZIP inspection against generated sensitive fields, `secretEnv`, opaque auth-state cookies/tokens and generic credential patterns before linkage, while unsafe/unscannable traces are deleted with value-free suppression metadata; reproductions retain value-free provider requirements and replay requires fresh exact causal confirmation | New evidence adapters must preserve the implemented redaction, retention, linkage and replay contract |
| §14 Persistence | IMPLEMENTED | immediate/reload/hard-reload/new-tab/clean-context/logout-login/app-restart/API read-back strategies; all seven scopes are executable through browser smoke, contract Level 6/7 checks and deterministic scope tests | Additional application-specific adapters may extend confirmation without changing the implemented scope contract |
| §15–17 Verdicts and evidence hierarchy | IMPLEMENTED | scan verdicts, regression `EXPECTED_CHANGE`/`REGRESSION` outcomes, separate `ENVIRONMENT_INVALID`/`BLOCKED`, priority resolution and Levels 0–7 across scan/contract/baseline/report evidence | New adapters must preserve this implemented hierarchy |
| §18 Detector system | IMPLEMENTED | all 58 catalogued RD001–RD1005 detectors have production classification plus unit, browser/contract, regression or environment broken/control evidence | New detectors require the same broken/control and observable-evidence policy |
| §19–20 Contracts and replay | IMPLEMENTED | versioned complex-step schemas, semantic source/target locators, assertions, cleanup declarations, fresh canaries/evidence and all five normative replay outcomes with `replay.json` | New step/expectation types must preserve schema compatibility and deterministic replay |
| §21 Report | IMPLEMENTED | HTML/JSON, finding timelines, screenshots/trace/video and dedicated network/snapshot/console/WebSocket/upload/download/contract/reproduction artifacts; application/environment/skipped/uncertain and expected/regression results remain distinct | Additional renderers may consume the implemented portable evidence model |
| §21 Report | IMPLEMENTED | HTML/JSON, finding timelines, screenshots/trace/video and dedicated network/snapshot/console/WebSocket/upload/download/contract/reproduction artifacts; a removed unsafe trace is never linked and appears only as sanitized suppression metadata; application/environment/skipped/uncertain and expected/regression results remain distinct | Additional renderers may consume the implemented portable evidence model |
| §22 Database adapters | IMPLEMENTED | zero-config SQLite plus PostgreSQL, Supabase, Firebase, MongoDB and Prisma/custom source connectors; read-only verification, schema/PK/soft-delete discovery, hash snapshots/diff, parameterization/mapping, TLS/remote policy, redaction and confirmed cleanup have integration, browser and external Conduit evidence | New adapters must preserve the implemented source contract and safety gates |
| §23 Provider adapters | IMPLEMENTED | maintained Stripe-test, Resend, SendGrid, Mailgun, S3, Supabase Storage and OAuth read-only adapters plus custom provider plugins; bounded requests, secret redaction and production guards pass integration/browser gates | New providers must preserve the implemented read-only, fail-closed contract |
| §24 Multi-role | IMPLEMENTED | isolated role storage states, UI visibility, Level 7 cross-role observation, API denial, direct route, cross-tenant read/write, revoked-role and invalidated-session probes | Application-specific role provisioning remains user configuration, not a missing verification capability |
| §25 Safety | IMPLEMENTED | interactive scans require one explicit project-level action consent and non-interactive CLI/MCP execution fails closed without `--yes`/user-owned server authorization; host/destructive/external opt-ins, form/endpoint/provider classification, file/cross-origin/popup boundaries, live pre-execution escalation, redaction, cleanup, read-only adapters, production guards and artifact secret gates remain independent | New action/provider classes must preserve the implemented consent and least-authority boundaries |
| §25 Safety | IMPLEMENTED | interactive scans require one explicit project-level action consent and non-interactive CLI/MCP execution fails closed without `--yes`/user-owned server authorization; host/destructive/external opt-ins, form/endpoint/provider classification, file/cross-origin/popup boundaries, live pre-execution escalation, redaction, cleanup, read-only adapters, production guards, immediate fail-closed trace retention and the aggregate artifact secret gate remain independent | New action/provider/artifact classes must preserve the implemented consent and least-authority boundaries |
| §26 Benchmark | IMPLEMENTED | precision, recall, FPR, discovery, expectation coverage, verdict/detector accuracy, replay, truncation, environment validity and confirmed cleanup success are machine-readable and release-gated | Future metrics must remain additive and deterministic |
| §27 Real-world cases | IMPLEMENTED | current main fingerprint `f3f65840…` has fresh repository-bound scans for TodoMVC, Actual Budget, Conduit/SQLite, Conduit/PostgreSQL 17 + Supabase Data API and Codex-generated Pocket Ledger; all nine normative capability classes have semantic validator-parsed evidence, including PostgreSQL CRUD cleanup, upload/export, Level 7 roles, multi-step and agent repair; signed hosted run `30195945498` passed Windows/macOS/Linux | Future engine fingerprints must repeat the hosted signed qualification |
| §28–31 Engineering/release/performance/UX | IMPLEMENTED | published npm package with registry-smoked `npx realdone scan`; installed-tarball first-scan gates start, verify and clean up metadata-free static and npm projects; one-command managed scan with bounded first-use browser bootstrap; CI, license/audit/package gates, quick/full budgets, coverage-balanced selection, bounded workers, final post-build affected-flow selection, trace-on-failure, snapshot dedup, environment/artifact/schema/raw-case gates and GitHub artifact-attestation workflow; main run `30195945498` passed hosted 15/15 across Windows/macOS/Linux, Node 20/22, PostgreSQL 17, MongoDB 8 and package smoke | Future releases must preserve the same release gates |
Expand Down
Loading