Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@ All notable changes are documented here. RealDone follows semantic versioning wh

## [Unreleased]

### Added

- Coverage-balanced quick-scan scheduling that represents more routes, action kinds, intents, and keyboard/submit activation paths within the same finite action budget, while placing known policy/environment denials after runnable actions.
- Additive action-selection telemetry in JSON and HTML reports, including eligible/selected/omitted counts, route coverage, semantic coverage, denial counts, the deterministic strategy version, and an explicit partial-scan explanation.
- Fresh fingerprint-bound qualification across TodoMVC, Actual Budget, Conduit/SQLite, Conduit/PostgreSQL 17 + Supabase Data API, and Codex-generated Pocket Ledger, including PostgreSQL create/update/delete cleanup, Level 7 roles, and a real Codex MCP baseline → RD901 regression → repair cycle.

### Fixed

- Report timelines and environment summaries now render arrows and separators as UTF-8 instead of mojibake.

## [1.3.3] - 2026-07-26

### Added
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ npx realdone scan

Best for a quick first check. Default mode uses one Chromium worker, safe actions only, no AI, no account, and no database credential.

Its finite budget is coverage-balanced rather than “the first controls in DOM order”: RealDone represents different routes, action kinds, intents, and Enter/submit paths, then reports exactly what was selected or omitted. Use `--full` when you need broader safe coverage.

### 2. Record and verify an important flow

```bash
Expand Down
2 changes: 2 additions & 0 deletions docs/ADVANCED.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Quick scan uses 8 pages, 24 actions, and a two-minute budget. Full safe audit ra
realdone scan --full
```

The 24-action quick budget is not “the first 24 controls.” RealDone deterministically balances the selection across discovered routes, action kinds, intents, and activation paths such as Enter-submit. Runnable actions are selected before actions already known to be denied by policy or environment checks; destructive and session-ending controls still remain last. The JSON and HTML reports show eligible, selected, omitted, and represented-route counts so partial coverage is explicit.

Explicit `--max-pages`, `--max-actions`, `--max-duration`, or policy budgets override the corresponding preset. If any budget is exhausted, the report is marked `truncated`; RealDone never presents a partial scan as complete.

## Project discovery, managed runtime, and environment validity
Expand Down
4 changes: 4 additions & 0 deletions docs/COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,7 @@ Chromium accepts a custom executable through `--browser-path`. Firefox and WebKi
The requested Playwright browser is downloaded automatically on first use when it is missing. Set `REALDONE_SKIP_BROWSER_INSTALL=1` to disable bootstrap and install browsers manually.

Codex and Claude Code integrations are command presets, not embedded SDKs. RealDone verifies the current documented non-interactive argument contracts; installed agent authentication and provider availability remain the user's responsibility.

## Unreleased report compatibility

Coverage-balanced action selection adds an optional `completeness.selection` object to `scan.json`. Existing `schemaVersion: "1.0"` fields and meanings are unchanged, old reports remain readable, and validators use passthrough/additive compatibility. Consumers may ignore the new telemetry or use it to distinguish discovered, selected, omitted, and route-represented action coverage.
2 changes: 2 additions & 0 deletions docs/PERFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Quick scan remains intentionally light: one Chromium worker, no provider/databas

`scan --full` raises the default safe budgets to 100 pages, 500 actions, and 30 minutes, enables deep persistence and trace-on-failure, but never enables destructive or external effects. Explicit budget flags or policy values still win, and exhausted budgets set `truncated`.

Finite scan budgets use the versioned `coverage-balanced-v1` selector. It spreads selected actions across routes and semantic action/activation classes instead of allowing one control-heavy page to consume the budget, prioritizes runnable actions over known denials, and preserves destructive/logout ordering. `scan.json` records additive selection telemetry and the HTML report explains any omitted actions; increase `--max-actions` or use `--full` when the represented coverage is insufficient for the release decision.

Baseline, regression, browser-matrix, and post-agent contract verification accept `--workers 1..16`. Results preserve deterministic input order; quick scan remains one worker because concurrent mutations against the same application can create misleading evidence. Content-addressed snapshot indexes use SHA-256 blobs so repeated states share one portable artifact while legacy per-finding snapshot JSON remains available.

Recorded verification can enforce an explicit budget:
Expand Down
3 changes: 3 additions & 0 deletions docs/PRODUCT_SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1392,13 +1392,16 @@ Full audit mới bật:
RealDone phải hỗ trợ:

* scan budget;
* coverage-balanced action selection: khi budget không đủ cho mọi action, không được chỉ lấy các phần tử đầu tiên theo DOM/discovery order; phải ưu tiên action chạy được và tối đa hóa độ phủ route, kind, intent và activation path một cách deterministic;
* incremental scan;
* affected-flow selection;
* snapshot deduplication;
* trace only on failure;
* parallel worker giới hạn;
* timeout rõ ràng.

Report phải ghi rõ số action đủ điều kiện, đã chọn, bị bỏ do budget, số route được đại diện và strategy version. Action đã biết chắc sẽ bị policy/environment chặn không được chiếm chỗ trước action có thể chạy trong cùng safety tier.

---

# 31. UX Quality
Expand Down
6 changes: 4 additions & 2 deletions docs/PRODUCT_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,18 @@ This file reports current evidence against the normative [`PRODUCT_SPECIFICATION

Release `v1.3.3` additionally ran ten pinned MIT projects and detected 10/10 injected no-op Create buttons as `NO_EFFECT / RD002`; commit, license, clean-control, run, and report hashes are in [`release/evidence/case-studies-10`](../release/evidence/case-studies-10/README.md). The five normative external cases were rerun and validator-bound to fingerprint `68d4a8cb…`, including PostgreSQL CRUD cleanup, Supabase Data API, Level 7 roles, and a real Codex MCP RD901 regression/repair cycle. Hosted run [`30189340006`](https://github.com/datzle123/RealDone/actions/runs/30189340006) passed all 15 gates across Windows/macOS/Linux and GitHub signed the aggregate for the released source commit.

The unreleased Phase I hardening adds coverage-balanced quick-scan scheduling and additive selection telemetry. Candidate fingerprint `f3f65840…` passed local type/unit/build (140 tests with 2 expected service-dependent skips), audit, full Chromium smoke, pack, installed-tarball smoke, and fresh repository-bound runs for all five normative external cases. TodoMVC exercised both Enter paths; Actual Budget preserved its safe history-dependent `UNCERTAIN`; Conduit passed untruncated SQLite and PostgreSQL 17 + Supabase Data API scans plus real create/update/delete cleanup; Pocket Ledger passed six visible actions, Level 7 roles, and a new Codex MCP RD901 regression/repair cycle. Aggregating those artifacts with the successful Linux/macOS/Windows attestations from candidate run `30193615824` passes 15/15 locally. Phase I is still not released until a fresh hosted run validates and signs the evidence commit itself.

| Specification area | Status | Evidence currently in the repository | Remaining normative gap |
| --- | --- | --- | --- |
| §4 Quick scan | IMPLEMENTED | `scan`, browser execution, report, reload checks, fixture smoke | Broader action and environment coverage is tracked below |
| §4 Quick scan | IMPLEMENTED | `scan`, browser execution, report, reload checks, fixture smoke, and deterministic coverage-balanced selection with explicit eligible/selected/omitted/route telemetry | Broader action and environment coverage is tracked below |
| §4 Deep scan | IMPLEMENTED | hard reload, new tab, clean context, versioned logout/login rehydration, managed app restart, API read-back, roles, trace/video, source snapshots, and explicit causal read-only provider confirmation; hosted run `29940370416` passed Windows/macOS/Linux | Application-specific adapters may extend the implemented orchestration without changing its evidence contract |
| §4 Record and verify | IMPLEMENTED | bounded semantic recording and deterministic verification for navigation, click/fill/select/check/keypress/upload/rich-text/drag, popup/download outcomes, masked rrweb, env-only secrets/files and non-ordinal locator fallbacks | Widget-specific adapters may extend recording without changing the implemented contract |
| §4 Baseline/regression | IMPLEMENTED | baseline, affected-flow selection, first-class `EXPECTED_CHANGE`/`REGRESSION` outcomes, RD901–RD905 classification, CI diff and Playwright export | Additional domain-specific diff adapters may extend the implemented contract |
| §4 Coding-agent verification | IMPLEMENTED | shared-core MCP tools retain workspace/action confinement, sealed contracts/baselines, final post-build attribution and fail-closed affected selection; authenticated Codex Desktop `0.143.0` produced a green baseline, selected 1 real RD901 regression, repaired the app without changing the contract, then selected 1 flow with 0 regressions; SHA-256-bound cycle artifacts are parsed by the release validator | New agent clients must preserve the implemented evidence boundary; agent prose is never verification evidence |
| §6 Project Discovery/Runtime Manager | IMPLEMENTED | `init` profiles single-package/monorepo projects; npm fallback without lock metadata; project-local Python virtual environments; zero-config static HTML plus conventional Node, Django/FastAPI/Flask, Laravel/PHP, Rails, ASP.NET Core, Spring Boot, Deno, Go and Rust managed runtimes; occupied-port rejection, development/production/Docker health-checks, logs, bounded restarts and process cleanup; installed-tarball scans start, verify and stop metadata-free static and npm projects in hosted run `29977292441` | Custom runtimes use an explicit HTTP URL; new ecosystem hints must add cross-platform broken/control and package evidence |
| §7 Environment Health Gate | IMPLEMENTED | main/route document, critical asset/content-type, bootstrap/render, health-endpoint and auth-state checks; separate `environment.json`; broken/control fixtures and TodoMVC defect copy | Additional ecosystem-specific diagnostics may be added without changing the implemented fail-closed contract |
| §8–11 Discovery, classification, data, executor | IMPLEMENTED | forms/links/buttons/native controls, Enter, hover/context, lazy-scroll, popup/download, opt-in same-origin iframe, complex-action recording boundaries, constraint-aware canaries, network-idle/stale/retry safety | Framework-specific discovery adapters may broaden coverage without changing the implemented safe-execution contract |
| §8–11 Discovery, classification, data, executor | IMPLEMENTED | forms/links/buttons/native controls, Enter, hover/context, lazy-scroll, popup/download, opt-in same-origin iframe, complex-action recording boundaries, constraint-aware canaries, network-idle/stale/retry safety, and budget selection that avoids route/semantic starvation and known-denial budget waste | Framework-specific discovery adapters may broaden coverage without changing the implemented safe-execution contract |
| §12–13 Evidence and snapshots | IMPLEMENTED | URL, redacted semantic DOM/control hashes, cookie hashes, local/session storage, bounded IndexedDB metadata, request/response, WebSocket frames, console/page error, upload/download digests, screenshots, trace/video/timeline, value-free source diffs, and redacted automatic provider evidence/artifacts; reproductions retain value-free provider requirements and replay requires fresh exact causal confirmation | New evidence adapters must preserve the implemented redaction, linkage and replay contract |
| §14 Persistence | IMPLEMENTED | immediate/reload/hard-reload/new-tab/clean-context/logout-login/app-restart/API read-back strategies; all seven scopes are executable through browser smoke, contract Level 6/7 checks and deterministic scope tests | Additional application-specific adapters may extend confirmation without changing the implemented scope contract |
| §15–17 Verdicts and evidence hierarchy | IMPLEMENTED | scan verdicts, regression `EXPECTED_CHANGE`/`REGRESSION` outcomes, separate `ENVIRONMENT_INVALID`/`BLOCKED`, priority resolution and Levels 0–7 across scan/contract/baseline/report evidence | New adapters must preserve this implemented hierarchy |
Expand Down
8 changes: 4 additions & 4 deletions docs/REAL_WORLD_VALIDATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,25 +48,25 @@ The Phase G rerun binds each compact evidence document to both the raw `scan.jso

| Pinned application | Environment | Observable result | Severe RealDone regressions |
| --- | --- | --- | --- |
| TodoMVC React | `VALID`, 3/3 assets | Enter mutation executed; known memory-only behavior remained `EPHEMERAL` with RD101/RD104/RD201 | 0 |
| TodoMVC React | `VALID`, 7/7 assets | both standalone Enter mutations executed; React remained `EPHEMERAL` and TypeScript React remained `BROWSER_LOCAL` | 0 |
| Actual Budget | `VALID`, 30/30 assets | 6 `VERIFIED`; one history-dependent missing target remained safely `UNCERTAIN` | 0 |
| Conduit + SQLite | `VALID`, untruncated | 17 `VERIFIED`; one generated-credential rejection remained safely `UNCERTAIN` | 0 |
| Conduit + PostgreSQL 17 + Supabase Data API | `VALID`, untruncated | 17 `VERIFIED`; signup was `SOURCE_OF_TRUTH_CONFIRMED` Level 6 through both adapters with the same added-row hash | 0 |
| Pocket Ledger | `VALID`, untruncated | all 6 visible actions `VERIFIED`, including focus-revealed skip link, multipart upload and non-empty CSV export | 0 |

The released `v1.3.3` engine fingerprint is `68d4a8cb4b087d702df15d609d0ee0f7c92efaf731fdccd0d726b462a9878abe`. Its repository-confined raw scans are stored under `release/evidence/raw/`; every primary case was `VALID` and untruncated. Compact documents are SHA-256-bound to the current engine, primary scans, supporting PostgreSQL CRUD scans, cleanup ledger, role verification and Codex cycle artifacts. The release gate requires semantic observable evidence for all nine §27 capability classes, not merely a minimum case count. Signed hosted run [`30189340006`](https://github.com/datzle123/RealDone/actions/runs/30189340006) passed all 15 gates across Windows/macOS/Linux for the released source and package path.
The current checked-in evidence is bound to Phase I candidate fingerprint `f3f65840e53777056fed4fc770853a522da85c46626f5bc910853515c266cc1a`. Every primary scan was `VALID` and untruncated. Compact documents bind the primary scans, PostgreSQL CRUD scans, cleanup ledger, Level 7 role verification and Codex cycle artifacts by SHA-256. Candidate-head run [`30193615824`](https://github.com/datzle123/RealDone/actions/runs/30193615824) already passed its Linux/macOS/Windows, Node 20/22, browser and package jobs; its first aggregate correctly failed because the previous external artifacts belonged to fingerprint `68d4a8cb…`. Combining those same successful platform attestations with the fresh candidate evidence now passes 15/15 locally, but Phase I remains unreleased until hosted CI validates and signs the evidence commit itself. Released `v1.3.3` and its signed run [`30189340006`](https://github.com/datzle123/RealDone/actions/runs/30189340006) remain the historical release qualification.

PostgreSQL qualification used the pinned upstream Conduit app with only a harness port mapping, native PostgreSQL 17.10, real create/update/delete row diffs and key-bounded cleanup. The Supabase source adapter read the same action through a local Supabase Data API-compatible PostgREST 14.15 endpoint; the official Supabase CLI was also checked, but its full Auth/Storage local stack could not start on this machine because Docker Desktop/WSL is absent. This evidence therefore claims the maintained Supabase Data API adapter boundary, not a hosted Supabase service.

Pocket Ledger was generated in a separate MIT repository by Codex, then exercised through the public scanner and a Level 7 role contract. Member state hid Reset and received API 403; admin state saw Reset. An authenticated Codex Desktop `0.143.0` session called RealDone MCP for a green baseline, produced a selected RD901 regression without changing the contract, repaired Increment, and finished with one selected contract and zero regressions. The release validator parses and cross-checks those bound artifacts instead of trusting the agent's final message.
Pocket Ledger was generated in a separate MIT repository by Codex, then exercised through the public scanner and a Level 7 role contract. Member state hid Reset and received API 403; admin state saw Reset. Authenticated Codex Desktop thread `019f8770-219f-7612-81eb-98670009f693` used Codex CLI `0.143.0` to call RealDone MCP for a green baseline, produced run `20260726T081932Z-0ea9` with one selected RD901 regression and an unchanged contract, repaired Increment, and finished in run `20260726T082050Z-ad9f` with one selected contract and zero regressions. The release validator parses and cross-checks those bound artifacts instead of trusting the agent's final message.

The intentional TodoMVC copy produced exactly 2 `CONTRADICTORY`, 1 `EPHEMERAL`, and 1 `NO_EFFECT` finding while its unbuilt Demo route remained `ENVIRONMENT_INVALID` instead of an app defect. The intentional Conduit copy produced its planted duplicate invoice as `BROKEN`/RD003, failed settings as `CONTRADICTORY`/RD001/RD302, fake delete as `CONTRADICTORY`/RD203/RD301, and inert button as `NO_EFFECT`/RD002.

The first current-engine Conduit rerun exposed an RD501 false positive: the public login page contained “Need an account?”, so an unchanged public state looked private after a real 404 login rejection. RD501 now requires an actual transition into private state or an explicit success claim. A broken fake-login control still triggers RD501, the public rejection control does not, and the real Conduit rerun returned to zero `CONTRADICTORY` findings.

The 2026-07-23 Actual Budget rerun then exposed a safety regression: its server target is a text input with an `https://example.com` placeholder rather than `type="url"`. RealDone initially filled a canary and clicked Connect, producing a DNS failure. URL-bearing placeholders now retain external-target classification; the rerun returned to 6 `VERIFIED`, 1 safe `UNCERTAIN`, 3 policy `SKIPPED`, and zero defects.

Machine-readable release inputs are in `release/external-cases.json` and `release/evidence/`; hosted run [`30189340006`](https://github.com/datzle123/RealDone/actions/runs/30189340006) passed all 15 gates, GitHub signed both aggregate JSON artifacts, and `gh attestation verify` accepted them for `v1.3.3`. The earlier `v1.3.2` package path remains qualified by hosted run [`29977292441`](https://github.com/datzle123/RealDone/actions/runs/29977292441).
Machine-readable candidate inputs are in `release/external-cases.json` and `release/evidence/`. For the released package, hosted run [`30189340006`](https://github.com/datzle123/RealDone/actions/runs/30189340006) passed all 15 gates, GitHub signed both aggregate JSON artifacts, and `gh attestation verify` accepted them for `v1.3.3`. Phase I receives its own signed run only after the candidate evidence commit passes hosted aggregation; it does not inherit the prior signature. The earlier `v1.3.2` package path remains qualified by hosted run [`29977292441`](https://github.com/datzle123/RealDone/actions/runs/29977292441).

### Actual Budget original application

Expand Down
Loading