feat: a second agent checks the P1 and P2 drafts before the reviewer sees them

README.md

@@ -361,6 +361,9 @@ On first run it writes `~/.diffity/inbox/config.json`:
 | `agent.mcpAllow` | The exact MCP tool names the agent may call, e.g. `mcp__claude_ai_Atlassian__getJiraIssue`. Empty (the default) means no MCP servers at all. Editable from the page. |
 | `agent.extraArgs` | Appended verbatim to the built command, for a flag this table does not cover. |
 | `agent.maxBudgetUsd` | `--max-budget-usd` for one run; `null` leaves it uncapped. A run that hits it is a failed attempt with that reason. Editable from the page. |
+| `validate.model` | The model that checks the drafted P1 and P2 findings before you see them; `null` (the default) runs no second pass. Editable from the page. |
+| `validate.timeoutMinutes` | How long the check may take before it is stopped and the draft goes out unchecked (default 15). Editable from the page. |
+| `validate.maxBudgetUsd` | `--max-budget-usd` for the checking run; `null` leaves it uncapped. Editable from the page. |
 | `waitForCi` | Whether a pull request waits for green CI before an agent is spent on it (default false). Editable from the page. |
 | `prepareTimeoutMinutes` | How long one preparation may take before it's abandoned. Editable from the page. |
 | `maxPrepared` | How many prepared reviews may wait for you at once (default 5). Each preparation is an agent run; the rest of the queue waits until a prepared review is posted or dismissed. Editable from the page. |
@@ -371,7 +374,9 @@ The command itself is not configurable: the daemon builds `claude -p --output-fo
 
 What CI made of the head goes into the prompt: every check that reported, with its verdict, a count of the ones a workflow skipped, and the instruction not to install, build, typecheck, lint or test — CI has done that, and the checkout is the author's code — so the agent reasons from the source and says in its summary when a check failed or is still running. Each card carries the same verdict as a dot before its title: green for passing, red for failing, amber while checks are still going, and no dot at all when nothing has decided — no checks, or only skipped and cancelled ones. With `waitForCi` on, a pull request whose checks are still running waits in the queue ("waiting: CI running (3 checks)") and one whose checks failed is set aside ("CI failed: check-job, pr-mgmt-job") — both re-decided at every poll, so passing checks bring one back to the queue on their own, unlike a filter skip. ↑ prepares it whatever CI says, and a review already prepared for an older head stays openable while its refresh waits.
 
-Every agent run is logged: the pull request and head it was for, which pass it was (`prepare` for a preparation, `answer` for a question asked in the page), the models it actually used, how long it took, its turns, its cost and its tokens, and how it ended (`prepared`, `skipped`, `answered`, `failed`, `timeout`, `rate-limited`). `diffity inbox runs` prints that log with totals — the record of what the inbox costs you. A prepared review's card carries its own share of it, "· 8 min · $1.20", with each run behind that head listed on hover, and the page's footer keeps a running total for today and for the last seven days.
+Name a model in `validate.model` and a second pass checks the draft before you see it. It runs only when the draft holds a P1 or P2 — the findings that would hold up a merge — and only on those: for each one it reads the lines the finding points at and what they depend on, then leaves it, rewrites it (`agent amend`) if the text or the severity is off, or dismisses it (`agent dismiss`) if it does not hold; it adds a finding only where checking one of these revealed another, and amends the general summary when its verdict or counts no longer stand. It gets no review skill and is told not to re-review the diff, and like the drafting pass it runs without the forge's credentials and behind the same deny list. A pass that times out, ends without its `VALIDATED` line, hits its budget or runs into your session limit does not lose you the review: the draft is exported as it stands and its card says "1 P1 · unchecked", so you know the findings are the first pass's alone. When a checking model is set, questions asked in the page are answered with it too — a question is about a finding, which is that model's job.
+
+Every agent run is logged: the pull request and head it was for, which pass it was (`prepare` for a preparation, `validate` for a check of its findings, `answer` for a question asked in the page), the models it actually used, how long it took, its turns, its cost and its tokens, and how it ended (`prepared`, `skipped`, `validated`, `answered`, `failed`, `timeout`, `rate-limited`). `diffity inbox runs` prints that log with totals — the record of what the inbox costs you. A prepared review's card carries its own share of it, "· 8 min · $1.20", with each run behind that head listed on hover, and the page's footer keeps a running total for today and for the last seven days.
 
 A run that ends on your Claude session limit is not the pull request's fault, so it is waited out rather than retried: the row goes back in the queue as "waiting: Claude session limit until 14:00", no failed attempt is counted against it, and no further preparation starts until the limit lifts. The reset time is read out of the agent's own message ("resets 2pm (Europe/Stockholm)"), or set half an hour ahead when the message names none. Polling and reconciling carry on meanwhile, so the page stays current and says how long the pause has left; the pause is kept with the inbox, so restarting the daemon does not spend another run rediscovering the limit.
 

packages/api/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@diffity/api",
-  "version": "0.10.24",
+  "version": "0.10.25",
   "private": true,
   "type": "module",
   "main": "./dist/index.js",

packages/cli/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@naturalcycles/diffity",
-  "version": "0.10.24",
+  "version": "0.10.25",
   "description": "Agent-agnostic, GitHub-style diff viewer and code review tool with a live agent loop",
   "type": "module",
   "bin": {

packages/cli/src/inbox/config.ts

@@ -24,6 +24,19 @@ export interface AgentConfig {
   maxBudgetUsd: number | null;
 }
 
+/**
+ * The second pass over a drafted review: a stronger model reads the P1 and P2 findings against the
+ * code and amends or dismisses the ones that do not hold. Off until a model is named.
+ */
+export interface ValidateConfig {
+  /** The model that checks the drafted findings; null runs no second pass at all. */
+  model: string | null;
+  /** How long the check may take before the agent is stopped and the draft goes out unchecked. */
+  timeoutMinutes: number;
+  /** `--max-budget-usd` for the checking run; null leaves it uncapped. */
+  maxBudgetUsd: number | null;
+}
+
 export interface InboxConfig {
   /** How often GitHub is asked; well inside its limits at a handful of calls per tick. */
   pollMinutes: number;
@@ -49,6 +62,7 @@ export interface InboxConfig {
    */
   alertPaths: string[];
   agent: AgentConfig;
+  validate: ValidateConfig;
   /** Whether a pull request waits for its CI to pass before an agent is spent on it. */
   waitForCi: boolean;
   prepareTimeoutMinutes: number;
@@ -75,6 +89,7 @@ export const DEFAULT_INBOX_CONFIG: InboxConfig = {
   alertWhen: '',
   alertPaths: [],
   agent: { model: null, effort: null, mcpAllow: [], extraArgs: [], maxBudgetUsd: null },
+  validate: { model: null, timeoutMinutes: 15, maxBudgetUsd: null },
   waitForCi: false,
   prepareTimeoutMinutes: 30,
   maxPrepared: 5,
@@ -107,7 +122,7 @@ export function parseInboxConfig(raw: unknown, source = 'inbox config'): InboxCo
     throw new Error(`${source} must be a JSON object`);
   }
   const obj = raw as Record<string, unknown>;
-  const config: InboxConfig = { ...DEFAULT_INBOX_CONFIG, alertPaths: [], agent: defaultAgent() };
+  const config: InboxConfig = { ...DEFAULT_INBOX_CONFIG, alertPaths: [], agent: defaultAgent(), validate: defaultValidate() };
 
   if (obj.prepare !== undefined) {
     throw new Error(`${source}: "prepare" was replaced by the "agent" block — delete it (the built-in command applies) and put extra flags in agent.extraArgs`);
@@ -145,6 +160,9 @@ export function parseInboxConfig(raw: unknown, source = 'inbox config'): InboxCo
   if (obj.agent !== undefined) {
     config.agent = parseAgentConfig(obj.agent, source);
   }
+  if (obj.validate !== undefined) {
+    config.validate = parseValidateConfig(obj.validate, source);
+  }
   if (obj.waitForCi !== undefined) {
     if (typeof obj.waitForCi !== 'boolean') {
       throw new Error(`${source}: waitForCi must be true or false`);
@@ -213,8 +231,31 @@ function parseAgentConfig(raw: unknown, source: string): AgentConfig {
   return agent;
 }
 
+function defaultValidate(): ValidateConfig {
+  return { ...DEFAULT_INBOX_CONFIG.validate };
+}
+
+function parseValidateConfig(raw: unknown, source: string): ValidateConfig {
+  if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) {
+    throw new Error(`${source}: validate must be a JSON object`);
+  }
+  const obj = raw as Record<string, unknown>;
+  const validate = defaultValidate();
+
+  if (obj.model !== undefined && obj.model !== null) {
+    validate.model = text(obj.model, 'validate.model', source);
+  }
+  if (obj.timeoutMinutes !== undefined) {
+    validate.timeoutMinutes = positive(obj.timeoutMinutes, 'validate.timeoutMinutes', source);
+  }
+  if (obj.maxBudgetUsd !== undefined && obj.maxBudgetUsd !== null) {
+    validate.maxBudgetUsd = positive(obj.maxBudgetUsd, 'validate.maxBudgetUsd', source);
+  }
+  return validate;
+}
+
 /** The settings the inbox page edits, kept in the config file beside the keys only the file holds. */
-export type InboxSettings = Pick<InboxConfig, 'filter' | 'alertWhen' | 'alertPaths' | 'maxPrepared' | 'pollMinutes' | 'live' | 'liveTimeoutMinutes' | 'prepareTimeoutMinutes' | 'waitForCi' | 'agent'>;
+export type InboxSettings = Pick<InboxConfig, 'filter' | 'alertWhen' | 'alertPaths' | 'maxPrepared' | 'pollMinutes' | 'live' | 'liveTimeoutMinutes' | 'prepareTimeoutMinutes' | 'waitForCi' | 'agent' | 'validate'>;
 
 /**
  * Writes the page-editable settings into the config file, leaving every other key as the reviewer

packages/cli/src/inbox/daemon.ts

@@ -78,7 +78,7 @@ export function settingsHost(config: InboxConfig, configPath: string | undefined
       filter: config.filter, alertWhen: config.alertWhen, alertPaths: config.alertPaths,
       maxPrepared: config.maxPrepared, pollMinutes: config.pollMinutes,
       live: config.live, liveTimeoutMinutes: config.liveTimeoutMinutes, prepareTimeoutMinutes: config.prepareTimeoutMinutes,
-      waitForCi: config.waitForCi, agent: config.agent,
+      waitForCi: config.waitForCi, agent: config.agent, validate: config.validate,
     }),
     update: settings => {
       // The config object is the one the tick, the prepares and the opens read from, so the change
@@ -135,6 +135,7 @@ export async function runDaemon(
     get waitForCi() { return config.waitForCi; },
     get alertPaths() { return config.alertPaths; },
     get agentModel() { return config.agent.model; },
+    get validateModel() { return config.validate.model; },
     pauseUntil: (until: string) => {
       store.pauseUntil(until);
       log(`preparing paused until ${localHhMm(until)} — Claude session limit`);

packages/cli/src/inbox/page.ts

@@ -144,6 +144,9 @@ export function inboxPage(): string {
         <option value="max">max</option>
       </select></label>
       <label>Budget per run ($)<input id="agentMaxBudgetUsd" type="number" min="0.5" step="0.5" placeholder="uncapped"></label>
+      <label>Checking model<input id="validateModel" type="text" placeholder="off"></label>
+      <label>Checking timeout (minutes)<input id="validateTimeoutMinutes" type="number" min="1" step="1"></label>
+      <label>Checking budget per run ($)<input id="validateMaxBudgetUsd" type="number" min="0.5" step="0.5" placeholder="uncapped"></label>
       <label class="check"><input id="live" type="checkbox"> Park a live agent on opened reviews</label>
       <label class="check"><input id="waitForCi" type="checkbox"> Hold a pull request until its CI has passed</label>
     </div>
@@ -365,13 +368,18 @@ export function inboxPage(): string {
       el('agentEffort').value = agent.effort || '';
       el('agentMcpAllow').value = (agent.mcpAllow || []).join('\\n');
       el('agentMaxBudgetUsd').value = agent.maxBudgetUsd == null ? '' : agent.maxBudgetUsd;
+      const validate = settings.validate || {};
+      el('validateModel').value = validate.model || '';
+      el('validateTimeoutMinutes').value = validate.timeoutMinutes;
+      el('validateMaxBudgetUsd').value = validate.maxBudgetUsd == null ? '' : validate.maxBudgetUsd;
     } catch (err) {
       el('settings-status').textContent = 'settings could not be loaded';
     }
   }
 
   async function saveSettings() {
     const budget = el('agentMaxBudgetUsd').value.trim();
+    const checkBudget = el('validateMaxBudgetUsd').value.trim();
     const next = {
       filter: el('filter').value,
       alertWhen: el('alertWhen').value,
@@ -390,6 +398,11 @@ export function inboxPage(): string {
         extraArgs: (settings.agent && settings.agent.extraArgs) || [],
         maxBudgetUsd: budget === '' ? null : Number(budget),
       },
+      validate: {
+        model: el('validateModel').value.trim() || null,
+        timeoutMinutes: Number(el('validateTimeoutMinutes').value),
+        maxBudgetUsd: checkBudget === '' ? null : Number(checkBudget),
+      },
     };
     const res = await fetch('/api/settings', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(next) });
     if (!res.ok) {