Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
eedc84d
test(skills): pin approval routing for delegated sub-agent turns
anconina Aug 12, 2026
4c30dee
fix(skills): let delegated runs resolve an approval callback owner
anconina Aug 12, 2026
e376778
refactor(agent,daemon): route delegated-endpoint checks through one p…
anconina Aug 12, 2026
c9e57df
test(daemon): pin the verdict for a turn that died without finalizing
anconina Aug 12, 2026
82a303c
fix(daemon): stop recall_miss claiming turns that died before finalizing
anconina Aug 12, 2026
8283ac5
test(daemon,orchestrator): pin the quarantine resolution trail
anconina Aug 12, 2026
74c1919
fix(daemon,orchestrator): make a resolved announcement quarantine leg…
anconina Aug 12, 2026
acc4f10
test(agent): drive the block-message round trip with a quote-bearing …
anconina Aug 12, 2026
ffa4cc9
fix(agent): stop the breaker block message nesting inside itself
anconina Aug 12, 2026
036ac14
test(agent): pin one coherent re-spawn directive for unreachable tools
anconina Aug 12, 2026
f563f36
fix(core): emit one coherent re-spawn directive for unreachable tools
anconina Aug 12, 2026
eba3462
test(memory): pin hard-over-soft precedence in the session rollup
anconina Aug 12, 2026
7275995
fix(memory): stop a soft degradation downgrading a hard failure
anconina Aug 12, 2026
4feb7bb
test(agent): pin a delivered sub-agent answer as delivered, not failed
anconina Aug 12, 2026
ad6485b
fix(agent): announce a delivered sub-agent answer as delivered
anconina Aug 12, 2026
cd7c0be
test(agent): pin a wrong-script reply as delivered-degraded, not term…
anconina Aug 12, 2026
6a4bde3
fix(agent): deliver a wrong-script answer instead of killing the turn
anconina Aug 12, 2026
5337f82
test(orchestrator): pin an operator lever for quarantined announcements
anconina Aug 12, 2026
494e322
feat(orchestrator): give the dead-letter queue list and release
anconina Aug 12, 2026
543fcc5
feat(daemon,cli): add `comis quarantine` — list and release parked an…
anconina Aug 12, 2026
d10890f
fix(orchestrator): load from disk before listing quarantined announce…
anconina Aug 12, 2026
d8f2fdc
test(agent): pin the step-limit message to the knob that actually bound
anconina Aug 12, 2026
5e764a1
fix(agent,daemon): name the step ceiling that actually bound the run
anconina Aug 12, 2026
a07f4a9
test(core): pin a sub-agent step ceiling that research can finish under
anconina Aug 12, 2026
19d4f15
feat(core): raise the sub-agent step ceiling default to 300
anconina Aug 12, 2026
656ecd8
refactor(orchestrator,agent): keep two files under the production lin…
anconina Aug 12, 2026
8b8c59e
test(daemon): pin that a run waiting on its children is not stuck
anconina Aug 12, 2026
a462bfa
fix(daemon): exempt a run waiting on live children from the stuck sweep
anconina Aug 12, 2026
2eae9da
test(live): add a deep-research delegation stress target
anconina Aug 12, 2026
19daad5
test(live): add browser fallback to the deep-research target
anconina Aug 12, 2026
28c7fed
test(core): spawn rejection must suggest the narrow group, not full
anconina Aug 12, 2026
304f87d
fix(core): suggest the narrowest ceiling that reaches the required tools
anconina Aug 12, 2026
dbc9286
test(live): R-05 must reject a full-privilege answer to a reachabilit…
anconina Aug 12, 2026
6572a52
fix(agent): cancel orphaned children and stop blaming the timeout knob
anconina Aug 12, 2026
0033057
fix(agent): kill a tree deepest-first so the cascade cannot steal att…
anconina Aug 12, 2026
717c5d3
test(integration): DLQ success is INFO, not DEBUG
anconina Aug 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/agents/resilience.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,11 @@ persistent dead-letter queue backed by a JSONL file at
automatically, delivering all queued messages
- **What the operator sees:** `announcement:dead_lettered` events when entries
are queued, `announcement:dead_letter_delivered` events when they are
successfully delivered on retry
successfully delivered on retry, and an INFO log line per entry that clears
- **File lifetime:** the JSONL is a snapshot of the queue, not an append-only
log — it is rewritten on every change and removed once the queue drains. An
absent file means nothing is pending, so a quarantine WARN with no file means
that quarantine already resolved

The dead-letter queue ensures that transient failures do not cause permanent
message loss. Even during extended provider outages, announcements are preserved
Expand Down
32 changes: 31 additions & 1 deletion docs/agents/subagent-lifecycle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,20 @@ classification. For example, an error-classified failed run is rendered as
`Failed`, so delivery does not erase the reason recorded by status and
observability surfaces.

A run that reached the end of its work counts as **delivered**, not halted —
including one that finished `completed_with_tool_errors`, where some tool call
failed but the sub-agent still produced an answer. Such a run is announced as
`Completed (completed_with_tool_errors)`, and its tool failures remain visible
as degradation on the observability surfaces. Only the genuine halts (the step
ceiling, the no-progress loop guard, context exhaustion, a budget stop, a hard
error) are announced as failures.

The sub-agent's output is always rendered as the **result**, never as an error.
A degraded run that still produced an answer shows that answer, with the
degradation carried by the status label. The one exception is a run that
abandoned background work it had launched: there the response cannot be trusted
as the result, so the failure is reported instead.

## Objective reinforcement

When a sub-agent's conversation grows long enough to trigger the context
Expand Down Expand Up @@ -273,7 +287,7 @@ security:
agentToAgent:
enabled: true
allowAgents: ["coder", "researcher"]
subAgentMaxSteps: 50
subAgentMaxSteps: 300
subAgentToolGroups: ["coding"]
subagentContext:
# -- Spawn limits --
Expand Down Expand Up @@ -340,6 +354,22 @@ returned by `session.run_status` and `subagent.wait`:
| `failed` | An unrecoverable error occurred during execution |
| `killed` | The sub-agent was force-terminated. The kill is attributed via `killedBy` (`parent` \| `health_monitor` \| `operator` \| `system`) on the failure record, the `subagent:killed` event, and the run status the parent polls -- a daemon health-monitor stuck-kill (no observed tool/LLM progress past `stuckKillThresholdMs`) never reads as a parent kill, and it delivers an LLM-free failure notification to the announce channel |
| `watchdog_timeout` | The sub-agent exceeded its wall-clock timeout and was force-failed by the watchdog timer. See [Resilience](/agents/resilience#sub-agent-watchdog). |

### Orphaned children are cancelled with their parent

When a run reaches a terminal state **abnormally** -- any end reason other than
`completed` -- every child it spawned that is still `running` or `queued` is
killed with `killedBy: "system"`, and the failure notification names the cause
(`its parent run ended (<endReason>) with no reader left for this result`).

A parent that ends abnormally can never consume what its children return, so
without the cascade they keep spending tokens on a result nobody will read. The
cascade walks the spawn tree once per node, bounded by `maxSpawnDepth`.

A **cleanly completed** parent cancels nothing. Background delegation is a
supported pattern: a child that announces to its own channel is expected to
outlive the turn that spawned it.

| `ghost_sweep` | The sub-agent was stuck in "running" state past the grace period and was force-failed by the periodic ghost sweep. See [Resilience](/agents/resilience#ghost-sweep). |

`swept` is reserved for result-file retention cleanup after `resultRetentionMs`;
Expand Down
4 changes: 3 additions & 1 deletion docs/operations/data-directory.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,9 @@ Because it carries the gateway token, the handle is written with mode `0o600` (o

### `dead-letters.jsonl`

Append-only JSONL of announcement-delivery failures that exhausted in-process retries (channel down, bot blocked, etc.). The delivery system replays these on provider recovery and expires entries after one hour. Only created when at least one announcement has been dead-lettered.
A JSONL **snapshot** of announcement-delivery failures that exhausted in-process retries (channel down, bot blocked, etc.), plus any parked parent-decision reservations. The delivery system replays these on provider recovery and expires entries after one hour.

The file tracks the queue rather than accumulating: every change rewrites it atomically, and it is **removed** as soon as the queue drains to zero. It therefore exists only while something is actually pending — if the daemon logged a quarantine WARN and this file is absent, the quarantine has since **resolved**, not vanished. Look for the matching resolution line (`Committed dead-letter operation removed without replay`, or `Dead-letter entry delivered successfully`) rather than treating the announcement as lost.

**Written by:** the announcement delivery subsystem.
**Safe to delete?** Yes. Pending re-deliveries are dropped; entries auto-expire after one hour anyway.
Expand Down
7 changes: 7 additions & 0 deletions docs/operations/multilingual.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,13 @@ Three config keys carry the multilingual surface; all are documented in the
URLs, and tool results preserved), and streaming consumers receive only the finalized response
while enforcement is active.

If the repair still cannot reach the expected script, **the answer is delivered anyway** — in
the script the model produced. A wrong writing system is a presentation defect, not an execution
failure: the turn answered the question, so discarding it would trade a cosmetic problem for a
total loss, and the degradation matrix below applies here too. The mismatch stays visible as a
WARN naming the resolver tier that set the target, and as an
`execution:recovery_attempted` event with `reason: "locale_fidelity"` and `succeeded: false`.

A contradicting script is detected two ways. **Bulk** foreign prose has to clear a share and a
unit floor, so that legitimately quoting a foreign name or term is not treated as a violation.
Separately, a single **token that welds two different non-Latin scripts** counts on its own, with
Expand Down
28 changes: 27 additions & 1 deletion docs/reference/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ icon: "terminal"

**What this is for:** the `comis` command is how you set up your daemon, manage agents and channels, browse memory, audit security, and operate the running system from the terminal. **Who it's for:** operators and developers running Comis on their own machine or a server.

Comis currently ships **32 top-level command groups**. Some commands call the
Comis currently ships **33 top-level command groups**. Some commands call the
running daemon over JSON-RPC; others inspect or update local state. The built-in
`--help` output is authoritative for the installed version.

Expand Down Expand Up @@ -1645,6 +1645,32 @@ comis cost export --provider openai --model gpt-4o-mini

---

### `comis quarantine`

Inspect and release **quarantined background-task announcements** — a completed task's outcome that the runtime could not prove reached its reader.

Nothing drains a quarantined announcement automatically: re-sending one whose delivery is unproven risks telling a user the same thing twice, so the runtime holds it and waits for a human. This command is that decision point. Backed by `ObsQuarantineListContract` (`obs.quarantine.list`) and `ObsQuarantineReleaseContract` (`obs.quarantine.release`), both admin-only.

```bash
# What is waiting, oldest first
comis quarantine list
comis quarantine list --format json

# Decide one
comis quarantine release <id> --outcome delivered
comis quarantine release <id> --outcome discarded
```

Use `delivered` when you have confirmed out of band that the reader already has the message, and `discarded` when it is not worth sending. Both remove the item — the queue exists to hold an *undecided* announcement — and the distinction is kept for the audit trail. Releasing an id that is already gone reports that plainly instead of failing, so re-running the command is safe.

The listing is **content-free**: it shows ids, route, timing, attempt count, the failure reason and the announcement's *length*, never its text. An operator deciding whether a reader was already informed needs the route and the reason, not the message body.

<Warning>
There is deliberately no offline mode. While the daemon is running it is the only authority over the queue: it holds the state in memory and rewrites `~/.comis/dead-letters.jsonl` from it on the next persist, so editing that file under a live daemon is silently undone. Always go through this command; see [`dead-letters.jsonl`](/operations/data-directory#dead-letters-jsonl).
</Warning>

---

### `comis explain`

Assemble an `IncidentReport` for a single agent session — a bounded, causal post-mortem (outcome, cost, per-tool stats, normalized failures, circuit-breaker timeline, large-result offloads, recall health — including degraded/failed recall lanes — session-wide activity-finalize tallies, and a deterministic likely root cause). The report is derived from log evidence only; no LLM is invoked, so the same session always yields the same verdict. Backed by `ObsExplainContract` (`obs.explain`).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/config-yaml.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1582,7 +1582,7 @@ Security configuration for log redaction, audit logging, permissions, action con
| `allowAgents` | `string[]` | `[]` | Allowed agent IDs for sub-agents (empty = all) |
| `subAgentRetentionMs` | `number` | `3600000` | Retention for completed sub-agent sessions (1 hour) |
| `waitTimeoutMs` | `number` | `60000` | Default timeout for wait mode (60 seconds) |
| `subAgentMaxSteps` | `number` | `50` | Default max steps for sub-agent execution |
| `subAgentMaxSteps` | `number` | `300` | Ceiling on tool-execution steps for a sub-agent run. A spawn's own `max_steps` is **clamped to this value** and cannot exceed it, so this is the only setting that raises the limit. Sized for delegated research, which spends a step per search and per fetch; lower it to tighten the runaway-loop bound. This is not a cost control — see `observability.spend` and the per-spawn token budget. Not to be confused with `agents.<id>.maxSteps`, which governs top-level turns only |
| `subAgentToolGroups` | `enum[]` | `["coding"]` | Default tool profile groups: `minimal`, `coding`, `messaging`, `supervisor`, `full` |
| `subAgentMcpTools` | `enum` | `"inherit"` | MCP tool inheritance: `inherit` or `none` |
| `tokenBudget` | `number \| null` | `null` | Per-spawn token budget for graph sub-agents. `null` (default) means inherit the graph share -- `budget.max_tokens` divided by the total node count -- but **only when the graph sets a token budget**; otherwise sub-agents are unbounded (today's behavior, unchanged). A positive integer caps every graph node's sub-agent at that many tokens. A graph node's own per-node budget overrides this default. A breach fails that node (honoring the graph's `on_failure`). See [Execution Graphs: Token budgets](/agents/execution-graphs#token-budgets). |
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/security-model.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -881,7 +881,7 @@ Top-level `security` configuration.
| `agentToAgent.allowAgents` | `string[]` | `[]` | Allowed agent IDs for sub-agent spawning (empty = allow all) |
| `agentToAgent.subAgentRetentionMs` | `number` | `3600000` (1 hour) | Retention period for completed sub-agent sessions |
| `agentToAgent.waitTimeoutMs` | `number` | `60000` (60s) | Default timeout for wait mode |
| `agentToAgent.subAgentMaxSteps` | `number` | `50` | Default max steps for sub-agent execution |
| `agentToAgent.subAgentMaxSteps` | `number` | `300` | Ceiling on tool-execution steps for a sub-agent run. A spawn's own `max_steps` is **clamped to this value** and cannot exceed it, so this is the only setting that raises the limit. Sized for delegated research, which spends a step per search and per fetch; lower it to tighten the runaway-loop bound. This is not a cost control — see `observability.spend` and the per-spawn token budget. Not to be confused with `agents.<id>.maxSteps`, which governs top-level turns only |
| `agentToAgent.subAgentToolGroups` | `string[]` | `["coding"]` | Default tool profile groups for sub-agents |
| `agentToAgent.subAgentMcpTools` | `"inherit"` \| `"none"` | `"inherit"` | MCP tool inheritance policy for sub-agents |

Expand Down
43 changes: 42 additions & 1 deletion packages/agent/src/bridge/bridge-safety-controls.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
import { describe, it, expect, vi } from "vitest";
import type { SessionKey, TypedEventBus, ComisLogger } from "@comis/core";

import { checkLoopLimit, emitLoopAbort, emitStepLimitAbort, buildAbortRedirectMessage, checkSpendLimit, emitSpendAbort } from "./bridge-safety-controls.js";
import { checkLoopLimit, emitLoopAbort, emitStepLimitAbort, buildAbortRedirectMessage, checkSpendLimit, emitSpendAbort, resolveStepLimitDetails } from "./bridge-safety-controls.js";
import { createStepCounter } from "../executor/step-counter.js";
import type { ExecutionPlan } from "../planner/types.js";
import type { SpendGateOutcome } from "../budget/budget-guard.js";
import { SpendError, type SpendWarn } from "../budget/spend-accumulator.js";
Expand Down Expand Up @@ -437,3 +438,43 @@ describe("emitSpendAbort", () => {
expect((logger.warn as ReturnType<typeof vi.fn>).mock.calls[0][0].hint).toMatch(/observability\.spend/);
});
});

describe("resolveStepLimitDetails — the knob it names must be the one that bound", () => {
// Live: a deep-research sub-agent ran 18 searches and 21 fetches, hit the
// ceiling at step 51, and told the operator:
// "I stopped after 51 tool-execution steps because agents.default.maxSteps=50.
// Simplify the workflow or increase agents.default.maxSteps before retrying."
// `agents.default.maxSteps` defaults to 150 and does not govern sub-agents at
// all. The ceiling that actually bound was
// `security.agentToAgent.subAgentMaxSteps` (default 50), so an operator who
// followed the guidance would raise a setting with no effect and hit the same
// wall. Naming the WRONG knob is worse than naming none.
it("names the sub-agent ceiling when a delegated run is what hit the limit", () => {
const counter = createStepCounter(50);
for (let i = 0; i < 50; i++) counter.increment();

const details = resolveStepLimitDetails(counter, "default", {
bindingKnob: "security.agentToAgent.subAgentMaxSteps",
});

expect(details.bindingKnob).toBe("security.agentToAgent.subAgentMaxSteps");
expect(details.cap).toBe(50);
expect(details.stepsExecuted).toBe(50);
});

it("names the caller's own override when the spawn passed max_steps", () => {
const counter = createStepCounter(20);
const details = resolveStepLimitDetails(counter, "default", {
bindingKnob: "sessions_spawn(max_steps)",
});

expect(details.bindingKnob).toBe("sessions_spawn(max_steps)");
});

it("still names the agent setting for an ordinary top-level turn", () => {
// Regression guard: the default path is unchanged.
const counter = createStepCounter(150);
expect(resolveStepLimitDetails(counter, "researcher").bindingKnob)
.toBe("agents.researcher.maxSteps");
});
});
21 changes: 20 additions & 1 deletion packages/agent/src/bridge/bridge-safety-controls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,31 @@ export interface AbortRedirectDetails {
stepLimit?: StepLimitDetails;
}

/**
* Where a run's step ceiling came from, when it is not the agent's own setting.
*
* A delegated run is governed by `security.agentToAgent.subAgentMaxSteps` (or by
* the `max_steps` its caller passed), NOT by `agents.<id>.maxSteps` — those are
* different keys with different defaults. Naming the agent setting for a
* sub-agent sends an operator to raise a knob that has no effect on the run that
* stopped, so the provenance travels with the counter rather than being guessed
* from the agent id. Mirrors `describeTimeoutKnob`, which already threads the
* timeout's source for exactly this reason.
*/
export interface StepLimitProvenance {
/** The config key or tool parameter that set the ceiling. */
readonly bindingKnob: string;
}

export function resolveStepLimitDetails(
stepCounter: StepCounter,
agentId: string,
provenance?: StepLimitProvenance,
): StepLimitDetails {
return {
bindingKnob: `agents.${agentId}.maxSteps`,
bindingKnob: provenance?.bindingKnob
?? stepCounter.getBindingKnob?.()
?? `agents.${agentId}.maxSteps`,
stepsExecuted: stepCounter.getCount(),
cap: stepCounter.getLimit?.() ?? stepCounter.getCount(),
};
Expand Down
11 changes: 0 additions & 11 deletions packages/agent/src/executor/executor-post-execution.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1228,17 +1228,6 @@ describe("tool-failure endReason and notice", () => {
.toBeLessThan(stripped.indexOf("synchronizeFinalAssistantResponse("));
});

it("source-grep — final model-status grounding reconciles locale failure before terminal classification", () => {
const stripped = readPostExecStripped();
const guardIndex = stripped.indexOf("enforceActiveModelSelfStatus(");
const reconcileIndex = stripped.indexOf("recoverFinalResponseLocaleFailure(");
const terminalIndex = stripped.indexOf("const finishReasonStr");

expect(guardIndex).toBeGreaterThanOrEqual(0);
expect(reconcileIndex).toBeGreaterThan(guardIndex);
expect(terminalIndex).toBeGreaterThan(reconcileIndex);
});

it("source-grep — the failure notice is built through the locale seam, not a literal", () => {
const stripped = readPostExecStripped();
// It used to be a bare English `[tool failure] <tool> reported an error`
Expand Down
23 changes: 0 additions & 23 deletions packages/agent/src/executor/executor-post-execution.ts
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,6 @@ import {
} from "./executor-response-filter.js";
import { BACKGROUND_POLLER_TOOL } from "../safety/background-failure-attribution.js";
import { parseContextExhaustionCause } from "../context-engine/errors.js";
import { recoverFinalResponseLocaleFailure } from "./prompt-runner/response-locale-enforcement.js";
import { buildSyntheticCriticDeps } from "./verification-gate-synth-deps.js";
import { resolveScaffoldDefaults } from "./scaffold-defaults.js";
import { generateCanaryToken } from "@comis/core";
Expand Down Expand Up @@ -1397,28 +1396,6 @@ export async function postExecution(params: PostExecutionParams): Promise<void>
},
});
}
if (
activeModelSelfStatus.corrected
&& recoverFinalResponseLocaleFailure(result, params.responseLocalePolicy)
) {
deps.logger.info(
{
step: "response-locale-recovery",
provider: params.provider,
modelId: params.modelId,
durationMs: 0,
},
"Final response guard satisfied the captured locale policy",
);
deps.eventBus.emit("execution:recovery_attempted", {
agentId: effectiveAgentId,
sessionKey: formattedKey,
reason: "locale_fidelity",
succeeded: true,
timestamp: deps.clock.now(),
});
}

// Derive effectiveFinishReason BEFORE the bookend log so it is visible there.
// The bookend must log effectiveFinishReason (not result.finishReason) so that
// an output_starved turn — which carries result.finishReason="stop" until promoted here —
Expand Down
3 changes: 2 additions & 1 deletion packages/agent/src/executor/prompt-assembly-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import type { TypedEventBus, WorkspaceFileName } from "@comis/core";
import {
formatSessionKey,
createMemoryRecallScope,
isDelegatedExecutionEndpoint,
scriptTokenFactor,
tryGetContext,
systemNowMs,
Expand Down Expand Up @@ -351,7 +352,7 @@ export async function assembleExecutionPrompt(params: PromptAssemblyParams): Pro
const isSubagentTurn = (
partition?.kind === "endpoint-conversation"
|| partition?.kind === "endpoint-conversation-principal"
) && partition.endpoint.channelType === "sub-agent";
) && isDelegatedExecutionEndpoint(partition.endpoint);
const memoryScope = turnScope === undefined
? err(new Error("RAG recall requires resolved turn authority"))
: createMemoryRecallScope(turnScope, !isSubagentTurn);
Expand Down
Loading
Loading