Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,14 @@ LOG_LEVEL=info
# DATABASE_SSL=false
# DATABASE_CA_CERT=/path/to/ca.pem

# --- Optional: Router ---

# Settle window (ms) for coalescing PM status-change webhooks into one dispatch.
# Any dispatch for the same project + work item inside the window supersedes the
# pending one (across agent types); the ack comment is deferred to job fire time.
# Default 10000. 0 disables. The legacy name PM_CREATE_COALESCE_WINDOW_MS is accepted.
# PM_COALESCE_WINDOW_MS=10000

# --- Optional: Security ---

# AES-256-GCM encryption key for credentials stored in the database.
Expand All @@ -48,7 +56,7 @@ LOG_LEVEL=info

# OAuth token for Claude Code SDK (Claude Max subscription).
# Generate with: claude setup-token
# See CLAUDE.md for full setup instructions.
# See docs/getting-started.md ("Choose Agent Engine") for setup.
# CLAUDE_CODE_OAUTH_TOKEN=

# --- Optional: Monitoring ---
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ All notable user-visible changes to CASCADE are documented here. The format is l

### Documentation

- **`CLAUDE.md` / `AGENTS.md` cut from 217 lines plus 95 KB of `@`-imports to ~75 lines; path-scoped notes moved to `docs/areas/`.** The root file is loaded by Claude Code and `cat`-injected through CASCADE's `contextFiles` context step (inline only under `CONTEXT_OFFLOAD_CONFIG.inlineThreshold`), so it now carries only universal commands, gotchas, hard invariants, an environment pointer and a "read this before touching that area" table. Per-area rules live in `docs/areas/{pm-integrations,router-dispatch,agents,backends}.md` (imperatives + links, ≤ 60 lines each); mechanism stays in `docs/architecture/`; env vars are catalogued in `.env.example`. Facts whose only home was `CLAUDE.md` were placed first: PR checkout by `refs/pull/N/head` → `01-services`; review-context budget and debugging (`REVIEW_DIFF_CONTEXT_TOKEN_LIMIT`, `PR context prepared`, informational CI status) → `03-trigger-system`; `claimReviewDispatch`, `UnrecoverableError`, the reviewer-persona check and `formatCrashReason` → `03`/`10-resilience`; integration-DB discovery order → `tests/README.md`; `PM_COALESCE_WINDOW_MS` → `.env.example`. The stale `## Git hooks` paragraph (it described an integration pre-push that never existed) is replaced with the real `lefthook.yml` contract. `tests/unit/architecture-docs.test.ts` now enforces the budget — ≤ 200 lines, under half of `CONTEXT_OFFLOAD_CONFIG.inlineThreshold`, no `@` imports, no ticket IDs / spec numbers / dates in `CLAUDE.md` or `docs/areas/`, every area doc ≤ 60 lines with an `**Applies to:**` scope line and linked from the root — and `tests/unit/repo-hygiene.test.ts` pins the shared root/`web/` Zod major. `src/integrations/README.md` lost its spec-changelog preamble and the "Provider migration status" / "Post-spec-010/011/012" tables (their durable facts were folded into the current-state sections; history lives here); the JIRA `authType` contract now has one home there, with `08-config-credentials` and `getting-started` pointing at it. `SECURITY.md`, `README.md` and `CONTRIBUTING.md` no longer point at `CLAUDE.md` for reference material. The `documentation-maintenance` prompt partial routes agents to the same homes; DB-seeded partials shadow disk, so deployment still requires an explicit `npm run db:seed-prompts`.

- **Friction reporting is now documented for operators and provider contributors.** Architecture docs cover the optional PM Friction slot (`lists.friction` for Trello, `statuses.friction` for JIRA/Linear), `ReportFriction`, and `cascade-tools pm report-friction --details-file -`. The integration guide explains that friction reports use existing provider `createWorkItem` plus optional `moveWorkItem`, so providers do not need a new adapter method or a DB-backed friction index. Resilience docs describe the JSONL sidecar/outbox retry path, missing-slot behavior, and non-blocking drain failures. See Trello card [Rvv7VVd5](https://trello.com/c/69ff6af3bc5c526cc5faa2d4).

- **Trigger architecture docs now describe the migrated trigger contracts.** Added guidance for canonical `TRIGGER_EVENTS`, shared PM/GitHub result builders, first-match dispatch, structured skip vs bare `null`, no-agent results, deferred bare-job re-checks, router outcome decision reasons, PM coalescing, capacity scope, dispatch failure compensation, and wedged-lock diagnostics. Migration note for future trigger contributors: new handlers should import event constants, use the shared builders, return structured skips for claimed-but-non-dispatched events, and reserve bare `null` for "continue to later handlers." See Trello card [qUbPtALY](https://trello.com/c/69fe2a950699baaf91688a5b).
Expand Down
251 changes: 55 additions & 196 deletions CLAUDE.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ docs: update self-hosting guide
refactor(agents): extract shared prompt builder
```

This is enforced by commitlint via lefthook pre-commit hooks.
This is enforced by commitlint via the lefthook `commit-msg` hook.

## Pull Request Workflow

Expand All @@ -117,7 +117,7 @@ This is enforced by commitlint via lefthook pre-commit hooks.

## Project Structure

See [CLAUDE.md](./CLAUDE.md) for a detailed architecture overview. Key directories:
See [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md) for a detailed architecture overview. Key directories:

- `src/router/` — Webhook receiver (enqueues jobs to Redis)
- `src/triggers/` — Event handlers (Trello, JIRA, Linear, GitHub)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ cascade projects update <id> --rebuild-worker-image # rebuild against a ref
cascade projects update <id> --clear-dockerfile # revert to the global default
```

For deeper documentation on all of these topics, see [CLAUDE.md](./CLAUDE.md).
For deeper documentation on all of these topics, see [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md) and [Getting Started](./docs/getting-started.md).

---

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Please include:

Cascade incorporates several security measures:

- **Credential encryption at rest**: AES-256-GCM encryption for all stored credentials when `CREDENTIAL_MASTER_KEY` is configured. See [CLAUDE.md](./CLAUDE.md#credential-encryption-at-rest) for details.
- **Credential encryption at rest**: AES-256-GCM encryption for all stored credentials when `CREDENTIAL_MASTER_KEY` is configured. See [Configuration and Credentials](./docs/architecture/08-config-credentials.md#credential-encryption) for details.
- **Dual-persona model**: Separate GitHub bot accounts for implementation and review prevent self-approval and feedback loops.
- **No env var fallback for secrets**: All project credentials are stored in the database — no secrets in environment variables or config files.
- **Session-based auth**: HTTP-only cookies with bcrypt password hashing for dashboard access.
Expand Down
3 changes: 2 additions & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

CASCADE is a PM-to-Code automation platform that connects project management tools (Trello, JIRA, Linear), source control (GitHub), and monitoring (Sentry) to AI-powered agents that autonomously implement features, review PRs, debug failures, and manage backlogs. Webhooks from external providers flow through a router, get queued in Redis, and are processed by ephemeral worker containers that run agents against cloned repositories.

> **Relationship to CLAUDE.md**: `CLAUDE.md` is the operational reference (commands, env vars, how-to). This document and its deep-dives cover the *system design* — how components fit together and why.
> **Relationship to CLAUDE.md**: `CLAUDE.md` is the short entry point loaded by Claude Code and injected through CASCADE's `contextFiles` context step — commands, gotchas, hard invariants and a pointer table. `docs/areas/` holds per-area working rules (imperatives plus links, read before editing that part of the tree). This document and its deep-dives cover the *system design* — how components fit together and why.

## System Overview

Expand Down Expand Up @@ -126,6 +126,7 @@ sequenceDiagram
| `src/utils/` | Logging, repo cloning, lifecycle/watchdog, env scrubbing |
| `src/types/` | Shared TypeScript types |
| `src/queue/` | BullMQ queue helpers |
| `docs/areas/` | Per-area working rules for contributors and agents (pm-integrations, router-dispatch, agents, backends) |

## Deep-Dive Documents

Expand Down
4 changes: 4 additions & 0 deletions docs/architecture/01-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,10 @@ The security scrub in step 8 prevents agent engines (which execute arbitrary LLM
- **Webhook jobs** (`trello`, `github`, `jira`, `linear`, `sentry`) — call the provider-specific webhook processor, which re-runs trigger dispatch and executes the matched agent
- **Dashboard jobs** (`manual-run`, `retry-run`, `debug-analysis`) — call `processDashboardJob()`, which loads project config and invokes the appropriate runner

### Repository checkout

Workers clone the target repository at runtime (`src/agents/shared/repository.ts` → `src/utils/repo.ts`); nothing from the host is mounted. When the job carries a `prNumber`, `setupRepository` fetches `+refs/pull/<N>/head:refs/remotes/pr/<N>` from `origin`, checks out `pr/<N>` detached, and — when `headSha` is also set — verifies that `git rev-parse HEAD` matches. This works for same-repo and external-fork PRs alike; the legacy `prBranch` field is kept for log readability but does not drive checkout, because fork branches do not exist on `origin` and a by-name checkout silently 404s. Any non-zero git exit throws — there is no warn-and-continue.

## Dashboard

**Entry point**: `src/dashboard.ts`
Expand Down
10 changes: 9 additions & 1 deletion docs/architecture/03-trigger-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,14 @@ Each trigger in a YAML agent definition can declare a `contextPipeline` — an o
| `pipelineSnapshot` | Fetch PM workflow/pipeline state and emit the single authoritative `PipelineSnapshotSummary` JSON context for backlog-manager |
| `alertingIssue` | Fetch Sentry issue and event details |

### prContext budget and debugging

The review agent receives a compact per-file diff context, not full file contents. `REVIEW_DIFF_CONTEXT_TOKEN_LIMIT` (`src/config/reviewConfig.ts`, 200k tokens) caps the whole context and each file gets at most 10 % of it. GitHub's changed-file API supplies the file list and change counts; patch bodies come from the checked-out workspace via `git diff origin/<base>...HEAD`. Files that cannot fit or cannot be locally verified (deleted, binary/no text patch, local diff failure or empty patch, oversized patch, budget exhausted) are listed under `SKIPPED FILES` with instructions to fetch on demand via `cascade-tools scm get-pr-diff --prNumber <N> --path <path>` (add `--outputFile <path>` for large or one-line JSON diffs that would truncate stdout), `Read`, or `Grep`.

When a review misses something, check the `PR context prepared` log entry: `included` / `skipped` / `skipReasons`, `patchSources`, `totalDiffTokens`, `perFileTokenCap`, and `localGitMismatches` (GitHub's API patch differed from the local one). Also check the context-offload logs if the diff was written under `.cascade/context/`.

CI check status is informational, not fatal: `fetchPRContextStep` wraps only `getCheckSuiteStatus` in a try/catch. If the reviewer PAT lacks the **Actions: Read** permission, the Actions API returns 403 and the `GetPRChecks` injection degrades to an explicit "CI check status UNAVAILABLE" message (deliberately distinct from "No CI checks configured") plus a `WARN CI check status unavailable` log. `getPR` and `getPRDiff` stay fatal — a review without the PR itself is meaningless.

## Shared Agent Execution

`src/triggers/shared/agent-execution.ts`
Expand Down Expand Up @@ -249,7 +257,7 @@ This includes:
- Work-item and PR traceability in `agent-work-items.ts`: create/update work-item records, maintain PR/work-item links before and after execution, fetch PR titles, and backfill run PR numbers.
- Agent execution in `agent-execution-runtime.ts`: call `runAgent()` with the resolved input plus project, config, and remaining budget.
- Post-run PM behavior in `agent-pm-summary.ts` and `agent-execution-lifecycle.ts`: post review/output summaries to the PM work item, handle artifacts, post budget warnings, clean up processing state, and call `handleSuccess` or `handleFailure`.
- Follow-up dispatch in `agent-execution-followups.ts`: dispatch review after a successful implementation PR once CI is passing and the review dedup key is claimed, and chain backlog-manager after a successful splitting run when the auto label/capacity checks allow it.
- Follow-up dispatch in `agent-execution-followups.ts`: dispatch review after a successful implementation PR once CI is passing and the review dedup key is claimed (`claimReviewDispatch` — the same key the `check-suite-success` trigger uses, so the two paths cannot double-enqueue; this fires before the container exits, so review dispatch does not depend on GitHub webhook timing), and chain backlog-manager after a successful splitting run when the auto label/capacity checks allow it.
- Auto-debug in `agent-auto-debug.ts`: fire-and-forget debug analysis for eligible failed or timed-out runs after callbacks and follow-up dispatch complete. It calls the shared `triggerDebugAnalysis()` runner, whose running/failed lifecycle is durable and cross-process — see [Debug-analysis status](#debug-analysis-status-durable-cross-process) below.

Credential scoping still happens before the facade runs. PM webhook handling enters provider credentials and PM provider scope before dispatch; GitHub and Sentry use `webhook-execution.ts` / `credential-scope.ts` to inject LLM keys, PM credentials, PM provider scope, and GitHub persona tokens as needed.
Expand Down
15 changes: 1 addition & 14 deletions docs/architecture/08-config-credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,20 +237,7 @@ await withLinearCredentials({ apiKey }, async () => {

`src/jira/api-host.ts`, `src/jira/authType.ts`

JIRA supports classic unscoped site tokens **and** Atlassian API tokens with scopes. The mode is selected by the optional `authType` field on the JIRA integration config (`project_integrations.config`) — a non-secret connection setting that mirrors `baseUrl`, **not** a credential role. Values: `'basic'` (or absent) and `'scoped'`. Both modes authenticate with **HTTP Basic** (`email:api_token`); `authType` selects the REST v3 *host*, not the auth scheme.

Every REST v3 call site routes through one shared resolver, `resolveJiraApiBaseUrl(creds)` — the JIRA analogue of the shared auth-header helper:

| `authType` | REST v3 host | Notes |
|---|---|---|
| `basic` / absent | tenant **site URL** (`creds.baseUrl`, e.g. `https://acme.atlassian.net`) | Classic behavior, unchanged. Every pre-existing config maps here. |
| `scoped` | Atlassian **gateway** (`https://api.atlassian.com/ex/jira/{cloudId}`) | `cloudId` is resolved from `${baseUrl}/_edge/tenant_info` (always the site URL, never the gateway) with the same Basic scoped token, cached per `baseUrl`. Direct site REST v3 calls can fail under scoped tokens, so the gateway is the supported path. |

The worker/CLI credential scope carries the mode across process boundaries via the `CASCADE_JIRA_AUTH_TYPE` env var (injected by `secretBuilder.augmentProjectSecrets`); `normalizeJiraAuthType` maps absent/unknown values back to `'basic'` so existing projects keep working. `accessible-resources` is intentionally **not** used to discover `cloudId` — it is OAuth 2.0 / 3LO guidance and returns `401` for scoped API tokens.

**Required scopes.** Read/write Jira work (classic OAuth `read:jira-work` + `write:jira-work`). Programmatic webhook management additionally needs webhook scopes — classic OAuth `manage:jira-webhook`, or granular `read:field:jira` + `read:project:jira` + `write:webhook:jira`. A scoped token without webhook scopes (or a non-app caller) gets `401`/`403` from `/rest/api/3/webhook`; the wizard then surfaces an actionable message pointing at manual webhook registration.

**Known limitation — ack reactions.** The "eyes" acknowledgment reaction uses Jira's internal `/rest/reactions/1.0/` API, which lives only on the tenant site URL and is not confirmed on the scoped gateway. Under `scoped` auth the reaction degrades quietly (one log line, then skip) — it is best-effort and never fails a run. Comments, status transitions, and label writes are unaffected.
JIRA's optional `authType` (`'basic'` | `'scoped'`, default `'basic'`) is a **non-secret connection setting** on the integration config, not a credential role; both modes authenticate with HTTP Basic and `authType` only selects the REST v3 host. The worker/CLI credential scope carries it across process boundaries as `CASCADE_JIRA_AUTH_TYPE` (injected by `secretBuilder.augmentProjectSecrets`), and `normalizeJiraAuthType` maps absent/unknown values to `'basic'`. The host-resolution contract (`resolveJiraApiBaseUrl`, cloudId lookup, gateway routing) is specified once in [`src/integrations/README.md`](../../src/integrations/README.md#jira-authentication-modes-scoped-tokens); required scopes and known limitations for operators are in [Getting Started](../getting-started.md#scoped-api-tokens-authtype).

## Credential Encryption

Expand Down
6 changes: 4 additions & 2 deletions docs/architecture/10-resilience.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ The backend adapter drains pending sidecar events after the engine returns, incl
The router queues `cascade-jobs` and `cascade-dashboard-jobs` with `attempts: 4` and exponential backoff. Dispatch errors before a worker container starts are classified in `src/router/dispatch-error-classifier.ts`:

- Transient: Docker socket `ECONNREFUSED` / `ECONNRESET` / `ENOTFOUND`, registry HTTP 429, container-name HTTP 409, and `SLOT_WAIT_TIMEOUT`.
- Terminal: validation errors (`TypeError`, `ZodError`) and image-not-found after fallback exhaustion.
- Terminal: validation errors (`TypeError`, `ZodError`) and image-not-found after fallback exhaustion. These are wrapped in BullMQ's `UnrecoverableError`, which skips the retry budget entirely.

Post-enqueue dispatch failures (Docker socket errors, slot-wait timeouts, container failures) flow through the BullMQ `failed` event and call `releaseLocksForFailedJob`, releasing the work-item lock, agent-type counter, and recently-dispatched mark. Webhook logs distinguish healthy backpressure (`Awaiting worker slot`) from the wedged-lock canary (`Work item locked (no active dispatch)`). Enqueue/schedule failures that occur before a BullMQ job exists are handled differently — see the split below.

Expand Down Expand Up @@ -168,6 +168,8 @@ Both GitHub persona usernames (implementer + reviewer) are resolved and cached.

- `respond-to-review` only fires when the **reviewer** persona submits `changes_requested`
- `respond-to-pr-comment` skips @mentions from **any** known persona
- `check-suite-success` looks for an approving review from the **reviewer** persona specifically
- Every SCM trigger handler filters self-events with `isCascadeBot(login)`; the one deliberate exemption is the self-directed `review_requested` case described in [03-trigger-system](./03-trigger-system.md)
- Trello/JIRA handlers check their bot member/account IDs similarly

### Self-authored event filtering
Expand All @@ -192,7 +194,7 @@ See [08-config-credentials](./08-config-credentials.md) — AES-256-GCM encrypti

Periodic scan for Docker containers that outlived their expected lifetime (watchdog timeout + buffer). Orphans are killed and their run records marked as failed.

When a worker container exits non-zero, the router inspects it before Docker AutoRemove can reap it and writes a grep-stable error reason: `Worker crashed with exit code N · OOMKilled=<true|false> · reason="<State.Error>"`. `OOMKilled=true` is the definitive cgroup OOM signal; exit 137 without that marker means something else sent the signal.
When a worker container exits non-zero, the router inspects it before Docker AutoRemove can reap it and writes a grep-stable error reason: `Worker crashed with exit code N · OOMKilled=<true|false> · reason="<State.Error>"`. `OOMKilled=true` is the definitive cgroup OOM signal; exit 137 without that marker means something else sent the signal. The string is produced by `formatCrashReason` in `src/router/active-workers.ts` and pinned by `tests/unit/router/container-manager-diagnostics.test.ts` — keep it grep-stable. The `[WorkerManager] Resolved spawn settings` log line emitted at every spawn records both `projectWatchdogTimeoutMs` and `globalWorkerTimeoutMs`, so a post-mortem can confirm whether the per-project override actually won.

## Worker Lifecycle Internals

Expand Down
Loading
Loading