Skip to content

Loop hardening from a real migration: safety, reliability, and model-output tolerance - #101

Merged
EmersonBraun merged 29 commits into
mainfrom
fix/loop-pilot-hardening
Sep 23, 2026
Merged

EmersonBraun merged 29 commits into
mainfrom
fix/loop-pilot-hardening

Conversation

@EmersonBraun

@EmersonBraun EmersonBraun commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes found by running the keep-pushing loop on a real multi-week repository migration (monorepo split into product repositories), with a single non-default provider at first and then a mixed one. Every item below was observed in that run and is covered by a test; the full list is under [Unreleased] in CHANGELOG.md.

Safety

  • A worker stopped at a tool-permission prompt is held for a person and never typed into — the idle nudge used to answer the dialog with its default, "Allow once".
  • The orchestrator (contract, plan interview, architect, votes, decompose) reads a harness-owned worktree of origin/<baseBranch>, not the operator's checkout, which can be on any branch at any age.
  • Workers start from origin/<baseBranch> fetched at dispatch, not the operator's stale local branch.
  • The worker brief no longer forbids editing the review-gated paths its contract asks it to edit; git stash is forbidden (the stash is shared by every worktree).
  • Gate lists (selfEditPaths, secretFilePatterns, requiredChecks) accumulate across config layers and shrink only by !entry.
  • The design gate refuses open objections unless --accept-objections carries them into decompose.
  • A worker out of usage is handed to another provider only after its terminal is closed — two agents never share a worktree.
  • A PR held for protected paths is released only by loop approve <issue> --head <sha> --by <you>: an attestation bound to that commit, recorded in the event log; deliberately not a PR label, which a worker holding the same credentials could add itself.

Reliability

  • The brief is handed over as a file (.ak-loop/brief.md) with a one-line pointer; typing tens of kilobytes into an agent TUI failed deterministically on content.
  • A brief counts as delivered only when the agent's turn starts (observe, press Enter, observe).
  • Ambiguous prompt sends are retried by Orca request id, spaced, instead of failing the dispatch.
  • Queue claims use --to-id via linear.people; merged issues shed blocked/needs-info; the merge record says who merged.
  • Model output is found in markers, a fenced JSON block, or the last valid JSON value — 5/5 models parse the interview (2/5 before); interview lists tolerate a string or an empty list.
  • decompose --create files the reviewed list, in linear.entryState (outside the queue), in the queue's project, with its labels, under --parent.
  • Phase artifacts are excluded from worker commits.
  • loop plan has its own time budget (worker.plan.stageTimeoutMs).
  • A worker whose provider ran out of usage (read from its screen) is handed to a builder from another provider; the provider cools down until the reset the CLI printed.
  • For agents whose turns Orca cannot observe (observation: unsupported), delivery of the brief is confirmed on screen and re-sent when the keystrokes were dropped.
  • Review rounds that only discover new findings do not spend maxFixRounds; persisting findings do; 2× is a hard ceiling.
  • On macOS, machine pressure uses the measured CPU busy share instead of the load average.
  • A PR closed without merge is escalated once; every deliver pass used to re-label the issue and move it back to returnState.
  • Approved PRD and design documents are written into the checkout only when it is the clean base branch; otherwise under the state directory, with a note.
  • Decompose marks work that is not a PR to project.repo (outside), files it with linear.outsideLabel, and the queue never dispatches it.

Verification

Rebased onto main (0.18.0) by cherry-picking the 21 commits; one was dropped because main already carries the same fix (#88, --to-id claims), and one release-manifest bump was superseded by 0.18.0.

  • Full suite on this branch: 150 files, 1479 tests passing; typecheck clean; capability manifest and generated docs reference regenerated.
  • Each of the 22 checks in .ak-harness/verification.json run directly on this branch: all exit 0.
  • The attested run (ak-verify) is pending a human: after the previous run completed, a new one needs ak-harness plan approved --by human, which the author of this PR will not do on its own behalf. Merge after that run is COMPLETE.

GitHub Actions does not run on this account at the moment (billing), so there is no CI signal on this PR.

Breaking: planned issues gain outside; the queue excludes linear.outsideLabel (default outside-loop). Gate lists accumulate across layers; approve-design requires --accept-objections when votes carry objections; decompose files issues in linear.entryState (default Backlog), which must not be one of linear.states.

AgentsKit Contributors and others added 29 commits September 22, 2026 23:09
Um segundo prazo de espera antes de desistir; sem idle, a dispatch falha
e o worktree meio-criado é removido, em vez de perder o prompt em silêncio.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 5042073)
Um overlay de máquina escrito para liberar .github/pr-intent.yaml substituiu
o selfEditPaths inteiro e apagou, em silêncio, o congelamento de packages/**
que o projeto adicionou cinco dias depois (agentskit-os #6238, F5).

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit d25e864)
…ontra

linear.entryState (default Backlog) não pode estar em linear.states; antes as
issues nasciam em states[0], que É a fila — o gate humano não existia. Agora
levam as labels da fila, o projeto que ela drena e --parent <épico>.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 42be4b4)
String vira lista de um item e lista vazia vira lacuna (prdGaps decide), em
vez de a rodada inteira falhar na validação — glm-5.3 respondia assim.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 42ffe3f)
…o prazo do planner por issue

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit ae5be3e)
… o loop nunca digita nele

O nudge de ociosidade caía num diálogo cujo default é "Allow once" — o Enter
aprovava justamente o comando que a config do agente marca como perigoso.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit ca320b9)
…rios, do CHANGELOG e dos testes

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 45c9fa3)
… operador

Contrato, entrevista, architect, votos e decompose rodavam o modelo em
project.root. Agora rodam num worktree desanexado de origin/<base> do próprio
harness, buscado e resetado antes do uso; fetch falho falha o estágio.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit d204a0a)
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 13867ef)
…coava 'no layers configured'

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 9b17b01)
…it stash

selfEditPaths é gate de revisão, não lista de 'nunca editar'. O stash é
compartilhado por todos os worktrees do repo.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 84ae6fb)
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit fddaa67)
…erdido

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 2d2ea62)
…entativas espaçadas

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 1f0e81a)
… recebe só o ponteiro

Colar ~44 KB numa TUI falhava de forma determinística conforme o conteúdo
(agent_session_ownership_unknown), e nenhum retry resolvia.

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit ce52fe6)
…o branch local do operador

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 254b728)
…quem mergeou

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit c227f4c)
…meça

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit c258505)
…t-objections

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 70d516a)
…rcadores, bloco json ou último JSON válido

Eval na entrevista: 5/5 modelos parseiam (antes 2/5).

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit ee8daf0)
…ooldown até o reset, terminal antigo fechado

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 507b67c)
…a 0.18.0

Co-Authored-By: Claude <noreply@anthropic.com>
…ela, com reenvio

Co-Authored-By: Claude <noreply@anthropic.com>
…PR retido por caminho protegido

Co-Authored-By: Claude <noreply@anthropic.com>
…fix rounds; persistente conta, teto 2×

Co-Authored-By: Claude <noreply@anthropic.com>
…vg; manifesto e referência regerados

Co-Authored-By: Claude <noreply@anthropic.com>
@EmersonBraun

Copy link
Copy Markdown
Member Author

ak-verify run 1790134946881-72831-g7byr3: COMPLETE, 22/22 checks passed on 48acd10 (contract approved by human 2026-09-23T03:42Z). Actions off (billing); evidence is local.

@EmersonBraun
EmersonBraun merged commit 3259d7f into main Sep 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant