Research spike: source of `from_*` tool calls in the main loop

[hoblin]

Problem to solve

Aoide's main loop occasionally surfaces tool calls named after sister/sub-agent message channels — e.g. from_melete_goal, from_zero-width-sleuth, from_whitespace-scout. These names are not registered as tools; they are the prefixes the system uses when delivering messages to the agent. Calling them produces:

{"error":"Tools::UnknownToolError: Unknown tool: from_<name>"}

…polluting the conversation and wasting tokens. Today this is a known papercut that requires the human to nudge the agent ("these are not tools").

We do not currently know whether the offending tool calls originate from the LLM itself or from somewhere inside Anima.

Two hypotheses

1. LLM hallucination. The model sees prior from_* blocks in the conversation context and pattern-matches them as callable tools. Anthropic's endpoint returns them in tool_use blocks because the model decided to invoke them.
2. Internal leak. Some part of Anima's pipeline (event subscriber, tool-execution dispatcher, registry assembly) generates these calls when it shouldn't — they never come from the LLM at all.

The fix differs significantly between the two:

• If hallucination: filter from_* tool_use blocks out of the dispatch path (lib/events/subscribers/llm_response_handler.rb#normalize_tool_uses) and pretend the LLM never called them. No error round-trip.
• If internal leak: find the source and stop emitting them.

Research approach

Wire up debug logging for Aoide's main loop the same way the muses are instrumented today (lib/melete/runner.rb:132-133 logs system prompt and user message at debug level). For Aoide we additionally need:

• Raw response payload from the Anthropic endpoint, including content blocks (text + tool_use)
• The list of tool_use blocks as returned by the API, before any normalization
• The tool name and ID of every dispatched tool execution, so we can correlate "what came in" vs "what got called"

Enable this in dev environment only.

Acceptance criteria

• After a session that exhibits the bug, the logs contain enough information to determine whether the from_* tool_use block was present in the API response or was synthesized later.
• Decision recorded in this issue (or a follow-up): hallucination vs internal leak.
• Follow-up implementation issue opened with the appropriate fix.

Suspected choke points (starting hints, not prescribed)

• lib/events/subscribers/llm_response_handler.rb — normalize_tool_uses is where the API response gets turned into dispatchable tool calls. Logging the raw response just before this method runs would give the cleanest signal.
• The decorators in app/decorators/pending_from_*.rb produce the from_* named blocks that appear in the agent's view. Worth checking whether the same names ever leak into the outgoing tool registry (they shouldn't, but if they do, the model sees them as legitimate tools).

Why this matters

Every spurious from_* call costs context (the failed tool result is persisted), wastes tokens, and erodes trust in the agent's apparent competence. It's one of the first weird behaviors a new agent encounters, and the human currently has to coach around it.

[hoblin]

Research-spike results — verdict: LLM hallucination

The instrumentation from PR #485 was run against a session deliberately constructed to trigger the bug (parallel sub-agents, agent waiting for push-delivered results — the documented hallucination trigger from PR #468 review). The bug fired three times in ~3 minutes of session 15.

Evidence

log/aoide.log for one of the hallucinations:

[10:55:53.240] DEBUG  session=15 raw response:
model: claude-opus-4-6
id: msg_01C9y8PSk28NLWE6qYinCXUb
type: message
role: assistant
content[2]:
  - type: text
    text: "Three subagents spawned in parallel — shell-runner (2s), sleep-echo-scout (4s), shell-sleeper (6s). Now we wait for the push delivery... 🎣"
  - type: tool_use
    id: toolu_015N2kYkP8M8HJXsoWTWhRwv
    name: from_shell-runner
    input:
      from: shell-runner
    caller:
      type: direct
stop_reason: tool_use
[10:55:53.240] DEBUG  session=15 raw tool_use blocks:
[1]:
  - type: tool_use
    id: toolu_015N2kYkP8M8HJXsoWTWhRwv
    name: from_shell-runner
[10:55:53.285] INFO  session=15 dispatching tool=from_shell-runner id=toolu_015N2kYkP8M8HJXsoWTWhRwv

The from_shell-runner tool_use block sits inside the content array of the raw Anthropic API response — including a fabricated input schema ({from: shell-runner}) and a fabricated caller: {type: direct} field that the model invented from whole cloth. The same id and name pass through normalize_tool_uses unchanged and reach dispatch, where Tools::Registry#execute raises UnknownToolError.

This rules out internal leak as the (#480-listed) alternative hypothesis. As suspected from the architectural prior, the outgoing tool registry never carries from_* names — but the model emits them anyway.

Pattern

All three hallucinations occurred immediately after the agent spawned sub-agents and was waiting for results. The agent literally said in her own thoughts in the same session:

"I just did the thing — I polled. The exact pattern documented in the Level 2 memory. 'Despite the documentation fix, during PR #468 review, Aoide repeatedly polled from_ tools 14+ times in rapid succession while explicitly instructed to wait.' And I just did it again. The harness talking. Wait for push delivery. Don't poll."

So: PR #445's from_* naming + system-prompt sister block + a Level 2 memory explicitly documenting the anti-pattern are jointly insufficient. The model knows the rule and breaks it anyway when it's expecting a result.

Path forward — silent strip

Since the from_* block is part of the API response itself, the only place to neutralise it is on our side. Plan for the implementation PR (will land with Closes #480):

1. Small pure filter — LLMResponse::PhantomCallFilter.call(hash) → hash — drops every block where type == "tool_use" && name.start_with?("from_") from response["content"]. No session, no AR, no events.
2. Wired into Events::Subscribers::LLMResponseHandler#emit at the top, immediately after log_raw_response so the diagnostic log keeps observing the true API response (we want to keep measuring hallucination rate post-fix).
3. Existing logic carries the rest — normalize_tool_uses and extract_text operate on the filtered content unchanged. Three edge cases to cover in specs:
   • Response with only from_* tool calls → falls through to response_complete! (no-op end of turn, no error round-trip)
   • Response with from_* + legitimate tool calls → only legitimate ones dispatch, text untouched
   • Response with text + from_* tool calls → text persists, no dispatch, end of turn

The follow-up question of whether to give the model in-context feedback (synthesised tool_result, Mneme nudge, etc.) is deliberately deferred — silent strip first, measure, then decide whether the residual hallucination rate justifies a feedback loop.