feat: agent-loop hooks — before/after LLM & tool calls + wrap_tool_call#255
Merged
Conversation
…ll (#253) Adds the missing interception points inside the agent loop: - before_llm_call: chain-modifiable request (model/messages), or LlmCallDecision.finish(text) to short-circuit (cost guards / call limits) - after_llm_call: observe-only terminal outcome (text/tool_calls/usage/ error/duration), fires once per call incl. error paths; streaming observed at accumulated final state, not per chunk - before_tool_call: per-invocation ToolCallDecision — proceed(args) folds argument changes, replace(result) / deny(message) short-circuit - after_tool_call: observe-only terminal outcome incl. deny/replace - wrap_tool_call: middleware-style continuation (call_next zero/one/many times) enabling retry, caching, human-in-the-loop — cf. LangChain middleware wrap_tool_call Design notes: - payload contracts in bub/agent_hooks.py; execution semantics in AgentHooks facade (bub/hook_runtime.py); wired via BubFramework.get_agent_hooks() -> ModelRunner/ToolExecutor - observer/chain hooks are fault-isolated (raising plugin logged+skipped, never fatal; veto only via decision objects); wrappers own the execution path so their exceptions surface as tool errors - ordering follows pluggy LIFO: last registered runs first / outermost - tool errors are normalized to BubError inside the continuation so wrappers catch a stable type; ToolExecutor() without hooks is unchanged Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
bub | d795a84 | Commit Preview URL Branch Preview URL |
Jul 07 2026, 12:47 AM |
…e after_llm_call, effective-call observation Review findings from PR #255 round 1: 1. before_llm_call rewrites now reach the provider AND the tape: completion_response accepts the effective max_tokens; both success-path record_chat calls record request.model instead of the pre-hook model. 2. after_llm_call is exactly-once on every exit path: the guard now catches BaseException so consumer aclose()/cancellation (GeneratorExit, CancelledError) still produce a terminal observation. 3. after_tool_call observes the effective invocation: the most recent ToolCall passed to call_next (wrappers may rewrite arguments or retry); wrappers that never invoke call_next are observed with the pre-wrap call. Contract documented in the wrap_tool_call hookspec. 4. Doc consistency: wrap nesting is pluggy LIFO (last registered = outermost) everywhere; fixed a literal backslash-n typo in the before_llm_call docstring and the misnamed nesting test. Adds 5 regression tests (rewritten model/max_tokens reach provider+tape, exactly-once on success and on early aclose, effective-call observation with and without call_next). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keeps the four before/after hooks (incl. LlmCallDecision.finish and the exactly-once / effective-request fixes). The continuation-style wrapper can return later as a separate proposal if retry/HITL use cases press. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per review direction: observers get the raised exception object instead of a stringified form — ToolCallResult.error is the original BubError (kind/message/details intact), LlmCallResult.error is the original BaseException (incl. GeneratorExit/CancelledError on close/cancel). Tests pin the preserved types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ned to BaseException Mirrors the LLM-side exactly-once guarantee: a cancelled tool invocation (CancelledError, BaseException) now produces a terminal after_tool_call observation with the original exception, then re-raises unwrapped. Regression: real Task.cancel() on a blocking async tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ld recipes (en/zh) - reference/hooks: four new table rows; 'Agent-loop hooks (AgentHooks)' section covering LIFO chaining, decision-only veto + fault isolation, exactly-once terminal observation (incl. aclose/cancellation), original exception preservation, run_id<->tape correlation, end-to-end request rewriting, and the removed wrap_tool_call boundary note - build/hooks: recipe section with model-routing/cost-guard, tool deny/replace policy, and terminal-observation metrics examples - zh-cn mirrors for both pages; website build verified Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
state persists across turns, so the counter is a session budget; the example now names it session_llm_calls, says so in the finish text, and notes how to build an actual per-turn budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bypasses by design Per review direction: catching BaseException for terminal observation is dropped on both the LLM and tool paths. Cancellation / consumer close (CancelledError, GeneratorExit) intentionally does not fire after hooks; success, Exception failure and deny/replace still observe exactly once. Result error types narrowed back to Exception; tests inverted to pin the not-observed semantics; en/zh reference and build docs updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #253.
Adds four hook points inside the agent loop (per the discussion in #253 and a use-case sweep of LangChain's middleware reference):
before_llm_callLlmCallDecision.finish(text)to skip the provider call (cost guards / call limits)after_llm_callaclose(), cancellation); streaming observed at accumulated final statebefore_tool_callproceed(args)folds argument changes,replace(result)/deny(message)short-circuit (enables #249-style plugins)after_tool_callSemantics decisions
run_idis threaded through all payloads for correlation with tape entries.wrap_tool_callwas prototyped and removed per review direction; it can return as a separate proposal if retry/HITL use cases press.wrap_model_calllikewise deferred (bub's model call streams events mid-flight; native candidate fallback already covers model fallback).Wiring: contracts in
bub/agent_hooks.py;AgentHooksfacade inhook_runtime.py;BubFramework.get_agent_hooks()→ModelRunner(hooks=…)→ToolExecutor(hooks=…).ToolExecutor()without hooks is unchanged.Tests: 15 hook tests + 5 review-regression tests (rewritten model/max_tokens reach provider+tape, exactly-once on success/early-close, chaining order, short-circuit, isolation). Full suite 243 passed; ruff + mypy clean.
🤖 Generated with Claude Code