feat(extract): make the extract-phase call timeout configurable - #59
Merged
Merged
Conversation
The 180s per-call extract timeout was a source constant. It is sized for a hosted model's generation speed, and "extract calls have predictable size" -- the reason it is tighter than the write phase's -- bounds the prompt, not the duration. A model served on localhost can be an order of magnitude slower, at which point extract fails on documents a hosted model handles without trouble: a local 12B model exhausted all three attempts on a 4386-character prompt, which is a small document by a real corpus's standards. Extract is also where a slow model fails earliest, because it runs on every document. So the phase that most needed the knob was the one that did not have it, and the only way past a timeout was to edit the source. KB_AI_EXTRACT_TIMEOUT_S mirrors KB_AI_WRITE_TIMEOUT_S deliberately, down to the fallback and the warn-once behaviour, so an operator who has learned one knob has learned both. It is read per call rather than at import, so setting it does not have to happen before kb_ai is imported. A value that cannot serve as a timeout -- zero, negative, non-numeric, NaN, or infinite -- is reported once on stderr and ignored: zero would fail every call instantly and infinite would silently remove the cap the default exists to impose, and failing a whole compile over a typo in an env var is the worse outcome. One way the two phases genuinely differ is documented rather than mirrored: extract retries in two places, because _phase2_with_retry re-dispatches its entire K-call set once on top of the LLM layer's two timeout retries. A hung phase-2 call therefore costs 6*timeout+60s to discover, not 3*timeout+30s, so the README sizes extract against six attempts rather than three. The autouse fixture that kept KB_AI_WRITE_TIMEOUT_S out of the suite's view now covers both variables, so the operator these knobs exist for does not get a red suite for a reason unrelated to their change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tybot02
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The extract phase's per-call LLM timeout was a source constant at 180s. That figure
is sized for a hosted model's generation speed, and the reason it is tighter than
the write phase's — "extract calls have predictable size (≤16K max_tokens)" — bounds
the prompt, not the duration.
A model served on localhost can be an order of magnitude slower. A local 12B model
exhausted all three attempts on a 4386-character prompt, which is a small document by
a real corpus's standards. The only way past it was to edit the source.
Extract is also where a slow model fails earliest, because it runs on every document.
So the phase that most needed the knob was the one that did not have it.
Change
KB_AI_EXTRACT_TIMEOUT_Smirrors theKB_AI_WRITE_TIMEOUT_Sadded in #58,deliberately down to the details, so an operator who has learned one knob has learned
both:
kb_aiisimported;
stderr and ignored, rather than letting a typo in an env var decide how a compile
behaves — zero would fail every call instantly, and infinite would silently remove
the cap the default exists to impose;
DEFAULT_CLIENT_TIMEOUT_S, since the client timeout is adefault and not a ceiling.
The defaults are unchanged, so the existing
_EXTRACT_CALL_TIMEOUT_S < _WRITE_CALL_TIMEOUT_S < DEFAULT_CLIENT_TIMEOUT_Sorderinginvariant still holds.
One place the phases genuinely differ
Documented rather than mirrored: extract retries in two places.
_phase2_with_retryre-dispatches its entire K-call set once on any failure, on top of the LLM layer's two
timeout retries. A hung phase-2 call therefore costs
6*timeout+60sto discover, not3*timeout+30s— about 1140s at the 180s default and 91 minutes at 900s. Both READMEssize extract against six attempts rather than three.
Verification
assert 180.0 == 1800.0, which is the defect itself: the env var was ignored.extract_knowledge_summarizedand
extract_knowledge_chunked) rather than the decorator in isolation, because thedispatch between them depends on document size and transcript detection — a knob
reaching one path and not the other would show up as a timeout only some documents
respect.
kb_ai.core.extractat 91% statement coverage, with the new code fully covered.infaccepted, zero/negativeaccepted, warn-once defeated, decorator reverted to the frozen constant) all killed,
each with a green control and reverse control.
🤖 Generated with Claude Code