docs(15.9): describe what the Ollama client retries and what it does not - #538
Merged
Merged
Conversation
The Ollama page said rag.llm.ollama.retry.max is the number of HTTP retries on 429 and 5xx errors. In codelibs/fess-llm-ollama it is the total number of attempts, the retried statuses are 429, 500, 502, 503 and 504, and I/O failures before a response (a refused or reset connection, a connection closed without a response, a connect timeout) are retried too. With codelibs/fess-llm-ollama#29 a response timeout is no longer retried, so a slow model needs a larger timeout, not more retries. Correct the timeout and retry.max rows and add a "Retries" subsection to config/llm-ollama.rst in the 15.9 tree for all seven languages: what is retried and what is not, the backoff, how long a failing request takes with the defaults, what to raise for a slow model, and the matching content_chunker.embedding.ollama.* keys.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Ollama page in the 15.9 tree (
config/llm-ollama.rst) describesrag.llm.ollama.retry.maxas the "maximum number of HTTP retry attempts (on429and5xxerrors)". That is not what the plugin does, and it leaves out the case that matters most for slow local models.What codelibs/fess-llm-ollama does on
main(OllamaLlmClient#executeWithRetry,OllamaEmbeddingClient#executeWithRetry):retry.maxis the total number of attempts, the first included (Math.max(1, getRetryMaxAttempts())). The default3means two retries, and1turns retries off.429,500,502,503and504(isRetryableStatus), not every5xx. Any other error status fails on the first attempt.IOExceptionis retried too: a refused or reset connection,NoHttpResponseException, a connect timeout. The page did not mention this at all.retry.base.delay.ms × 2^(n-1)plus a jitter of up to ±20% of the base delay, capped at 60 seconds (sleepBackoff,MAX_BACKOFF_MS).codelibs/fess-llm-ollama#29 changes one case: a read (response) timeout is no longer retried. It is rethrown from the first attempt, so a model that never answers holds a request for one
timeoutinstead of three full timeouts plus the backoff. A slow first answer, such as a model that is still loading, needs a largerrag.llm.ollama.timeout(orcontent_chunker.embedding.ollama.timeout), not more retries.Changes, all in
config/llm-ollama.rstof the 15.9 tree, for ja, en, de, es, fr, ko and zh-cn:rag.llm.ollama.timeoutrow calls it the response (read) timeout and says that a request that runs into it is not retried.rag.llm.ollama.retry.maxrow says it counts attempts per request to Ollama, the first included.rag.llm.ollama.timeoutrather than a largerretry.max;content_chunker.embedding.ollama.*keys insystem.properties.The key names and defaults on the page (
timeout60000,connect.timeout5000,retry.max3,retry.base.delay.ms2000) match the plugin source and are unchanged. No existing heading changes, so no anchor moves.Requires codelibs/fess-llm-ollama#29 for the read-timeout part; merge it first. The rest of the section describes the plugin as it is on
maintoday.Verification
python3 tools/check_headings.pyon all seven15.9trees: passes.main: each page has one more section (33 instead of 32) and three more bullet lists, all tables keep their shape, and the only new system message is the one for the added:doc:role, which plain docutils does not know. The structure is identical across the seven languages.dummybuild of each language's15.9tree (built-in extensions only), compared withmain: no warning appears only after the change, the page's own warnings are the same 14 in every language, and the:doc:link tosearch-semanticresolves. A copy with that link pointed at a missing document reportsunknown document, so the build does check the link.OllamaLlmClientandOllamaEmbeddingClientonmain, and the read-timeout behaviour against the diff of fix: stop retrying a read timeout from Ollama fess-llm-ollama#29.The non-English wording has not been reviewed by native speakers.