Skip to content

refactor: define all retry strategies once in vespa.retries (#1344) - #1346

Merged
thomasht86 merged 2 commits into
masterfrom
thomasht86/unify-retry-strategies
Sep 4, 2026
Merged

thomasht86 merged 2 commits into
masterfrom
thomasht86/unify-retry-strategies

Conversation

@thomasht86

@thomasht86 thomasht86 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Context

🤖 Code written by Claude, reviewed and tested by me

Retry strategies have drifted across the codebase, and we want to gather them in one place, both to make them easier to change, and to reduce introducing new strategies and/or strategy drift in calling methods.

What's changed?

Every tenacity policy in the library now lives in vespa/retries.py as a module-level Retrying/AsyncRetrying constant. Call sites use POLICY.wraps or POLICY.copy()(fn, ...), so the constants are never mutated and tests have a stable patch target.

Behaviour-preserving, with two deliberate exceptions called out below.

  • QUERY_RETRY: default for VespaAsync.query (5 attempts, random exp wait).
  • DOCV1_RETRY + THROTTLE_RETRY: the two-layer policy for the async document/v1 methods, now applied once in a shared _docv1_request helper instead of four copies of stacked decorators.
  • SYNC_REQUEST_RETRY: replaces the hand-rolled loop in VespaSync._request_with_retry; wait_golden_jitter reproduces the old 0.1 * 1.618**n + uniform(0, 1) formula and num_retries_429 still bounds it.
  • VISIT_RETRY, CONTROL_PLANE_RETRY, URL_VALIDATION_RETRY: the decorators from Vespa.visit, VespaCloud and vespa.models, unchanged.

Deliberate changes:

  • THROTTLE_RETRY spells out stop=stop_never. The 429 retry was already unbounded by omission; it is now explicit and documented, because it is the only backpressure mechanism on feed_async_iterable.
  • The 429 wait differed between feed_data_point (wait_random_exponential(multiplier=1, max=3)) and its three siblings (wait_exponential(multiplier=1, max=10)). All four now share wait_random_exponential(multiplier=1, max=10): jittered so concurrent tasks that hit 429 together do not retry in lockstep, with the larger 10s cap. Release note: 429 retries on the async document API now use random exponential backoff capped at 10s.

Also, minor fixes:

  • test_query_exhausts_retries[None-5] patched a tenacity private that had no effect and slept for real (9-34s). It now patches vespa.application.QUERY_RETRY and runs in ~0.1s.
  • New tests/unit/test_retries.py covers policy shape, sync 429/connection retry, unbounded docv1 429, 503 exhaustion and exception re-raise.
  • query.ipynb no longer claims queries retry 429/500/503/504 three times, which was true of neither client. API docs gain a vespa.retries page.

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

Every tenacity policy in the library now lives in vespa/retries.py as a
module-level Retrying/AsyncRetrying constant. Call sites use POLICY.wraps
or POLICY.copy()(fn, ...), so the constants are never mutated and tests
have a stable patch target.

Behaviour-preserving, with two deliberate exceptions called out below.

- QUERY_RETRY: default for VespaAsync.query (5 attempts, random exp wait).
- DOCV1_RETRY + THROTTLE_RETRY: the two-layer policy for the async
  document/v1 methods, now applied once in a shared _docv1_request helper
  instead of four copies of stacked decorators.
- SYNC_REQUEST_RETRY: replaces the hand-rolled loop in
  VespaSync._request_with_retry; wait_golden_jitter reproduces the old
  0.1 * 1.618**n + uniform(0, 1) formula and num_retries_429 still bounds it.
- VISIT_RETRY, CONTROL_PLANE_RETRY, URL_VALIDATION_RETRY: the decorators
  from Vespa.visit, VespaCloud and vespa.models, unchanged.

Deliberate changes:
- THROTTLE_RETRY spells out stop=stop_never. The 429 retry was already
  unbounded by omission; it is now explicit and documented, because it is
  the only backpressure mechanism on feed_async_iterable.
- feed_data_point's 429 wait was wait_random_exponential(multiplier=1, max=3)
  while its three siblings used wait_exponential(multiplier=1, max=10).
  All four now share the latter.

Also:
- test_query_exhausts_retries[None-5] patched a tenacity private that had
  no effect and slept for real (9-34s). It now patches
  vespa.application.QUERY_RETRY and runs in ~0.1s.
- New tests/unit/test_retries.py covers policy shape, sync 429/connection
  retry, unbounded docv1 429, 503 exhaustion and exception re-raise.
- query.ipynb no longer claims queries retry 429/500/503/504 three times,
  which was true of neither client. API docs gain a vespa.retries page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sebastiannberg
sebastiannberg previously approved these changes Sep 3, 2026
Comment thread vespa/retries.py
Comment thread vespa/retries.py
…429 retry

- Replace the Sphinx-only "#:" comments on the policy constants with string
  literal attribute docstrings, which both Sphinx autodoc and mkdocstrings
  (griffe) recognise. The constants were hidden on the rendered API page
  because griffe saw no docstring and show_if_no_docstring defaults to false.

- THROTTLE_RETRY now uses wait_random_exponential(multiplier=1, max=10)
  instead of wait_exponential(multiplier=1, max=10). Many feed tasks that
  hit 429 together would otherwise sleep the same 1, 2, 4, 8, 10s and retry
  in lockstep, indefinitely given stop_never. This keeps the jitter
  feed_data_point had and the larger cap its siblings had. The test asserts
  the wait type and cap.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@thomasht86
thomasht86 merged commit 37e53c0 into master Sep 4, 2026
46 of 47 checks passed
@thomasht86
thomasht86 deleted the thomasht86/unify-retry-strategies branch September 4, 2026 07:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants