Skip to content

fix(link): prevent unbounded requestdl API calls on persistent CDN 404 - #381

Open
TwistedRat wants to merge 2 commits into
sirrobot01:betafrom
TwistedRat:fix/requestdl-flood-cooldown
Open

TwistedRat wants to merge 2 commits into
sirrobot01:betafrom
TwistedRat:fix/requestdl-flood-cooldown

Conversation

@TwistedRat

Copy link
Copy Markdown
Contributor

Problem

Two commits together cause unbounded requestdl API calls that can trigger TorBox's cooldown_until mechanism:

  1. 4a7777c (already in beta via this PR) — CDN 404 was changed from CategoryPermanent to CategoryRefetchable so that expired CDN URLs are re-fetched rather than cached as permanent failures. This is the correct behaviour for a stale URL, but it has a side-effect:

  2. The loop: invalidateAndRefetch deletes the link from the account-level cache and calls requestdl to get a fresh CDN URL. The new URL is returned but not stored in s.validated. On the next rclone/Plex poll (every few seconds), s.validated has no entry → HEAD check → 404 → another invalidateAndRefetch → another requestdl call. This repeats on every poll for as long as the file returns 404 — which can be minutes while TorBox is still processing a newly-added torrent.

The result is hundreds of requestdl calls per file per hour. TorBox's API terms treat this as automated abuse and apply an extended cooldown_until freeze on the API key.

Fix

pkg/manager/link/errors.go — no change from what landed in beta; 404 stays CategoryRefetchable.

pkg/manager/link/service.go — adds a per-file refetch cooldown map:

  • After any 404-triggered invalidateAndRefetch, the file key (infohash:filename) is recorded with a 5-minute expiry.
  • Subsequent polls within that window return the cached error immediately without calling requestdl.
  • On a successful HEAD validation the cooldown is cleared, so a file that becomes available within the window is served promptly.

The cooldown is intentionally short (5 min) — long enough to stop the flood, short enough that a newly-processed torrent is retried automatically.

Also included

  • Exponential backoff for transient errors (429, 502, 503, 504) during link validation — prevents a single rate-limit spike from being cached as a permanent failure.
  • 502/504 added as CategoryRetryable (previously fell through to CategoryPermanent).
  • Retryable errors are no longer stored in s.validated — a 502 that clears after 10 s would otherwise block a file for the lifetime of the process.

Reproducer

  1. Add a torrent to TorBox that takes > 30 s to process.
  2. Radarr/Sonarr immediately requests the file via the rclone FUSE mount.
  3. Without this fix: requestdl is called every few seconds until TorBox finishes processing.
  4. With this fix: one requestdl call on first 404, then silence for 5 min, then one retry.

Testing

Confirmed against a live TorBox Pro account. The cooldown_until field in /v1/api/user/me was set by the pre-fix behaviour and expired after the fix was deployed.

TwistedRat and others added 2 commits August 3, 2026 21:53
A 404 during CDN link validation was classified as CategoryPermanent,
causing the error to be cached indefinitely in the link service's
in-memory validated map. If a file's CDN URL returned 404 at any point
(e.g. Plex probed a season pack episode before decypharr finished
processing it), every subsequent request for that file returned the
cached failure without ever retrying — even after the file became
available seconds later.

The fix reclassifies HTTP 404 as CategoryRefetchable. On the next
request the link service invalidates the cached failure, calls
fetchDownloadLink to generate a fresh CDN URL, and validates that URL
instead. This correctly handles the "file not yet ready" race without
permanently poisoning the link cache.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After the 404→refetchable fix (4a7777c), a file whose CDN URL persistently
returns 404 (still processing or genuinely gone) triggered one requestdl call
per rclone/Plex poll, indefinitely. invalidateAndRefetch clears the account
cache on every call, so each subsequent poll sees a cache miss and hits the
provider API again — no upper bound.

Add a per-file refetch cooldown (5 min) tracked in a new refetchCooldowns map
keyed by infohash:filename. After the first 404 refetch, further requestdl
calls for that file are blocked until the window expires. The cooldown is
cleared immediately when the file becomes accessible, so recovery is instant
once the CDN URL is valid.

The CDN-expiry path (line 121) is intentionally excluded — that is a normal
3-hour rotation, not a failure loop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant