Skip to content

Graceful shutdown can interrupt in-flight symlink placement for an entry already logged "downloaded", leaving it permanently stuck with no error #407

Description

@colehellman

Is there an existing issue for this?

Current Behavior

A graceful shutdown of decypharr (docker stop/docker compose stop, or any restart) can interrupt in-flight symlink placement for a torrent entry that has already been logged as "downloaded". The destination folder is left permanently empty, the queue entry is never retried, and nothing — not decypharr's own state, not Sonarr/Radarr's queue — ever surfaces an error for it.

Sequence observed in the log around the incident:

[manager] Processing torrent Action=symlink ... Provider=torbox
... submitted to torbox ...
... downloaded ...
[http] Shutting down gracefully...
[decypharr] Decypharr has been stopped gracefully.
[manager] Closing active download queue
Job queue stopped

Whatever entry was mid-processing (between being logged "downloaded" and its symlink actually being created) at the moment Closing active download queue runs never gets a second attempt after the next start. There's no re-scan of "downloaded but not yet symlinked" entries on startup, and no periodic reconciliation catches it either.

Because the torrent is already reported downloaded by the provider (RealDebrid/TorBox), decypharr's own state and the arr apps' queue both look completely healthy: Sonarr/Radarr show sizeleft: 0 (100%), trackedDownloadStatus: ok, no error message — indistinguishable from a normal completed download, except the destination folder is empty and stays empty indefinitely.

Expected Behavior

Either:

  • The graceful-shutdown handler should finish (or cleanly abort and re-queue) any in-flight symlink placement before Closing active download queue completes, or
  • On startup, decypharr should reconcile "downloaded" entries against actual symlink/filesystem state and re-run symlink placement for any that are missing it, or
  • At minimum, an entry whose symlink target never landed should eventually surface as an error/failed state instead of staying silently "ok" forever.

Steps To Reproduce

Reliably reproducing the exact shutdown timing is impractical, but the failure mode is structural and reproducible in effect:

  1. Have decypharr processing an action=symlink torrent (RD or TorBox backend) — provider has confirmed the torrent downloaded, decypharr is in the process of creating the destination symlink(s).
  2. Send a graceful shutdown signal to the decypharr process at that moment (docker compose stop, docker restart, a Watchtower update, an out-of-band script restart — any of these trigger the same handler).
  3. Let decypharr come back up.
  4. Check the destination folder for that release: it exists but is empty. Check Sonarr/Radarr's queue for the corresponding item: sizeleft: 0, trackedDownloadStatus: ok, no error — permanently stuck in this state with no further action from decypharr ever occurring.

In our environment the actual trigger for the specific incident we traced could not be pinned down (it wasn't our own watchdogs, wasn't Watchtower's schedule, wasn't a cron/systemd timer) — the point is that any restart at the wrong moment causes this, not one specific trigger.

Scope (measured, not assumed)

We were cautious not to assume every empty download folder is this bug. We swept all folders under our TV download directory (2,236 total, 2,232 empty) and cross-referenced every one against Sonarr's own /api/v3/parse endpoint (matching releases to actual episode records, not fuzzy filename matching):

  • 1,796 (80.5%) were harmless — the matched episode already has a file elsewhere from a prior successful import; an empty folder is decypharr's normal post-import end state.
  • 278 (12.5%) were this exact bug — episode confirmed to have no file anywhere, folder empty, sizeleft: 0.
  • 12 (0.5%) partially-imported pack edge cases.
  • 144 (6.5%) didn't resolve to a specific episode via parse.

So this is a real, non-trivial leak (278 confirmed-stuck items accumulated over roughly a week in a moderately active instance), not a rare corner case — but it's also not "most empty folders are broken," which is worth stating explicitly since the raw 2,232 number is misleading on its own.

Environment

  • OS: Debian (Proxmox LXC)
  • Version: 2.5-stable (ghcr.io/sirrobot01/decypharr:latest)
  • Docker Install: Yes
  • Browser: N/A

What branch are you running?

Main/Latest

Trace Logs? Not Optional

We do not have trace-level logs for this — our instance runs at info level and this was found via retrospective log analysis, not a live capture set up in advance to catch it. The exact info-level log sequence around the incident (torrent processing → downloaded → graceful shutdown, in that order, with no further mention of the affected release after restart) is quoted above and is the full evidence we have. Happy to re-run at trace level and attach fresh logs if a maintainer can point at what would most help isolate this — the failure is intermittent (tied to shutdown timing, not a specific title), so we can't guarantee reproducing it on demand even with tracing on, but can leave tracing enabled here for a while and follow up if it recurs.

Trace Logs have been provided as applicable

  • I have read and followed the steps in the documentation link and provided the required trace logs — not checked: we only have info-level logs, explained above, not the required trace logs. Filing anyway per the checkbox's own carve-out ("required as applicable") since we believe the info-level evidence here is specific enough to be actionable, but flagging honestly rather than checking a box that isn't true.

We worked around this on our side with a script that detects queue items on a decypharr client with sizeleft == 0, past a stall-age threshold, where the arr app's own hasFile field says the episode/movie genuinely has no file — then blocklists+retries the release from the arr side. That's a mitigation from the consumer side, not a fix to the underlying interrupted-shutdown behavior, which is why we're filing this upstream.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions