Stream file transfers with cancellation and atomic downloads

[qwrobins]

Summary

• Stream uploads/downloads with backpressure and replay upload streams safely across validated redirects.
• Add a shared transfer deadline (--transfer-timeout, default 120 seconds) and SIGINT/SIGTERM/caller cancellation.
• Stage downloads on the destination filesystem and atomically replace only after success, preserving existing files and cleaning up failures.
• Preserve HTTPS, redirect limits, and cross-host credential filtering; update CLI help, reference, and bundled skills.

Validation

• Full test suite: 674 passed, 1 skipped
• Bun-focused transfer/command tests: 40 passed
• Typecheck, TypeScript build, compiled binary build, and binary help smoke test passed
• Covers large transfers, backpressure, redirects, stalled/truncated bodies, cancellation, and failed writes with existing destinations

Closes #184

Greptile Summary

Streams file uploads and downloads with bounded memory, cancellation, shared deadlines, validated redirects, and atomic destination replacement.

• Replays upload streams safely across redirects while preserving cross-host credential filtering.
• Stages downloads beside their destination and commits them only after successful completion.
• Adds --transfer-timeout, signal-driven cancellation, updated command metadata, documentation, bundled skills, and transfer-focused tests.
• Improves recoverable composite-workflow failure output and preserves completed resources and meaningful exit codes.
• Adds pagination failure checkpoints and validates atomic Unix installer replacement.

Greptile automatically discovered a related ticket that helped explain the purpose of this PR: file downloads should respect backpressure while preserving cancellation, cleanup, and final rename semantics.

Confidence Score: 5/5

The PR appears safe to merge; the previous transfer findings are fully addressed and no new actionable failures remain.

Successful uploads now require clean source-stream completion, and staging cleanup can no longer alter a committed download’s outcome or mask its primary error. The subsequent workflow, pagination, CLI-output, installer, and HTTP-classification changes preserve their documented contracts without introducing a blocking defect.

Important Files Changed

Filename	Overview
src/core/io/file-transfer.ts	Implements deadline-bound, cancellable streaming transfers, redirect validation, upload replay, and atomic staged downloads.
src/commands/file.ts	Integrates streaming transfers and timeout validation while preserving recoverable upload-and-attach workflow state.
src/core/runtime/workflow.ts	Models successful, failed, and skipped workflow steps with structured errors, completed resources, and meaningful exit codes.
src/core/pagination/page.ts	Centralizes page validation and attaches last-committed pagination progress to request failures.
install.sh	Stages and verifies Unix binaries before atomically replacing an existing installation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[File command] --> B{Upload or download?}
  B -->|Upload| C[Open regular file and record size]
  C --> D[Request signed upload target]
  D --> E[Stream PUT under shared deadline]
  E --> F{Redirect?}
  F -->|Yes| G[Validate HTTPS and filter cross-host headers]
  G --> E
  F -->|No, success| H[Await complete upload stream]
  H --> I[Optionally create attachment]
  B -->|Download| J[Validate Linear upload URL]
  J --> K[Stream GET under shared deadline]
  K --> L[Write private staging file with backpressure]
  L --> M[Atomically rename over destination]
  E --> N[Cancellation or failure cleanup]
  K --> N
  L --> N

Loading

_{Reviews (3): Last reviewed commit: "Merge main and preserve structured uploa..." | Re-trigger Greptile}

Context used (4)

• Linear — Respect backpressure for S3 downloads
• Knowledge Base — Curated Linear resource commands
• Knowledge Base — Command runtime and GraphQL transport
• Knowledge Base — Output, errors, and dry runs

[qwrobins]

Greploop complete (GitHub): 3 review rounds, final confidence 5/5, 2 findings fixed/resolved, 0 unresolved comments.

Final validation after integrating main:

• Full suite: 810 passed, 1 skipped
• Bun-focused transfer/command/workflow tests: 94 passed
• Typecheck, TypeScript build, and binary build passed
• All PR CI checks passed (Ubuntu, Windows, and Unix installer tests on Ubuntu/macOS).