fix: keep collecting after empty top retry timeout - #39
Conversation
9f3c108 to
6768357
Compare
axisrow
left a comment
There was a problem hiding this comment.
Requesting changes. The retry fallback is aimed at the same-name overwrite race, but it recovers from every DownloadTimeoutError. This can accept an empty top export after a genuinely failed or ambiguous retry, so the retry failure is lost.
axisrow
left a comment
There was a problem hiding this comment.
Requesting changes. The exception classification is improved, but the overwrite fallback still can produce inconsistent output when the retry actually overwrites the original file in place.
|
Addressed review 5062769720 in commit 4257ea5. Before retrying, the collector copies the first CSV to a safe output-root temporary file. If the distinct DownloadNoNewPathError occurs, the fallback uses that preserved file, so parquet, manifest, and optional raw CSV all describe the same original export even if Chrome overwrote the live path. Added regression coverage for in-place overwrite. Verification: 179 passed; ruff clean. |
|
Addressed the cleanup coverage gap in commit acc6aa6. Extended the successful empty-export retry test to assert that output_root contains no retry temporary files after _collect_one completes. The test fails when finally cleanup is replaced with pass. Verification: 179 passed; ruff clean. |
axisrow
left a comment
There was a problem hiding this comment.
Requesting changes. The data-consistency fix is correct on the normal path, but temporary backup cleanup is not exception-safe.
|
Addressed review 5062787940 in commit b0526a5. Backup creation and copy2 now run inside the cleanup scope, with retry_backup initialized to None; a copy failure removes the created temporary file before propagating the original error. Added regression coverage for copy2 failure and leftover retry files. Verification: 180 passed; ruff clean. |
axisrow
left a comment
There was a problem hiding this comment.
Approving. The latest revision addresses the prior findings: retry failures now have a distinct no-new-path signal, the original CSV is backed up and restored before the fallback, and backup cleanup covers copy failures. The added tests cover ambiguous failures, in-place overwrite consistency, and cleanup.
Closes #38
Summary
Verification
uv run pytest -q(175 passed)uv run ruff check src/wordstat/collector.py tests/test_collector_batch.pyLive artifacts were written only under
/tmp/wordstat-issue38-DSuHtn.