Skip to content

fix: keep collecting after empty top retry timeout - #39

Merged
axisrow merged 10 commits into
mainfrom
ao/wordstat-26/root
Aug 31, 2026
Merged

axisrow merged 10 commits into
mainfrom
ao/wordstat-26/root

Conversation

@axisrow

@axisrow axisrow commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Closes #38

Summary

  • fall back to the first legitimate empty top export when the same-name retry times out
  • keep retry behavior for successful top exports and fail-closed dynamics behavior unchanged
  • add regression coverage for a timed-out top retry

Verification

  • uv run pytest -q (175 passed)
  • uv run ruff check src/wordstat/collector.py tests/test_collector_batch.py
  • live CDP run on :9233 with the issue parameters: 3/3 phrases completed; all 12 parquet files written; top views recorded as empty

Live artifacts were written only under /tmp/wordstat-issue38-DSuHtn.

@axisrow
axisrow force-pushed the ao/wordstat-26/root branch from 9f3c108 to 6768357 Compare August 31, 2026 02:31

@axisrow axisrow left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The retry fallback is aimed at the same-name overwrite race, but it recovers from every DownloadTimeoutError. This can accept an empty top export after a genuinely failed or ambiguous retry, so the retry failure is lost.

Comment thread src/wordstat/collector.py Outdated

@axisrow axisrow left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The exception classification is improved, but the overwrite fallback still can produce inconsistent output when the retry actually overwrites the original file in place.

Comment thread src/wordstat/collector.py Outdated
@axisrow

axisrow commented Aug 31, 2026

Copy link
Copy Markdown
Owner Author

Addressed review 5062769720 in commit 4257ea5. Before retrying, the collector copies the first CSV to a safe output-root temporary file. If the distinct DownloadNoNewPathError occurs, the fallback uses that preserved file, so parquet, manifest, and optional raw CSV all describe the same original export even if Chrome overwrote the live path. Added regression coverage for in-place overwrite. Verification: 179 passed; ruff clean.

@axisrow

axisrow commented Aug 31, 2026

Copy link
Copy Markdown
Owner Author

Addressed the cleanup coverage gap in commit acc6aa6. Extended the successful empty-export retry test to assert that output_root contains no retry temporary files after _collect_one completes. The test fails when finally cleanup is replaced with pass. Verification: 179 passed; ruff clean.

@axisrow axisrow left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The data-consistency fix is correct on the normal path, but temporary backup cleanup is not exception-safe.

Comment thread src/wordstat/collector.py Outdated
@axisrow

axisrow commented Aug 31, 2026

Copy link
Copy Markdown
Owner Author

Addressed review 5062787940 in commit b0526a5. Backup creation and copy2 now run inside the cleanup scope, with retry_backup initialized to None; a copy failure removes the created temporary file before propagating the original error. Added regression coverage for copy2 failure and leftover retry files. Verification: 180 passed; ruff clean.

@axisrow axisrow left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The latest revision addresses the prior findings: retry failures now have a distinct no-new-path signal, the original CSV is backed up and restored before the fallback, and backup cleanup covers copy failures. The added tests cover ambiguous failures, in-place overwrite consistency, and cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant