Skip to content

test: pin page dataset collector inputs - #653

Closed
castrojo wants to merge 1 commit into
mainfrom
fix/page-dataset-collector-tests
Closed

test: pin page dataset collector inputs#653
castrojo wants to merge 1 commit into
mainfrom
fix/page-dataset-collector-tests

Conversation

@castrojo

Copy link
Copy Markdown
Collaborator

Summary

  • Pin the matrix and applications collector shape tests to a checked-in legacy-input fixture.
  • Make Bazaar fallback scenario assertions order-independent while preserving exact names and casing.

Findings

  • Tests matrix: the collector was counting valid legacy rows with published passed/failed status and last_run. The expected 14 came from the historical inputs at 084b6fde3f; replaying those inputs produces 64 rows, 14 completed, and 50 waiting. Live docs/results later gained completed lane evidence (including bluefin-stable-smoke), so changing the collector to force 14 would under-report QA coverage. No collector bug.
  • Bazaar fallback: the live result update at 2044b9031f reordered the two matching names; the matcher preserves failed_scenarios order and still returns both distinct checks. The testsuite also now labels the related flatpak check Required system Flatpaks are installed, while bazaar user service is available remains in the common scripts suite. This is fixture/order drift, not a wrong signal match.

Validation

  • just lint
  • pytest -q tests/unit/test_page_dataset_collector.py ✅ 33 passed
  • pytest -q tests/unit ⚠️ 384 passed, 5 pre-existing unrelated failures in Zot/RECC tests

Use historical page inputs for matrix and Bazaar fallback shape assertions so live QA refreshes cannot change the expected coverage or scenario order.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo

Copy link
Copy Markdown
Collaborator Author

Superseded by #654, which landed these exact test changes as part of unblocking main.

main had become un-mergeable: every open PR failed the required test-validation check on the same 7 pre-existing baseline failures, and each PR fixed only a subset — so none could go green on its own. #654 collected every test-side fix into a single change to break that deadlock.

Verified: every file in this PR is now byte-identical to main, so there is nothing left to merge. The analysis here was correct and is preserved in #654's history — closing only to avoid a no-op merge commit. Thanks!

@castrojo castrojo closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant