test: pin page dataset collector inputs - #653
Closed
castrojo wants to merge 1 commit into
Closed
Conversation
Use historical page inputs for matrix and Bazaar fallback shape assertions so live QA refreshes cannot change the expected coverage or scenario order. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
|
Superseded by #654, which landed these exact test changes as part of unblocking
Verified: every file in this PR is now byte-identical to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Findings
passed/failedstatus andlast_run. The expected 14 came from the historical inputs at084b6fde3f; replaying those inputs produces 64 rows, 14 completed, and 50 waiting. Livedocs/resultslater gained completed lane evidence (includingbluefin-stable-smoke), so changing the collector to force 14 would under-report QA coverage. No collector bug.2044b9031freordered the two matching names; the matcher preservesfailed_scenariosorder and still returns both distinct checks. The testsuite also now labels the related flatpak checkRequired system Flatpaks are installed, whilebazaar user service is availableremains in the common scripts suite. This is fixture/order drift, not a wrong signal match.Validation
just lint✅pytest -q tests/unit/test_page_dataset_collector.py✅ 33 passedpytest -q tests/unit