Make the test suite build its own binary, and create directories parents-first - #4
Merged
Merged
Conversation
…ing $PATH The suite shelled out to a bare "tree2scaffold", so it exercised whichever copy happened to be installed in $GOBIN and passed on any developer machine that had one. CI has no such binary, so the first run of the new native-release workflow failed five tests outright. A package-level TestMain now builds cmd/tree2scaffold once into a temp dir and every test invokes that. The suite tests the working tree, and local runs and CI finally mean the same thing.
Apply ranged over the directory set directly, and that set is a map, so the creation order was randomised on every run. When a nested directory came up before its own parent, and that parent still existed as a file awaiting force conversion, MkdirAll failed with "not a directory". This is why force_mode_with_hidden_files failed intermittently rather than never: it only lost when .github/workflows was drawn ahead of .github. Sorting the directories puts every parent ahead of its children, since a parent path is always a prefix of its child. The regression test repeats the scenario 50 times, because a single pass would clear the old code roughly half the time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The
native releaseworkflow added in the distribution work ran for the first time onmainand failed immediately.mainis currently red. Two independent defects, both of which the suite had been hiding.1. The tests never actually ran the code under test
Five tests shelled out to a bare
"tree2scaffold":That resolves through
$PATH, so on any machine with the tool installed in$GOBINthe suite passed — while testing the installed binary, not the working tree. CI has no such binary:A package-level
TestMainnow buildscmd/tree2scaffoldonce into a temp dir and every test invokes that.tree2scaffold_integration_test.goalready did this correctly and now shares the one build instead of doing its own.2. Directory creation order was randomised
With the PATH problem fixed,
force_mode_with_hidden_filesstill failed — intermittently, which is why it had survived.Applyranged over the directory set directly, and that set is amap:Go randomises map iteration. When
.github/workflowscame up before.github, and.githubstill existed as a file awaiting force conversion,MkdirAllfailed withnot a directory. It only lost the coin flip about half the time.Measured before the fix — 5 runs of that one test on
main, and on the branch with only fix #1 applied:main(pre-existing)Sorting the directory list puts every parent ahead of its children, since a parent path is always a prefix of its child.
Test plan
$GOBINremoved fromPATH— i.e. under CI conditionsTestApplyCreatesParentsBeforeChildrenrepeats 50× (one pass would clear the old code ~half the time); verified it fails at iteration 0 against the unfixed codego vetclean,gofmtcleangolangci-lint: 11 issues before, 11 after — all pre-existing, none addedNote
This does not touch the 11 pre-existing lint issues or the unused
verifyGeneratedFilesUnusedhelper — separate cleanup.