Skip to content

Make the test suite build its own binary, and create directories parents-first - #4

Merged
lancekrogers merged 2 commits into
mainfrom
fix/tests-build-their-own-binary
Aug 4, 2026
Merged

Make the test suite build its own binary, and create directories parents-first#4
lancekrogers merged 2 commits into
mainfrom
fix/tests-build-their-own-binary

Conversation

@lancekrogers

Copy link
Copy Markdown
Owner

Why

The native release workflow added in the distribution work ran for the first time on main and failed immediately. main is currently red. Two independent defects, both of which the suite had been hiding.

1. The tests never actually ran the code under test

Five tests shelled out to a bare "tree2scaffold":

cmd := exec.Command("tree2scaffold", "-root", tmp, "-yes")

That resolves through $PATH, so on any machine with the tool installed in $GOBIN the suite passed — while testing the installed binary, not the working tree. CI has no such binary:

integration_checksum_test.go:306: scaffold failed:
  exec: "tree2scaffold": executable file not found in $PATH

A package-level TestMain now builds cmd/tree2scaffold once into a temp dir and every test invokes that. tree2scaffold_integration_test.go already did this correctly and now shares the one build instead of doing its own.

2. Directory creation order was randomised

With the PATH problem fixed, force_mode_with_hidden_files still failed — intermittently, which is why it had survived. Apply ranged over the directory set directly, and that set is a map:

for dir, isDir := range paths {

Go randomises map iteration. When .github/workflows came up before .github, and .github still existed as a file awaiting force conversion, MkdirAll failed with not a directory. It only lost the coin flip about half the time.

Measured before the fix — 5 runs of that one test on main, and on the branch with only fix #1 applied:

result
main (pre-existing) 1 FAIL / 5
PATH fix only 2 FAIL / 5
both fixes, 8 full-suite runs 0 FAIL / 8

Sorting the directory list puts every parent ahead of its children, since a parent path is always a prefix of its child.

Test plan

  • Full suite passes 8/8 with $GOBIN removed from PATH — i.e. under CI conditions
  • New TestApplyCreatesParentsBeforeChildren repeats 50× (one pass would clear the old code ~half the time); verified it fails at iteration 0 against the unfixed code
  • go vet clean, gofmt clean
  • golangci-lint: 11 issues before, 11 after — all pre-existing, none added

Note

This does not touch the 11 pre-existing lint issues or the unused verifyGeneratedFilesUnused helper — separate cleanup.

…ing $PATH

The suite shelled out to a bare "tree2scaffold", so it exercised
whichever copy happened to be installed in $GOBIN and passed on any
developer machine that had one. CI has no such binary, so the first run
of the new native-release workflow failed five tests outright.

A package-level TestMain now builds cmd/tree2scaffold once into a temp
dir and every test invokes that. The suite tests the working tree, and
local runs and CI finally mean the same thing.
Apply ranged over the directory set directly, and that set is a map, so
the creation order was randomised on every run. When a nested directory
came up before its own parent, and that parent still existed as a file
awaiting force conversion, MkdirAll failed with "not a directory".

This is why force_mode_with_hidden_files failed intermittently rather
than never: it only lost when .github/workflows was drawn ahead of
.github. Sorting the directories puts every parent ahead of its
children, since a parent path is always a prefix of its child.

The regression test repeats the scenario 50 times, because a single
pass would clear the old code roughly half the time.
@lancekrogers
lancekrogers merged commit d3f356b into main Aug 4, 2026
2 checks passed
@lancekrogers
lancekrogers deleted the fix/tests-build-their-own-binary branch August 4, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant