Keep an experiment's code out of the commands the gate runs - #129
Merged
Conversation
Record 0009 decided that nothing which runs on its own builds or executes anything under experiments/. It named no mechanism, and every command in the guide and in the workflows walked the whole module, so the sentence was true of the intention and not of the tree. At 9807dc7 `go list ./...` returned the experiment's package next to the runner's seven, and six build entries, three suite entries, the vet job and the static analysis compiled it on every pull request. What that would have cost is the failure 0009 was written against. A prototype somebody abandoned half-written holds the board red on six platforms it was never about, and the two ways out of a red board are deleting the evidence or rewriting the answer. The commands now name the runner's two roots. gofmt takes cmd and internal, because it reads directories rather than a package pattern, and it is the one command here that fails on an experiment without compiling it. CodeQL moves from autobuild to a manual build of the same two roots: autobuild builds the module, and the first line of that file says its subject is the runner's own source. Record 0017 carries the mechanism, the three options it was chosen over and what each would have cost. The largest of those is measured rather than argued. An experiment as its own module, or behind a build constraint, stops `go run ./experiments/reading-a-tree-of-records` resolving from the root of a checkout, and that command is written twice in an answered record which gains lines rather than having them replaced. The command was run at this commit and still reports its three numbers. What nothing refuses is written in the record rather than left to be found: a Go file at the root, or under a root directory the layout names that is neither cmd/ nor internal/, is outside these patterns. Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com>
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #122
What this changes
Every command this repository is gated by now names the runner's two roots
rather than the whole module:
./cmd/... ./internal/...where a command takesa package pattern, and
cmd internalwhere gofmt takes directories. The guide,the build and test workflow and the headless workflow all run the same strings.
CodeQL moves from autobuild to a manual build of the same two roots.
docs/decisions/0017-what-the-gate-reads-of-an-experiment.mdis the record. Itsupplies the mechanism that record
0009decided the property for, names thethree options it was chosen over, and says what each of those would have cost.
What failure it prevents
Record
0009says nothing that runs on its own builds or executes anythingunder
experiments/. It named no mechanism, and an experiment lives insidethis module, so the sentence was true of the intention and not of the tree.
The failure that shape produces is the one
0009was written against. Aprototype abandoned half-written holds the board red on six platforms it was
never about, and the two ways out of a red board are deleting the evidence or
rewriting the answer. Both destroy what the record exists to preserve.
What was run
At the commit being pushed,
24723778bdcdae1b52d7886cab8cc5cc2c5a389e.The state before, on
origin/mainat9807dc7:The four commands the guide asks for, at this commit:
The suite executed 93 top-level tests, skipped none, and printed the
integration-hardware disclosure once, which is what the two workflow jobs count:
The record checks:
The command the rejected options would have broken, run at this commit from the
root of a checkout. It is written twice in an answered record, and it still
resolves and still reports its three numbers:
That is one machine and one platform, and the numbers are not the point here.
Whether the command resolves at all is.
The CodeQL change is the one thing above that was not run locally. Manual build
mode is exercised by this pull request's own run and nowhere else, and if that
check goes red the change is wrong rather than the check.
What this does not do
It refuses nothing new. The mechanism is a pattern in five files, and a Go file
at the root of the tree, or under a root directory the layout names that is
neither
cmd/norinternal/, sits outside all of them with nothing to say so.root-holds-a-directory-the-layout-does-not-namerefuses a new root directory,so what escapes is a package added under a directory already named. That hole is
written into the record rather than left to be found, and it is smaller than the
one being closed rather than absent.
It does not change what the record checks read. Reading an experiment's record
as text is
0009's own carve-out and is untouched.It takes nothing back from
0009and it does not decide anything that recordleft to its own reopening condition.
Nobody else has read this change. What stands behind it is the evidence above
rather than a second reader.