Skip to content

Say which commit a measurement in an answer was produced at (#123) - #128

Merged
iderex merged 1 commit into
mainfrom
records/an-answer-names-the-commit-it-measured
Aug 12, 2026
Merged

Say which commit a measurement in an answer was produced at (#123)#128
iderex merged 1 commit into
mainfrom
records/an-answer-names-the-commit-it-measured

Conversation

@iderex

@iderex iderex commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Closes #123

What this changes

docs/decisions/0016-an-answer-names-the-commit-it-measured.md takes the field
of the three options the issue set out, with its reasons and what each rejected
option would have cost. It supersedes 0008 for that one thing and answers to
0013, so the field is optional and an absence is never refused. 0008 gains
the one line 0000 allows a superseded record to take.

Measurement-Commit carries the object name of the commit the measurement was
produced at, written in full. That is the shape record 0004 already writes for
the neighbouring case, where an answered experiment's code is removed and the
record gains a line naming the commit that removed it with the full hash.

The runner refuses a value that is not the shape of an object name, in
internal/check/measurement.go. The field and its check are declared together
rather than beside the four fields 0008 fixes, which is what headerDateFields
already argues for at its own list: a field a later record adds arrives with the
check that reads it.

The template names the field in its prose rather than carrying it in its header,
for the reason Answer-Written is named the same way. A template that ships a
field filled in teaches every new record to declare a value it does not have.

What failure it prevents

An answer names the command, the platform, the architecture and the toolchain,
and no version of the code. The prototype it points at is allowed to change, and
record 0004 lets it be removed entirely. When either happens the answer keeps
its numbers and loses the thing that produced them, and it does so silently,
because the record still reads as complete. A reader who runs the command again
cannot tell whether they are reproducing the measurement or measuring something
else.

The abbreviation is what the fixture is built from because it is the mistake
somebody actually makes. The short name is what git prints in a log, it is what
a hand copies, it resolves on the machine it was copied on, and it stops
resolving on a repository that has grown into a collision. It also cannot be
told from a typo, since seven characters that resolve to nothing and seven
characters somebody mistyped are the same string.

What was run

At 19ac2bf0442b8d9e1ffcbe594d5befb04c1c7031.

$ go build ./... && go vet ./... && gofmt -l .
(no output from any of the three)

$ go test -count=1 ./...
ok  	github.com/Flowfin/lab/cmd/lab	5.515s
ok  	github.com/Flowfin/lab/cmd/pullrequest	1.767s
?   	github.com/Flowfin/lab/experiments/reading-a-tree-of-records	[no test files]
ok  	github.com/Flowfin/lab/internal/check	2.217s
ok  	github.com/Flowfin/lab/internal/hardware	2.141s
ok  	github.com/Flowfin/lab/internal/invariants	2.355s
ok  	github.com/Flowfin/lab/internal/prose	1.555s
ok  	github.com/Flowfin/lab/internal/pullrequest	3.265s

$ go run ./cmd/lab check .
examined .
1 experiment directory walked, 1 record read
17 decision records read
the time this run read is 2026-08-12T01:27:27Z
0 refused

The check was deleted from the walk and the suite went red on its fixture, which
is what says the fixture proves the check rather than passing beside it:

--- FAIL: TestCases/record-with-a-measurement-commit-abbreviated
        expected refusal not produced: record-measurement-commit-is-not-a-commit

The absence boundary is load-bearing rather than decorative. Making an absent
field refusable reddens the existing corpus, because almost every record on the
board and in the fixtures carries no measurement:

--- FAIL: TestCases/record-with-an-empty-state
--- FAIL: TestCases/a-promotion-naming-a-branch
--- FAIL: TestCases/record-with-a-question-dated-on-the-day-of-the-run
--- FAIL: TestCases/record-abandoned-with-a-reason
--- FAIL: TestCases/two-experiments-answering-to-their-own-slugs
--- FAIL: TestCases/a-record-that-needs-nothing-and-registers-no-test
        refusal produced that no case expected: record-measurement-commit-is-not-a-commit

Both edits were reverted and the suite above was run afterwards. That second run
is also what record 0013 predicts: a check that makes older records red is the
wrong check, and this one is written so that it cannot.

The means is Go, in the package that already walks the tree and reads the
header, and Markdown for the record, in the shape record 0000 fixes. It
carries the three rules: the property is refusable through the Refusal the
package already has, one fixture trips exactly it with a near neighbour that
differs only in the value, and the numbers above come from commands. It adds no
language, runtime or dependency this tree does not carry.

This change has no second reader. Nobody but me has read it, and the evidence
above stands in place of that rather than beside it.

What this does not do

It does not resolve the name. Nothing asks git whether the object is in this
repository, whether it is a commit rather than a blob, or whether the command in
the answer was ever run against it, because the runner reads a checkout and
opens no connection. Forty hexadecimal characters that resolve to nothing pass.
Written at the check and in the record.

It does not make the fact guaranteed. A record quoting a measurement and
carrying no Measurement-Commit is legal and always will be, which is record
0013's price paid deliberately. The template, the review and record 0016 are
what make the field usual instead.

It does not touch the one record already on the board. Adding the field to it is
the migration record 0013 refuses, so it was left exactly as it was and the
field has no user in the tree today.

It does not decide anything about a measurement produced against another
project's code, which has the same shape and a different answer.

An answer quotes a command and its output and says nothing about which
version of the code the command ran against. Record 0004 lets that code
be removed entirely afterwards, and when it is, the answer keeps its
numbers and loses the thing that produced them silently, because the
record still reads as complete.

Record 0016 takes the field of the three options the issue set out, with
the reasons and what each rejected option would have cost. It supersedes
0008 for that one thing and answers to 0013, so the field is optional and
an absence is never refused. Record 0008 gains the one line record 0000
allows a superseded record to take.

Measurement-Commit carries the object name in full, which is what record
0004 already writes for the neighbouring case. The runner refuses a value
that is not the shape of an object name. The abbreviation is the fixture
because it is the mistake somebody actually makes: it is what git prints,
it resolves on the machine it was copied on, it stops resolving on a
repository that has grown into a collision, and it cannot be told from a
typo.

Two things are written at the check rather than left to a green run.
Nothing here asks git whether the object exists, is a commit, or has
anything to do with the command in the answer, because the runner reads
a checkout and opens no connection. And the absence stays unrefusable, so
the field makes the fact recordable and never guaranteed.

The one record on the board is left exactly as it was. Record 0013 says a
record already on the default branch is not edited, not migrated and not
marked, and adding the field to it afterwards is the migration that rule
refuses.

The template names the field in its prose rather than carrying it in its
header, for the reason Answer-Written is named the same way.

Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com>
@iderex
iderex merged commit 9807dc7 into main Aug 12, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Say which commit a measurement in an answer was produced at

1 participant