- Live service: https://label-verifier-173761965521.us-central1.run.app
- What was built, and the assumptions and limits: APPROACH.md
- Every measurement, in the order it was made: docs/measurements/README.md
Checks whether a drinks label carries what its application filed, and whether it carries the statutory health warning of 27 CFR 16.21.
It reads the label, compares what it read to what was filed, and answers per claim with the
evidence: the text it read, the spelling it compared that to, how far apart they were, and a crop
of the part of the label the answer rests on. Where the evidence does not decide, it says so.
REVIEW and NOT_FOUND are answers, not failures — the thing this is built to avoid is asserting
something about a label that the label does not bear out.
Everything runs in one process. There is no external model, no API key and no network call at run
time: the reader's weights are embedded in the binary and /health reports their SHA-256.
Nothing is stored — an uploaded image lives as long as the request that brought it.
Needs Go 1.24 and the ONNX Runtime shared library, version 1.24 or later (the Go binding asks for API version 24; older runtimes refuse).
# Linux
curl -L https://github.com/microsoft/onnxruntime/releases/download/v1.27.1/onnxruntime-linux-x64-1.27.1.tgz | tar xz
mkdir -p third_party/onnxruntime && cp onnxruntime-linux-x64-1.27.1/lib/libonnxruntime.so* third_party/onnxruntime/On Windows put onnxruntime.dll in third_party/onnxruntime/. Anywhere else, point
TREASURY_ORT_LIB at the library.
go build ./...
go test -short ./...The service and its page
go run ./cmd/serve # http://localhost:8080Open it and you get one screen, with two panels. One label: pick it, type what the application filed, press the button. Results come back as a row per claim — what it is in plain words, the verdict as a word, what was filed, what the label appears to say, and a picture of where on the label that came from. Many labels: a CSV of claims and a ZIP of images, answered as a table that fills in as each label is read, a row per label with its claims openable underneath.
One label from the command line
go run ./cmd/decode -ttb eval/real50/0047.json eval/real50/0047.pngIn a container
Every v* tag publishes one, built by the same Dockerfile:
docker run --read-only -p 8080:8080 ghcr.io/elijahmontenegro/treasury:latestDistroless, non-root, read-only filesystem, 117 MB. It listens on PORT, and /health reports
the commit it was built from and the SHA-256 of each model inside it, so what is running can
always be traced to a tag.
To build it yourself, which is what the release does:
docker build -t treasury \
--build-arg COMMIT=$(git rev-parse HEAD) \
--build-arg COMMIT_TIME=$(git show -s --format=%cI HEAD) .Two registries, and why. The release publishes to GHCR, under this repository, so the artefact sits beside the code. The deployed service pulls from Artifact Registry, because Cloud Run will not pull from GHCR, and pushing to Artifact Registry from the release workflow would mean putting a Google credential in a workflow that already holds a token which can publish. Deploys are manual, so the copy is too — one command, and the layers are the ones GHCR already has:
docker buildx imagetools create \
--tag us-central1-docker.pkg.dev/PROJECT/cloud-run-source-deploy/label-verifier:1.0.1 \
ghcr.io/elijahmontenegro/treasury:1.0.1deploy/artifact-cleanup.json is the retention policy in force on that registry: tagged images
are kept, so is anything a tagged index references, so are the five most recent versions, and
untagged ones older than thirty days go. It is in the tree rather than only in a console, so what
deletes an image is something you can read.
The contract is api/openapi.yaml, written by hand; the types and the server
interface are generated from it, and CI fails if the two have drifted. The running service serves
the document at /openapi.yaml.
GET / |
the operator's page |
POST /verify |
multipart: image (PNG or JPEG) and claims (JSON). Verdicts, evidence, crops, timings, build identity |
POST /verify/batch |
multipart: claims (CSV) and images (ZIP). application/x-ndjson, one line per label as it finishes |
GET /health |
readiness and which build is answering, with the model hashes |
GET /openapi.yaml |
the specification |
curl -X POST localhost:8080/verify \
-F image=@eval/real50/0047.png \
-F claims=@eval/real50/0047.json
curl -X POST localhost:8080/verify/batch \
-F claims=@claims.csv \
-F images=@labels.zipThe batch CSV's first row is its column names. One column must be image, naming a file inside
the ZIP; the rest may be beverage, brand, class, producer, address, origin, abv and
net_ml. Answers stream — a line is written as each label finishes rather than the whole batch
being assembled at the end, so a caller sees progress and nothing accumulates with the size of the
batch: the archive is read where the multipart form put it rather than into memory, and what is
held is its directory and the one entry a worker is reading. A batch may name at most a thousand
labels, and an archive that declares more than two gigabytes of contents is refused before
anything is unpacked, both before the engine runs. Verdicts
come back without crops, since three hundred labels of them would be a response measured in
gigabytes; each says whether one exists, and /verify will show it for that label alone.
Brand, class or type, the permittee and its address, origin, alcohol content, net contents, and the statutory warning with its heading.
A claim is compared to the label's own text, not to a guess about it. The rules that decide are
the same seven the engine has accumulated under measurement, each one added because a specific
false assertion made it necessary — a number must be read exactly as a whole run of digits, a name
taken from inside a longer line must be delimited at both ends and must be the whole name, a
reading with characters missing may not contradict, and so on. docs/measurements/README.md records each
with the label that forced it.
The code:
verify/ the decision: distance, margin, refusal, evidence
internal/ocr/ the reader: PP-OCRv4 detection and recognition, embedded, run on the CPU
ttb/ the domain: what the regulation says, and nothing about mechanism
api/ the specification, and the code generated from it
internal/httpapi/ the service and the page
cmd/serve the service cmd/decode one label
cmd/eval the measurement cmd/whymissed where a claim was lost
The evidence:
eval/real50/ fifty real COLA registry labels, their filed claims and what they print
eval/real10/ the ten step 4 started with, kept because its tables are in the record
testdata/ one synthetic label and an augmented copy, for the tests
docs/measurements/ every table in the order it was measured, and the analyses behind them
docs/evidence/ crops the record points at, where a sentence needed a picture
deploy/ the registry retention policy, which is configuration and belongs in the tree
eval/real50 and eval/real10 hold each label's image, the claims its application filed, and a
transcription of what the label actually prints, which is what recall is scored against. The
synthetic corpus the tables also report is generated rather than stored, and is not in the
repository. The reported tables were made with the system's own font families, from which the
evaluation half is held out:
go run ./cmd/gen set -n 500 -seed 1 -fontdir /path/to/fonts # writes synth/Not here, and named because the record refers to them: python/digits and python/encoder
trained the digit classifier and the glyph encoder of the retired engine, and were deleted with it
at step 19a.
docs/measurements/README.md carries every table in the order it was measured, including the ones that were
later corrected, and says which. The short version: on fifty real registry labels the engine
verifies 156 of the 192 claims they carry, with precision 1.00 across 550 labels and not one
false assertion; median 1.3 s and 95th percentile 3.9 s per label on four cores, and 4.5 s
and 10.1 s end to end against the deployed service, warm, where a vCPU is about half a core of
the machine those local figures come from.