feat(check): cap every finding list in the JSON payload - #60
Merged
Merged
Conversation
tybot02
approved these changes
Aug 26, 2026
tybot02
force-pushed
the
feat/check-output-cap
branch
from
August 26, 2026 10:23
1375b93 to
89a832a
Compare
Pointed at a 1024-document knowledge base, `check` printed 482KB of JSON,
of which 1024 rows were `{"document": ..., "reason": "missing"}` -- the same
reason every time, for a condition check_extractions itself documents as not
a fault. The one thing the run had to say (0 mismatched, and grounding unable
to check any of the 698 articles it found) was buried in it.
Every finding list is now cut to 20 items by default. The cap is a display
limit and never a measurement: `count` stays the true total whatever --limit
does, and `truncated` says outright that rows were dropped, so a capped list
cannot be misread as a complete one. `--limit 0` restores the full payload.
The cap is uniform, so the actionable lists (`mismatched`, `unsourced`) are
cut at the same 20 as the benign `missing` that motivated it -- the right
default, since a KB with thousands of genuine faults has a bigger problem
than its report length.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tybot02
force-pushed
the
feat/check-output-cap
branch
from
August 26, 2026 11:35
89a832a to
da9d971
Compare
tybot02
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Pointed at a 1024-document knowledge base,
kb-ai checkprinted 482KB of JSON, ofwhich 1024 rows were
{"document": ..., "reason": "missing"}— the same reason everytime, for a condition
check_extractionsitself documents as not a fault.A diagnostic that takes a screenful to say "nothing is wrong" does not get read. The
one thing that run had to say — 0 mismatched, and grounding unable to check any of the
698 articles it found — was buried in it.
Change
Every finding list is cut to 20 items by default.
The cap is a display limit and never a measurement:
countstays the true total whatever--limitdoes;truncatedsays outright that rows were dropped, so a capped list cannot be misreadas a complete one;
--limit 0restores the full payload,--limit Ntakes a different slice.Truncating into a bare list would have been worse than not truncating: twenty rows that
look like the whole set is a wrong answer, where twenty rows labelled "of 1024" is a
short one.
The cap is uniform, so the actionable lists (
mismatched,unsourced) are cut at thesame 20 as the benign
missingthat motivated it. That is the right default — a KBwith thousands of genuine faults has a bigger problem than its report length — but it
is why
--limit 0exists.Verification
kb_ai.commands.checkat 100% statement coverage (50 statements, 0 missing).--limitunset/
0/5/1000,countis identical for every list, rows equalmin(limit, count),truncatedis true exactly when rows were dropped, and thestderr summaries are byte-identical.
--limitvalues (negative, non-numeric, float) all exit 2 with argparse usage.🤖 Generated with Claude Code