feat(mdcode): let a judge read the model's own tables to settle a guard by libei · Pull Request #426 · GoogleCloudPlatform/knowledge-catalog

libei · 2026-09-14T19:43:30Z

Follows #423, now merged as e6d34fd. This branch is rebased onto it and carries the reading-judge feature plus a rewrite of the actions guide.

#423 left CreditWithinOrderTotal as an expression because a judge handed only the attempted call has no evidence for it: the order's total is on record and the call never states it. This gives the judge the model's own tables, so that rule gets a judged twin too.

$ ../../../dist/kcmd action run IssueCredit --judge --judge-reads-store --arg order=12346 --arg amount=3.00 --arg memo="Coupon applied late"
Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_agent_demo...
  rules stated in words go to gemini-2.5-flash (us-central1)
  it may read commerce's tables to settle them
  the judge reads: SELECT total FROM Orders WHERE order_id = 12346
  order: '12346' -> Order 12346
Committed at 2026-09-14T19:12:04.996343Z.

Nobody wrote that SQL. The rule names the order's total in words, and the judge was told which tables hold this model's data. Ask for more than the order is worth and the same guard refuses, comparing $20.00 against the $15.00 then on record — under the $25 ceiling, so no threshold rule catches it.

What the judge is allowed to do

It is shown the model rather than the database. The entities, tables, columns and dialect in its system instruction come from the selected binding profile, the same source the lookup tools come from, so a column the model does not bind is a column the judge is not told exists. One sentence in commerce.yaml produces SELECT total FROM Orders under the Spanner profile and SELECT order_total FROM purchase_order under the AlloyDB one.

Every statement is checked. A single read beginning with SELECT or WITH, checked after comments and string literals are blanked out so that a ; inside a quoted memo cannot split one statement into two.

Every statement is wrapped, and the wrap is the part carrying the weight rather than the keyword check. WITH gone AS (DELETE ... RETURNING *) SELECT * FROM gone begins with WITH and deletes rows. What runs is SELECT * FROM (...) AS judge_read LIMIT 21, and PostgreSQL refuses a data-modifying CTE inside a subquery:

$ psql "host=$PGHOST dbname=semantic_agent_demo" -c "SELECT * FROM (
WITH gone AS (DELETE FROM order_line WHERE order_id = 12346 RETURNING *)
SELECT * FROM gone
) AS judge_read LIMIT 21"
ERROR:  WITH clause containing a data-modifying statement must be at the top level

That ran against the live cluster, and the rows it named were still there afterwards. It matters most on AlloyDB, where the store's query path runs in an implicit transaction and a write reaching it would commit; on Spanner the read path is read-only and could not write whatever was sent.

Every read is printed and every result is capped. At most 20 rows, each cell clipped, and the judge is told when its answer was cut short. A judge that went and looked did something on the caller's behalf that the transcript has to show.

What it costs

Gemini refuses tools and responseSchema in one request, so the exchange is two-phase: a read loop with function declarations and no schema, then a verdict call with the schema and no tools. A guard that reads nothing costs two model calls rather than one, and each round of reading adds another, up to four reads. The demo's committed run put four guards to Gemini, one of which read once, for nine calls.

Verified against real instances

Every transcript the README pastes is a capture. On Spanner: the $3 credit above, the $20 refusal, the same call unsettled when the judge cannot read, and the whole action refused when there is no judge at all. On AlloyDB: both outcomes again in the other dialect against the other tables, plus the wrapped CTE refusal above. AlloyDB returns rows without column names, so the judge is handed values and told nothing about which column each came from — it settled the rule anyway, because the statement it wrote selected one column and it knew which one it asked for.

What this does not fix

A reading judge inherits the timing every guard has. Guards settle before the transaction opens, so it sees committed state, and two credits racing each other can each read a total that neither will leave behind. That is the strongest argument for keeping the arithmetic twin, which can eventually run inside the write. OrderTotalMatchesLineItems stays an expression for the same reason and a stronger one: it constrains the state the write leaves behind, so no judge settles it however much it can read.

A rule worded to read a memo column would be reading text a customer or an agent wrote, inside the prompt that decides whether a write proceeds. The judge is told to compose every statement itself and never to treat a value it read back as an instruction, results are capped and clipped, and every statement is printed. None of that is a guarantee, and the README says so.

Also

--judge-reads-store says what a judge may do rather than hiring one, so it needs --judge and is an error alone.
The actions guide gains A guard that reads a row; the reference gains the flag. Both drop the stale claim that the only store an action runs against is Spanner.
The three expressions stay declared and unreferenced.

The actions guide, rewritten

Nine commits at the end of the branch rewrite docs/semantic-model/actions.md. The guide runs to 1,553 lines and had settled into a whitepaper register: third person throughout, mechanics stated before motive, and headings that named objects rather than the job a reader came to do.

All 44 prose passages are rewritten. Your model rather than a model, the problem before the YAML that solves it, sentence-case headings under an 85-character cap on H2 and 50 on H3, captions on all eight visuals, and no nested lists. Three headings changed and the file's only H4 became an H3; nothing in the repo links to any of them.

Four sentence-level sweeps ran over it, each fixing an instance and then the class: garden-path openings, counts announced without their members, sentences whose subject is a reduced relative, and paragraphs that grant a permission before naming the situation that makes a reader want it.

The last of those is worth calling out because it survived the first pass. "You can also write no executor anywhere" tells you what the format allows and not why anybody would want it, so the option lands as trivia. Three paragraphs opened that way and each now opens on the case instead.

The rewrite is prose-only by construction: the file is split into alternating prose and fenced segments and only the prose is replaced, so all 564 lines of its 43 fenced blocks pass through byte-identical. Three exceptions are deliberate, all inside hand-drawn illustrations, where "exactly one" became "one kind only" and "a single row" with the column alignment preserved. Those keep the constraint they state. The two remaining uses sit inside captured kcmd agent tools output and are left verbatim, because captured output is a recording.

No prose line exceeds 80 columns. npm test is unchanged at 5 suites, 0 fail, 1106 tests.

A rule can name a value the caller never states. A credit that must not exceed the total of the order it is applied to is settled by comparing an argument against a stored number, and a judge handed only the attempted call has no evidence for it. The previous commit left `CreditWithinOrderTotal` as an expression for that reason. It now has a judged twin, because `kcmd action run --judge --judge-reads-store` gives the judge the model's own tables and lets it compose and run its own read. What the judge may do is derived rather than configured. The entities, tables, columns and dialect in its instructions come from the selected binding profile, the same source the lookup tools come from, so a column the model does not bind is a column the judge is not told exists. One sentence in `commerce.yaml` produces `SELECT total FROM Orders` under the Spanner profile and `SELECT order_total FROM purchase_order` under the AlloyDB one, with nobody writing either. Every statement is checked to be a single read beginning with `SELECT` or `WITH`, after comments and string literals are blanked out so that a `;` inside a quoted memo cannot split one statement into two. What runs is that text inside `SELECT * FROM (...) AS judge_read LIMIT 21`, and the wrap is the part carrying the weight rather than the keyword check: `WITH gone AS (DELETE ... RETURNING *) SELECT * FROM gone` begins with `WITH` and deletes rows, and PostgreSQL accepts a data-modifying CTE at the top level of a statement while refusing one inside a subquery. That matters on AlloyDB, where the store's query path runs in an implicit transaction and a write reaching it would commit; on Spanner the read path is read-only and could not write whatever was sent. At most 20 rows come back, every cell is clipped, the judge is told when its answer was cut short, and every statement is printed, because a judge that went and looked did something on the caller's behalf that the transcript has to show. Gemini refuses `tools` and `responseSchema` in one request, so the exchange is two-phase: a read loop carrying function declarations and no schema, then a verdict call carrying the schema and no tools. The cost follows from that shape. A guard that reads nothing costs two model calls rather than one, and each round of reading adds another, up to four reads. The wording of a judged rule and the capabilities of the judge have to be chosen together. A rule says the value is on record and has to be read, because the judge decides for itself whether to look. Put to a judge that cannot read, the same rule comes back as not holding, with a reason naming the value it could not get -- which is the safe direction, and still a refusal of every call. The demo agent guards on a fourth judgment and passes `--judge-reads-store`, and every transcript the README pastes is a capture of a run against a real instance. On Spanner: a $3 credit committed after the judge read the order total, a $20 credit on the same order refused against the $15 then on record, the same call unsettled when the judge cannot read, and the whole action refused when there is no judge at all. On AlloyDB: the same two outcomes in the other dialect against the other tables, plus the wrapped data-modifying CTE refused by the server with the rows it named still present afterwards. AlloyDB returns rows without column names, so the judge is handed values and told nothing about which column each came from; it settled the rule anyway, because the statement it wrote selected one column and it knew which one it asked for. `OrderTotalMatchesLineItems` is still an expression and still unreferenced, and no judge settles it however much it can read: it constrains the state the write leaves behind, and guards are settled before the transaction opens. That timing is also what a reading judge inherits. It sees committed state, so two credits racing each other can each read a total that neither will leave behind, which is the strongest argument for keeping the arithmetic twin. The actions guide gains `A guard that reads a row`, the reference gains the flag, and both drop the claim that the only store an action runs against is Spanner.

…either way Re-running section 3 against a real Spanner database showed the committed capture is a real run that does not reliably repeat. The memo `Coupon applied late` gives a cause without naming a thing that went wrong, which leaves it near the line `CreditMemoNamesAServiceFailure` draws: across eight runs of the identical call the judge held the rule five times and flagged it three times, printing an advisory warning above the commit on those three. The write lands either way, because the rule is advisory, so nothing about the outcome changes. A reader following the page would see output the page does not show and have no way to tell whether something had gone wrong. One paragraph now says that a judged rule is settled afresh for every call, so a case close to the line a rule draws has no one fixed answer. Everything else in the section reproduced exactly against the real store, including the judge composing its own read and comparing the argument against the row rather than against what the caller said.

A review of the read path found three, all in the layer between the judge's statement and the store. A `#` line comment is a comment in GoogleSQL and was not treated as one. `SELECT 1 # ;` and a newline was refused as two statements, which costs the judge one of the four reads it is allowed and tells it nothing it can act on. Worse, an apostrophe inside such a comment opened a quoted run that blanked everything after it, so a `;` behind the apostrophe became invisible to the single-statement check and both pre-checks were bypassed. The subquery wrap still caught what followed, which is why this was a hole rather than an incident. The first-word check read the first token whole, and split on parentheses as well as spaces. A union of two reads is written `(SELECT ...) UNION ALL (SELECT ...)` and came back as beginning with 'nothing'; `SELECT*FROM t` came back as beginning with 'SELECT*FROM'. Leading parentheses now come off and the keyword is matched as a prefix. The read budget was spent per turn rather than per statement. Gemini puts several function calls in one turn, and each call in the turn was executed, so four turns of three calls ran twelve reads against a limit the prompt told the model was four. Reads are now counted, the statements over the limit are answered with the reason rather than dropped, and the notice at the end reports what was actually spent. The file header claimed more than the code does, in two places. The subquery wrap is called a guarantee, and a query calling a function that writes is still a query: AlloyDB sends statements outside any transaction this client opened, so such a call would commit. And the wrap was said to bound an injected judge to the tables it was shown, when nothing holds a statement to the schema -- a read reaches whatever the credentials behind the action reach, and its rows can be quoted back in the verdict's reason. Both now say what is true.

…sted `kcmd action list` ends each action with the command that runs it, and names `--judge` when a guard is settled by judgment, because a suggested command certain to be refused is worse than no suggestion. Giving a judge tables to read broke that: `IssueCredit` in the commerce demo is guarded by a judgment comparing the credit against the order's recorded total, and the line the listing printed was refused for exactly the reason the demo page then uses as its negative example. Nothing in a constraint's wording says whether settling it takes a look at the store, so the line now offers `--judge-reads-store` wherever the model has a table the profile binds and a statement can name. A judge with nothing to look up looks nothing up, and the offer costs one model call. Where nothing is bound the flag fails at setup, so it is left off. The demo README's captured listing is re-captured from a live run.

The "A guard that reads a row" section led with four paragraphs of mechanism and showed a command whose captured output stopped at three progress lines, so a reader learned how the flag works without ever seeing what it does. It now opens with why the call alone cannot settle the rule, states the two prerequisites, shows the rule as it is actually written in the demo model, and then shows both outcomes live: the credit that commits, and the $200 credit refused against a $162.85 total the judge read for itself while ignoring the $900 the memo asserts. The mechanism notes follow, each led by what it means for the reader rather than by what the code does. The limits are stated honestly alongside them: the subquery wrap does not stop a query calling a function that writes, so the section says to give the action credentials no wider than the tables the model binds. The reference row for the flag gains the setup error it can raise.

The header opened on an essay about two kinds of rule and never stated what the module exports, what shape a JudgeStore has, or who constructs one. A reader arriving from a stack trace had to reconstruct all three from the code before any of the reasoning below made sense. It now leads with the two things a JudgeStore holds -- the schema text and the read method -- says which function builds it and which flag causes one to exist, and maps every other symbol in the file onto one of those two. The three steps read performs are given as a numbered list in the same shape run_action.ts uses for its pipeline, and the reasoning that used to come first now follows them: which of the three steps actually holds, what an injected judge is and is not limited by, and the timing the design does not fix. modelJudgeStore's docstring leads with what it returns rather than with how it fails; readOnly's names its step; read gets a roadmap line. GeminiJudge gains the call shape on the class, where a reader of decide lands, rather than only on the private method that explains it. No behaviour changes.

The guide explained the same facts several times over, each time from scratch, which is most of why it reads long. "Nothing evaluates an expression, so a guarded call is refused" was explained in full in four places. It is now explained once, in the subsection that exists for it, and pointed at from the other three. "Guards settle before the transaction opens" was stated three times; it is stated once, in the section on judging, and now carries the concurrency consequence that a later paragraph used to re-derive. Section 2's two status blocks said the same thing either side of one paragraph and are merged. Section 5 listed five habits for wording a judgment with three paragraphs about what a judge can settle wedged between the second and the third, so a reader hunting for the third crossed them. The five are now contiguous and the three follow. The reading-judge notes drop the parser mechanics -- comment blanking, why a subquery refuses a data-modifying CTE -- which tell a maintainer something and tell a reader of this page nothing. Both now live in judge_store.ts, where somebody who needs them is already looking. Renamed "A guarded action is refused, not run unchecked", an antithesis apposition this guide's own style bans, and fixed another at the entity lookup rules. Two quoted rule texts inside transcripts are elided; the sentences are in the YAML above them. No captured command output changed. 11,279 words to 10,811.

"An action does not contain the write" opened the guide with an abstract negation, using a definite reference ("the write") to something not yet introduced, and saying what an action is not before saying what it is. It is also untrue for one of the four executor kinds. Section 1 says of `sql`: "The fourth kind, `sql`, contains the write instead." The same words, the opposite claim. The paragraph now leads with what an action does and puts the split where it is accurate for all four kinds: how the write happens is kept separate, in the executor, which is the physical part and may come from a profile. With `sql` that executor carries the DML, which no longer contradicts anything.

The sentence used three forms of "describe" in one breath -- "the data this model describes ... that operation described where the data is described" -- and its second condition, wanting the operation recorded with the data, restated the purpose of the page as a reason to keep reading it. The sentence after it already makes that point, better. What is left is the condition that carries information, with three concrete examples in place of "an operation that changes the data this model describes".

"Two files decide everything below" pointed at material the reader has not reached, "one runnable thing" named the result with a placeholder, and "everything a caller or an agent gets is derived from them" asserted in the abstract what the diagram underneath shows concretely. The sentence now names what each file holds and what comes out of the pair, in the same terms as the diagram's nodes. "Nothing is written twice: each key appears in one of those two files" said the same thing a second time, after the diagram had said it, and is untrue: an executor declared in the model may also appear in a profile, which overrides it. That rule is stated where it applies, at the executor binding, so the coda is removed rather than corrected.

The first mention of `executor` in actions.md listed it beside the name and the parameters as though all three were settled in the model. A profile can replace it, and the guide said so ninety lines later. The opening sentence now says which parts a profile may touch. Added which executor kind suits a model-level default: an `mcp`, `rest` or `grpc` executor names an operation in another system and usually does not change with the store, so the model is its place, which is what the example already shows; a `sql` executor is written in one database's own table and column names, so it belongs in that database's profile. "An action the profile does not mention keeps whatever the model declared, so the executor written above..." -- the block written above is the profile's, and the sentence means the model's. Both are now named by kind. "Exactly one kind, where an executor is written at all." was a noun phrase punctuated as a sentence, and its second clause restated a point made in full two paragraphs later.

The paragraph introduced `mcp` in full, grouped `rest` and `grpc` as "the other two remote kinds" without saying what their fields mean, and reached `sql` as "a fourth". Four parallel things now read as four items, each with its field pair. Also replaced "No joins, no ranges, no aggregation, no ordering", a noun phrase punctuated as a sentence, in the lookup-tool paragraph. A sweep of actions.md and the demo README for verbless sentences found no others.

"An action the profile does not mention keeps whatever the model declared" drops the relative pronoun, so "the profile does not mention" reads as a fresh subject and verb, the real verb "keeps" arrives six words late, and "whatever the model declared" stacks a second embedded clause on top. It now reads "If a profile says nothing about an action, that action keeps the model's executor." Swept the whole file for the same shape and for the two habits that travel with it: a colon splicing a claim to its own gloss, and a clause fronted before the reader has a subject to attach it to. 43 sentences rewritten across all eight sections. Where a reduced relative sits in subject position the pronoun is restored; in object position it costs nothing and was left alone. Also fixed "Four stages, and a rule that stops at the first one does nothing", a verbless count-led opener of the kind already banned elsewhere in the file. No code block changed: all 564 fenced lines are byte-identical to the previous commit.

"A model gives everything two names, and a metric is written in the first of them" announces a count, never names either member, and then points at one by position. The reader holds "two names" open, reaches "the first of them" with no order established, and has to reconstruct both from the example that follows. It now reads "Every entity and field therefore carries two names, one in the model and one in the database." Two more of the same kind: - "Five rules, three outcomes, two that no query settles." Three bare counts, no verb, and nothing named. Now two sentences that say which. Checked against the listing above it: five rules, the distinct outcomes are escalate, reject and warn, and rules 4 and 5 are the two marked `judgment`. - "`invoke` answers with three states rather than two." The "two" is a set the reader has not been given. Dropped, and the three states are now named in order. Counts followed straight by a colon and the list are the good form and were left alone.

The guide read like a whitepaper: third-person throughout, mechanics before motive, and headings that named objects rather than the job a reader came to do. This rewrites all 44 prose passages in a developer-facing voice. - Second person for the reader and first person for the platform. "A semantic model says..." becomes "Your semantic model defines...". - Every section states the problem before the YAML that solves it, and every paragraph names the situation before granting the permission that answers it. Three paragraphs opened by granting permission; each now opens on the case. - Sentence case for headings, an 85-character cap on H2 and 50 on H3. Three headings changed and the file's only H4 became an H3; no inbound anchor pointed at any of them. - Captions on all eight visuals: three figures and five tables. - No nested lists, and generic concepts are no longer capitalized. Prose only, with three exceptions inside hand-drawn illustrations where "exactly one" became "one kind only" and "a single row". Those preserve the constraint they state and the column alignment around them. The two remaining uses sit inside captured `kcmd agent tools` output and are left verbatim. All 564 fenced lines are otherwise byte-identical, no prose line exceeds 80 columns, and npm test is unchanged at 5 suites, 0 fail, 1106 tests.

libei requested review from amirhormati and dlychagin-gg September 14, 2026 19:43

libei force-pushed the judge-sql branch from 04ee288 to 0407b1b Compare September 14, 2026 20:37

libei added 14 commits September 14, 2026 21:33

libei marked this pull request as ready for review September 15, 2026 06:15

libei merged commit 36f777c into GoogleCloudPlatform:main Sep 15, 2026
7 checks passed

Navigation Menu

Uh oh!

feat(mdcode): let a judge read the model's own tables to settle a guard - #426

feat(mdcode): let a judge read the model's own tables to settle a guard#426
libei merged 15 commits into
GoogleCloudPlatform:mainfrom
libei:judge-sql

libei commented Sep 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

libei commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What the judge is allowed to do

What it costs

Verified against real instances

What this does not fix

Also

The actions guide, rewritten

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

libei commented Sep 14, 2026 •

edited

Loading