feat(mdcode): guard the demo agent on rules stated in words

toolbox/mdcode/demo/semantic-model/agent/agent.ts

@@ -7,15 +7,22 @@
 //                 offset the charge."
 //
 // Nothing here mentions credits, orders, database tables or an operations desk.
-// Four steps: create the runtime, derive the tools, adapt them to ADK, run.
-// Point it at another semantic model and it is another agent, with no edit to
-// this file. That is the property it exists to test -- so the moment something
-// about this business has to be written here, the model was missing it and the
-// fix belongs there.
+// Five steps: create the runtime, hire a judge, derive the tools, adapt them to
+// ADK, run. Point it at another semantic model and it is another agent, with no
+// edit to this file. That is the property it exists to test -- so the moment
+// something about this business has to be written here, the model was missing
+// it and the fix belongs there.
+//
+// The judge is a capability rather than a policy: it can settle a rule stated
+// in words, and WHICH rules it is asked are the ones the model names in
+// `guards`. So supplying one here says nothing about commerce, the same way
+// supplying a database connection does not.
 
 import {FunctionTool, InMemoryRunner, LlmAgent} from '@google/adk';
 import {Type} from '@google/genai';
 
+import {ApiContext} from '../../../src/libts/gcp/context';
+import {GeminiJudge} from '../../../src/libts/gcp/gemini';
 import {callableTools, modelTools} from '../../../src/libts/semantic/runtime/agent_tools';
 import {createSemanticRuntimes} from '../../../src/libts/semantic/runtime/runtime';
 import {closeStore} from '../../../src/libts/semantic/runtime/store';
@@ -42,15 +49,34 @@ process.env.GOOGLE_GENAI_USE_ENTERPRISE ??= 'true';
 process.env.GOOGLE_CLOUD_PROJECT ??= runtime.store.project;
 process.env.GOOGLE_CLOUD_LOCATION ??= 'us-central1';
 
-// 2. Derive what the model offers, and keep what this binding can serve.
-//    `kcmd agent tools` prints all of it before a language model is involved.
+// 2. Hire something that can settle a rule stated in words. Pointed at the
+//    same project the store is in, so the binding profile is still the only
+//    place that says where any of this runs.
+//
+//    It may well be the same Gemini model the agent runs on, and it is not
+//    the same call: the judge is asked one rule about one set of attempted
+//    arguments, under a system instruction of its own, outside the agent's
+//    conversation. So there is nothing in the transcript for the agent to
+//    argue with, and no turn in which it can talk the gate round.
+const judge = new GeminiJudge(ApiContext.default(), {
+  project: process.env.GOOGLE_CLOUD_PROJECT,
+  location: process.env.GOOGLE_CLOUD_LOCATION,
+  model: process.env.DEMO_JUDGE_MODEL,
+});
+
+// 3. Derive what the model offers, and keep what this binding can serve.
+//    `kcmd agent tools --judge` prints all of it before an agent exists. The
+//    judge goes to the derivation rather than to each call: it is what decides
+//    whether a guarded action is offerable at all, and a tool offered on the
+//    strength of a judge and then called without one would be refused
+//    mid-call.
 const {callable, withheld, instruction} =
-    callableTools(modelTools({runtime}));
+    callableTools(modelTools({runtime, judge}));
 for (const tool of withheld) {
   console.error(`(withheld) ${tool.name}: ${tool.unavailable}`);
 }
 
-// 3. Adapt each one to ADK. A derived parameter already carries a JSON type and
+// 4. Adapt each one to ADK. A derived parameter already carries a JSON type and
 //    a description, which is the whole of a function declaration -- so this
 //    changes the shape and none of the content.
 const tools = callable.map(
@@ -69,8 +95,11 @@ const tools = callable.map(
       execute: (args: unknown) => tool.invoke(args as Record<string, unknown>),
     }));
 
-// 4. Run, printing each call and each answer so the transcript shows which
-//    tools the agent chose and what the store said back.
+// 5. Run, printing each call and each answer so the transcript shows which
+//    tools the agent chose and what the store said back. A refusal and a
+//    warning both come back inside a tool answer, so both are already in the
+//    transcript; what the agent does with them is the derived instruction's
+//    business rather than this file's.
 //
 // The instruction is the model's, not this file's: what this business asks of
 // anything acting on it is in `ai_context` in commerce.yaml, and how to use a

toolbox/mdcode/demo/semantic-model/agent/catalog/EntryGroups/commerce_demo/commerce.yaml

@@ -2,8 +2,8 @@
 #
 # The business is a small ecommerce operation: customers, their orders, and the
 # lines that make up an order. One thing can be done to it -- credit a customer
-# against an order -- and three policy rules say when that is allowed. The
-# README runs one request against it, the kind a support desk gets every day: a
+# against an order -- and the rules below say when that is allowed. The README
+# runs one request against it, the kind a support desk gets every day: a
 # customer was charged for shipping that should have been free.
 #
 # Nothing here says where the data lives, what the columns are called, or what
@@ -14,12 +14,12 @@
 # This file is the same bytes under both, which is the claim the pair exists to
 # make.
 #
-# Three constraints are declared and none is referenced. A constraint is inert
-# until an action names it in `guards`, and today's runtime evaluates none of
-# them -- so naming one here would make the action unrunnable rather than
-# checked. They are written down now because the policy exists now; wiring them
-# up is the next step, not a forgotten one. Add `guards: [CreditUnderReviewThreshold]`
-# to the action and re-run `kcmd agent tools` to see exactly what that costs today.
+# The same policy is written twice, once in arithmetic and once in words. A
+# constraint carries either an `expression` a query settles or a `judgment` a
+# language model settles, and `IssueCredit` guards on the judgments, because
+# those are the ones something can answer today. The expressions are declared
+# and named by nothing, which makes them inert: a constraint takes effect where
+# something references it.
 
 version: "0.2.0.dev0/google"
 
@@ -115,30 +115,102 @@ semantic_model:
         affects:
           - {concept: LineItem, operation: create, fields: [type, amount, memo]}
           - {concept: Order, operation: modify, fields: [total]}
+        # Which rules this call must pass, and the only place a constraint
+        # takes effect. All three are judgments, so every gate here costs a
+        # model call -- the loader says so at load time, and it is the price of
+        # gating on the half of the policy something can settle today.
+        guards:
+          - CreditUnderReviewThresholdWithJudge
+          - CreditMemoNamesAServiceFailure
+          - CreditIsNotSplitToAvoidReview
         ai_context:
           instructions: >-
             Give the order as its number, the amount in dollars, and a memo
             saying why. Look the order up first if you were given a customer
             name rather than a number: an Order is identified by its key alone.
+            Say in the memo what actually went wrong on the order -- a late
+            delivery, a damaged item, a shipping charge applied in error --
+            and, if this credit is part of a larger amount owed, say that and
+            give the total. A rule reads the memo, so a fact left out of it is
+            a fact the rule cannot weigh.
 
+    # Rules a query settles. All three are correct, none is referenced, and
+    # nothing evaluates one yet -- naming one in `guards` would make the action
+    # unrunnable rather than checked. They stay because they are the rules as
+    # they should end up: arithmetic a query settles costs no model call and
+    # cannot answer two identical calls differently.
     constraints:
       - name: CreditWithinOrderTotal
         expression: amount <= Order.total
         on_violation: escalate
         description: >-
           A credit cannot exceed the total of the order it credits. Lower the
           credit amount, or split it across the orders it actually covers.
+        # No counterpart in words, because the judge this demo hires is handed
+        # the attempted call and nothing else, and `Order.total` is in the
+        # store. That is this judge rather than judges in general: `Judge` is
+        # an interface, and one built over a store connection would settle
+        # this. What argues for leaving it an expression is that a query
+        # answers the comparison for nothing and answers it the same way
+        # twice.
 
       - name: CreditUnderReviewThreshold
         expression: amount <= 25
         on_violation: escalate
         description: >-
           A credit over $25 is above the self-service ceiling. A supervisor
           decides it.
+        # The one rule here that a judge CAN settle, because the threshold is
+        # read off the call's own `amount`. Stated in words below as
+        # `CreditUnderReviewThresholdWithJudge` -- the same rule, down to the
+        # description, differing only in who settles it -- and that twin is
+        # what the demo guards on. When the expression evaluator lands, the
+        # twin is the one to drop, which is why its name says how it is
+        # settled and this one's does not.
 
       - name: OrderTotalMatchesLineItems
         expression: Order.total == SUM(LineItem.amount)
         on_violation: reject
         description: >-
           An order's total must equal the sum of its line items, with credits
           subtracted. Nobody can approve an order that does not add up.
+        # No counterpart in words either, and out of reach of any judge, however
+        # much it can read: this constrains the state the write leaves behind,
+        # and guards are settled before the transaction opens. It wants to be
+        # checked inside the transaction, or declared in the schema.
+
+      # Rules stated in words, settled by a judge reading the attempted call.
+      # These are what `IssueCredit` guards on, and they carry all three
+      # consequences: escalate, warn and reject.
+      - name: CreditUnderReviewThresholdWithJudge
+        judgment: >-
+          The credit amount requested must not exceed 25 dollars, which is the
+          self-service ceiling for this desk. Read the amount as dollars.
+        on_violation: escalate
+        description: >-
+          A credit over $25 is above the self-service ceiling. A supervisor
+          decides it.
+
+      - name: CreditMemoNamesAServiceFailure
+        judgment: >-
+          The memo argument of this call must name a specific thing that went
+          wrong on the order: a late delivery, a damaged item, a shipping
+          charge applied in error. A memo saying only that the customer asked,
+          or that the credit is goodwill, or giving no reason at all, names no
+          failure and does not satisfy this rule.
+        on_violation: warn
+        description: >-
+          Say in the credit memo what actually went wrong with the order.
+
+      - name: CreditIsNotSplitToAvoidReview
+        judgment: >-
+          The credit requested must be the whole of what this order is owed,
+          not one piece of a larger amount divided to stay under the 25-dollar
+          self-service ceiling. A memo argument calling the credit a part, a
+          half, an instalment, the first or second of several, or a remainder,
+          or naming a total larger than the amount argument, does not satisfy
+          this rule.
+        on_violation: reject
+        description: >-
+          Raise this as a single credit for the full amount and send it for
+          supervisor review.

toolbox/mdcode/docs/semantic-model/actions.md

@@ -391,8 +391,29 @@ about handling a breach lives elsewhere.
 **Name fields model-qualified.** Write `LineItem.memo` rather than "the memo".
 `kcmd` resolves every `Entity.field` token in the text against the model and
 fails the push when the entity declares no such field, so a rename cannot leave
-the sentence pointing at nothing. The qualified name also tells the judge
-exactly which value to read.
+the sentence pointing at nothing. The qualified name also tells the judge which
+value to read.
+
+One caution comes with it. A guard is settled from the attempted call's
+arguments and nothing else, so a sentence phrased as a rule about stored data
+can be read as a rule the judge has no evidence for. The judge is told to answer
+that such a rule does not hold and to say in its reason what is missing, which
+refuses the call rather than passing it. That instruction binds a model rather
+than the runtime, so the rule can come back held instead, which costs a rule
+that never fires and says nothing. The rule above is settled correctly under
+this wording, but a guard that reads awkwardly as a statement about the call is
+worth rephrasing to name the argument, and worth testing against a case it
+should refuse. A rule that truly needs stored rows — comparing a credit against
+the order total, say — has no wording that reaches a judge given only the call,
+and belongs in an `expression` unless the judge supplied can query the store
+itself. Nothing in the `Judge` interface forbids one that does; the
+implementation shipped here makes a single model call with no tools.
+
+A rule about the state a write *leaves behind* is a different matter, and no
+judge settles it however much it can read. Guards are settled before the
+transaction opens, so a rule such as "an order's total equals the sum of its
+lines" has nothing to look at yet. That rule belongs inside the transaction or
+in the schema.
 
 **Say what does not count.** A rule with no negative example is graded against
 whatever the model guesses the author had in mind. "A memo that states only that
@@ -1218,10 +1239,31 @@ names a guard stated as an expression, nothing here evaluates one, and so the
 reported here instead — before any agent exists, rather than inside a
 transaction.
 
-`kcmd agent tools` supplies no judge, so an action guarded by a judgment is
-marked the same way and for the same reason: the derivation reports what the
-runtime would do with the judge it holds, and it holds none. Passing a judge
-through to the tools an agent is handed is the next step and is not taken yet.
+An action guarded by a judgment is marked the same way when the derivation
+holds no judge, and for the same reason: what is reported is what the runtime
+*would* do with what it is holding, and with no judge it would refuse. Supply
+one and the same action is offerable, with the same description and the same
+parameters:
+
+```console
+$ kcmd agent tools --judge
+Rules stated in words go to gemini-2.5-flash (us-central1).
+...
+  action  issue_credit  (IssueCredit)
+```
+
+The flag takes an optional model name, the same way
+[`kcmd action run --judge`](#a-guard-settled-in-words) does. What it does not do
+is call one: a judge settles a rule when an action runs, and printing what an
+agent is offered runs none, so this listing costs nothing however many guarded
+actions it names.
+
+The judge belongs to the derivation rather than to each invocation, which is the
+one place this is easy to get wrong. Whether a guarded action can be offered *at
+all* depends on holding a judge, so the same object has to answer `runnable` and
+answer the call. A tool derived with a judge and then invoked without one would
+be advertised as callable and refused mid-call, which is the drift the next
+paragraph is about.
 
 The tool is still returned, still named and still described. An action the
 model declares should not vanish from what the model offers; what it is waiting
@@ -1263,6 +1305,11 @@ if (!runtime.store) throw new Error(runtime.storeError);
 const {callable, withheld, instruction} = callableTools(modelTools({runtime}));
 ```
 
+`modelTools` also takes `judge`, and passing one is what makes an action guarded
+by a judgment callable at all. `GeminiJudge` implements the seam over Vertex AI;
+anything with a `decide` method does. Omit it and such an action is still
+derived, still named and still described, and reported in `withheld`.
+
 One call returns a runtime for every model document in the entry group. Each
 carries the store its deployment target names, the profile it was built under,
 and the document it was authored in, so a message about one model can say which
@@ -1327,12 +1374,23 @@ not. The place to put it is the model.
 ### A worked example
 
 `demo/semantic-model/agent/` is an agent built this way, running against a live
-operational store: a commerce model, a binding profile, and one file of 56 lines
-that names no table, no column and no business term. Thirteen of those lines are
-the adapter onto the agent framework. Its README walks the same four steps and
-states what the run cannot yet do — the $30 credit it issues is over the model's
-declared $25 self-service ceiling and is written anyway, because that ceiling is
-an expression and nothing evaluates one.
+operational store: a commerce model, a binding profile, and one file of 66 lines
+that names no table, no column, no business term and no dollar threshold.
+Thirteen of those lines are the adapter onto the agent framework. Its README
+walks the same steps and reaches all three of `on_violation`'s outcomes against
+that store: a $30 credit held because the model's $25 self-service ceiling is
+`escalate`, a credit written with a warning because the memo names no service
+failure, and a credit refused outright because the memo admits it is one piece of
+a larger amount.
+
+It also states what that costs and what it cannot do. The model guards on three
+judgments, so the demo pays three model calls per attempted write and loads with
+the all-judged warning; the two rules that compare the call against stored rows
+are declared but not enforced, one because the judge it hires reads no store and
+one because it constrains the state after the write. And the split-credit
+rule fires only because the model tells callers to disclose a split in the memo,
+which makes it a check on honest mistakes rather than a control — the version
+that would hold regardless is an expression over what is already stored.
 
 ## What is not modeled yet
 

toolbox/mdcode/docs/semantic-model/reference.md

@@ -101,9 +101,14 @@ runs nothing.
 A tool the runtime cannot call is listed and marked `[NOT RUNNABLE]` rather than
 dropped, with the reason in its description, so a refusal is visible before any
 agent exists. To see what a guard costs today, add a constraint to an action's
-`guards` and run this again. This command supplies no judge, so an action
-guarded by a `judgment` is marked unrunnable here even though
-`kcmd action run --judge` would settle it.
+`guards` and run this again. What is offerable depends on what the caller holds,
+so an action guarded by a `judgment` is marked unrunnable without `--judge` and
+callable with it.
+
+| Flag | Effect |
+|------|--------|
+| `--profile [name]` | Read the model under this binding profile. Defaults to `default_profile`, else the model's inline bindings. |
+| `--judge [model]` | List what an agent holding a judge is offered: an action guarded by a rule stated in words is callable rather than `[NOT RUNNABLE]`. Takes a Gemini model id, defaulting to `gemini-2.5-flash`, on the same region rule as [`action run --judge`](#action). No model is called either way — a judge settles a rule when an action runs, and this listing runs none. |
 
 A model whose profile names no Spanner database offers no tools, because calling
 one needs a store. That model is reported as offering none and the rest of the