Read at 3222207.
What happens
In three examples the variable that sets the agent's model is also the judge's model, so changing one moves the other:
examples/e2e_demo.py — L74 MODEL_NAME = os.environ.get("MODEL_NAME", "gemini-3-flash-preview") → L322 LlmAgent(model=MODEL_NAME) → L439 Client(endpoint=MODEL_NAME) → L484 LLMAsJudge.correctness() → client.evaluate. The judge query is rendered with endpoint=self.endpoint (src/bigquery_agent_analytics/client.py:2326-2332), which is MODEL_NAME.examples/context_graph_adcp_demo.ipynb — same wiring: cell 6 MODEL_NAME, cell 10 LlmAgent(model=MODEL_NAME), cell 15 Client(endpoint=MODEL_NAME), cell 39 client.evaluate(...) (cells 0-indexed).examples/nba_agent_trace_analysis_notebook.ipynb — cell 4 MODEL_NAME = "gemini-2.5-flash" (no env override), cell 8 LlmAgent(model=MODEL_NAME), cell 14 LLMAsJudge.correctness(threshold=0.6, model=MODEL_NAME).
Nothing in those files or the README says the judge is the agent's model.
Silent defaults
With no endpoint or model passed, both grading paths use a hardcoded "gemini-2.5-flash": Client(endpoint=None) → DEFAULT_ENDPOINT (client.py:531, evaluators.py:52) on the caller's BigQuery connection; LLMAsJudge(model=None) → "gemini-2.5-flash" (evaluators.py:726) via genai.Client() on ambient credentials. No warning on either path when that coincides with the agent's model.
What this is not
The repo also shows the right shape, which is why this is filed as three examples and not a design complaint: examples/e2e_notebook_demo.ipynb separates AGENT_MODEL from SDK_ENDPOINT (cells 5/8/12); examples/skill_evolution_lab has dedicated AGENT_MODEL / ANALYST_MODEL / JUDGE_MODEL with different defaults (run_e2e_demo.sh); examples/agent_improvement_cycle uses one model_id for both roles and README.md:685 says so. No claim is made about score inflation — nothing was executed.
Reproduce
git checkout 3222207
grep -n 'MODEL_NAME' examples/e2e_demo.py # 74, 322, 439
grep -n 'DEFAULT_ENDPOINT\|or "gemini-2.5-flash"' src/bigquery_agent_analytics/evaluators.py src/bigquery_agent_analytics/client.py
Suggested fix
In the three examples, a second variable for the judge (JUDGE_MODEL / a separate endpoint=) defaulting to a different model, as e2e_notebook_demo already does. Optionally, one line in the Client and LLMAsJudge docstrings naming the default judge, and a note that endpoint=MODEL_NAME makes the agent grade itself.
Read at
3222207.What happens
In three examples the variable that sets the agent's model is also the judge's model, so changing one moves the other:
examples/e2e_demo.py— L74MODEL_NAME = os.environ.get("MODEL_NAME", "gemini-3-flash-preview")→ L322LlmAgent(model=MODEL_NAME)→ L439Client(endpoint=MODEL_NAME)→ L484LLMAsJudge.correctness()→client.evaluate. The judge query is rendered withendpoint=self.endpoint(src/bigquery_agent_analytics/client.py:2326-2332), which isMODEL_NAME.examples/context_graph_adcp_demo.ipynb— same wiring: cell 6MODEL_NAME, cell 10LlmAgent(model=MODEL_NAME), cell 15Client(endpoint=MODEL_NAME), cell 39client.evaluate(...)(cells 0-indexed).examples/nba_agent_trace_analysis_notebook.ipynb— cell 4MODEL_NAME = "gemini-2.5-flash"(no env override), cell 8LlmAgent(model=MODEL_NAME), cell 14LLMAsJudge.correctness(threshold=0.6, model=MODEL_NAME).Nothing in those files or the README says the judge is the agent's model.
Silent defaults
With no endpoint or model passed, both grading paths use a hardcoded
"gemini-2.5-flash":Client(endpoint=None)→DEFAULT_ENDPOINT(client.py:531,evaluators.py:52) on the caller's BigQuery connection;LLMAsJudge(model=None)→"gemini-2.5-flash"(evaluators.py:726) viagenai.Client()on ambient credentials. No warning on either path when that coincides with the agent's model.What this is not
The repo also shows the right shape, which is why this is filed as three examples and not a design complaint:
examples/e2e_notebook_demo.ipynbseparatesAGENT_MODELfromSDK_ENDPOINT(cells 5/8/12);examples/skill_evolution_labhas dedicatedAGENT_MODEL/ANALYST_MODEL/JUDGE_MODELwith different defaults (run_e2e_demo.sh);examples/agent_improvement_cycleuses onemodel_idfor both roles andREADME.md:685says so. No claim is made about score inflation — nothing was executed.Reproduce
Suggested fix
In the three examples, a second variable for the judge (
JUDGE_MODEL/ a separateendpoint=) defaulting to a different model, ase2e_notebook_demoalready does. Optionally, one line in theClientandLLMAsJudgedocstrings naming the default judge, and a note thatendpoint=MODEL_NAMEmakes the agent grade itself.