Problem
runtime/python/pyproject.toml has litellm>=1.0.0,<1.50.0 as a hard dependency, pulled in solely for the llm_judge grader. Every user installs a heavy transitive tree whether or not they ever use a judge, and our version pin becomes theirs.
It is also why test_llm_judge_fails_without_api_key and test_llm_judge_uses_configured_model fail on a clean checkout without it.
Proposal
[project.optional-dependencies]
judge = ["litellm>=1.0.0,<1.50.0"]
graders.py already handles the import being absent and returns a clear "install with pip install litellm" message — so the fallback path exists and is tested.
Acceptance criteria
Problem
runtime/python/pyproject.tomlhaslitellm>=1.0.0,<1.50.0as a hard dependency, pulled in solely for thellm_judgegrader. Every user installs a heavy transitive tree whether or not they ever use a judge, and our version pin becomes theirs.It is also why
test_llm_judge_fails_without_api_keyandtest_llm_judge_uses_configured_modelfail on a clean checkout without it.Proposal
graders.py already handles the import being absent and returns a clear "install with pip install litellm" message — so the fallback path exists and is tested.
Acceptance criteria