fix(ci): vendor autodoc prompt templates so docs regeneration works again - #11
Merged
Merged
Conversation
…gain .runtime_ci/autodoc.json points its templates at scripts/prompts/*.dart in this repo, but the autodoc scaffolder never wrote those files; the tooling instead resolved them from the runtime_ci_tooling package root via the consumer repo's package_config.json. 5e00fe5 removed runtime_ci_tooling from dev_dependencies (per the tooling's own "global activation only" guidance), which broke that resolution: manage_cicd ran `dart run scripts/prompts/...` on files that do not exist, got empty stdout, and skipped every module with "Empty prompt". Hashes never advanced, so docs-freshness fails on every run since the sampling-capability change touched lib/src/server and lib/src/types, and the release workflow (gated on CI success on main) cannot fire. Vendor the four prompt scripts from runtime_ci_tooling v0.23.13 lib/src/prompts (verbatim, formatted at 120 cols) at the exact paths autodoc.json already declares. Template resolution now hits the primary consumer-repo branch and prompt generation produces non-empty output for both quickstart and api_reference locally. scripts/ is excluded from analysis to keep the vendored files diffable against upstream. The tooling-side fix (resolve templates from the globally activated package) belongs in runtime_ci_tooling.
With the vendored templates in place, autodoc got past prompt generation
and hit the next failure: `npm install -g @google/gemini-cli@latest` now
resolves to 0.56.x, whose PolicyEngine rejects the
`--allowed-mcp-server-names=` flag that runtime_ci_tooling's
docsOnlyPolicyArgs passes for MCP isolation ("Invalid policy rule:
mcpName is required if specified (cannot be empty)"), so every Gemini
pass dies at CLI startup. The last successful regen (2026-03-25) ran on
0.35.0, the @latest of that day and the version runtime_ci_tooling
v0.23.13 was written against.
Pin 0.35.0 everywhere the generated workflows install the CLI (ci,
issue-triage, release) so one upstream CLI release cannot break all
Gemini-backed jobs at once. These files are template-generated; the
durable fix (pin in the templates, drop or adapt the empty
--allowed-mcp-server-names flag) is tracked in
open-runtime/runtime_ci_tooling#70.
Author
|
Executive summary, since the diff looks bigger than it is. What actually changed (556 lines, ~10 of them decisions):
How it broke, and when:
Why fix it at all:
Blast radius: CI/docs tooling only. No |
Author
|
I'm open to different approached to fixing this, but currently I'm downgrading this in priority to focus on cost saving issues and will come back to this. |
mark-at-pieces
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs-freshnessfails onmainsince the sampling-capability merge, and because the release workflow gates on CI success onmain, releases cannot fire. The staleness itself is expected (lib/src/serverandlib/src/typeschanged); the defect is thatauto-autodoc's regeneration skips every module withEmpty prompt for <type>, skipping, so the docs never regenerate and the hashes never advance. This PR makes regeneration work again by vendoring the prompt templates at the paths the config already declares.Root Cause
.runtime_ci/autodoc.jsondeclares its templates as consumer-repo paths (scripts/prompts/autodoc_quickstart_prompt.dart, ...) -- that is what the autodoc scaffolder writes -- but the scaffolder never writes the script files themselves. Resolution relied on a fallback inmanage_cicd: when the relative path does not exist in the consumer repo, it resolves theruntime_ci_toolingpackage root from the consumer repo's.dart_tool/package_config.jsonand reads the scripts fromlib/src/prompts/there.5e00fe5removedruntime_ci_toolingfrom dev_dependencies, following the tooling's own v0.23.13 guidance ("use global activation only -- do not add as a dev_dependency"). Globally activated packages are not in the consumer repo'spackage_config.json, so the fallback returns null,templatePathstaysscripts/prompts/..., and the prompt step runsdart runon a file that does not exist:The sub-process failure is swallowed (
runSyncreturns empty stdout), which is why the log says "Empty prompt" instead of "file not found". The last successful regen (00ef6ee, 2026-03-25 02:21 UTC) ran hours before5e00fe5(04:55 UTC) landed; every run since has this failure. The v0.23.13 guidance and the v0.23.13 template resolution contradict each other -- that tooling-side fix (resolve templates from the globally activated package, e.g. via its embedded-prompts extraction) belongs inruntime_ci_tooling.Second layer, exposed once prompts generated: the workflows install the Gemini CLI unpinned (
npm install -g @google/gemini-cli@latest). Current latest (0.56.x) rejects the--allowed-mcp-server-names=(empty) flag thatruntime_ci_tooling'sdocsOnlyPolicyArgspasses for MCP isolation --Invalid policy rule: mcpName is required if specified (cannot be empty)-- so every Gemini pass died at CLI startup (first commit'sauto-autodocrun on this PR). The tooling's comment documents the old contract ("Gemini CLI 0.34+ behavior: empty list -> none"), which no longer holds.Fix
Vendor the four prompt scripts from
runtime_ci_toolingv0.23.13lib/src/prompts/--_ci_config.dart,autodoc_quickstart_prompt.dart,autodoc_api_reference_prompt.dart,autodoc_examples_prompt.dart-- verbatim (formatted at 120 cols) intoscripts/prompts/, the exact pathsautodoc.jsonalready declares. Template resolution now takes the primary consumer-repo branch (File('$repoRoot/$templatePath').existsSync()), before the broken fallback is ever consulted. The scripts are self-contained (dart:io,dart:convert, and the sibling_ci_config.dartonly).scripts/**is excluded from analysis so the vendored files stay byte-diffable against upstream.The source variants are vendored rather than the tooling's embedded (AOT) variants: the embedded quickstart prompt is trimmed (no changelog-context section), and the source variants are what produced the existing March docs.
Second commit: pin
@google/gemini-cli@0.35.0(the@latestof the last successful regen, 2026-03-25, and the version v0.23.13's flags were written against) at all 9 install sites acrossci.yaml,issue-triage.yaml, andrelease.yaml, so one upstream CLI release cannot break every Gemini-backed job at once. These are template-generated files; the durable fix belongs in the templates (tracked with the resolver bug in open-runtime/runtime_ci_tooling#70).Test plan
dart run scripts/prompts/autodoc_quickstart_prompt.dart Types <repo>/lib/src/types <repo>/lib/src/types-- exit 0, 8904 bytes of prompt (previously: file did not exist, empty stdout)autodoc_api_reference_prompt.dart-- exit 0, 1871 bytesdart pub global activate ... --git-ref v0.23.13+manage_cicd autodoclocally -- reaches the Gemini gate ("Gemini CLI not installed") instead of failing at prompt generation; the Gemini half only runs in CI where the CLI andGEMINI_API_KEYexistdart analyze-- 71 issues, identical tomainbaseline (scripts/**excluded)dart format --line-length 120 --set-exit-if-changedon the vendored files -- 0 changesauto-autodocrun: prompt generation succeeded for all modules (Pass 1: Generating QUICKSTART.md...with noEmpty prompt), proving the vendored templates resolve; Gemini then crashed at startup on the unpinned-CLI flag incompatibility aboveauto-autodocrun on the second commit is the end-to-end proof: regenerateserver/types/top_leveldocs, push abot(autodocs)commit to this branch,docs-freshnessgreen (reviewer-visible in this PR's checks)