Skip to content

ci: sync the vendored metric preset list with the Vespa CLI - #1349

Draft
thomasht86 wants to merge 2 commits into
masterfrom
thomasht86/sync-metric-presets
Draft

thomasht86 wants to merge 2 commits into
masterfrom
thomasht86/sync-metric-presets

Conversation

@thomasht86

Copy link
Copy Markdown
Collaborator

Follow-up to #1348 (PR 2 of the production tests plan). Based on that branch for now so the resource files exist; GitHub retargets this to master once #1348 merges and its branch is deleted.

What

  • .github/workflows/sync-metric-presets.yml: scheduled Mon-Thu at midnight (same as release-vespacli.yml) plus workflow_dispatch. Runs the sync script and, when the vendored file changed, re-runs the two vendored unit tests and opens a PR from sync-metric-presets/<tag> to the default branch. Re-runs for the same tag force-push the branch and reuse an open PR.
  • vespa/utils/sync_metric_presets.py: stdlib-only. Reads repository/path/ref from vespa/resources/metric-presets.source.json, resolves the latest Vespa CLI release tag (skipping lsp-*, drafts and prereleases), fetches the raw file at that tag, byte-compares with vespa/resources/metric-presets.json. On drift it overwrites the file, bumps ref, prints added/removed preset names and writes changed/tag/previous_ref/summary to GITHUB_OUTPUT. A 404 (tags before v8.753.16 have no file) is reported as nothing to do. --tag and --dry-run for local use.
  • tests/unit/test_sync_metric_presets.py: tag picking, diff/summary and the sync paths (identical, 404, drift, dry run) with mocked fetches.

Verified locally

  • Latest tag (v8.753.16): byte-identical, nothing to do.
  • --tag v8.751.13: 404 path, nothing to do.
  • Locally edited copy: drift reported with added/removed names, file restored byte-identical and ref rewritten in the same format (no diff in metric-presets.source.json).

Notes

  • Uses GITHUB_TOKEN via gh, so the repo setting "Allow GitHub Actions to create and approve pull requests" must be on, and the auto-PR will not trigger the PR CI workflows (standard GITHUB_TOKEN limitation). Switch to a PAT/app token if CI on the auto-PR is wanted.
  • The PR-opening path is only exercised in CI once this is on master with real drift.

🤖 Generated with Claude Code

Add a scheduled workflow (Mon-Thu midnight, like release-vespacli.yml, plus
workflow_dispatch) that resolves the latest Vespa CLI release tag, fetches
client/go/internal/cli/cmd/metric-presets.json at that tag and, if the bytes
differ from vespa/resources/metric-presets.json, overwrites the vendored copy,
bumps the ref in metric-presets.source.json and opens a PR.

The script only uses the standard library and reads repository/path/ref from
the source file. Tags before v8.753.16 do not have the file (HTTP 404), which
is reported as nothing to do. `lsp-*` tags are skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Base automatically changed from thomasht86/production-tests to master September 21, 2026 08:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant