Skip to content

Add build-on-tag CI pipeline for grpc-tunnel-server image, fix server vulns - #137

Merged
shawnburke merged 5 commits into
mainfrom
claude/docker-image-tag-validation-yxd4om
Aug 31, 2026
Merged

Add build-on-tag CI pipeline for grpc-tunnel-server image, fix server vulns#137
shawnburke merged 5 commits into
mainfrom
claude/docker-image-tag-validation-yxd4om

Conversation

@shawnburke

@shawnburke shawnburke commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

TLDR

Adds a build pipeline for the grpc-tunnel server image

  • Refactors the trivy workflow to be shared b/t agent and tunnel
  • Adds BUILD_VERSION tagging to tunnel
  • Fixes an existing trivy issue where it runs on main vs the PR/branch SHAs
    • Also fixes some vulns that popped up as part of this work

Summary

This change adds build version tracking to the tunnel server, allowing operators to identify which release is running in a deployed pod without needing to inspect the image tag or shell into the container.

Key Changes

  • Config: Added BuildVersion field to Config struct that reads from AXON_BUILD_VERSION environment variable, defaulting to "dev" for unstamped builds
  • Health endpoint: Updated /healthcheck handler to include build_version in the JSON response alongside existing fields like status, server_id, and clients
  • Startup logging: Added explicit log line at startup showing the build version
  • Docker image: Added BUILD_VERSION build argument to Dockerfile that gets set as AXON_BUILD_VERSION environment variable
  • CI/CD:
    • Created new tunnel-server-release.yml workflow to handle release tag builds (separate from the path-filtered tunnel-server-image.yml)
    • Updated tunnel-server-image.yml to accept workflow_call trigger and generate semver tags for releases
    • Configured build args to stamp <ref>-<sha> into release images
  • Tests: Added comprehensive test coverage for build version handling in both config and health handler

Implementation Details

  • The getBuildVersion() function mirrors the agent's implementation to ensure consistency across the platform
  • Empty or unset AXON_BUILD_VERSION values are normalized to "dev" rather than blank strings to avoid confusion in logs and API responses
  • The health handler was extracted into a named function (newHealthHandler) to make its response shape testable
  • Release images are stamped at build time by CI, while local builds default to "dev"

https://claude.ai/code/session_01LVEy7CPtRVgPYSLHdFy2Rb

claude added 2 commits August 31, 2026 04:14
Tagging the repo built and published a version-stamped agent image but not
a tunnel server one. docker.yml triggers on `tags: v*` and emits semver
tags; tunnel-server-image.yml triggers only on branch pushes, and its
metadata emitted only branch and sha tags. So `git tag v1.2.3` produced
cortex-axon-agent:1.2.3 with no matching cortex-axon-tunnel-server:1.2.3 —
there was no way to stamp a version covering both halves of the system.

Add the semver tag patterns to the tunnel server image metadata (inert on
branch builds, which have no tag ref) and introduce
tunnel-server-release.yml to trigger the build on `v*` tags.

The tag trigger is a separate file rather than a `tags:` entry on
tunnel-server-image.yml because that workflow's push trigger is filtered by
`paths:`. A tag push carries no file diff for a path filter to match, so a
`tags:` entry beside `paths:` would never fire. The new workflow calls the
existing one via `workflow_call`, so there is still a single build
definition.

`latest` semantics are unchanged: the tunnel server's `latest` still tracks
main only.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVEy7CPtRVgPYSLHdFy2Rb
The agent image takes a BUILD_VERSION build-arg, exposes it as
AXON_BUILD_VERSION, and serves it from /api/v1/info. The tunnel server image
had no equivalent, so a deployed server could not say which build it was
running -- the only signal was the image tag on the deployment, which is
trusted rather than observed and goes stale the moment a tag is re-pushed.

Mirror the agent's contract on the server:

- server/docker/Dockerfile takes BUILD_VERSION (default "dev") and exports
  AXON_BUILD_VERSION, and tunnel-server-image.yml passes <ref>-<sha>, the
  same value docker.yml passes for the agent.
- config.Config carries BuildVersion, read from AXON_BUILD_VERSION with an
  unset or empty value falling back to "dev" so the field is never blank.
- The server logs it on its own startup line and includes it in the startup
  config line.
- /healthz and /healthcheck report it as build_version.

The health closure in main became newHealthHandler so its response shape can
be tested; the tests assert on the parsed body rather than the format string,
which also covers the hand-rolled JSON actually parsing.

Verified by running the built server with AXON_BUILD_VERSION set: the stamp
appears in both startup log lines and in the /healthcheck body.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVEy7CPtRVgPYSLHdFy2Rb
Three divergences between the two image pipelines, all closed here.

Trivy: the tunnel server image was never scanned. The agent gets a blocking
CRITICAL/HIGH gate on PRs and a registry scan on main and tags; the
internet-facing half had none. The scan steps are now one reusable workflow
(trivy-scan.yml) that both pipelines call, because a second copy of those
steps is how the two drifted apart in the first place. trivy.yml keeps the
daily sweep and now covers both images at :main.

That refactor also fixes a bug in the agent's release scan: trivy.yml
hardcoded cortex-axon-agent:main as the image-ref, so a tag build's scan
reported on main rather than on the image that tag published. Both pipelines
now pass the primary tag metadata-action resolved, so a release scans the
release. SARIF uploads carry a per-image category; two uploads sharing a
category on one commit overwrite each other in the Security tab, and the
agent's PR and release scans share one so alerts track a single series.

latest: the agent's latest tracked the last release (metadata-action's
default latest=auto) while the tunnel server's tracked main HEAD, so a
deployment pinned to :latest got a different kind of thing depending on which
half it was. Dropping the flavor block leaves the agent's behavior on both.

build_version: the agent served it from /api/v1/info while the tunnel server
serves it from /healthcheck, so "which build is this" was a different probe
per component. The agent's /healthcheck now carries it too, keeping /info
unchanged. serve_test.go pinned the exact health body bytes, which made
adding a field a test failure rather than a decision; it now parses the body
and asserts the fields it cares about.

tunnel-server-release.yml grants security-events: write because the build
workflow chains into the scan, and a nested reusable workflow cannot hold a
permission its caller withheld.

Verified: actionlint clean on all five changed workflows, full agent and
server suites pass.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVEy7CPtRVgPYSLHdFy2Rb
Comment thread .github/workflows/trivy.yml Fixed
Comment thread .github/workflows/trivy.yml Fixed
Comment thread .github/workflows/tunnel-server-image.yml Fixed
@github-advanced-security

Copy link
Copy Markdown
Contributor

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

claude added 2 commits August 31, 2026 04:36
The Trivy gate reports one fixable CRITICAL/HIGH in the agent image:
CVE-2026-74662 in linux-libc-dev. It is not this branch's: the scheduled
scan of cortex-axon-agent:main failed with the identical single finding on
2026-08-30, with no PR involved, after succeeding daily before that.

This is the case the Dockerfile's APT_CACHE_BUST comment describes -- the
scan flagging an OS-package CVE whose fix is already in the archive, with
buildx serving a stale apt layer. The scan runs with ignore-unfixed, so a
reported finding means Debian has published a fix. Bumping the value
invalidates that layer's cache key and lets apt-get upgrade pull it, leaving
the rest of the build (Go, npm, snyk-broker clone) on cache.

If the fixed package turns out not to be in the archive yet, this no-ops and
the gate stays red; the follow-up then is to find what still pulls
linux-libc-dev into the runtime stage, since the build-essential note says it
was meant to be absent entirely.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVEy7CPtRVgPYSLHdFy2Rb
The new Trivy gate on the tunnel server image found 8 fixable CRITICAL/HIGH
vulnerabilities, all in the server binary's Go dependencies:

  golang.org/x/net      CVE-2026-25681, -27136, -33814, -39821, -46600
  golang.org/x/text     CVE-2026-56852
  google.golang.org/grpc CVE-2026-33186, GHSA-hrxh-6v49-42gf

These are real and pre-existing; the image had simply never been scanned, so
nothing surfaced them. The agent already ships x/net v0.56.0, x/text v0.39.0
and grpc v1.82.1 and its scan reports none of these -- the server had drifted
behind on the same dependencies. Bumping it to the agent's versions is the
fix and closes the drift rather than picking arbitrary newer ones.

Also add explicit permissions blocks to the four jobs that call the scan,
which CodeQL flagged on this PR. They grant exactly what the scan needs:
pull the image, upload SARIF. A called workflow cannot hold a permission its
caller withheld, so these have to name the scan's real needs rather than
narrow to nothing.

Verified: full server suite passes on the bumped dependencies, actionlint
clean on all five changed workflows.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVEy7CPtRVgPYSLHdFy2Rb
@shawnburke shawnburke changed the title Add build version tracking to tunnel server Add build-on-tag CI pipeline for grpc-tunnel-server image, fix server vulns Aug 31, 2026
@shawnburke
shawnburke enabled auto-merge (squash) August 31, 2026 04:40

Copy link
Copy Markdown
Collaborator Author

Both scans are green on d0a9adb, resolving the two findings above.

  • Run Trivy PR scan (local image) ✅ — the APT_CACHE_BUST bump cleared CVE-2026-74662. It was the stale-apt-layer case the ARG's comment describes, so no follow-up on linux-libc-dev is needed. This also fixes the same failure on main, which has been red since the 2026-08-30 scheduled scan.
  • Run Trivy Container scan / Scan image ✅ — the tunnel server image now passes its first-ever scan. The 8 CVEs were golang.org/x/net, x/text and grpc in the server binary; the agent already shipped fixed versions of all three, so the server had simply drifted behind on shared dependencies rather than needing anything novel.

Rest of CI is green too: test, test-sdk, docker-tests, docker-tests-grpc, soak, helm-test, CodeQL, Analyze (go/python/actions). The three CodeQL permissions findings are fixed and resolved.

No severity thresholds or ignore-unfixed settings were touched — every fix is a real version bump.


Generated by Claude Code

@shawnburke
shawnburke merged commit 83dd704 into main Aug 31, 2026
23 checks passed
@shawnburke
shawnburke deleted the claude/docker-image-tag-validation-yxd4om branch August 31, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants