Skip to content

Cache musl toolchain in GHCR for GitHub System Tests - #451

Closed
pawelchcki wants to merge 1 commit into
masterfrom
pawel/ghcr-musl-toolchain
Closed

pawelchcki wants to merge 1 commit into
masterfrom
pawel/ghcr-musl-toolchain

Conversation

@pawelchcki

Copy link
Copy Markdown
Contributor

Summary

  • Publish the digest-pinned musl toolchain image from public ECR to GHCR when the pinned digest or publisher workflow changes on master.
  • Have GitHub System Tests use the GHCR copy when it exists, with public ECR as a bootstrap fallback before the first publication.
  • Publish using the repository's GITHUB_TOKEN, so the package is linked to this repository and the build job can pull it with packages: read.

This avoids repeated anonymous downloads from public ECR, which caused toomanyrequests: Data limit exceeded before compilation started. GitLab already uses an internal mirror; its pipeline is unchanged.

Rollout

The publisher workflow runs on the first master push that includes this file. Before it succeeds, System Tests continue using the current public ECR image. After publication, they use the same pinned manifest digest from GHCR. The publisher can also be rerun with workflow_dispatch if the initial public ECR copy hits the existing quota.

Validation

  • yq parsed both workflow files.
  • shellcheck -s bash passed on the new workflow scripts.
  • git diff --check passed.
  • make -n build-extended-image confirmed that MUSL_TOOLCHAIN_IMAGE from the workflow overrides the Makefile default.

@pawelchcki
pawelchcki requested a review from a team as a code owner September 29, 2026 14:53
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T14:55:38.748282Z c4483e4 PR opened
🔒 Security Review ✅ Completed 2026-09-29T14:57:41.691983Z c4483e4 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pawelchcki

Copy link
Copy Markdown
Contributor Author

I had some flakes regarding using public.ecr repository

@datadog-datadog-prod-us1

Copy link
Copy Markdown

Tests

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 69.29% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c4483e4 | Docs | View more details | Give us feedback!

@xlamorlette-datadog xlamorlette-datadog left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#455 is a simpler solution.

I think it is also more reliable:

  • Datadog public registry is built for heavy traffic, and it fixes all non-GitLab build, not just GitHub CI, notably local builds.
  • Here, the publisher copies the image from public.ecr.aws, so, as stated in the PR description, we can hit the same rate limit.

What do you think?

@pawelchcki

Copy link
Copy Markdown
Contributor Author

nice good to know

@pawelchcki pawelchcki closed this Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants