Build corpus images last, in a separate workflow step - #44
Open
tadamcz wants to merge 1 commit into
Open
Conversation
agent_corpus is by far the slowest target (~9 GB corpus fetch + build). Splitting it into its own step, after all other images, means the fast images are never delayed behind it, and the job's step list shows when everything except the corpus is done. Same runner and buildx builder, so agent_corpus still gets its FROM agent layers as cache hits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
agent_corpustarget is exceptionally slow (~9 GB corpus fetch + corpus build). This splits the build loop in two:agentandcomparatorfor every FC pinagent_corpusfor every FC pin, always lastThe discovery step now emits two lists (
builds/corpus_builds); the build logic is otherwise unchanged, and the Dockerfile is untouched.Benefits:
gh run view --job) shows at a glance when everything except the corpus is done.Both steps run in the same job on the same buildx builder, so
agent_corpusstill gets itsFROM agentlayers as cache hits from the first step — only the corpus stages actually run in step two.