Add ci channel for per-commit Build Cache Service framework resolution

[LoopedBard3]

Adds a new ci channel that resolves .NET binaries from the Build Cache Service (BCS) instead of NuGet feeds, giving per-commit granularity for performance-regression bisection instead of feed cadence.

On the ci channel, crank always overlays BOTH shared frameworks in every job — the base runtime (Microsoft.NETCore.App, from dotnet/runtime) and the ASP.NET Core shared framework (Microsoft.AspNetCore.App, from dotnet/aspnetcore). Both default to latest, and each repo can be pinned independently to bisect one while holding the other at latest.

Named ci (not buildcache) per review feedback — "buildcache" is an implementation detail; ci describes where the binaries come from, consistent with the other channel names.

CLI surface

Rather than add new arguments, the ci channel reuses the existing runtimeVersion / aspNetCoreVersion arguments — on this channel they carry a commit SHA instead of a feed version:

• empty → latest cached build for that repo on main
• 8–40 hex chars → pin/bisect that repo at that commit
• a feed version string → rejected with a clear error (version-string pinning is not supported on ci; conversely the other channels accept a version string and reject a SHA)

No new arguments are introduced. Latest always resolves against main — the only branch the BCS pipeline builds — so there's no branch knob to configure. To target a specific point, pin a commit SHA; SHA lookups are branch-independent, so a commit that happens to live on a release branch still resolves. (If the pipeline starts caching release branches later, branch selection can be added back as a purely additive argument.)

Usage:

# latest of both on main
--application.channel ci

# bisect aspnetcore, runtime stays latest
--application.channel ci --application.aspNetCoreVersion <aspnetcore-sha>

# bisect the base runtime, aspnetcore stays latest
--application.channel ci --application.runtimeVersion <runtime-sha>

# pin both
--application.channel ci --application.runtimeVersion <runtime-sha> --application.aspNetCoreVersion <aspnetcore-sha>

Controller (Job.cs) — reuses RuntimeVersion / AspNetCoreVersion (no new properties); on the ci channel they carry a commit SHA.

Agent (Startup.cs / BuildCacheClient.cs) — new CLI:

• --build-cache-base-url (default: https://pvscmdupload.z22.web.core.windows.net)
• --build-cache-disabled

How resolution works

The agent still builds the app with real NuGet packages (so restore/publish behave normally), then overlays BCS bits per publish kind:

• Framework-dependent: a per-job isolated dotnet home is created at <temp>/crank-buildcache/home-…/, mirrored from the global _dotnethome, and overlaid with the BCS runtime and aspnetcore archives. StartProcess resolves from this home; the global _dotnethome is never mutated (concurrent jobs / other channels unaffected).
• Self-contained: BCS files are overlaid onto the published outputFolder; the SDK-bound apphost is preserved.

The base runtime archive is raw build output, so BCS bits are overlaid onto a feed-installed runtime. The aspnetcore archive is the runtime-pack nupkg stored verbatim (carrying deps.json + runtimeconfig.json), so that framework folder is built entirely from BCS and the job fails loudly if the pack is incomplete. If a requested commit isn't in the cache, crank fails rather than silently falling back.

Version surfaces (bisection distinguishability)

Every ci run stamps the resolved commits so bisection points never collide, and the stamp reaches the perf database:

• results measurements NetCoreAppVersion / AspNetCoreVersion (what lands in the controller --json and the SQL/ES document as application.netCoreAppVersion / aspNetCoreVersion) → {feedVersion}+ci.{12-char sha} on the ci channel; non-ci runs are unchanged ({feedVersion}+{12-char sha}). The +ci. marker is folded into the existing fields rather than added as new keys, so bisection runs are self-identifying without any downstream schema change.
• job.RuntimeVersion / job.AspNetCoreVersion (the DTO, visible on the agent job object) → {feedVersion}+ci.{8-char sha}
• Dependency records carry the full 40-char CommitHash + repo URL for both runtime and aspnetcore

The 12- and 8-char forms and the full Dependency hash all encode the same underlying commit; the numeric version prefix stays feed-coherent (so runtimeconfig/roll-forward stay valid) and the +ci.{sha} suffix is the per-commit distinguisher.

Cache correctness

RuntimeVersion and AspNetCoreVersion are part of BuildKeyData, so --application.options.reuseBuild cache-busts when a pinned SHA changes. The reuse path also records the +ci.{sha} version measurements (it returns before the fresh-build recording, so without this a reused ci run would surface no version).

Backward compatibility

Fully gated on --application.channel ci. DefaultChannel is unchanged (current). No new controller properties are added — the existing runtimeVersion / aspNetCoreVersion arguments only change meaning on the ci channel; every other channel takes the exact same path as before, so old controllers/agents interoperate unchanged. Audited diff-level against main: purely additive, no opt-out-path behavior change.

Verification

• dotnet build clean; 119/119 unit tests pass (BuildCacheClientTests.cs adds 40 methods covering ParseLatestBuilds, commit-SHA validation incl. path-traversal, the SHA-vs-version channel rules, dual-repo overlay, isolated-home concurrency, SelectHighestManagedDir numeric ordering, cleanup, etc.).
• Local both-overlay hello-world runs (win-x64) confirmed the three ci paths end-to-end:
  • latest resolution engages the correct latest/main/latestBuilds.json path per repo;
  • pinned aspnetcore + latest runtime overlays both and runs, with results surfacing aspNetCoreVersion = {ver}+ci.8f51f037469f and netCoreAppVersion = {ver}+ci.6550ccf7827c (matching the resolved commits), while the non-ci load job carries no ci. marker (gating intact);
  • a feed version string on ci fails fast with the documented error.

Post-merge dependency: the aspnetcore latestBuilds.json index (published by the BCS indexer / dotnet-performance pipeline) must exist for latest-aspnetcore resolution; until then the agent fails loudly rather than silently falling back.

[Copilot]

Pull request overview

This PR adds a new buildcache channel to Crank’s agent/runtime resolution pipeline so the runtime binaries can be sourced per-commit from the Build Caching Service (BCS), enabling finer-grained performance regression bisection than VMR feed cadence allows.

Changes:

• Added build-cache-related properties to Job for selecting commit/branch/config.
• Added BuildCacheClient for resolving latest builds, downloading/extracting artifacts, and overlaying runtime binaries.
• Updated agent startup/version resolution logic and docs to support the new buildcache channel and new agent CLI options.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/Microsoft.Crank.Models/Job.cs	Adds job-level inputs for BCS commit/branch/config selection.
src/Microsoft.Crank.Agent/Startup.cs	Adds CLI options and integrates buildcache into runtime version resolution + post-publish overlay.
src/Microsoft.Crank.Agent/BuildCacheClient.cs	Implements BCS querying/downloading/extraction and overlay helpers.
docs/dotnet_versions.md	Documents buildcache channel usage and configuration knobs.
docs/build_cache_requirements.md	Describes required BCS-side blob layout/access for the integration.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Suppressed comments (6)

src/Microsoft.Crank.Agent/BuildCacheClient.cs:1135

• This maps every non-ARM64 Linux host to linux-x64, so the linux-musl-x64 configuration declared above is never selected. On a musl agent, buildcache therefore downloads glibc runtime and ASP.NET artifacts instead of failing as the documentation promises for unsupported musl ASP.NET builds, leading to incompatible binaries. Detect musl explicitly (or reject it before resolution).

            return RuntimeInformation.ProcessArchitecture == Architecture.Arm64 ? "linux-arm64" : "linux-x64";

src/Microsoft.Crank.Agent/Startup.cs:3049

• This per-job extract is not registered on JobContext until line 3368. If the ASP.NET download, SDK/runtime installation, cancellation check, or home creation fails first, DeleteJobAsync sees null and leaves this potentially multi-GB directory behind. Register the extract immediately so the existing final cleanup owns it on every later return path.

                        runtimeBuildCacheExtractDir = await BuildCacheClient.DownloadAndExtractAsync(
                            _buildCacheBaseUrl, BuildCacheClient.RepoNameRuntime, runtimeBuildCacheCommitSha, runtimeBuildCacheConfigResolved, cancellationToken);

src/Microsoft.Crank.Agent/Startup.cs:3066

• Like the runtime extract, this directory is only assigned to JobContext after all installation and isolated-home work succeeds. Any intervening failure bypasses the end-of-job cleanup and leaks the extraction. Register it here immediately after creation.

                        aspNetCoreBuildCacheExtractDir = await BuildCacheClient.DownloadAndExtractAsync(
                            _buildCacheBaseUrl, BuildCacheClient.RepoNameAspNetCore, aspNetCoreBuildCacheCommitSha, aspNetCoreBuildCacheConfigResolved, cancellationToken);

src/Microsoft.Crank.Agent/BuildCacheClient.cs:227

• User SHAs of 8–39 characters are accepted, but ResolveCommitAsync returns them unchanged and this constructs an exact blob path from that value. Static blob storage cannot expand a commit prefix, so an advertised prefix requests a prefix-named directory and returns 404 instead of resolving the full commit. Either require the exact SHA length used by BCS or resolve prefixes through an index/API before downloading.

            var artifactUrl =
                $"{normalizedBaseUrl}/builds/{Uri.EscapeDataString(repoName)}/buildArtifacts/" +
                $"{Uri.EscapeDataString(commitSha)}/{Uri.EscapeDataString(buildCacheConfig)}/{Uri.EscapeDataString(artifactFile)}";

src/Microsoft.Crank.Models/Job.cs:83

• The PR description still advertises the controller properties BuildCacheCommitSha, BuildCacheBranch, and BuildCacheConfig, plus a --build-cache-repo-name agent option, but this API instead exposes four repo-specific properties and no config override or repo-name option. Update the PR wire-up and usage sections to match the implemented dual-repository contract so consumers do not use nonexistent options.

        public string BuildCacheRuntimeCommitSha { get; set; } = "";
        public string BuildCacheAspNetCoreCommitSha { get; set; } = "";
        public string BuildCacheRuntimeBranch { get; set; } = "";
        public string BuildCacheAspNetCoreBranch { get; set; } = "";

src/Microsoft.Crank.Agent/BuildCacheClient.cs:275

• Downloaded archives are intentionally retained with no size, age, or count bound. Per-commit bisection—the primary use case—adds runtime and ASP.NET archives for every tested SHA, so a long-lived agent's temp volume grows indefinitely and can eventually exhaust disk space. Add a bounded cache/eviction policy while preserving active extracts.

        /// <summary>
        /// Deletes a previously-extracted directory. Safe to call multiple times. Archives in the
        /// parent commit dir are intentionally NOT deleted so subsequent jobs for the same commit
        /// can reuse the download.

[sebastienros]

I would prefer to name it ci, because buildcache sounds too much like an implementation detail.

[LoopedBard3]

That makes sense, especially with the other channel names. Went ahead and updated this everywhere user-facing, but kept some of the internals as they truly refer to the buildCache.

[sebastienros]

Since we use a custom channel, which makes sense (what feed is the runtime coming from), would it be possible to reuse the exist "version" arguments instead of adding new ones? They will be shas and not actual versions.

And not setting a version would pick the latest one from the ci channel (buildcache)

[sebastienros]

I could see a world where we could set either a version or a sha, since I assume all builds actually have a version.

[LoopedBard3]

Moved over to this approach. The ci channel only supports shas because the Cache is keyed by commit and currently doesn't have any mapping or relation to the longer version strings. We would have to decide how we want to map versions to sha's as the VMR does not have a version for every ASP.NET or Runtime commit, and I am not aware of any other place that would have a version for every SHA (though maybe the mapping doesn't have to cover every SHA for general use).

[sebastienros]

Does it mean we can target a release branch and it will pick up the latest available build for this branch?

I was assuming it would only be for the main branches,

[sebastienros]

I don't see why this would be two different variables, no way we would use different branches for aspnet/dotnet.

Another option would be to reuse the existing version arguments which already have the convention with patterns. Creating a custom format for wildcard versions.

[LoopedBard3]

Yea, this was an idea of where this could be down the line with the idea that custom branches would be selectable from the cache for testing. Instead, I have removed ciBranch and this flow as a whole as we are currently only building for main so the functionality would not actually work for anything. If we find we need this in the future we can add it then, otherwise specific commits can be used from wherever they are built.

[sebastienros]

this is the kind of idea I am having when suggesting to use existing arguments, and base the logic on the channel.

[LoopedBard3]

Landed on this IIUC.