Skip to content

Add shallow-clone (depth) support to GitFetcher - #1186

Open
stanhu wants to merge 1 commit into
chef:mainfrom
stanhu:sh-git-fetcher-shallow-clone
Open

stanhu wants to merge 1 commit into
chef:mainfrom
stanhu:sh-git-fetcher-shallow-clone

Conversation

@stanhu

@stanhu stanhu commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a depth source option to git sources so components can be fetched with a shallow clone instead of downloading the full repository history. This can significantly cut fetch time and disk usage for large repositories where full history isn't needed.

Behavior

When a git source sets a positive integer depth:

source git: "https://example.com/repo.git", depth: 1

the fetcher initializes an empty repo and fetches the resolved revision (SHA) directly at that depth, instead of git cloneing the full history:

git init --quiet .
git fetch --depth N <url> <sha>

Fetching the resolved SHA (rather than a branch/tag ref) means shallow mode:

  • works for tag, branch, and SHA pins alike, and
  • is immune to a pinned branch tip moving between resolution and fetch.

A missing or non-positive depth preserves the existing full-clone behavior, so this is fully backward compatible.

Submodules

In shallow mode there is no clone --recursive, so git_checkout uses submodule update --init --recursive. It deliberately does not pass --depth: a shallow submodule fetch targets the submodule's branch tip and fails when the pinned submodule commit isn't near it, so submodules are fetched at full depth.

Requirements / caveats

  • The remote must allow fetching a reachable SHA (uploadpack.allowReachableSHA1InWant / allowAnySHA1InWant).
  • Components whose build depends on git history (e.g. git describe) should not use this option.

Changes

  • lib/omnibus/fetchers/git_fetcher.rb — add clone_depth; git_clone does init + git_fetch in shallow mode; git_fetch fetches the resolved SHA with --depth; git_checkout initializes submodules at full depth.
  • lib/omnibus/software.rb — allow :depth as a valid git source key and document the option.
  • spec/unit/fetchers/git_fetcher_spec.rb — specs covering no/zero/positive depth, larger depth, submodule handling in both modes, and the fetch-resolved-SHA behavior.

Testing

bundle exec rspec spec/unit/fetchers/git_fetcher_spec.rb21 examples, 0 failures. chefstyle clean on all changed files.

🤖 Generated with Claude Code

@stanhu
stanhu requested review from a team as code owners August 4, 2026 21:56
@stanhu
stanhu force-pushed the sh-git-fetcher-shallow-clone branch 2 times, most recently from c43e7d4 to a32bba0 Compare August 8, 2026 20:33
Introduce a `depth` source option on git sources so components can be
fetched with a shallow clone instead of downloading the full history.
This can significantly cut fetch time and disk usage for large
repositories where full history is not needed.

When `source git: url, depth: N` (positive integer) is set, the fetcher
initializes an empty repo and fetches the *resolved revision* (SHA)
directly at that depth, rather than `git clone`ing the full history:

    git init --quiet .
    git fetch --depth N <url> <sha>

Fetching the resolved SHA (instead of a branch/tag ref) means shallow
mode works for tag, branch, and SHA pins alike, and is immune to a
pinned branch tip moving between resolution and fetch. It does require
the remote to allow fetching a reachable SHA
(uploadpack.allowReachableSHA1InWant / allowAnySHA1InWant).

For submodules, shallow mode uses `submodule update --init` (since there
was no `clone --recursive`) but deliberately does not pass `--depth`:
a shallow submodule fetch targets the submodule's branch tip and fails
when the pinned submodule commit is not near it, so submodules are
fetched at full depth.

A missing or non-positive depth preserves the existing full-clone
behavior, so this is fully backward compatible. Components whose build
depends on git history (e.g. `git describe`) should not use this option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Stan Hu <stanhu@gmail.com>
@stanhu
stanhu force-pushed the sh-git-fetcher-shallow-clone branch from a32bba0 to dca1e51 Compare August 8, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant