Problem
The current .buildkite/scripts/build_pr.sh script clones full repositories (e.g., kibana) when running documentation builds for pull requests. For large repositories, this results in excessive data transfer (>4GB) and increases build times, even though only the PR head and some history are required.
Proposed Improvements
- Use
git fetch --depth=<n> (shallow fetch) to avoid cloning the complete history and tags.
- Consider partial clone (
--filter=blob:none) for further bandwidth reduction where supported.
- Only fetch the PR head and target branch with sufficient depth to support
git diff against the target.
- Avoid creating unnecessary local branches—consider detached checkouts of the PR head when a branch isn’t needed.
- Document and test minimum required depth for reliable diff/merge-base computation across supported repositories.
Goal: Reduce network usage and accelerate build times in CI, especially for very large repositories.
Related context: Relevant snippet from the script:
git fetch origin pull/$GITHUB_PR_NUMBER/head:pr_$GITHUB_PR_NUMBER &&
See related discussion for previous optimizations.
Problem
The current
.buildkite/scripts/build_pr.shscript clones full repositories (e.g.,kibana) when running documentation builds for pull requests. For large repositories, this results in excessive data transfer (>4GB) and increases build times, even though only the PR head and some history are required.Proposed Improvements
git fetch --depth=<n>(shallow fetch) to avoid cloning the complete history and tags.--filter=blob:none) for further bandwidth reduction where supported.git diffagainst the target.Goal: Reduce network usage and accelerate build times in CI, especially for very large repositories.
Related context: Relevant snippet from the script:
See related discussion for previous optimizations.