Skip to content

Resolve commit SHAs from one rev-list per repository - #330

Merged
cboos merged 4 commits into
mainfrom
dev/sha-link-resolution
Sep 11, 2026
Merged

cboos merged 4 commits into
mainfrom
dev/sha-link-resolution

Conversation

@cboos

@cboos cboos commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Closes #327.

Rendering with a git repository in context spent most of its "Markdown rendering" time in git subprocesses resolving commit-SHA links. Every distinct SHA-shaped token ran git branch -r --contains (plus git rev-parse when found), memoised per process, so each render worker repeated the whole set. Over a third of those tokens are not commits at all (task ids, UUID fragments, digit runs), so much of that time went into proving negatives.

The change

git_remote.py now reads git rev-list --remotes once per working directory and resolves each candidate by prefix search in the sorted list (binary ids in one bytes object, 20 bytes per commit; SHA-256 repositories work too). One function, _commit_for, owns the rule: a token matching [0-9a-f]{7,40} that abbreviates exactly one commit reachable from a remote-tracking ref links by that commit's full id; everything else stays plain text. resolve_sha's interface is unchanged.

Same answers

  • Old and new resolver in one process, over 1788 distinct SHA-shaped tokens from real transcripts of this project: 0 differences. The old resolver spent 23.8 s on the 568 hits and 14.8 s on the 1220 misses; the new one a 27 ms list read plus ~5 ms of lookups.
  • End to end, the rendered pages of a whole project are byte-identical between old and new (below), with 1599 commit links in both.

Timings

Serial, uncached CLI conversion (CLAUDE_CODE_LOG_RENDER_JOBS=1, --no-cache, CLAUDE_CODE_LOG_DEBUG_TIMING=1) of a frozen 24-file / 40 109-message project, old and new module interleaved:

run wall Markdown bucket Pygments (control)
old 59.4 s 32.1 s 6.6 s
new 36.9 s 7.7 s 7.1 s
old 67.9 s 36.6 s 8.3 s
new 37.0 s 7.4 s 7.2 s

The Markdown and Pygments memo hit counts are identical in all four runs, so every run did the same work. The slower old run is slow on every bucket, Pygments included, so its extra time is machine load rather than the resolver. This is a different corpus from the one in #327, so compare the ratio, not the absolute figures.

Decisions worth knowing

  • Ambiguity is judged among remote-reachable commits only. Git would also count local-only commits, so a prefix shared by a pushed and an unpushed commit now links to the pushed one instead of to nothing. On this repository: 0 local-only commits whose 7-char prefix collides with a remote one (1589 remote, 275 local-only), and 0 ambiguous 7-char prefixes within the remote set. When it does happen, the result is a link to a real pushed commit, not a broken one.
  • Fail-closed when the list cannot be read. If git rev-list fails or times out (30 s cap, since it runs once per directory), nothing links. Linking every SHA-shaped token unvalidated instead would turn a skipped check into a page of dead links — over a third of candidates are not commits — with nothing saying the check was skipped.
  • Freshness for long-lived processes (watch, the TUI). The old per-token check saw commits fetched after startup; a list read once would not. A lookup miss re-reads the list once it is older than max(60 s, 20x its own read time); a hit never re-reads. So commits that leave the remote (force-push, deleted branch) keep linking until restart — deliberate, since re-reading on hits would bring back a git call per token. Both directions are pinned by tests.
  • Each render worker reads its own list rather than being fed one through the pool initializer: ~20 ms here, ~0.1 s on a 13k-commit repository, against ~1 s of spawn and import per worker.

Unchanged

The missing-checkout precondition: a transcript whose working directory no longer exists (or has no origin) still gets no commit links, because the remote URL comes from that checkout. Covering it would need the remote URL stored per project or a configured template, which is a separate feature.

Interaction with #329

#329 replaces the Markdown linkifier that calls this resolver; its Interaction with #327 section covers the combined tree.

Review state

Approved at 70c7bca, rebased onto 6eb8696 as f383f45 with an identical patch. The approval was given at 7787d9a and extended across two follow-ups: 8a0d820 is docstring-only (+4/-1 in _commit_for, recording the hit/miss asymmetry above; the code with docstrings stripped is AST-identical), and 70c7bca is test-only (the uppercase shape test no longer depends on the commit's SHA having a letter in its first 7 chars).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KBfajJZtCtVDRA7hnMBp3E

Summary by CodeRabbit

  • Bug Fixes

    • Improved commit link resolution for short and full commit identifiers.
    • Prevented local-only or ambiguous identifiers from being linked incorrectly.
    • Improved handling for repositories without remote references and repositories using SHA-256.
    • Added more reliable recovery when commit information is temporarily unavailable or outdated.
  • Performance

    • Reduced repeated repository lookups, improving responsiveness when resolving multiple commit references.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: dec234c4-2259-422d-895d-4898b64427fb

📥 Commits

Reviewing files that changed from the base of the PR and between 8a0d820 and f383f45.

📒 Files selected for processing (1)
  • test/test_commit_linkifier.py

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

The resolver now reads remote commit ids once per working directory, caches them with bounded staleness, and resolves SHA prefixes through binary lookup. Tests cover remote reachability, ambiguity, retries, SHA-256 repositories, and cache eviction.

Changes

Remote commit resolution

Layer / File(s) Summary
Commit lookup contract
claude_code_log/git_remote.py
Defines SHA-shape validation and binary prefix lookup for sorted remote commit ids.
Remote commit loading and cache
claude_code_log/git_remote.py
Loads git rev-list --remotes results, refreshes stale entries, and maintains a bounded thread-safe per-working-directory cache.
Resolver integration and validation
claude_code_log/git_remote.py, test/test_commit_linkifier.py
Routes resolve_sha through _commit_for, updates cache clearing, and tests resolution, retries, ambiguity, SHA-256 support, and cache eviction.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Refactor · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant Renderer
  participant resolve_sha
  participant RemoteCommitCache
  participant Git
  Renderer->>resolve_sha: resolve SHA token
  resolve_sha->>RemoteCommitCache: request commits for cwd
  RemoteCommitCache->>Git: run rev-list --remotes
  Git-->>RemoteCommitCache: remote commit ids
  RemoteCommitCache-->>resolve_sha: matching full commit id
  resolve_sha-->>Renderer: formatted commit URL
Loading

Merge Risk: ⚪ Minimal · up to f383f

The commit-resolution cache change is ready to merge with no identified material regression.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 34.21% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Issue #327 requires replacing per-token Git subprocesses with one repository-level listing and local SHA checks while preserving link behavior. git_remote.py now reads git rev-list --remotes once …
Out of Scope Changes check ✅ Passed The changed resolver code and tests directly support issue #327. Cache management, stale-list handling, bounded per-working-directory storage, and documentation support the required repository-level l…
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: resolving commit SHAs with one git rev-list operation per repository.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev/sha-link-resolution

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

cboos and others added 4 commits September 11, 2026 20:46
Rendering with a repository in context spent most of its Markdown time
in git: every distinct SHA-shaped token ran `git branch -r --contains`
(~22 ms), plus `rev-parse` when found, memoized per process so every
render worker repeated it. Over a third of those tokens are not commits
at all (task ids, UUID fragments, digit runs), so much of the cost went
into proving negatives.

The resolver now reads `git rev-list --remotes` once per working
directory and resolves each candidate by prefix search in the sorted
list: an unambiguous abbreviation of a remote-reachable commit links
with its full id, anything else stays plain text. Over 1788 distinct
tokens from this project's transcripts the answers are identical to the
old resolver's; the old one spent 23.8 s on the 568 hits and 14.8 s on
the 1220 misses, the new one a 27 ms list read and ~5 ms of lookups.

When the list cannot be read at all but the remote is known, SHA-shaped
tokens link as written, unvalidated. Both behaviours go through one
function, `_commit_for`, which also enforces the SHA shape. A long-lived
process re-reads a list when a lookup misses and the list has outlived
its trust window, so commits fetched after it started are still found.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBfajJZtCtVDRA7hnMBp3E
The first cut linked every SHA-shaped token unvalidated when
`git rev-list --remotes` failed or timed out. Over a third of the
candidates in real transcripts are not commits, so that turned a
skipped check into a page of dead links with nothing saying the check
was skipped — the failure direction the per-candidate resolver never
had. An unreadable list now finds nothing, and a later miss past the
trust window retries the read.

This also leaves the unvalidated behaviour with no trigger: a missing
checkout cannot reach it (no remote URL to build a link from), so it
belongs with a stored remote URL, if that is ever wanted. The shape
check stays in `_commit_for` as the resolver's own contract, since the
prefix search alone would accept uppercase hex and short abbreviations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBfajJZtCtVDRA7hnMBp3E
The list is re-read on a stale miss, never on a hit, so a commit that
leaves the remote (force-push, deleted branch) stays linked for the
life of a long-lived process. Say so where the rule lives, so it reads
as a decision rather than an oversight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBfajJZtCtVDRA7hnMBp3E
The test uppercased a pushed commit's 7-char prefix and expected the
shape gate to reject it. When that prefix is all digits (about 4% of
SHAs), uppercasing changes nothing, the token is valid, and it links:
a failure that depends only on the commit's random id. Use the
shortest prefix of at least 7 chars that contains a letter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBfajJZtCtVDRA7hnMBp3E
@cboos
cboos force-pushed the dev/sha-link-resolution branch from 70c7bca to f383f45 Compare September 11, 2026 18:47
@cboos
cboos merged commit 0421092 into main Sep 11, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SHA-link resolution dominates render time: 549 git subprocesses per project, per worker

1 participant