disk cache: verify CAS digests fetched from a proxy backend - #922
Open
martinpitt wants to merge 1 commit into
Open
martinpitt wants to merge 1 commit into
martinpitt wants to merge 1 commit into
Conversation
Previously, blobs fetched from a proxy backend were committed after only a size comparison. A backend answering with same-length wrong bytes got them stored under the digest they were asked for, and served to every client after that. Verify proxy reads by -hashing their blobs and comparing before committing. A hash mismatch is logged, dropped and reported as a cache miss, similar to a size mismatch. Client uploads already *are* verified that way, so this closes the verification gap for proxy mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, blobs fetched from a proxy backend were committed after only a size comparison. A backend answering with same-length wrong bytes got them stored under the digest they were asked for, and served to every client after that.
Verify proxy reads by -hashing their blobs and comparing before committing. A hash mismatch is logged, dropped and reported as a cache miss, similar to a size mismatch.
Client uploads already are verified that way, so this closes the verification gap for proxy mode.
I'm trying to set up two bazel-remote instances chained over the HTTP protocol, as in #524: a developer-side instance with
--http_proxy.urlpointing at a shared one,--num_uploaders 0so it never writes back. The shared instance is published read-only by a reverse proxy restricted toGETandHEAD(which makes it read-only). A mistake there lets anyonePUT, and with no digest check on the fetch side nothing downstream would notice.Related: