Skip to content

containerz: add signed bundle transfer to Deploy RPC - #351

Open
sumitk31 wants to merge 5 commits into
openconfig:mainfrom
sumitk31:docker_cosign
Open

containerz: add signed bundle transfer to Deploy RPC#351
sumitk31 wants to merge 5 commits into
openconfig:mainfrom
sumitk31:docker_cosign

Conversation

@sumitk31

@sumitk31 sumitk31 commented Aug 3, 2026

Copy link
Copy Markdown

Summary

Extend the Containerz Deploy RPC to transfer an opaque signed bundle alongside the container image tarball. This enables offline/air-gapped image onboarding where the target verifies a signature before loading the image.

The proto intentionally does not name a specific signing tool (cosign, notary, etc.). It defines only what is transferred; how the target verifies the bundle remains implementation-specific.

Changes

  • signed_bundle_size on ImageTransfer
  • signed_bundle_content and SignedBundleTransferEnd on DeployRequest
  • SignedBundleTransferReady / SignedBundleTransferProgress on DeployResponse
  • gnoi_version bumped to 0.3.0
  • Documented fail-fast server behavior when signed bundle transfer is not supported

Server behavior when signing is not supported

When signed_bundle_size = 0, behavior is unchanged — backward compatible with targets that do not implement signed bundle transfer.

When signed_bundle_size > 0 and the target does not support signed bundle transfer or verification, the target must:

  1. Respond to ImageTransfer with image_transfer_error (google.rpc.Status code FAILED_PRECONDITION)
  2. Do so before sending ImageTransferReady
  3. Not accept image or bundle content on that stream

Clients can detect lack of support without uploading the image tarball.

Deploy flow (when signed_bundle_size > 0 and target supports signing)

There is one terminal success for the entire Deploy operation: ImageTransferSuccess. The signed bundle phase uses Ready / Progress / End messages only (no separate bundle success message). Failures at any stage are reported via image_transfer_error.

Phases

Phase Client → Server Server → Client
0. Capability check ImageTransfer (signed_bundle_size > 0) image_transfer_error if unsupported, or continue
1. Image transfer ImageTransfer ImageTransferReady
content (chunks) ImageTransferProgress
ImageTransferEnd
2. Signed bundle transfer SignedBundleTransferReady
signed_bundle_content (chunks) SignedBundleTransferProgress
SignedBundleTransferEnd
3. Verify + load (server-side) ImageTransferSuccess or image_transfer_error

When signed_bundle_size = 0, phase 0 does not apply and behavior matches the pre-0.3.0 flow.

Call flow diagram

sequenceDiagram
    participant Client
    participant Server

    Client->>Server: ImageTransfer (image_size, signed_bundle_size)
    alt Target does not support signed bundle transfer
        Server->>Client: image_transfer_error (FAILED_PRECONDITION)
    else Target supports signed bundle transfer
        Server->>Client: ImageTransferReady (chunk_size)
        loop Image chunks
            Client->>Server: content
            Server->>Client: ImageTransferProgress (bytes_received)
        end
        Client->>Server: ImageTransferEnd

        Server->>Client: SignedBundleTransferReady (chunk_size)
        loop Signed bundle chunks
            Client->>Server: signed_bundle_content
            Server->>Client: SignedBundleTransferProgress (bytes_received)
        end
        Client->>Server: SignedBundleTransferEnd

        Server->>Server: Verify signed bundle against image
        Server->>Server: Load image into registry / plugin store
        alt Success
            Server->>Client: ImageTransferSuccess (name, tag, image_size)
        else Failure (verify or load)
            Server->>Client: image_transfer_error
        end
    end
Loading

Motivation

  • Supply chain integrity — only signed images should be loaded on network devices
  • Air-gapped deployments — common on NOS platforms; registry-based verification is often unavailable
  • Portable gNOI contract — bundle transfer over the same Deploy stream, not side-channel file copies
  • Signing-tool agnosticsigned_bundle_* naming keeps the RPC opaque to the verifier implementation
  • Graceful capability mismatch — fail-fast image_transfer_error avoids uploading a large image to an unsupported target

Implementation notes

  • Go stubs regenerated for containerz (containerz.pb.go, containerz_grpc.pb.go)
  • Downstream NOS and reference implementations must enforce fail-fast behavior when signed_bundle_size > 0 is not supported

Test plan

  • Review proto naming and backward compatibility (signed_bundle_size = 0)
  • Regenerate Go stubs via regenerate-files.sh
  • Update Containerz reference client/server implementations

@google-cla

google-cla Bot commented Aug 3, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Extend DeployRequest to stream Sigstore cosign bundles alongside container
images for offline signature verification during image onboarding.

- Add cosign_bundle_size to ImageTransfer
- Add cosign_bundle_content and CosignBundleTransferEnd to DeployRequest
- Add CosignBundleTransferReady/Progress to DeployResponse
- Bump gnoi_version to 0.3.0
Address review feedback: keep the signing mechanism opaque in the
Deploy RPC by using signed_bundle_* field and message names instead of
cosign-specific naming.
@sumitk31 sumitk31 changed the title feat(containerz): add cosign bundle transfer to Deploy RPC containerz: add signed bundle transfer to Deploy RPC Aug 4, 2026
Run regenerate-files.sh (bazel 8.4.2) to update containerz.pb.go and
containerz_grpc.pb.go after signed_bundle_* proto rename.
@sumitk31
sumitk31 force-pushed the docker_cosign branch 2 times, most recently from 45b2757 to 472a38f Compare August 4, 2026 11:53
Match the protoc version stamp on upstream main for containerz stubs only, without bumping the repo-wide protobuf dependency.

@nburnwalcisco nburnwalcisco left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing if have permission to approve.

Comment on lines +273 to +286
// SignedBundleTransferReady indicates to the client that the target is ready
// to receive signed bundle content after the image transfer completes.
message SignedBundleTransferReady {
// Indicates the size of the chunks that the client should break the bundle
// into as it is transferred.
int32 chunk_size = 1;
}

// SignedBundleTransferProgress is sent periodically to the client during the
// signed bundle transfer.
message SignedBundleTransferProgress {
// The number of signed bundle bytes transferred so far.
uint64 bytes_received = 1;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these need to be a separate message? Could we integrate this in InageTransferReady/Progess?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We added dedicated bundle Ready/Progress types mainly for clarity at the protocol boundary:

Different phases — ImageTransferReady is sent before image content; bundle transfer only starts after ImageTransferEnd. A second ready is a phase transition, not a repeat of the first handshake.
Clear progress semantics — ImageTransferProgress.bytes_received today means image bytes. During bundle upload it would mean bundle bytes. Separate types make that explicit in generated code, logs, and client state machines.
Consistent pattern — Image transfer uses Ready → Progress → End; we mirrored that for the bundle after ImageTransferEnd.
DeployResponse oneof — Distinct response variants let clients branch on message type without inferring phase from stream position alone.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the size of a signature bundle is typically ~10KB, is it worth adding this as a separately streamed and chunked message?

Could it instead be simply included in the ImageTransfer message?

Document that targets must return image_transfer_error with
FAILED_PRECONDITION in response to ImageTransfer when
signed_bundle_size is non-zero but signed bundle transfer is not
supported, before ImageTransferReady.
@brianneville

Copy link
Copy Markdown
Contributor

High level question: could we get the same guarantee of integrity just by updating the ImageTransferEnd message to include a hash of the file?

Like:

message ImageTransferEnd {
  // optional hash of the image tar. If present, the server should 
  // verify that the hash matches the hash of the content transferred.
  gnoi.types.HashType hash = 1;
}

similar to how the File/Put RPC works:

gnoi.types.HashType hash = 3; // hash of the file.

That way, we can be sure that the gNOI client and server both have the same tar file present, and then the verification on that tar file can be performed from the client's end prior to transfer.
That way we could avoid needing to build verification logic into the Deploy RPC flow like this.

@sumitk31

sumitk31 commented Sep 8, 2026

Copy link
Copy Markdown
Author

High level question: could we get the same guarantee of integrity just by updating the ImageTransferEnd message to include a hash of the file?

Like:

message ImageTransferEnd {
  // optional hash of the image tar. If present, the server should 
  // verify that the hash matches the hash of the content transferred.
  gnoi.types.HashType hash = 1;
}

similar to how the File/Put RPC works:

gnoi.types.HashType hash = 3; // hash of the file.

That way, we can be sure that the gNOI client and server both have the same tar file present, and then the verification on that tar file can be performed from the client's end prior to transfer. That way we could avoid needing to build verification logic into the Deploy RPC flow like this.

Thanks for the suggestion — the File/Put hash pattern is a good reference for transfer integrity.

We can consider that, but a hash on ImageTransferEnd and a signed bundle address different guarantees:

  • Hash (HashType on ImageTransferEnd) — confirms the server received the same bytes the client sent (integrity), similar to File/Put.
  • Signed bundle — carries cryptographic provenance so the target can verify authenticity before loading the image, which matters for air-gapped/on-box policy (e.g. only load images that pass signature verification on the device).

If verification is done only on the client before transfer, a compromised or misconfigured client could still push an unsigned image and a device that only checks a hash would accept it. The signed-bundle flow lets the target enforce supply-chain policy without relying on the client to have verified correctly beforehand.

That said, adding an optional hash on ImageTransferEnd as a complement (integrity check in addition to signature verification) could be reasonable follow-up. For this PR, the goal is specifically on-target signature verification for offline image onboarding.

@sumitk31 sumitk31 closed this Sep 8, 2026
@sumitk31 sumitk31 reopened this Sep 8, 2026
@brianneville

Copy link
Copy Markdown
Contributor

If verification is done only on the client before transfer, a compromised or misconfigured client could still push an unsigned image and a device that only checks a hash would accept it. The signed-bundle flow lets the target enforce supply-chain policy without relying on the client to have verified correctly beforehand.

I think it would be good to formalize this in some way so that we dont have clients sending blobs where the blob verification is completely opaque and out-of-scope.

Doing the verification like this on the server-side like this feels like its going to cause issues for a network operator which uses multiple vendors.
Network operators may then need to maintain different signed_bundle_content blobs per vendor (e.g. one vendor supports notary, one supports cosign), and know which one to send to each containerz server that they're deploying across.

@sumitk31

sumitk31 commented Sep 8, 2026

Copy link
Copy Markdown
Author

If verification is done only on the client before transfer, a compromised or misconfigured client could still push an unsigned image and a device that only checks a hash would accept it. The signed-bundle flow lets the target enforce supply-chain policy without relying on the client to have verified correctly beforehand.

I think it would be good to formalize this in some way so that we dont have clients sending blobs where the blob verification is completely opaque and out-of-scope.

Doing the verification like this on the server-side like this feels like its going to cause issues for a network operator which uses multiple vendors. Network operators may then need to maintain different signed_bundle_content blobs per vendor (e.g. one vendor supports notary, one supports cosign), and know which one to send to each containerz server that they're deploying across.

Thanks — agreed we should avoid leaving verification fully opaque for operators.

The proto stays tool-agnostic on the wire, but in practice Cosign/Sigstore is the industry standard for container signing; Notary is deprecated and not a good baseline for a multi-vendor profile. If openconfig standardizes on a Sigstore bundle as the recommended signed_bundle_content format, operators send one blob type to all compliant targets; per-vendor differences stay in trust/policy config, not bundle encoding.

We can document that profile alongside this PR and optionally add a format hint on ImageTransfer in a follow-up. Hash on ImageTransferEnd remains complementary integrity, not a substitute for on-box signature policy.

@sumitk31
sumitk31 requested a review from alshabib September 9, 2026 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants