Skip to content

ci(e2e): add CentOS Stream 10 Docker/Podman/SELinux suite#2093

Open
ericcurtin wants to merge 1 commit into
NVIDIA:mainfrom
ericcurtin:test-podman-docker
Open

ci(e2e): add CentOS Stream 10 Docker/Podman/SELinux suite#2093
ericcurtin wants to merge 1 commit into
NVIDIA:mainfrom
ericcurtin:test-podman-docker

Conversation

@ericcurtin

@ericcurtin ericcurtin commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in E2E suite that boots a CentOS Stream 10 VM under QEMU/KVM
on the existing self-hosted linux-amd64-cpu8 runner, installs Docker
Engine alongside Podman, verifies SELinux is enforcing, and runs the
standard e2e:docker and e2e:podman suites against both engines on the
same SELinux-enforcing guest, per the discussion in #2092.

Related Issue

Follow-up to the CI request in #2092 (comment).

Changes

  • New reusable workflow .github/workflows/e2e-centos-selinux.yml:
    • Runs on the existing self-hosted linux-amd64-cpu8 runner (the same
      class already used by driver-vm-linux.yml with --privileged) —
      no new org-provisioned runner class is required.
    • Downloads a CentOS Stream 10 GenericCloud qcow2 image, seeds it with
      cloud-init over a generated ISO, and boots it under hardware-accelerated
      QEMU/KVM. Fails fast if /dev/kvm isn't exposed to the job.
    • Copies the checked-out repo into the guest over scp/tar and drives the
      rest of the suite entirely over SSH.
    • Installs Docker via get.docker.com alongside Podman (RHEL-family
      distros ship Podman but not Docker) and applies the
      --ip-forward-no-drop dockerd override so both engines coexist.
    • Fails fast if SELinux isn't Enforcing, so the suite can't silently
      pass on a permissive guest.
    • Runs mise run e2e:docker then mise run e2e:podman against the
      built supervisor image inside the guest.
    • Greps the guest audit log for AVC denials recorded during the run and
      fails the job if any are found, surfacing SELinux/bind-mount
      relabeling regressions (e.g. the :z/:Z support being added in
      feat(docker,podman): add SELinux label support for bind mounts #2092).
  • Wires the suite into branch-e2e.yml behind a new test:e2e-centos
    label (renamed from test:e2e-rhel), following the existing
    test:e2e-kubernetes pattern (optional, not part of the required CI
    gate, since it boots a VM inside the job rather than using standard
    runner infrastructure).
  • Updates e2e-label-help.yml, CI.md, and CONTRIBUTING.md to document
    the renamed label.

Note on the earlier RHEL-runner approach

An earlier version of this suite required an org admin to provision a
GitHub-hosted RHEL 10 "larger runner", which isn't something CI changes in
this repo can provision on their own. This revision replaces that with a
CentOS Stream 10 VM booted under QEMU/KVM on the existing self-hosted
runner instead, so the suite needs no new runner class and can run as soon
as this PR merges.

Testing

  • Validated all touched/added workflow YAML with actionlint (no new
    findings; pre-existing runner-label warnings are unrelated, caused by
    no actionlint.yaml runner-label config in the repo)
  • mise run markdown:lint passes
  • License header check (update_license_headers.py --check) passes
  • Not runnable end-to-end from this session: no /dev/kvm-capable
    runner available, and no Rust/Python source changed, so mise run pre-commit's Rust/Python checks weren't re-run

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

@copy-pr-bot

copy-pr-bot Bot commented Jul 1, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ericcurtin

Copy link
Copy Markdown
Contributor Author

I think an org admin has to turn on RHEL 10 runners though

@maxamillion

Copy link
Copy Markdown
Collaborator

This looks good to me, but I think you're right on needing an admin. cc @drew @TaylorMutch

@ericcurtin

Copy link
Copy Markdown
Contributor Author

This looks good to me, but I think you're right on needing an admin. cc @drew @TaylorMutch

We could theoretically do CentOS Stream VMs to work around this, would be good enough...

@maxamillion

Copy link
Copy Markdown
Collaborator

@ericcurtin yeah, good point. It probably makes sense too since CentOS Stream more community upstream oriented than RHEL. I do love me some RHEL but this is probably a better fit for CentOS Stream.

PR NVIDIA#2092 adds SELinux relabeling (:z/:Z) support to the Docker and
Podman driver bind mounts, but our existing E2E lanes run inside the
Ubuntu-based ghcr.io/nvidia/openshell/ci container image, which does
not enforce SELinux. That leaves the new relabeling behavior, and any
future SELinux-sensitive change, without a host that can actually deny
a mislabeled mount.

The original version of this suite required an org admin to provision
a GitHub-hosted RHEL 10 "larger runner", which isn't something CI
changes in this repo can provision. Per discussion on PR NVIDIA#2093, swap
that for a CentOS Stream 10 VM booted under QEMU/KVM on the existing
self-hosted linux-amd64-cpu8 runner (already used by
driver-vm-linux.yml with --privileged), driven entirely over SSH. This
needs no new runner class: it downloads a CentOS Stream 10 GenericCloud
image, seeds it with cloud-init over a generated ISO, boots it with
hardware-accelerated QEMU, and copies the checked-out repo in over
scp/tar.

Once the guest is up, the suite installs Docker Engine and Podman side
by side, confirms SELinux is Enforcing, runs the standard e2e:docker
and e2e:podman suites, and fails the job if the audit log shows AVC
denials during the run. It remains wired in as optional/non-blocking
via the test:e2e-centos label (renamed from test:e2e-rhel), matching
the existing test:e2e-kubernetes pattern.

Documentation in CI.md and CONTRIBUTING.md is updated to reflect the
renamed label and workflow.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
maxamillion
maxamillion previously approved these changes Jul 2, 2026
Comment thread .github/workflows/e2e-rhel-selinux.yml Outdated
description: "GitHub Actions runner label (must be a RHEL 10 image)"
required: false
type: string
default: "linux-amd64-rhel10"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These need to be enabled at an org-level. We will have to follow-up internally as to whether this is possible.

@ericcurtin ericcurtin force-pushed the test-podman-docker branch from 0a5b5ed to 0831f62 Compare July 3, 2026 11:14
@ericcurtin ericcurtin changed the title ci(e2e): add RHEL 10 Docker/Podman/SELinux compatibility suite ci(e2e): add CentOS Stream 10 Docker/Podman/SELinux suite Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants