Skip to content

KUBERNETES_ENABLE_POD_UID does not add pod UID label in DRA-attributed metrics #718

Description

@rajathagasthya

Description

When GPU allocation happens through DRA (KUBERNETES_ENABLE_DRA=true), setting DCGM_EXPORTER_KUBERNETES_ENABLE_POD_UID=true does not add the UID label to metrics. The same configuration on a device-plugin node adds it as expected.

Expected behavior

With both KUBERNETES_ENABLE_DRA=true and
DCGM_EXPORTER_KUBERNETES_ENABLE_POD_UID=true, metrics attributed to a pod holding a DRA ResourceClaim should carry the UID label, the same as device-plugin allocated GPUs.

Actual behavior

DRA-attributed series carry pod, namespace, container, the dra_* labels, and (when enabled) pod labels, but no UID, regardless of the pod-UID setting.

Steps to reproduce

  1. Node with DRA GPU allocation (e.g. GPU Operator with the DRA stack; NVIDIA DRA driver publishing ResourceSlices).
  2. Run dcgm-exporter with KUBERNETES_ENABLE_DRA=true, DCGM_EXPORTER_KUBERNETES_ENABLE_POD_UID=true, a ServiceAccount token, and read access to resource.k8s.io/resourceslices and pods.
  3. Start a pod that requests a GPU via a ResourceClaim and scrape /metrics.
  4. Metrics for that pod have pod/namespace/container and dra_* labels but no UID label. Repeating with a device-plugin allocation on the same exporter produces the UID label.

Environment

Observed with dcgm-exporter 4.6.0-4.8.3 (GPU Operator DRA stack, Kubernetes with resource.k8s.io/v1).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions