docs(examples): add managed GPU inference codelab

[Ganeshkumar Ashokavardhanan (ganeshkumarashok)]

What this PR adds

This PR adds a guided example for serving Qwen/Qwen2.5-7B-Instruct with two vLLM replicas on AKS-managed NVIDIA GPU nodes.

The codelab covers the complete workflow:

• Validate CLI versions, preview registration, regional SKU availability, and quota.
• Create a CPU-only AKS cluster and a fixed two-node managed A100 pool.
• Verify CUDA allocation and managed DCGM metrics on every GPU node.
• Stage and verify model weights once on a shared Azure Blob NFS volume.
• Deploy two private vLLM replicas across separate nodes and send an OpenAI-compatible request.
• Observe device and service metrics, then safely remove GPU capacity or the complete example.

Design and safety

The example keeps the inference Service private and uses kubectl port-forward instead of creating an unauthenticated public endpoint. The scripts are safe to rerun and include ownership markers, subscription-specific kubeconfig contexts, existing-resource compatibility checks, fail-safe cleanup, and useful diagnostics for long-running failures.

The serving manifest includes bounded resources, tagged images, startup/readiness/liveness probes, topology spread, a PodDisruptionBudget, and maxSurge: 0 so a rollout does not require a third GPU.

This uses the guided codelab structure discussed in #5965, but is self-contained under examples/managed-gpu-inference-service/.

Validation

The documented serving configuration was validated end to end with AKS 1.35, two Standard_NC24ads_A100_v4 nodes, vllm/vllm-openai:v0.28.0, shared Blob NFS model storage, two ready replicas on separate nodes, and successful completion requests.

The contribution also passed:

• bash -n and ShellCheck for every script.
• YAML parsing and required Kubernetes field checks for every manifest.
• Compilation checks for all embedded Python blocks.
• Relative Markdown link validation.
• The repository-style codespell check.
• Container tag resolution and current Azure CLI flag checks.

Scope

This example intentionally does not add public ingress, authentication, autoscaling, or multi-region routing. Those production concerns and the preview support limitations are documented in the README.

[Copilot]

🟡 Changes recommended

Critical cleanup safety issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a self-contained AKS managed GPU inference codelab for serving Qwen with two private vLLM replicas, shared Blob NFS storage, and managed A100 nodes.

Changes:

• Adds guided setup, validation, inference, observability, and cleanup documentation.
• Adds repeatable Azure and Kubernetes automation scripts.
• Adds manifests for GPU checks, model staging, storage, and vLLM serving.

File summaries

File	Reviewed changes
examples/managed-gpu-inference-service/scripts/lib.sh	Shared configuration and ownership helpers
examples/managed-gpu-inference-service/scripts/90-cleanup.sh	Workload, pool, and resource cleanup
examples/managed-gpu-inference-service/scripts/50-deploy-vllm.sh	vLLM deployment and request validation
examples/managed-gpu-inference-service/scripts/40-stage-model.sh	Model download and verification
examples/managed-gpu-inference-service/scripts/30-verify-gpu.sh	CUDA and DCGM validation
examples/managed-gpu-inference-service/scripts/20-create-managed-gpu-pool.sh	Managed GPU pool setup
examples/managed-gpu-inference-service/scripts/10-create-cluster.sh	AKS cluster creation
examples/managed-gpu-inference-service/scripts/00-preflight.sh	Environment, feature, SKU, and quota checks
examples/managed-gpu-inference-service/README.md	Codelab overview and workflow
examples/managed-gpu-inference-service/modules/07-cleanup.md	Cleanup procedures
examples/managed-gpu-inference-service/modules/06-observability.md	Metrics guidance
examples/managed-gpu-inference-service/modules/05-inference-service.md	vLLM deployment
examples/managed-gpu-inference-service/modules/04-model-storage.md	Model staging
examples/managed-gpu-inference-service/modules/03-verify.md	GPU validation
examples/managed-gpu-inference-service/modules/02-managed-gpu-nodepool.md	Managed GPU pool setup
examples/managed-gpu-inference-service/modules/01-cluster.md	CPU cluster setup
examples/managed-gpu-inference-service/modules/00-prerequisites.md	Prerequisites and preflight
examples/managed-gpu-inference-service/manifests/vllm-serving.yaml	vLLM Deployment, Service, and PDB
examples/managed-gpu-inference-service/manifests/namespace.yaml	Dedicated namespace
examples/managed-gpu-inference-service/manifests/model-storage.yaml	Blob NFS storage resources
examples/managed-gpu-inference-service/manifests/model-stage-job.yaml	Model staging Job
examples/managed-gpu-inference-service/manifests/gpu-smoke-test.yaml	GPU validation pod

Review details

Suppressed comments (3)

examples/managed-gpu-inference-service/modules/05-inference-service.md:20

• A PodDisruptionBudget limits voluntary evictions; it cannot guarantee that one replica is always available during crashes, node loss, or other involuntary failures. Describe this as allowing at most one voluntary disruption so the module does not promise stronger availability than the manifest provides.

- A PodDisruptionBudget with one replica always available.

examples/managed-gpu-inference-service/modules/06-observability.md:8

• The verifier reaches 19400 through hostNetwork and localhost, not through a Kubernetes NodePort Service. Calling this a node port can lead readers to look for a NodePort resource or use the wrong access path; describe it as a node-local port instead.

| DCGM exporter | Node port `19400` | GPU activity, memory, temperature, power, and hardware errors |

examples/managed-gpu-inference-service/scripts/lib.sh:55

• version_ge compares only extracted numeric tuples, so the stable 19.0.0 compares lower than the configured 19.0.0b29 minimum even though a stable release supersedes that prerelease. Once the preview extension ships that stable version, preflight will incorrectly reject it; parse prerelease/stable ordering explicitly.

def parts(value):
    return tuple(int(part) for part in re.findall(r"\d+", value))

raise SystemExit(0 if parts(sys.argv[1]) >= parts(sys.argv[2]) else 1)

• Files reviewed: 22/22 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.