docs(examples): add end-to-end GPU inference codelab

examples/end-to-end-AI-codelabs/README.md

@@ -0,0 +1,24 @@
+# End-to-end AI codelabs on AKS
+
+Build complete AI scenarios on Azure Kubernetes Service (AKS). Each codelab
+starts with Azure infrastructure and ends with a working application and an
+observable result.
+
+These examples are designed as hands-on learning experiences. Each module has a
+goal, an estimated time, commands, a checkpoint, focused troubleshooting, and a
+clear next step.
+
+## Codelabs
+
+| Codelab | What you build | Time |
+|---|---|---:|
+| [Provision GPU inference with Kueue](kueue-gpu-inference-autoscaling/) | Start with zero GPU nodes, submit a three-pod inference workload through Kueue, and watch the AKS cluster autoscaler provision all three GPU nodes before the models serve requests | 45–60 minutes |
+
+## Before you begin
+
+The codelabs create billable Azure resources. Review the prerequisites and cost
+notes for a codelab before you start, and run its cleanup module when you finish.
+
+The examples are intentionally kept in Markdown, shell scripts, and Kubernetes
+manifests. This makes them easy to run from GitHub now and suitable for a more
+interactive lab experience later.

examples/end-to-end-AI-codelabs/kueue-gpu-inference-autoscaling/README.md

@@ -0,0 +1,115 @@
+# Provision GPU inference with Kueue on AKS
+
+Start with an empty GPU node pool, submit a three-pod inference workload, and
+watch Azure Kubernetes Service (AKS) add all three GPU nodes before Kueue allows
+the workload to run.
+
+## What you'll build
+
+```mermaid
+sequenceDiagram
+    participant You
+    participant Kueue
+    participant PR as ProvisioningRequest
+    participant CAS as AKS cluster autoscaler
+    participant GPU as GPU node pool
+    participant vLLM
+
+    You->>Kueue: Submit suspended three-pod inference Job
+    Kueue->>PR: Request three GPUs
+    PR->>CAS: Best-effort atomic scale-up
+    CAS->>GPU: Scale from zero to three nodes
+    GPU-->>CAS: All three GPU nodes are ready
+    CAS-->>PR: Provisioned=True
+    PR-->>Kueue: Admission check ready
+    Kueue->>vLLM: Unsuspend all three pods
+    vLLM->>vLLM: Each pod loads Qwen2.5-0.5B and runs a request
+    vLLM-->>You: INFERENCE_VALIDATED × 3
+```
+
+This codelab demonstrates three separate responsibilities:
+
+- **Kueue** decides when the workload can start.
+- **ProvisioningRequest and the AKS cluster autoscaler** add node capacity.
+- **vLLM** loads a real model and serves an OpenAI-compatible request.
+
+## Learning objectives
+
+After you finish, you'll be able to:
+
+- Configure a Kueue queue that gates GPU workloads on capacity provisioning.
+- Identify each state between workload submission and Kueue admission.
+- Scale an AKS GPU node pool from zero before pods are created.
+- Validate GPU inference with a real model request.
+- Distinguish workload admission, node autoscaling, and inference replica scaling.
+
+## Time and cost
+
+- **Time:** 45–60 minutes, including cluster and GPU node provisioning.
+- **Cost:** The cluster uses two CPU system nodes. Up to three GPU nodes are
+  billed while the pool is above zero. Model download and cold start can take
+  several minutes.
+- **Cleanup:** Run `./scripts/90-cleanup.sh --all` as soon as you finish.
+
+Azure GPU availability and quota vary by subscription and region. The preflight
+script checks both before creating resources.
+
+## Important managed GPU limitation
+
+This codelab uses a conventional GPU node pool with the AKS-installed driver and
+an explicitly installed NVIDIA device plugin. It does **not** use
+`--enable-managed-gpu=true` because managed GPU node pools don't support the
+cluster autoscaler during preview.
+
+The fully managed GPU experience and automatic GPU node scaling can't currently
+be demonstrated on the same pool. This codelab prioritizes the end-to-end
+Kueue-to-autoscaler path.
+
+## Modules
+
+| # | Module | Goal | Time |
+|---|---|---|---:|
+| 1 | [Check prerequisites](modules/01-check-prerequisites.md) | Confirm tools, Azure access, GPU SKU availability, and quota | 5 minutes |
+| 2 | [Create the cluster](modules/02-create-cluster.md) | Create a Kubernetes 1.35 AKS cluster with CPU system nodes | 15 minutes |
+| 3 | [Create the GPU pool](modules/03-create-gpu-pool.md) | Add an autoscaling GPU pool that starts at zero | 5 minutes |
+| 4 | [Install the controllers](modules/04-install-controllers.md) | Install Kueue and the NVIDIA device plugin | 5 minutes |
+| 5 | [Configure provisioning](modules/05-configure-provisioning.md) | Connect Kueue to the AKS cluster autoscaler | 3 minutes |
+| 6 | [Run inference](modules/06-run-inference.md) | Trigger scale-up and make a real model request | 10–20 minutes |
+| 7 | [Observe and troubleshoot](modules/07-observe-and-troubleshoot.md) | Read each control-plane handoff and diagnose stalls | 10 minutes |
+| 8 | [Clean up](modules/08-cleanup.md) | Remove GPU capacity and all lab resources | 5 minutes |
+
+## Fast path
+
+Follow the modules for explanations and checkpoints. After reading them once,
+use this sequence to repeat the lab:
+
+```bash
+./scripts/00-preflight.sh
+./scripts/10-create-cluster.sh
+./scripts/20-create-gpu-pool.sh
+./scripts/30-install-controllers.sh
+./scripts/40-configure-queue.sh
+./scripts/50-submit-inference.sh
+./scripts/60-watch-flow.sh
+./scripts/90-cleanup.sh --all
+```
+
+## Validation status
+
+The complete checked-in scenario was validated in `eastasia` using three
+`Standard_NV6ads_A10_v5` nodes:
+
+- The pool started with zero nodes.
+- Kueue created one Workload and a ProvisioningRequest with `count: 3`.
+- CAS changed the pool from zero to three.
+- All three nodes became Ready and advertised `nvidia.com/gpu`.
+- The ProvisioningRequest reached `Provisioned=True` with reason
+  `CapacityIsProvisioned`.
+- Kueue admitted the Job only after all capacity was ready.
+- The Job completed `3/3`, and every pod returned `INFERENCE_VALIDATED`.
+- Submission through Job completion took 418 seconds.
+
+The run established the settings required by a 4-GB A10-4Q profile: 70% GPU
+memory utilization, a 2,048-token context, four sequences, and eager execution.
+The watcher reports Azure pool count and Ready Kubernetes node count separately
+so node-bootstrap failures remain visible.

examples/end-to-end-AI-codelabs/kueue-gpu-inference-autoscaling/manifests/inference-job.yaml

@@ -0,0 +1,114 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: vllm-inference-check
+  namespace: gpu-inference
+  labels:
+    app.kubernetes.io/name: vllm-inference-check
+    app.kubernetes.io/part-of: kueue-gpu-inference-codelab
+    kueue.x-k8s.io/queue-name: gpu-inference
+spec:
+  suspend: true
+  parallelism: 3
+  completions: 3
+  activeDeadlineSeconds: 2400
+  backoffLimit: 0
+  ttlSecondsAfterFinished: 3600
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: vllm-inference-check
+        app.kubernetes.io/part-of: kueue-gpu-inference-codelab
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: inference
+          image: vllm/vllm-openai:v0.10.2
+          command:
+            - bash
+            - -lc
+            - |
+              set -euo pipefail
+              LOG=/tmp/vllm.log
+
+              vllm serve Qwen/Qwen2.5-0.5B-Instruct \
+                --served-model-name qwen \
+                --host 127.0.0.1 \
+                --port 8000 \
+                --gpu-memory-utilization 0.70 \
+                --max-model-len 2048 \
+                --max-num-seqs 4 \
+                --enforce-eager >"$LOG" 2>&1 &
+              server_pid=$!
+              trap 'kill "$server_pid" 2>/dev/null || true' EXIT
+
+              echo "Waiting for vLLM to load the model..."
+              for attempt in $(seq 1 120); do
+                if python3 - <<'PY'
+              import urllib.request
+              urllib.request.urlopen("http://127.0.0.1:8000/health", timeout=2)
+              PY
+                then
+                  break
+                fi
+                if ! kill -0 "$server_pid" 2>/dev/null; then
+                  echo "vLLM exited before becoming ready"
+                  cat "$LOG"
+                  exit 1
+                fi
+                if [[ "$attempt" == "120" ]]; then
+                  echo "Timed out waiting for vLLM"
+                  cat "$LOG"
+                  exit 1
+                fi
+                sleep 5
+              done
+
+              python3 - <<'PY'
+              import json
+              import urllib.request
+
+              body = json.dumps({
+                  "model": "qwen",
+                  "messages": [{"role": "user", "content": "Reply with exactly: AKS inference ready"}],
+                  "max_tokens": 16,
+                  "temperature": 0,
+              }).encode()
+              request = urllib.request.Request(
+                  "http://127.0.0.1:8000/v1/chat/completions",
+                  data=body,
+                  headers={"Content-Type": "application/json"},
+              )
+              with urllib.request.urlopen(request, timeout=120) as response:
+                  result = json.load(response)
+              content = result["choices"][0]["message"]["content"].strip()
+              if not content:
+                  raise RuntimeError("The model returned an empty response")
+              print(f"MODEL_RESPONSE: {content}")
+              print("INFERENCE_VALIDATED")
+              PY
+          env:
+            - name: HF_HOME
+              value: /models/cache
+          resources:
+            requests:
+              cpu: "2"
+              memory: 8Gi
+              nvidia.com/gpu: "1"
+            limits:
+              cpu: "4"
+              memory: 16Gi
+              nvidia.com/gpu: "1"
+          volumeMounts:
+            - name: model-cache
+              mountPath: /models/cache
+            - name: dshm
+              mountPath: /dev/shm
+      volumes:
+        - name: model-cache
+          emptyDir:
+            sizeLimit: 20Gi
+        - name: dshm
+          emptyDir:
+            medium: Memory
+            sizeLimit: 4Gi

examples/end-to-end-AI-codelabs/kueue-gpu-inference-autoscaling/manifests/kueue-gpu-queue.yaml

@@ -0,0 +1,87 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: gpu-inference
+  labels:
+    aks-codelab: kueue-gpu-inference-autoscaling
+    app.kubernetes.io/part-of: kueue-gpu-inference-codelab
+---
+apiVersion: kueue.x-k8s.io/v1beta2
+kind: ResourceFlavor
+metadata:
+  name: gpu-inference
+  labels:
+    aks-codelab: kueue-gpu-inference-autoscaling
+spec:
+  nodeLabels:
+    agentpool: gpupool
+  tolerations:
+    - key: sku
+      operator: Equal
+      value: gpu
+      effect: NoSchedule
+---
+apiVersion: kueue.x-k8s.io/v1beta2
+kind: ProvisioningRequestConfig
+metadata:
+  name: gpu-inference
+  labels:
+    aks-codelab: kueue-gpu-inference-autoscaling
+spec:
+  provisioningClassName: best-effort-atomic-scale-up.autoscaling.x-k8s.io
+  managedResources:
+    - nvidia.com/gpu
+---
+apiVersion: kueue.x-k8s.io/v1beta2
+kind: AdmissionCheck
+metadata:
+  name: gpu-inference-provisioning
+  labels:
+    aks-codelab: kueue-gpu-inference-autoscaling
+spec:
+  controllerName: kueue.x-k8s.io/provisioning-request
+  parameters:
+    apiGroup: kueue.x-k8s.io
+    kind: ProvisioningRequestConfig
+    name: gpu-inference
+---
+apiVersion: kueue.x-k8s.io/v1beta2
+kind: ClusterQueue
+metadata:
+  name: gpu-inference-cluster-queue
+  labels:
+    aks-codelab: kueue-gpu-inference-autoscaling
+spec:
+  namespaceSelector:
+    matchLabels:
+      app.kubernetes.io/part-of: kueue-gpu-inference-codelab
+  queueingStrategy: BestEffortFIFO
+  resourceGroups:
+    - coveredResources:
+        - cpu
+        - memory
+        - nvidia.com/gpu
+      flavors:
+        - name: gpu-inference
+          resources:
+            - name: cpu
+              nominalQuota: "12"
+            - name: memory
+              nominalQuota: 48Gi
+            - name: nvidia.com/gpu
+              nominalQuota: "3"
+  admissionChecksStrategy:
+    admissionChecks:
+      - name: gpu-inference-provisioning
+        onFlavors:
+          - gpu-inference
+---
+apiVersion: kueue.x-k8s.io/v1beta2
+kind: LocalQueue
+metadata:
+  name: gpu-inference
+  namespace: gpu-inference
+  labels:
+    aks-codelab: kueue-gpu-inference-autoscaling
+spec:
+  clusterQueue: gpu-inference-cluster-queue