build(sglang): migrate runtime to 0.5.19

[CjhHa1]

Summary

Migrate UniRL from SGLang 0.5.12.post1 to 0.5.19 and narrow the integration to contracts that are still missing upstream:

• update the CUDA 13 / Torch 2.13 / Transformers 5.12 dependency baseline and AR ServerArgs names
• delete fork-era diffusion patches superseded by 0.5.19 native post-training, LoRA, sleep/wake, and scheduler APIs
• remove legacy packaging/config aliases and duplicate LoRA target ownership; use model_config.lora_target_modules for ServerArgs leaf targets and adapters for pipeline components
• bind native AR control calls to the exact 0.5.19 request API, fail closed on missing patch contracts, and reject grouped-trajectory misalignment instead of silently reusing rows
• retain only UniRL-specific distributed broadcasts, flattened tensor payloads, TeaCache reset, driver-authoritative rollout/noise/condition patches, and CUDA staging required by IPC

Related Issue

N/A

Test Plan

• uv pip compile pyproject.toml --extra sglang --extra train --extra infer --python-version 3.12 --prerelease allow --no-header --no-annotate — passed during migration validation
• uv pip compile pyproject.toml --extra vllm --extra train --extra infer --python-version 3.12 --prerelease allow --no-header --no-annotate — passed during migration validation
• SKIP=no-commit-to-branch uvx pre-commit run --all-files --show-diff-on-failure — passed after rebase and final simplification
• one-off CPU assertions — passed:
  • exact native abort/pause/resume request contracts
  • unknown-argument warn/strict behavior
  • grouped trajectory concat/slice plus malformed partial-field rejection
  • multi-component LoRA success and partial-failure state
  • model-owned LoRA target lowering and YAML construction checks
  • diffusion LoRA rejects non-standard PEFT scaling and full-weight updates on LoRA-configured engines
• Hydra resolution — passed:
  • Qwen3 AR (DATA_PATH=/tmp/data)
  • SD3, Qwen-Image, FLUX.2-Klein, and WAN2.2 SGLang diffusion recipes

Checkpoint-backed Validation Summary

All GPU gates used real, complete pretrained checkpoints rather than random, tiny, or layer-truncated models. Here, "full checkpoint" describes the loaded model; it does not mean that every gate performed full-parameter training.

• Qwen3-4B deterministic rollout/replay used the full base checkpoint with no LoRA. It covered seeded repeatability, sleep/wake, rollout-versus-replay log-probability drift, and a signed backward pass.
• Qwen3-4B TP2 started from the full checkpoint and exercised both synchronization modes in one gate: unchanged and mutated base/full-weight synchronization, then zero and mutated LoRA synchronization.
• Qwen3-30B-A3B loaded the full MoE checkpoint with no LoRA and covered deterministic TP2/EP2 generation. The unsupported rl_on_policy_target=fsdp + fused MoE KV combination was intentionally excluded because production does not use it.
• SD3.5 alignment used the full base transformer with use_lora=false. It covered grouped rollout, shared initial noise, per-output trajectory separation, replay drift, and signed backward. Training-side replay skipped VAE loading because image decoding is not part of the policy/replay comparison.
• SD3.5 synchronization started from the complete checkpoint with a PEFT adapter attached and tested both paths: base/full-weight mutation and LoRA mutation. The final Beijing gate then verified native sleep/wake while the mutated LoRA state was active.
• These are checkpoint-backed inference, replay, gradient, mutation, and lifecycle gates. They do not include optimizer steps, long-running training, or reward-curve convergence.

Prior H20 validation remains applicable to the retained runtime paths:

• package/import smoke: unirl-sglang-0519-smoke-0917-v5
• Qwen3-4B deterministic lifecycle: unirl-sglang-0519-align-ar-gz-0917-v2
• Qwen3-4B production numeric gate: unirl-sglang-0519-align-ar-prod-oracle-gz-0918
• Qwen3-4B TP2 full-weight/LoRA mutation: unirl-sglang-0519-tp2-sync-gz-0918-v4
• Qwen3-30B-A3B TP2/EP2 deterministic gate: unirl-sglang-0519-qwen30b-ep2-gz-0918-v6
• SD3.5 alignment and mutation: unirl-sglang-0519-align-sd3-gz-0918-v3, unirl-sglang-0519-sd3-sync-gz-0918-v4
• SD3.5 native sleep/wake after custom memory-saver retirement: unirl-sglang-0519-native-sleep-gz-0920, instance 8b1d805da0af10b001a0bd3474a915a6; sleep→wake rollout bitwise exact, rollout/replay absdiff mean 1.2930483e-04, max 5.2893162e-04, signed backward grad norm 0.2479593; exit code 0
• SD3.5 blocker regression with P=2 prompts / K=2 outputs: unirl-sglang-0519-blocker-regression-gz-0920, instance 8b1d810aa0af112801a0bd6669c2153c; batch shape [4,5,16,32,32], sleep→wake bitwise exact, distinct grouped trajectories, rollout/replay absdiff mean 7.8734010e-05, max 4.3755770e-04, signed backward grad norm 0.1263615; exit code 0
• SD3.5 final full-weight / LoRA mutation and native sleep/wake on Beijing H20: unirl-sglang-final-on-occupy-bj-0920 on qwen38-bt-invariance-occupy-8x8-bj-0920 launcher ts-8b1d810aa0af112801a0be34c11116fd-launcher; seeded grouped repeat, unchanged full-weight sync, and zero-LoRA sync bitwise exact; full-weight and LoRA mutations changed output; post-LoRA native sleep→wake bitwise exact; exit code 0
• SD3.5 production LoRA scale regression for LoRA scale (alpha/rank) silently wrong on rename-based models in SGLang rollout #155: unirl-sglang-sd3-lora-scale-bj-0921 on the same Beijing H20 launcher; the production attention adapter mapped all 191 layers with skipped=0; an isolated terminal-layer probe used the production rank/alpha (rank=32, alpha=64, expected scale 2.0) and measured a 2.1025x LoRA-effect L2 ratio versus alpha=32, while the production-alpha trajectory was closer to the bf16 merged-weight reference (mean absdiff 7.8393e-04 vs 8.8442e-04); exit code 0

The repository intentionally keeps no committed top-level test tree, so focused migration harnesses were removed after recording the reproducible assertions and GPU task identifiers above.

Compatibility / Risk

• Diffusion sleep/wake uses 0.5.19's native MemoryOccupationController. UniRL patches the pinned implementation so rollback includes the current module, nested unregistered tensors follow residency changes, and TP/SP/CFG/DP failures propagate to the caller. The removed tagged CUDA-VM layer could not be enabled on 0.5.19 because diffusion ServerArgs has no enable_memory_saver field.
• Driver-owned initial noise/audio/seeds are partitioned by prompt and expanded output before execution; grouped trajectory and denoising metadata are concatenated and sliced with strict alignment checks.
• AR recipes use cuda_graph_max_bs_decode; the retired alias is removed so stale configs fail through the existing unknown-argument path.
• Diffusion LoRA ServerArgs leaf targets now have one owner (model_config); weight-sync pipeline components remain adapter-owned (transformer, plus transformer_2 for WAN2.2).
• Diffusion tensor LoRA accepts standard PEFT LoRA with one global alpha. rsLoRA, DoRA, per-layer alpha patterns, and auxiliary PEFT features fail closed. Sync mode is config-owned for the engine lifetime: use_lora=true permits adapter sync only; full-weight sync requires use_lora=false and a trainer-side merged adapter.
• Multi-output rollout trajectories now fail fast on partial fields, mismatched schedules, or invalid batch indices instead of returning misaligned policy data.
• Official SD3.5 CLIP folders ship both model.safetensors and model.fp16.safetensors. SGLang 0.5.19's customized CLIP loader refuses that layout and falls back to native CLIPTextModel, dropping text_projection. This is an upstream 0.5.19 gap; the same filter already exists for transformers and shipped for text encoders in SGLang 0.5.20 (sgl-project/sglang#37616). The Beijing mutation/sleep gates passed on that fallback path.
• No checkpoint or data-format migration is introduced.

The production Qwen gate passes its mean <0.05 / max <0.5 limits. Its remaining mean rollout/replay difference (0.03673) is attributable to bf16 head/log-softmax precision plus incremental Triton KV-cache decode versus full-sequence flex-attention teacher forcing, with partial cancellation.

Reviewer Notes

Rebased onto current upstream/main. The final diff removes more code than it adds and has been re-reviewed against PR_REVIEW_REDUNDANCY_CHECKLIST.md. Review the exact-version AR control calls, native diffusion rollback/rank aggregation, multi-prompt driver routing, and grouped trajectory invariants first.

AI-assisted implementation; the submitter reviewed the final diff. No agent was added as a commit co-author.

Checklist

• I reviewed the changed code and removed unrelated/generated artifacts.
• I updated tests, docs, and configs where needed, or explained why not.

[Jayce-Ping]

Most changes are configs/docs only. Code changes are about SGLang migration. Since the real GPU tests have passed, I think it can be merged.

[celve]

Double check this, is this intentional?