test(reward): add H20 MPS qualification harness and baseline

[Zcchill]

Summary

• extend the existing reward-service load test with batch/concurrency sweeps,
  repetitions, score capture, and a compact JSON summary
• treat missing, empty, and non-finite reward responses as request failures
• add dedicated and shared H20 example configs under configs/

This is the benchmark/qualification slice of #463. It does not add production
MPS lifecycle or scheduling behavior. Generated result JSON is not committed.

H20 qualification

Fresh rerun on unirl-v4-8-gpus-gz-0920 (UniRL v4 image): NVIDIA H20 96 GB,
driver 535.247.01; Python 3.12.11; Ray 2.46.0; Torch 2.7.1; Transformers
4.56.0.dev0. Each mode ran CLIP and PickScore at batch sizes 1/4/8 and
concurrency 1/4/16 for 1,620 requests (7,020 items). Mean items/s is the
average of the 27 runs.

• dedicated, two GPUs: 1,620/1,620 valid; 137.30 items/s
• fractional Ray without MPS, one GPU: 1,620/1,620 valid; 82.48 items/s;
  +20.1% mean rewards/GPU-hour over dedicated
• MPS at 100%, one GPU: 1,620/1,620 valid; 97.27 items/s; +17.9% mean raw
  throughput over fractional no-MPS and +41.7% mean rewards/GPU-hour over
  dedicated
• consolidated control, one GPU: 1,620/1,620 valid; 143.73 items/s; +109.4%
  mean rewards/GPU-hour over dedicated
• all 81 candidate run-level parity checks among dedicated / no-MPS / MPS
  passed at atol=1e-3; including consolidated, the maximum absolute score
  delta was 4.56e-4

Active-thread limit finding

CUDA_MPS_ACTIVE_THREAD_PERCENTAGE=50 produced non-finite FP16 CLIP scores on
this H20/driver stack, while PickScore FP16 and both scorers in FP32 remained
finite. FP16 CLIP and PickScore both passed at 100%. This report therefore
qualifies the tested recipe only at 100%.

Validation

• fresh four-mode H20 rerun on unirl-v4-8-gpus-gz-0920: 6,480/6,480
  requests valid
• PYTHONPATH=. python3 -m pytest tests/test_bench_concurrent.py -q (2 passed)
• both H20 YAML files pass load_config
• git diff --check

[jyizheng]

Ran the checked-in recipe on one B300 (driver 580.105, torch 2.14+cu130, transformers 4.57.6), same sweep, 1620/1620 valid in every mode:

mode	GPUs	items/s	rewards/GPU-h vs dedicated
dedicated	2	127.5	—
fractional, no MPS	1	122.2 (-4%)	+92%
fractional, MPS 100%	1	124.9 (-2%)	+96%

GPUs sit at 12% busy in the dedicated layout, so this panel is CPU/HTTP-bound here and colocation is nearly free; the H20 numbers are a GPU-bound regime. Worth stating which regime applies before quoting the multiplier.

Two things to flag before the next rerun:

1. Both actors load PickScore_v1, so clip is the PickScore backbone with a different divisor, and the consolidated control is the expected ~2x from halving the work. Pointing clip at openai/clip-vit-large-patch14 would make it a real two-backbone panel.
2. The fp16 CLIP NaN at active-thread 50% does not reproduce on B300: 32 combos (PickScore_v1 / CLIP-L-14 × fp16 / fp32 × off / 100 / 50 / 25), all finite, values identical to 5 decimals. It looks specific to the H20 / driver 535 / torch 2.7.1 stack rather than to MPS partitioning. Happy to share the probe script.

Also: the PR body still cites the consolidated arm, 81 parity checks and atol 1e-3, but that tooling and the report file were removed in the last two pushes.

[Zcchill]

Superseded by #492. The benchmark commits are already in #492's branch history; #492 is being retargeted to main so implementation and qualification can be reviewed together.