feat(dyref): add multi-reference SFT and RL training

[Ideny42]

Summary

Add the DyRef two-stage training path on top of #417's ordered ImageSet / ImageSets foundation.

• Convert official DyRef metadata into one raw-media manifest shared by SFT and RL, preserving ordered references while keeping the clean target out of model conditioning.
• Train Qwen-Image-Edit-2511 and fixed-N FLUX.2-Klein with multi-reference flow-matching SFT, including Qwen N=1 compatibility and SGLang rollout/replay condition parity.
• Add SigLIP2 semantic similarity, CSD style similarity, discriminative reward scaling, and difficulty-aware advantage reweighting on top of FlowGRPO.
• Warm-start Stage II from an exported Stage-I PEFT adapter and synchronize that adapter before rollout 0.

This is a stacked draft. It depends on #417 and should be rebased onto main after #417 lands.

Related Issue

Refs #470

Depends on #417.

Test Plan

• Full repository hooks:
  SKIP=no-commit-to-branch pre-commit run --all-files --show-diff-on-failure — passed.
• Python syntax:
  python3 -m compileall -q unirl datasets/dyref — passed.
• Hydra composition: all four new Qwen/FLUX SFT and RL configs composed and resolved with local placeholder paths — passed.
• CPU contract harnesses with synthetic N=1/N=2/N=3 data:
  • DyRef conversion, ordered reference typing, target isolation, and fixed-N filtering — passed.
  • Multi-reference diffusion SFT track construction and clean-target VAE shapes — passed.
  • Qwen text placeholders, source-latent packing, img_shapes, SGLang capture splitting, replay reconstruction, and legacy N=1 wire compatibility — passed.
  • SigLIP2 DRS F2, CSD style-reference selection, DAR weighting, and first-rollout weight sync — passed.
  • PEFT Stage-I-to-Stage-II adapter loading plus rank/alpha mismatch rejection — passed.
  • FLUX legacy stretch parity, DyRef aspect-preserving crop, and one-of-{1,2,3} SDE scheduling — passed.
• Not run: real-checkpoint GPU SFT/RL. This draft is being opened before the planned 8×H20 validation (SFT on 8 GPUs; RL on 7 train/rollout GPUs + 1 reward GPU).

Compatibility / Risk

• Existing manifest fields remain valid. DyRef rows add one role="target" image that is projected only into reward metadata; it never becomes a model input.
• LoraConfig.initial_adapter_path is optional. It accepts standard PEFT adapter directories and intentionally rejects rank/alpha mismatch and meta-initialized loading.
• Legacy flat Qwen N=1 latent conditions and flat one-image-per-prompt SGLang requests are normalized without changing N=1 model numerics.
• FLUX keeps stretch as the default condition resize mode; only the DyRef recipes opt into aspect-preserving center crop.
• CSD requires its external ViT-L checkpoint. The adapted model wrapper retains the upstream MIT notice; checkpoint licensing remains governed by its model card.
• The public DyRef code's dar_min_weight=0.5 collapses typical published similarity values to uniform weights. These recipes use 0.01 so DAR remains active; GPU validation must report the resulting weight distribution.
• The Qwen SGLang path may redundantly encode shared references across expanded outputs; GPU profiling will determine whether a follow-up deduplication patch is needed.

Reviewer Notes

• Review this PR's five commits after feat(types): add row-aligned image sets #417 head fe40529; the earlier commits belong to the prerequisite PR.
• Review target isolation first (unirl/data/data_source.py), then Qwen rollout/replay parity, reward math, and adapter warm-start.
• The broader RFC also names HunyuanImage3, BAGEL, Qwen3-Omni, and Janus-Pro capability fixes. Those are intentionally excluded and will be submitted separately.
• Duplicate-work check: feat(flux2_klein): support multiple reference images end to end #351 was superseded by feat(types): add row-aligned image sets #417; no separate open DyRef implementation was found.

Checklist

• I reviewed the changed code and removed unrelated/generated artifacts.
• I updated tests, docs, and configs where needed, or explained why not.