feat(ltx2): add CLAP-only T2AV reward recipe

[leviking98z-rgb]

Summary

• Add an LTX-2.3 T2AV FlowGRPO recipe that optimizes generated audio with a configurable CLAP reward.
• Add an explicit audio policy log-prob weight for joint audio/video SDE trajectories. None preserves latent-element weighting, while 0.0..1.0 selects an explicit modality mixture; the AudioCaps recipe uses 1.0 (audio-only policy objective).
• Support a composite CLAP reward with configurable matched-cosine and hardest-negative retrieval-margin weights. The recipe uses matched cosine + retrieval margin with weights 1.0 + 1.0.
• Use official human-written AudioCaps captions as both the LTX-2.3 generation prompt and matched CLAP text, with seven deterministic captions from other official records as negatives.
• Add an AudioCaps manifest converter pinned to upstream commit d004db3ea1b01cf4fd0347dd8d27db90cadc8809.
• Use deterministic ODE evaluation with prompt-stable, per-sample initial noise so checkpoint curves isolate policy changes instead of eval-time SDE noise.
• Load the checkpoint-declared LTX-2.3 vocoder class, preserve the 48 kHz audio path, and support CLAP processor signatures used by transformers 4.x and 5.x.

Related Issue

N/A

Data

• Dataset: AudioCaps, introduced at NAACL 2019 and built from AudioSet clips.
• Training manifest: all 49,838 official train clips, one human caption per clip.
• Validation manifest: 64 distinct clips selected deterministically from the official validation split, one of the five human captions per clip.
• Train/validation clip overlap: 0.
• Generated JSONL files are local artifacts and are not committed. Recreate them with:

python datasets/audiocaps/prepare_audiocaps.py --out-dir data/audiocaps

The positive CLAP text is read directly from each record's standard prompt field. Seven deterministic captions from other records in the same split are stored as negative candidates. For each generated waveform:

matched_cosine = cosine(audio, matched_caption)
retrieval_margin = matched_cosine - max(cosine(audio, negative_caption_i))
reward = matched_cosine + retrieval_margin

This is still an audio-only reward: CLAP embeds the generated waveform and text captions. "Audio-only policy" separately means that the GRPO importance ratio uses the audio trajectory log-prob; LTX-2.3 continues to jointly denoise audio and video.

Test Plan

• pre-commit run --all-files --show-diff-on-failure
• Recipe target validation: 2,733 _target_ paths resolved.
• Hydra compose plus OmegaConf.resolve, with assertions for batch size 8, group size 8, audio-only log-prob, constant 1e-4 LR, zero warmup, per-group advantage normalization, independent rollout x_T, and deterministic eval.
• Constant-scheduler assertion: learning rate remained exactly 1e-4 for 500 synthetic optimizer steps.
• CLAP scorer checks on the training node, including per-record negatives and one-time unique-text encoding.
• Manifest validation: 49,838 unique train clips, 64 unique validation clips, and zero clip overlap.
• 8x H20 end-to-end baseline run at commit e13b1ac4: 100 rollouts, batch size 8 prompts, group size 8 samples per prompt, 49 frames, 4 denoise steps, 2 SDE steps, 256x384, 2 optimizer updates per rollout, exit code 0.
  • W&B: https://wandb.ai/leviking98z-zhejiang-university/unirl_video_zw23/runs/ucigolhe
  • Dynamic-prompt rollout reward: first 20 mean -0.10874, last 20 mean +0.00322; matched component -0.01884 -> +0.05051 and margin component -0.08990 -> -0.04730.
  • All 200 optimizer updates had finite, nonzero loss and positive gradient norm. Gradient norm range: 0.00452..0.08740.
  • Learning rate stayed exactly 1e-4 for all 200 updates: constant schedule, zero warmup, no decay.
  • Zero-variance group ratio stayed 0; clipping stayed 0; observed ratio range was 0.99878..1.00009.
• 8x H20 targeted ablation inspired by feat(minimax-h3): t2va model package for Flow-GRPO (trainside FSDP) #347: 100 rollouts with only adv_use_global_std=true and sampling.init_same_noise=true changed; batch/group size, reward, audio-only policy, optimizer, LR, and rollout budget were unchanged. Exit code 0.
  • W&B: https://wandb.ai/leviking98z-zhejiang-university/unirl_video_zw23/runs/oyd8myqf
  • The ablation was not adopted because the deterministic endpoint and curve linearity regressed.
• Deterministic ODE checkpoint sweep over both runs at rollouts 0/20/40/60/80/100: 32 fixed AudioCaps prompts, 4 samples per prompt, 4 denoise steps, 256x384, eval_eta=0.0, and prompt-stable per-sample x_T.
  • Comparison W&B: https://wandb.ai/leviking98z-zhejiang-university/unirl_video_zw23/runs/w0u97h72

Training strategy	Composite 0 -> 100	OLS slope / rollout	R2	Best composite
recipe: per-group std, independent x_T	-0.1784 -> -0.0433 (+0.1351)	+0.001494	0.9119	-0.0433 @ 100
#347 ablation: global std, shared x_T	-0.1784 -> -0.1199 (+0.0585)	+0.000943	0.4797	-0.0580 @ 80

For the retained recipe, deterministic matched cosine improved -0.0666 -> +0.0075, retrieval margin improved -0.1118 -> -0.0508, and 8-way retrieval top-1 improved 16.41% -> 32.03% (best 35.94% at rollout 80).

Compatibility / Risk

• Defaults are backward-compatible: omitted audio_policy_logp_weight keeps latent-element weighting, matched_cosine_weight defaults to 1.0, and retrieval_margin_weight defaults to 0.0.
• The recipe requires an LTX-2.3 audio-capable checkpoint and 8 GPUs for the documented launch configuration.
• A single cosine similarity is bounded by [-1, 1]. The configured composite reward is not: with unit weights its mathematical range is [-3, 3], because the retrieval margin is also included. Absolute score should not be interpreted as a percentage or benchmark grade.
• The 64-clip validation manifest, with 32 prompts sampled per evaluation, demonstrates optimization behavior but is not a replacement for a full AudioCaps benchmark with generation-quality metrics such as FAD.
• Audio-only policy weighting targets the generated audio trajectory directly, but the audiovisual transformer still co-denoises both modalities and audio/video quality can remain coupled.
• The rollout remains stochastic at eta=0.7; evaluation uses eval_eta=0.0. The recipe deliberately uses a constant learning rate with no warmup and no decay.

Reviewer Notes

• PR feat(ltx2.3): add LTX-2.3 T2AV support with joint audio+video SDE policy #115 provides the joint audio/video SDE policy and PR feat(ltx2): add T2AV audio reward (CLAP + composite, ImageBind optional) #168 provides waveform decoding and audio reward plumbing. This PR exercises that merged path with a CLAP-only reward model.
• Using the paired human caption as the positive CLAP target is the conventional text-audio alignment setup. The retrieval margin adds a contrastive signal against other real AudioCaps captions so that raising similarity to every caption equally does not improve the full reward.
• PR feat(minimax-h3): t2va model package for Flow-GRPO (trainside FSDP) #347's batch-wide advantage std and shared initial noise were tested together as a controlled ablation. They improved some middle checkpoints but regressed at rollout 100, so this recipe intentionally retains per-group normalization and independent rollout initial noise.
• Earlier in-training eval used eta=0.7; although its initial noise was stable, its per-step SDE noise changed with the eval step. The ODE sweep above is the comparable checkpoint curve, and the recipe now defaults to that evaluation mode.
• Terminal summaries may display aggregate GRPO loss as 0.0000 because signed normalized advantages cancel at a ratio near 1 and the logger prints four decimals. Raw W&B history contains finite, nonzero loss values.
• AI assistance was used. The full diff and experiment outputs were reviewed. Open PRs/issues were searched for CLAP, AudioCaps, and LTX audio overlap; no duplicate implementation was found.

Checklist

• I reviewed the changed code and removed unrelated/generated artifacts.
• I updated tests, docs, and configs where needed, or explained why not.

[leviking98z-rgb]

100-rollout H20 validation completed successfully.

• W&B: https://wandb.ai/leviking98z-zhejiang-university/unirl_video_zw23/runs/h9yi7m8o
• reward mean/std: 0.092621 / 0.027247
• reward range: 0.0179..0.1439
• grad norm mean/range: 0.003803 / 0.0012..0.0140
• exit code: 0
• no traceback, non-finite values, or vocoder loading mismatch warnings