[RFC] Community refresh of main-repository documentation

[CjhHa1]

Summary

UniRL's runtime capabilities have expanded faster than several user-facing and package-local README files. This RFC coordinates a community refresh of the main framework repository documentation and splits the work into independent, reviewable tasks.

The scope of this issue is Tencent-Hunyuan/UniRL. Changes to the separate unirl-project/unirl documentation site should be tracked independently.

Goals

• Make the top-level README, installation guide, examples index, and package-local READMEs agree with the current code and checked-in recipes.
• Give every documented entrypoint, algorithm, model, rollout path, and reward/data workflow a traceable source of truth.
• Let contributors claim one focused task and submit it as an independent PR.
• Avoid broad rewrites or formatting-only churn outside the claimed task.

Sources of truth

Use implementation/configuration sources before existing prose:

• Python/CUDA versions and extras: pyproject.toml
• Entrypoints and built-in defaults: unirl/train_*.py
• Runnable configurations: examples/**/*.yaml
• Public algorithms: unirl/algorithms/__init__.py
• Model integration: unirl/models/<model>/ plus at least one matching recipe
• Rollout engines: unirl/rollout/engine/
• Reward names: unirl/reward/local/registry.py and unirl-reward-service/reward_service/scorers/registry.py
• Benchmark registrations: benchmarks/core/registry.py

A model package existing by itself is not sufficient evidence for end-to-end support. Document restrictions such as SFT-only, experimental, engine-specific, or service-dependent explicitly.

Work order

Complete DOC-MAIN-01 first. Tasks 02–06 can then proceed in parallel. A maintainer performs DOC-MAIN-07 after the focused PRs land.

[x] DOC-MAIN-01 — Entry points, installation, and canonical recipes @MinhaoLi0318 (#491)

Difficulty: small to medium
Files: README.md, INSTALL.md, examples/README.md

• Reconcile the training-domain table with all current unirl/train_*.py entrypoints, including SFT, agentic, and async AR/diffusion.
• Make built-in default recipes consistent with the @hydra.main(config_name=...) declarations.
• Resolve conflicting prompt-enhancer defaults.
• Check optional extras against pyproject.toml, including engine-specific, training, evaluation, dataset preparation, Cosmos3, FastVideo, and VeOmni dependencies.
• Keep pyproject.toml as the dependency/version source of truth; identify legacy install paths as non-recommended where appropriate.

Acceptance: every listed entrypoint and canonical recipe exists; installation extras match pyproject.toml; example commands use current nested recipe paths.

[ ] DOC-MAIN-02 — Algorithm reference matrix @aKMSHSF

Difficulty: medium
Files: unirl/algorithms/README.md and links from README.md

• Replace fixed or incomplete family counts with a matrix derived from public algorithm exports and recipes.
• Cover supervised objectives, AR algorithms, diffusion/flow algorithms, teacher-distillation objectives, and unified-model objectives.
• Record domain, advantage requirements, multi-update support, and anchor/recompute behavior.
• Clarify that DanceGRPO and MixGRPO are recipe-level FlowGRPO variants rather than independent algorithm classes.
• Link team algorithm tutorials without duplicating their full content.

Acceptance: every public objective exported from unirl/algorithms/__init__.py is represented or intentionally excluded with a reason; claims agree with class capabilities and a checked-in recipe where applicable.

[x] DOC-MAIN-03 — Model support matrix @MinhaoLi0318 (#501)

Difficulty: medium
Files: README.md, unirl/models/README.md

• Inventory current model bundles and matching recipes.
• Distinguish bundle availability from an end-to-end runnable path.
• Add currently omitted families where supported, and state restrictions such as SFT-only, bundle-only, engine-specific, or scorer-dependent.
• Verify model names against package and recipe naming rather than marketing aliases alone.

Acceptance: each support row links to implementation and/or a runnable recipe; no bundle-only package is presented as generally runnable.

[ ] DOC-MAIN-04 — Training paths: SFT, async, HSDP, and VeOmni @MrDongsls

Difficulty: medium
Files: unirl/trainer/README.md, unirl/train/readme.md, unirl/train/sft/README.md, relevant sections of examples/README.md

• Document the SFT path from manifest to entrypoint, train stack, checkpoint, and resume behavior.
• Document async AR and async diffusion entrypoints, canonical recipes, publication/quiesce boundaries, and limitations.
• Add user-facing HSDP and VeOmni guidance tied to existing recipes.
• Cross-link trainer orchestration and train-stack implementation details instead of duplicating them.

Acceptance: a reader can trace each path from data/config to entrypoint and checkpoint behavior; every command points to a checked-in recipe.

[ ] DOC-MAIN-05 — Reward, runtime data, datasets, and benchmarks @Dmaner

Difficulty: medium
Files: unirl/reward/README.md, unirl-reward-service/README.md, a new unirl/data/README.md, datasets/README.md, benchmarks/README.md

• Reconcile local and remote scorer lists with their registries and note disabled or deployment-specific scorers.
• Document the runtime data boundary separately from offline dataset conversion.
• Refresh dataset and benchmark indexes against the current directories/registries.
• Mark internal development logs and resume prompts as maintainer handoff material rather than user guides.

Acceptance: scorer and benchmark names match registries; dataset/runtime responsibilities are unambiguous; index links resolve locally.

[ ] DOC-MAIN-06 — Rollout and distributed runtime @GrassHeadd

Difficulty: medium
Files: unirl/rollout/README.md, unirl/rollout/engine/README.md, engine-specific READMEs, unirl/distributed/README.md, unirl/distributed/weight_sync/README.md

• Refresh the engine matrix and direct/separate/colocated deployment descriptions.
• Document current sampling ownership and replay/anchor-sensitive behavior.
• Cover FastVideo WAN 2.1/2.2 support accurately.
• Separate rollout-output transport (including TransferQueue) from trainer-to-rollout weight synchronization.
• Document async barriers/quiescence where they affect rollout and synchronization.

Acceptance: engine, transport, and sync names match current classes/config fields; deployment diagrams and examples do not conflate data transport with weight sync.

[ ] DOC-MAIN-07 — Maintainer consistency pass

Difficulty: medium
Owner: repository maintainer after tasks 01–06 merge

• Review the combined diff against the then-current main branch.
• Remove duplicated tables, conflicting defaults, dead links, and unrelated cleanup.
• Verify cross-links among top-level, examples, and package-local README files.
• Confirm terminology is consistent across domains.

Acceptance: the same entrypoint/version/recipe receives one consistent answer across all affected documents.

How to claim a task

Comment with the task ID, the files you plan to touch, and whether you need a domain reviewer. A maintainer can record the assignee in this issue. Prefer one task per PR; reference this issue and the task ID in the PR description.

If a task is already claimed, coordinate before editing overlapping files. In particular, README.md and examples/README.md are shared by several tasks and should receive final integration edits in DOC-MAIN-07.

Validation

Run the checks relevant to the files you touched:

# Resolve each canonical recipe documented by your PR
python -m unirl.train_<entry> --config-name=<domain>/<recipe> --cfg job --resolve

# Repository checks
SKIP=no-commit-to-branch pre-commit run --all-files --show-diff-on-failure

Also verify local Markdown links and anchors in the changed files. GPU training is not required for documentation-only PRs; note any command that could not be composed locally and why.

Non-goals

• Reorganizing or rewriting the separate Fumadocs site.
• Changing runtime behavior solely to match stale prose.
• Adding subjective support claims without a checked-in implementation/configuration source.
• Combining unrelated code cleanup with documentation PRs.

[EthemKD]

I'd like to take DOC-MAIN-01. I'll reconcile README.md, INSTALL.md, and examples/README.md against the current train entrypoints, Hydra defaults, checked-in recipes, and pyproject.toml extras. I'll keep the PR limited to DOC-MAIN-01 and call out any ambiguity instead of guessing.

[CjhHa1]

I'd like to take DOC-MAIN-01. I'll reconcile README.md, INSTALL.md, and examples/README.md against the current train entrypoints, Hydra defaults, checked-in recipes, and pyproject.toml extras. I'll keep the PR limited to DOC-MAIN-01 and call out any ambiguity instead of guessing.

For Sure! I'll assign you and record it

[aKMSHSF]

I’d like to take DOC-MAIN-02 — Algorithm reference matrix.

Files to touch:
unirl/algorithms/README.md
Linked files from README.md

Plan:
I’ll derive the matrix from the public algorithm exports, implementation classes, and checked-in recipes.

Domain reviewer: Not required.

[CjhHa1]

I’d like to take DOC-MAIN-02 — Algorithm reference matrix.

Files to touch: unirl/algorithms/README.md Linked files from README.md

Plan: I’ll derive the matrix from the public algorithm exports, implementation classes, and checked-in recipes.

Domain reviewer: Not required.

For sure!

[MrDongsls]

@CjhHa1 Hi, I’d like to take this. I’ll document the SFT and async training paths and add HSDP/VeOmni guidance across the listed READMEs, based on the current implementations and checked-in recipes. Could you assign it to me?

[GrassHeadd]

@CjhHa1
I'd like to take DOC-MAIN-06 (rollout and distributed runtime).

Plan: go through the engine READMEs and distributed docs, fix anything that doesn't match the current code: engine matrix, deployment modes, sampling ownership and replay/anchor behavior, async barriers/quiescence, FastVideo WAN support, and untangle rollout transport (TransferQueue) from weight sync. Will verify everything against the actual classes/configs.

Will clarify any problems here if i get stuck

Files to touch:

• unirl/rollout/README.md
• unirl/rollout/engine/README.md + engine-specific READMEs
• unirl/distributed/README.md
• unirl/distributed/weight_sync/README.md

[MinhaoLi0318]

@CjhHa1 like to take DOC-MAIN-04 — Training paths: SFT, async, HSDP, and VeOmni.

I plan to audit the relevant training entrypoints, trainer/train-stack documentation, and checked-in recipes, with particular attention to the async AR/diffusion paths, publication/quiesce boundaries, checkpoint/resume behavior, and current HSDP/VeOmni recipes.

Files I expect to touch:

• unirl/trainer/README.md
• unirl/train/readme.md
• unirl/train/sft/README.md
• relevant sections of examples/README.md

I’ll use the implementation and checked-in recipes as the source of truth and keep the changes scoped to DOC-MAIN-04.

A trainer/async-runtime domain reviewer would be helpful if available.

[MinhaoLi0318]

@CjhHa1 I’d like to take DOC-MAIN-03 — Model support matrix.

I plan to audit the current model bundles against checked-in recipes and actual runnable paths, and update:

• README.md
• unirl/models/README.md

I’ll verify model names against the implementation/configuration and explicitly distinguish:

• model bundle availability
• end-to-end runnable support
• SFT-only paths
• engine-specific integrations
• scorer-dependent or experimental paths

I’ll also make sure each support row is backed by an implementation path and/or a checked-in runnable recipe, rather than inferring support from the existence of a model package alone.

A model-integration domain reviewer would be helpful if available.

[Dmaner]

@CjhHa1 I’d like to take DOC-MAIN-05 — Reward, runtime data, datasets, and benchmarks.

I plan to reconcile the local and remote scorer lists with their registries, clarify runtime data handling versus offline dataset conversion, and refresh the dataset and benchmark indexes, including any disabled or deployment-specific components.

Files I expect to touch:

• unirl/reward/README.md
• unirl-reward-service/README.md
• unirl/data/README.md
• datasets/README.md
• benchmarks/README.md

I’ll use the current implementations and checked-in configurations as the source of truth, verify local links, and keep the changes scoped to DOC-MAIN-05.

A reward/data domain reviewer would be helpful if available.