Skip to content

FEP(sig-network): FlagCX new features for FlagOS 2.2 - #85

Open
wbavon wants to merge 6 commits into
mainfrom
fep-flagcx-2.2
Open

FEP(sig-network): FlagCX new features for FlagOS 2.2#85
wbavon wants to merge 6 commits into
mainfrom
fep-flagcx-2.2

Conversation

@wbavon

@wbavon wbavon commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

FEP for the FlagCX features in the FlagOS 2.2 cycle, on top of v0.13.0:

  • Vendor adaptation — T-Head (PPU) backend merged ([PAL] Add PPU support FlagCX#512), 13th in-tree chip backend, testable now. Device API adaptor interface extended to 2 more domestic chips: Kunlunxin first (vendor PR expected ~09-10), second vendor targeted within the window with schedule risk.
  • PD disaggregation (GLM5.2) — T-Head and MetaX, 1P+1D 8+8 cards; ≥3% end-to-end gain vs. Mooncake TransferEngine (default). Runs on T-Head at ~parity with optimization in progress; MetaX gated on its base platform optimization (GLM not yet running there).
  • Distributed operators — AllGather, ReduceScatter, AllGather+GEMM, GEMM+ReduceScatter, targeting intra-/inter-node parity with Triton-distributed; NVIDIA sm90+ this cycle, built on the FlagCX Device API + IR bindings (#539/#545). Adapted on NVIDIA, most scenarios beat torch-native. The fused implementations live in FlagTree (02-allgather-gemm with a benchmark harness vs. torch-native, 03-gemm-allreduce); standalone AG/RS and GEMM+RS code locations plus the Triton-distributed comparison method are open TODOs.
  • Ascend under Non-Goals: blocked on a PCI-probe interface issue, tracked with the sglang-plugin Huawei P0 work.

Test Plan carries the pre-testing inputs from development (official command doc, per-vendor API compatibility list — CI currently covers NVIDIA/MetaX/Hygon) and the 2.2 scope rule (existing features tested to 2.1 scope more thoroughly; new features only within the stated support scope).

Status stays Provisional until the open TODOs are filled: second Device API vendor, ≥3% metric definition, operator code locations, Triton-distributed baseline method. Owner: @MC952-arch. Feature Freeze 2026-08-31.

@wbavon
FEP(sig-network): FlagCX new features for FlagOS 2.2
f94c815 
Signed-off-by: wbtiger <28288271@qq.com>
@CLAassistant

CLAassistant commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@wbavon
Rename FEP file to 0085
89651c0 
Signed-off-by: wbtiger <28288271@qq.com>
@orixzhu2025

orixzhu2025 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Design vendors to adapt FlagCX SVT test cases (new chip vendor with 9 comm operators,40 functions)
Design distributed operator with FlagCX support SVT test cases (T-Head + NV) (T-Head + Metax)
Design PD separation with FlagCX support SVT test cases (T-Head + Cambricon) (GLM 5.2)
Design FlagOS inference integration(vllm-plugin +FlagCX+/+FlagGems+FlagTree) testing
Design FlagOS training integration(FlagCX+Megatron+/+FlagGems+FlagTree) testing
Desgin Stability test for FlagCX crossing different chip vendors(24H+)(Large scale concurrency)(fault injection)

@orixzhu2025 orixzhu2025 added this to the FlagOS 2.2 milestone Jul 30, 2026
@wbavon wbavon added FEP FlagOS Enhancement Proposal sig/network SIG: network target/2.2 Target release: FlagOS 2.2 labels Aug 1, 2026
@wbavon wbavon removed this from the FlagOS 2.2 milestone Aug 1, 2026
@wbavon
fep-0085: sync scope with development status and set Owner
0587d81 
Per the 2026-08-24 sync with development:

- Owner set to @MC952-arch.
- G2 names Kunlunxin as the first Device API vendor (PR ~09-10, second
  vendor carries schedule risk).
- G3 PD disaggregation narrows to T-Head (1P+1D, 8+8 cards); the >=3%
  baseline is Mooncake TransferEngine default. MetaX becomes a stretch
  target: its base optimization is unfinished and GLM does not yet run
  there.
- G4 states the actual status: adapted on NVIDIA, beats torch-native in
  most scenarios, Triton-distributed parity still being optimized;
  vendor scope this cycle is NVIDIA.
- Ascend deferral (PCI-probe issue, tracked with the sglang-plugin
  Huawei P0) recorded under Non-Goals.
- Test Plan adds the pre-testing inputs development provides (command
  doc, per-vendor API compatibility list) and the 2.2 scope rule; CI
  currently covers NVIDIA/MetaX/Hygon.
- Related PRs adds the UIL groundwork (#539/#545) and the two pending
  vendor PRs. Unconfirmed items stay as TODO markers.
@wbavon

wbavon commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

recheck

@wbavon
fep-0085: rewrite G4 to match the FlagTree code
84414fa 
- The fused operators live in FlagTree, not FlagCX:
  python/tutorials/tle/raw/nvshmem/02-allgather-gemm and
  03-gemm-allreduce. The FlagCX side of the feature is the Device API +
  IR bindings the kernels call.
- The operators that exist are AllGather+GEMM and GEMM+AllReduce.
  Standalone AllGather/ReduceScatter and GEMM+ReduceScatter are not in
  the tree; moved to Non-Goals with a TODO to confirm scope.
- G4's test row now uses the real harness: benchmark.py sweeps seven
  layer shapes at M=8192, gates correctness (atol/rtol 1e-3), times six
  variants against torch-native, and dumps a per-shape speedup CSV.
- No Triton-distributed baseline exists anywhere in the tree; kept as
  TODO (add a third variant or run their upstream benchmark, version to
  be fixed) instead of claiming a comparison method.

Signed-off-by: wbavon <wbtigergithub@126.com>
@wbavon
fep-0085: restore Goals to the development feature list
f072d6f 
The development-side 2.2 feature list is the authoritative scope; the
previous revision had narrowed Goals down to the 08-24 status snapshot.
Now Goals state the committed targets and status is noted separately:

- G3: PD disaggregation targets T-Head and MetaX (per the feature list);
  the 08-24 status stays — runs on T-Head at parity, MetaX gated on its
  base platform optimization.
- G4: all four operators are in scope (AllGather, ReduceScatter,
  AllGather+GEMM, GEMM+ReduceScatter) with the Triton-distributed parity
  target. The verified FlagTree state stays: 02-allgather-gemm with its
  benchmark harness, 03-gemm-allreduce without one; standalone AG/RS and
  GEMM+RS have no located code yet and the Triton-distributed
  comparison method is undefined — both kept as TODOs, with test-plan
  rows carrying [TODO] rather than invented commands.
- Non-Goals: GEMM+ReduceScatter and standalone AG/RS removed from the
  exclusion list (they are goals); MetaX removed from the PD exclusion.

Signed-off-by: wbavon <wbtigergithub@126.com>
@wbavon

wbavon commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

recheck

@wbavon
fep-0085: fold status notes into plain prose
b06d387 
Remove the "Status 2026-08-24:" labels and similar framing from G3, G4,
Features 2/3, and the G3/G4 test-plan intros; current state now reads as
normal sentences in place. Collapse the duplicate 08-25 history entries
into one.

Signed-off-by: wbavon <wbtigergithub@126.com>
MC952-arch
MC952-arch reviewed Aug 27, 2026
View reviewed changes
Comment thread fep/sig-network/0085-flagcx-flagos-2.2-new-features.md
2. **PD-disaggregation support and optimization for inference** — run
prefill-decode disaggregation for GLM5.2 on T-Head and MetaX with ≥3%
end-to-end gain over the Mooncake TransferEngine baseline, building on the
P2P Engine introduced in v0.13.0

@MC952-arch MC952-arch Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GLM5.2 PD分离测试目前仅在T-Head上跑通，后续高优开展优化工作，沐曦有风险

MC952-arch
MC952-arch reviewed Aug 27, 2026
View reviewed changes
Comment thread fep/sig-network/0085-flagcx-flagos-2.2-new-features.md
([FEP-0021](0021-flagcx-v0.13.0-new-features.md)).
3. **Distributed operators** — AllGather and ReduceScatter, plus the fused
AllGather+GEMM and GEMM+ReduceScatter operators, targeting intra-node and
inter-node performance on par with Triton-distributed.

@MC952-arch MC952-arch Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议添加Torch原生对比

MC952-arch
MC952-arch reviewed Aug 27, 2026
View reviewed changes
Comment thread fep/sig-network/0085-flagcx-flagos-2.2-new-features.md
([FEP-0021](0021-flagcx-v0.13.0-new-features.md)).
3. **Distributed operators** — AllGather and ReduceScatter, plus the fused
AllGather+GEMM and GEMM+ReduceScatter operators, targeting intra-node and
inter-node performance on par with Triton-distributed.

@MC952-arch MC952-arch Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

三组测试：

  • intra/inter-node ReduceScatter
  • intra/inter-node AllGather
  • intra/inter-node GEMM+ReduceScatter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MC952-arch MC952-arch MC952-arch left review comments
@Al2O3Al2O3 Al2O3Al2O3 Awaiting requested review from Al2O3Al2O3

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

FEP FlagOS Enhancement Proposal sig/network SIG: network target/2.2 Target release: FlagOS 2.2

Projects

FlagOS FEP Tracker
Status: Todo

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wbavon @CLAassistant @orixzhu2025 @MC952-arch