Skip to content

tools: Omega-QVLA dit_svdquant_v1 pack → E0M3/UE4M3 converter + format doc + synthetic fixture - #179

Merged
LiangSu8899 merged 2 commits into
flashrt-project:mainfrom
Lmy271828:pr1-omega-e0m3-converter
Aug 23, 2026
Merged

tools: Omega-QVLA dit_svdquant_v1 pack → E0M3/UE4M3 converter + format doc + synthetic fixture#179
LiangSu8899 merged 2 commits into
flashrt-project:mainfrom
Lmy271828:pr1-omega-e0m3-converter

Conversation

@Lmy271828

@Lmy271828 Lmy271828 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Scope

Offline tooling for converting Omega-QVLA dit_svdquant_v1 rank-0 packs into the packed E0M3 weight and tile-interleaved UE4M3 SFB format consumed by FlashRT's existing SM110 GEMM.

This PR does not add a frontend, pipeline, runtime consumer, server, CUDA graph, CMake source, pybind symbol, or public Python API. It has no effect on existing model inference or default builds.

Included:

  • tools/convert_omega_pack_e0m3.py: schema validation and offline conversion.
  • tools/check_omega_e0m3_layer.py: CPU emulation plus direct converted-artifact GEMM verification on Thor.
  • tools/gen_omega_pack_fixture.py: deterministic four-record synthetic fixture.
  • docs/omega_pack_e0m3.md: Milestone 1 format and reproduction contract.

Audit follow-up

Commit 7355c3c6 addresses the merge review:

  • Removes documentation for runtime/serving/graph files that are not present in this PR. External runtime experiments are no longer represented as repository functionality.
  • Validates every selected record before CUDA work: format, rank=0, W4A4 metadata, N/K metadata, low-rank shapes, positive finite scale tables, rotation shapes, finite tensors, and a real int64 permutation.
  • Fails instead of skipping unsupported records. An unfiltered complete pack requires 252 records by default; fixtures and filtered subsets require complete selected coverage.
  • Adds CPU-only --validate-only for schema and coverage checks.
  • Records schema version, selected layers, and source/selected counts in the artifact. weights, aux, and selected_layers must cover the same set.
  • Writes through a unique sibling temporary file and atomically replaces the destination only after complete conversion. The source pack cannot be used as --out.
  • Adds --artifact to the checker. This path quantizes only the activation and feeds the artifact's stored packed and sfb tensors directly to cutlass_fp4_gemm_e0m3w; it fails when artifact-vs-fp16 cosine is below --min-artifact-cos.
  • Narrows .gitignore from global *.pt to /artifacts/omega_e0m3/*.pt.
  • Adds 13 CPU contract tests.

Reproduce

CPU validation and emulation:

python tools/gen_omega_pack_fixture.py --out /tmp/fixture_pack.pt
python tools/convert_omega_pack_e0m3.py \
    --pack /tmp/fixture_pack.pt --out /tmp/unused.pt --validate-only
python tools/check_omega_e0m3_layer.py \
    --pack /tmp/fixture_pack.pt --mode emulate

Thor direct artifact round-trip:

python tools/convert_omega_pack_e0m3.py \
    --pack /tmp/fixture_pack.pt --out /tmp/fixture_e0m3.pt --fold none
python tools/check_omega_e0m3_layer.py \
    --pack /tmp/fixture_pack.pt --artifact /tmp/fixture_e0m3.pt \
    --mode kernel --min-artifact-cos 0.98

Full pack conversion defaults to a strict 252/252 coverage gate. Use --expected-records only for a deliberately different complete pack.

Validation

  • Tool contract tests: 13 passed.
  • Changed Python scripts: py_compile passed.
  • git diff --check: passed.
  • Default fixture generation: passed; four schema records load with weights_only=True.
  • Fixture CPU validation: 4/4 passed.
  • Fixture CPU emulation: passed.
  • Private path scan: clean.

The Thor host was offline during this audit follow-up, and the real 252-record pack was not available locally. Before merge, rerun the two hardware gates introduced here: fixture packed/SFB direct GEMM round-trip and real-pack 252/252 conversion plus representative direct artifact GEMM checks. Cross-platform model E2E is not required because this PR has no runtime integration.

@Lmy271828
Lmy271828 requested a review from LiangSu8899 as a code owner August 18, 2026 07:58
@Lmy271828
Lmy271828 force-pushed the pr1-omega-e0m3-converter branch from c12e822 to 32f02a5 Compare August 22, 2026 13:11
@LiangSu8899

Copy link
Copy Markdown
Member

Audit fixes are pushed in 7355c3c6.

The PR is now limited to its actual Milestone 1 surface. The converter validates all selected records and fails on unsupported format/rank/shape/rotation/permutation/scale metadata; full-pack conversion defaults to a strict 252-record gate. Output is atomic and records explicit schema/coverage metadata.

The checker now has a real artifact round-trip: --artifact loads the converted packed and sfb tensors and feeds them directly to the SM110 GEMM. It does not regenerate the weight from weight_res_q, and it returns failure below --min-artifact-cos.

Validation completed locally:

  • 13 tool contract tests passed.
  • Default four-record fixture generated and loaded with weights_only=True.
  • CPU validation reported 4/4.
  • CPU emulate checker passed.
  • py_compile and git diff --check passed.
  • Merge simulation against current main is clean.

The Thor host was offline and the real 252-record pack was not present locally. The PR body now lists the remaining pre-merge hardware gates precisely: direct fixture packed/SFB GEMM round-trip, then real-pack 252/252 conversion and representative direct artifact GEMM checks. No model E2E is needed for this offline-only PR.

@LiangSu8899
LiangSu8899 merged commit 4b45669 into flashrt-project:main Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants