Skip to content

Reduce CUDA wheel size for PyPI publishing - #6

Merged
andimarafioti merged 3 commits into
mainfrom
reduce-cuda-wheel-size
Jun 25, 2026
Merged

Reduce CUDA wheel size for PyPI publishing#6
andimarafioti merged 3 commits into
mainfrom
reduce-cuda-wheel-size

Conversation

@andimarafioti

@andimarafioti andimarafioti commented Jun 25, 2026

Copy link
Copy Markdown
Owner

Summary

PyPI publishing is failing because the CUDA wheels are above the project's 100 MB per-file upload limit. The failed v0.2.0 CUDA 12.8 wheels were about 157 MiB each, almost entirely from qwentts_cpp/lib/libggml-cuda.so.0.

This PR keeps the public PyPI package on CUDA 12.8 and reduces the CUDA architecture matrix used by the publish workflow:

  • CUDA 12.8 publish wheels: 75-virtual;86-real;90-real;120-real;120-virtual

The PR/dispatch wheel workflow still builds downloadable validation artifacts for CUDA 12.8, CUDA 13.0, and CPU. CUDA 13.0 artifacts use:

  • CUDA 13.0 validation wheels: 75-virtual;80-real;86-real;90-real;121-real

That keeps native CUDA 13 cubins for A100 (sm_80), A10G/A10-class Ampere (sm_86), Hopper (sm_90), and DGX Spark / GB10 (sm_121), plus PTX fallback for older supported GPUs. These CUDA 13 wheels are artifacts only; they are not uploaded to PyPI.

Hardware tradeoffs for PyPI CUDA 12.8 wheels

  • A10G (sm_86): native cubin, expected fast first load/TTFA.
  • Hopper (sm_90): native cubin.
  • CUDA 12.8 Blackwell baseline (sm_120): native cubin.
  • Ada (sm_89): can use the sm_86 cubin through CUDA same-major binary compatibility.
  • A100 (sm_80): PTX fallback, so first use may pay CUDA JIT cost.
  • DGX Spark / GB10 (sm_121): PTX fallback in the public CUDA 12.8 wheel.

Validation

  • Parsed .github/workflows/publish.yml and .github/workflows/wheels.yml with PyYAML.
  • Confirmed the failing publish log reports: File too large. Limit for project 'qwentts-cpp-python' is 100 MB.
  • Inspected the failed v0.2.0 artifacts:
    • x86_64 CUDA 12.8 wheel: 157.0 MiB
    • aarch64 CUDA 12.8 wheel: 156.9 MiB
    • bundled libggml-cuda.so.0: about 156.2 MiB compressed in each wheel
  • Built and benchmarked CUDA 13/aarch64 comparison wheels locally on DGX Spark to verify the PTX/native first-run behavior:
    • PTX fallback: fresh-cache first TTFA 15.87s, total 16.94s; warm-cache first TTFA 466ms.
    • Native sm_121: fresh-cache first TTFA about 0.5s, total about 1.6s.
    • Broad CUDA 13 artifact policy 75-virtual;80-real;86-real;90-real;121-real: 96.49 MiB locally on aarch64, first TTFA 521ms, total 1.57s.

Notes

This should unblock PyPI without waiting for a file-size-limit increase. The package remains optimized for the CUDA 12.8 server path, while CUDA 13 wheels remain available from GitHub Actions artifacts for Spark and CUDA 13 testing.

@andimarafioti
andimarafioti force-pushed the reduce-cuda-wheel-size branch from b578afa to 5b1b072 Compare June 25, 2026 09:47
@andimarafioti
andimarafioti force-pushed the reduce-cuda-wheel-size branch from 5b1b072 to de4ae9d Compare June 25, 2026 09:49
@andimarafioti
andimarafioti marked this pull request as ready for review June 25, 2026 13:15
@andimarafioti
andimarafioti merged commit f2d5187 into main Jun 25, 2026
6 checks passed
@andimarafioti
andimarafioti deleted the reduce-cuda-wheel-size branch June 25, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant