Reduce CUDA wheel size for PyPI publishing - #6
Merged
Conversation
andimarafioti
force-pushed
the
reduce-cuda-wheel-size
branch
from
June 25, 2026 09:47
b578afa to
5b1b072
Compare
andimarafioti
force-pushed
the
reduce-cuda-wheel-size
branch
from
June 25, 2026 09:49
5b1b072 to
de4ae9d
Compare
andimarafioti
marked this pull request as ready for review
June 25, 2026 13:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PyPI publishing is failing because the CUDA wheels are above the project's 100 MB per-file upload limit. The failed
v0.2.0CUDA 12.8 wheels were about 157 MiB each, almost entirely fromqwentts_cpp/lib/libggml-cuda.so.0.This PR keeps the public PyPI package on CUDA 12.8 and reduces the CUDA architecture matrix used by the publish workflow:
75-virtual;86-real;90-real;120-real;120-virtualThe PR/dispatch wheel workflow still builds downloadable validation artifacts for CUDA 12.8, CUDA 13.0, and CPU. CUDA 13.0 artifacts use:
75-virtual;80-real;86-real;90-real;121-realThat keeps native CUDA 13 cubins for A100 (
sm_80), A10G/A10-class Ampere (sm_86), Hopper (sm_90), and DGX Spark / GB10 (sm_121), plus PTX fallback for older supported GPUs. These CUDA 13 wheels are artifacts only; they are not uploaded to PyPI.Hardware tradeoffs for PyPI CUDA 12.8 wheels
sm_86): native cubin, expected fast first load/TTFA.sm_90): native cubin.sm_120): native cubin.sm_89): can use thesm_86cubin through CUDA same-major binary compatibility.sm_80): PTX fallback, so first use may pay CUDA JIT cost.sm_121): PTX fallback in the public CUDA 12.8 wheel.Validation
.github/workflows/publish.ymland.github/workflows/wheels.ymlwith PyYAML.File too large. Limit for project 'qwentts-cpp-python' is 100 MB.v0.2.0artifacts:libggml-cuda.so.0: about 156.2 MiB compressed in each wheelsm_121: fresh-cache first TTFA about 0.5s, total about 1.6s.75-virtual;80-real;86-real;90-real;121-real: 96.49 MiB locally on aarch64, first TTFA 521ms, total 1.57s.Notes
This should unblock PyPI without waiting for a file-size-limit increase. The package remains optimized for the CUDA 12.8 server path, while CUDA 13 wheels remain available from GitHub Actions artifacts for Spark and CUDA 13 testing.