Skip to content

ci: run CPU core tests on pull requests - #306

Open
kasikci wants to merge 3 commits into
mainfrom
codex/cpu-core-ci
Open

kasikci wants to merge 3 commits into
mainfrom
codex/cpu-core-ci

Conversation

@kasikci

@kasikci kasikci commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a GitHub-hosted CPU Core job covering graph I/O, resource runner, admission failure handling, micro-scheduler, worker drain, and ragged-attention head-dimension padding.
  • Move the two existing padding test functions (six cases) from test_ragged_attention.py into test_ragged_attention_cpu.py, so they run without the original module's CUDA requirement. Preserve all moved assertions and all remaining GPU tests.
  • Use Python 3.12 and pinned CPU PyTorch, dependency caching, a 15-minute timeout, cancellation of superseded same-PR runs, timing output, and a seven-day JUnit artifact.
  • Document the exact command in CONTRIBUTING.md. No production code or dependencies change.

The job uses the existing PR open/update/reopen triggers. Its explicit six-module list avoids collecting unrelated GPU-dependent tests. Making CPU Core a required merge check remains a separate repository setting.

Files changed

  • .github/workflows/ci.yml
  • CONTRIBUTING.md
  • test/modular/test_ragged_attention.py
  • test/modular/test_ragged_attention_cpu.py

Local validation

Fresh disposable Linux aarch64 container, Python 3.12.14, CPU PyTorch 2.9.1.

python -m pip check
HF_HUB_OFFLINE=1 PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 \
python -m pytest -q -ra --strict-markers --durations=20 \
  test/modular/test_ragged_attention_cpu.py
HF_HUB_OFFLINE=1 PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 \
python -m pytest -q -ra --strict-markers --durations=20 \
  --junitxml=cpu-core.xml \
  test/modular/test_graph_io.py \
  test/modular/test_resource_runner.py \
  test/modular/test_admit_failure_handling.py \
  test/modular/test_micro_scheduler.py \
  test/modular/test_worker_drain.py \
  test/modular/test_ragged_attention_cpu.py
python -m ruff check .
git diff --check
  • Padding module alone: 6 passed, zero skipped, pytest 0.03 seconds (1.19 seconds including startup).
  • Full CPU Core selection: 102 passed, zero skipped, pytest 1.07 seconds (2.02 seconds including startup).
  • pip check, Ruff, and diff whitespace checks passed.
  • Workflow YAML parsed; every shell run block passed bash -n.
  • AST comparison confirmed all moved and remaining test/helper function bodies and decorators are unchanged.
  • No GPU tests were executed locally; their code is preserved. Dense-attention expansion is outside this change.

GitHub validation

CPU Core passed on GitHub-hosted Ubuntu for commit 1bb8ffb71dd3ef988578e07c021b07b6aa3110ff:

  • 102 passed, zero skipped; pytest reported 1.89 seconds.
  • Test step: 5 seconds.
  • Dependency installation: 29 seconds, with the pip download cache restored.
  • Complete job: 49 seconds, excluding queue time.
  • JUnit artifact uploaded successfully.

These are timings from one run, not a latency guarantee. All four CI jobs passed: CPU Core, Ruff/build, Dynamo (48 tests), and Rust transport (17 native tests and 35 Python interoperability tests, plus both builds).

@kasikci
kasikci requested a review from NSagan271 September 22, 2026 15:50

@NSagan271 NSagan271 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment, and also something that came to mind for a next step in the testing restructure: a lot of the tests have to mock components of the system (the engine, the api server data worker, the worker, etc.), an a centralized system for dealing with those would be good to have.

Comment thread .github/workflows/ci.yml Outdated
run: |
python -m pytest -q -ra --strict-markers --durations=20 \
--junitxml=cpu-core.xml \
test/modular/test_graph_io.py \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer something more automated for adding more tests to the core set, either a test/core directory, or a file listing all of the core tests.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense

Comment thread .github/workflows/ci.yml Outdated
test/modular/test_admit_failure_handling.py \
test/modular/test_micro_scheduler.py \
test/modular/test_worker_drain.py \
test/modular/test_ragged_attention_cpu.py

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the rationale for the core set of tests? I think most seem fundamental, but test_ragged_attention_cpu a test for a specific, not heavily-used resource. And maybe some others should be added, like test/modular/test_shm_tensor_comm.py, test/modular/test_worker_speculation_via_graph_api.py could be added, though coming up with an actual set of core tests would involve heavily cleaning up the tests that we have.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides test_ragged_attention_cpu, the other are fundamental and should be there. test_ragged_attention_cpu is the first attempt to disentangle CPU vs GPU tests (previously it was a bit of a mash for this test). There isn't a special reason this is here, but it's first in a sequence of more refactorings. We can decide to keep or remove bunch of tests once we have a clean CPU suite.

@kasikci

kasikci commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

One comment, and also something that came to mind for a next step in the testing restructure: a lot of the tests have to mock components of the system (the engine, the api server data worker, the worker, etc.), an a centralized system for dealing with those would be good to have.

That makes sense to me. I will assign myself an issue to deal with this once I determine a full core set of CPU tests.

@NSagan271 NSagan271 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants