Skip to content

[Bindings] Track python shutdown issue on Macos - #2088

Draft
tdavidcl wants to merge 25 commits into
Shamrock-code:mainfrom
tdavidcl:cursor/macos26-runtestpy-light-ci-11e5
Draft

tdavidcl wants to merge 25 commits into
Shamrock-code:mainfrom
tdavidcl:cursor/macos26-runtestpy-light-ci-11e5

Conversation

@tdavidcl

Copy link
Copy Markdown
Member

Enable runtestpy on macos-generic.acpp / macos-26 and run that env_tests job during light CI so the python-module shutdown crash can be tracked on PRs without enabling the full env_tests matrix.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks @tdavidcl for opening this PR!

You can do multiple things directly here:
1 - Comment pre-commit.ci run to run pre-commit checks.
2 - Comment pre-commit.ci autofix to apply fixes.
3 - Add label autofix.ci to fix authorship & pre-commit for every commit made.
4 - Add label full-ci to run the full test suite (default is light CI; full CI also runs on Mergify merge-queue branches).
5 - Add label trigger-ci to create an empty commit to trigger the CI.

Once the workflow completes a message will appear displaying informations related to the run.

Also the PR gets automatically reviewed by gemini, you can:
1 - Comment /gemini review to trigger a review
2 - Comment /gemini summary for a summary
3 - Tag it using @gemini-code-assist either in the PR or in review comments on files

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

cursoragent and others added 3 commits August 18, 2026 06:26
Enable runtestpy on macos-generic.acpp / macos-26 and run that env_tests
job during light CI so the python-module shutdown crash can be tracked
on PRs without enabling the full env_tests matrix.

Assisted-by: Cursor Agent
GitHub Actions does not expose matrix in job-level if expressions.
Build the env_tests matrix in a set_matrix job and keep only macos-26
when light_ci is true, matching the Intel/LLVM workflow pattern.

Assisted-by: Cursor Agent
Python-module runs never called close(), so DeviceScheduler queues
were destroyed during process-exit static teardown after AdaptiveCpp
had already shut down. On macOS that throws
std::system_error("mutex lock failed: Invalid argument") and aborts.

Register atexit only in lib mode (embed already closes from main /
shamtest), make close() idempotent, and reset schedulers first.

Assisted-by: Cursor Agent
@cursor
cursor Bot force-pushed the cursor/macos26-runtestpy-light-ci-11e5 branch from faf096f to 47aed84 Compare August 18, 2026 06:27
@tdavidcl tdavidcl changed the title [CI] Enable macos-26 python tests in light CI [Bindings] Track python shutdown issue on Macos Aug 18, 2026
tdavidcl and others added 6 commits August 18, 2026 08:17
Introduce shamcomm::MpiInstance, which initializes MPI on construction
when it is not already started and finalizes it on destruction or close()
only if this instance owns the session.

Assisted-by: Cursor Agent
Keep the existing start_mpi and close_mpi API and drive MPI_Init /
MPI_Finalize through a unique_ptr to the RAII MpiInstance.

Assisted-by: Cursor Agent
Use a name that matches MPIInitInfo and makes the RAII init/finalize
role explicit.

Assisted-by: Cursor Agent
Print the MPI_Init pointer arguments plus *argc/argv values, and log
MPI_Finalize(), before the corresponding MPI calls in MPIInitGuard.

Assisted-by: Cursor Agent
Keep macos-26 runtestpy and light_ci on the env_tests matrix from Shamrock-code#2089.

Assisted-by: Cursor Agent
Comment thread .github/workflows/main_workflow.yml Outdated
tdavidcl and others added 4 commits August 18, 2026 17:15
MacOS Homebrew venvs change pybind11 include-path strings versus the
first configure, so ninja rebuilt every pybind11-using TU during CI
pip install. Bake PYTHON_EXECUTABLE at new-env time, pass it through
setup.py, and point macOS shamconfigure at absolute acpp/libomp paths.

Assisted-by: Cursor Agent
This reverts commit 91a7a66.

Assisted-by: Cursor Agent
Comment thread .github/workflows/env_tests.yml Outdated
tdavidcl and others added 4 commits August 19, 2026 09:43
Keep runtestpy for PYTHONPATH import smoke tests. Skip pip install on
macos-26, where it currently rebuilds the whole tree.

Assisted-by: Cursor Agent
@github-actions

Copy link
Copy Markdown
Contributor

Workflow report

workflow report corresponding to commit b04e3bd
Commiter email is 37929162+mergify[bot]@users.noreply.github.com
You are using github private e-mail. This prevent proper tracing of who contributed what, please disable it (see Keep my email addresses private).

Light CI is enabled (the default for pull requests). This will only run the basic tests and not the full tests.
Full CI runs if the full-ci label is set, or automatically on Mergify merge-queue branches (mergify/merge-queue/*).
The merge gate job "on PR / all" is skipped in this case. Queue entry uses "on PR / all_light"; full CI runs in the merge queue.

Pre-commit check report

Pre-commit check: ✅

trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check for merge conflicts................................................Passed
check that executables have shebangs.....................................Passed
check that scripts with shebangs are executable..........................Passed
check for added large files..............................................Passed
check for case conflicts.................................................Passed
check for broken symlinks................................................Passed
check yaml...............................................................Passed
detect private key.......................................................Passed
No-tabs checker..........................................................Passed
Tabs remover.............................................................Passed
cmake-format.............................................................Passed
Validate GitHub Workflows................................................Passed
clang-format.............................................................Passed
ruff check...............................................................Passed
ruff format..............................................................Passed
Check doxygen headers....................................................Passed
Check license headers....................................................Passed
Check #pragma once.......................................................Passed
Check SYCL #include......................................................Passed
No ssh in git submodules remote..........................................Passed
No UTF-8 in files (except for authors)...................................Passed

Test pipeline can run.

Clang-tidy diff report


/__w/Shamrock/Shamrock/src/shambindings/src/pybindings.cpp:141:23: warning: loop variable is copied but only used as const reference; consider making it a const reference [performance-for-range-copy]
  141 |             for (auto fct : *static_init_shamrock_pybind) {
      |                       ^
      |                  const  &
/__w/Shamrock/Shamrock/src/shambindings/src/pybindings.cpp:141:18: note: FIX-IT applied suggested code changes
  141 |             for (auto fct : *static_init_shamrock_pybind) {
      |                  ^
/__w/Shamrock/Shamrock/src/shambindings/src/pybindings.cpp:141:22: note: FIX-IT applied suggested code changes
  141 |             for (auto fct : *static_init_shamrock_pybind) {
      |                      ^

73 warnings generated.
clang-tidy applied 2 of 2 suggested fixes.
Suppressed 70 warnings (70 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.

Suggested changes

Detailed changes :
diff --git a/src/shambindings/src/pybindings.cpp b/src/shambindings/src/pybindings.cpp
index 000bd0a2..02d043d3 100644
--- a/src/shambindings/src/pybindings.cpp
+++ b/src/shambindings/src/pybindings.cpp
@@ -138,7 +138,7 @@ namespace shambindings {
         }
 
         if (static_init_shamrock_pybind) {
-            for (auto fct : *static_init_shamrock_pybind) {
+            for (const auto& fct : *static_init_shamrock_pybind) {
                 fct(m);
             }
         }
# Doxygen diff with `main` Removed warnings : 30 New warnings : 30 Warnings count : 8244 → 8244 (0.0%)
Detailed changes :
- src/shampylib/include/shampylib/pyNodeInstance.hpp:27: warning: Member register_pymodules(py::module &m) (function) of namespace shamsys::instance is not documented.
+ src/shampylib/include/shampylib/pyNodeInstance.hpp:29: warning: Member register_pymodules(py::module &m) (function) of namespace shamsys::instance is not documented.
- src/shamsys/include/shamsys/NodeInstance.hpp:127: warning: Member get_compute_device() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/include/shamsys/NodeInstance.hpp:130: warning: Member get_compute_device() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:100: warning: Member device_compute (variable) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:101: warning: Member device_alt (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:101: warning: Member device_compute (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:102: warning: Member device_alt (variable) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:103: warning: Member ctx_compute (variable) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:104: warning: Member ctx_alt (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:104: warning: Member ctx_compute (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:105: warning: Member ctx_alt (variable) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:106: warning: Member sched_compute (variable) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:107: warning: Member sched_alt (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:107: warning: Member sched_compute (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:108: warning: Member sched_alt (variable) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:109: warning: Member callback_mem_perf_info() (function) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:110: warning: Member callback_mem_perf_info() (function) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:116: warning: Member init_device_scheduling() (function) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:117: warning: Member init_device_scheduling() (function) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:135: warning: Member init_queues(std::string search_key) (function) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:136: warning: Member init_queues(std::string search_key) (function) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:147: warning: Member finalize() (function) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:148: warning: Member finalize() (function) of namespace syclinit is not documented.
- src/shamsys/src/NodeInstance.cpp:163: warning: Member compute_queue_eu_count (variable) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:165: warning: Member get_compute_queue_eu_count(u32 id=0) (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:173: warning: Member compute_queue_eu_count (variable) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:175: warning: Member get_compute_queue_eu_count(u32 id=0) (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:207: warning: Member print_device_list_debug() (function) of namespace shamsys::instance::tmp is not documented.
+ src/shamsys/src/NodeInstance.cpp:217: warning: Member print_device_list_debug() (function) of namespace shamsys::instance::tmp is not documented.
- src/shamsys/src/NodeInstance.cpp:229: warning: Member start_sycl_auto(std::string search_key) (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:239: warning: Member start_sycl_auto(std::string search_key) (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:295: warning: Member init_strategy (variable) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:309: warning: Member init_strategy (variable) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:374: warning: Member get_compute_scheduler() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:376: warning: Member get_alt_scheduler() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:378: warning: Member get_compute_scheduler_ptr() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:382: warning: Member get_alt_scheduler_ptr() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:384: warning: Member print_device_info(const sycl::device &Device) (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:391: warning: Member print_device_list() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:394: warning: Member get_compute_scheduler() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:396: warning: Member get_alt_scheduler() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:397: warning: Member print_mpi_capabilities() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:398: warning: Member get_compute_scheduler_ptr() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:399: warning: Member print_mpi_comm_info() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:401: warning: Member check_dgpu_available() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:402: warning: Member get_alt_scheduler_ptr() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:404: warning: Member print_device_info(const sycl::device &Device) (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:411: warning: Member print_device_list() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:417: warning: Member print_mpi_capabilities() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:419: warning: Member print_mpi_comm_info() (function) of namespace shamsys::instance is not documented.
+ src/shamsys/src/NodeInstance.cpp:421: warning: Member check_dgpu_available() (function) of namespace shamsys::instance is not documented.
- src/shamsys/src/NodeInstance.cpp:53: warning: Member print_device_list() (function) of namespace shamsys::instance::details is not documented.
+ src/shamsys/src/NodeInstance.cpp:54: warning: Member print_device_list() (function) of namespace shamsys::instance::details is not documented.
- src/shamsys/src/NodeInstance.cpp:98: warning: Member initialized (variable) of namespace syclinit is not documented.
+ src/shamsys/src/NodeInstance.cpp:99: warning: Member initialized (variable) of namespace syclinit is not documented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants