Description
mx.distributed.init(backend="jaccl") segfaults immediately (signal 11) during
2-node Mac↔Mac initialization, even when both hosts have working RDMA devices
(ibv_devices enumerates rdma_en2) and Thunderbolt 40 Gb/s links confirmed.
The segfault is 100% reproducible on fresh builds.
This differs from #3777 (closed by #4180) which covered the no-verbs-device
case. In our case, RDMA devices ARE present — the crash happens deeper in the
initialization path.
The same 2-node setup works fine with MlxRing (socket-based backend).
Environment
- Hardware: 2× Apple Silicon Macs, 16 GB RAM, connected by direct Thunderbolt cable (40 Gb/s confirmed via
system_profiler SPThunderboltDataType)
- macOS: 26.6.2 (latest)
- MLX:
0.32.0.dev20260806+cc3f3e60 (from rltakashige/mlx-jaccl-fix-small-recv fork, branch address-rdma-gpu-locks, commit cc3f3e60)
- Both hosts verified:
rdma_ctl status → enabledibv_devices → rdma_en2 present (real verbs devices)- Thunderbolt link: active, 40 Gb/s
Steps to Reproduce
- Connect two Apple Silicon Macs via Thunderbolt cable
- Enable RDMA on both hosts (
rdma_ctl enable, cold boot if needed)
- Verify
ibv_devices shows rdma_en2 on both hosts
- Run
mx.distributed.init(backend="jaccl") with both hosts in the group
Expected Behavior
JACCL initialization completes, allowing distributed operations over Thunderbolt RDMA.
Actual Behavior
Segfault immediately during initialization:
Fatal Python error: Segmentation fault
signal=11 (SIGSEGV)
Stack trace points into libjaccl.dylib — the shared-buffer / protection-domain
registration path during mx.distributed.init(backend="jaccl").
Isolation Evidence
| Test |
Result |
2-node MlxJaccl (RDMA), min_nodes: 2 |
Segfault signal 11 on both ranks |
2-node MlxRing (socket), min_nodes: 2 |
Works — round-trip inference confirmed |
| Single-node on each Mac |
Works |
The crash is isolated to JACCL initialization over RDMA. The Ring backend
uses the same topology and works. This isolates the bug to the JACCL/RDMA
initialization path specifically.
Related
Description
mx.distributed.init(backend="jaccl")segfaults immediately (signal 11) during2-node Mac↔Mac initialization, even when both hosts have working RDMA devices
(
ibv_devicesenumeratesrdma_en2) and Thunderbolt 40 Gb/s links confirmed.The segfault is 100% reproducible on fresh builds.
This differs from #3777 (closed by #4180) which covered the no-verbs-device
case. In our case, RDMA devices ARE present — the crash happens deeper in the
initialization path.
The same 2-node setup works fine with
MlxRing(socket-based backend).Environment
system_profiler SPThunderboltDataType)0.32.0.dev20260806+cc3f3e60(fromrltakashige/mlx-jaccl-fix-small-recvfork, branchaddress-rdma-gpu-locks, commitcc3f3e60)rdma_ctl status→enabledibv_devices→rdma_en2present (real verbs devices)Steps to Reproduce
rdma_ctl enable, cold boot if needed)ibv_devicesshowsrdma_en2on both hostsmx.distributed.init(backend="jaccl")with both hosts in the groupExpected Behavior
JACCL initialization completes, allowing distributed operations over Thunderbolt RDMA.
Actual Behavior
Segfault immediately during initialization:
Stack trace points into
libjaccl.dylib— the shared-buffer / protection-domainregistration path during
mx.distributed.init(backend="jaccl").Isolation Evidence
MlxJaccl(RDMA),min_nodes: 2MlxRing(socket),min_nodes: 2The crash is isolated to JACCL initialization over RDMA. The Ring backend
uses the same topology and works. This isolates the bug to the JACCL/RDMA
initialization path specifically.
Related