Skip to content

Fix CUDA termination crash via deterministic worker teardown and lazy stream evaluation - #4514

Open
aleroot wants to merge 2 commits into
ml-explore:mainfrom
aleroot:threads_and_strems_4506
Open

aleroot wants to merge 2 commits into
ml-explore:mainfrom
aleroot:threads_and_strems_4506

Conversation

@aleroot

@aleroot aleroot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

As described in #4506 a termination crash occurs when a detached worker thread outlives the CUDA runtime and attempts to clean up CUDA resources, triggering a std::terminate.

To resolve this I have made some changes so that the worker class is restructured therefore the detached thread only captures a new, inert State struct containing standard threading primitives. Additionally the CUDA resources remain in the Worker, which is now managed by the CommandEncoder as a unique_ptr to ensure deterministic destruction before the runtime unloads. The destructor ~CudaHandle has been modified to safely ignore errors during shutdown, and eval_impl now selects streams lazily to prevent accidental worker instantiation.

Fixes #4506

  • ☑️ I understand it is strictly prohibited to use AI to write PR description
  • AI usage disclosure:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] threads and streams issues

1 participant