Skip to content

Error at the fist attempt #355

Description

@AlbertoCarrai

This is the error I get

built empty Qwen3ForCausalLM via AutoModelForCausalLM (attn=sdpa)
Traceback (most recent call last):
File "c:\Users\acarr\Sviluppo\AirLLM\Chat.py", line 5, in
model = AutoModel.from_pretrained("Qwen/Qwen3-32B")
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\auto_model.py", line 63, in from_pretrained
return class_(pretrained_model_name_or_path, *inputs, **kwargs)
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\airllm_base.py", line 186, in init
self.init_model()
~~~~~~~~~~~~~~~^^
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\airllm_base.py", line 362, in init_model
set_module_tensor_to_device(self.model, buffer_name, self.running_device, value=buffer)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\accelerate\utils\modeling.py", line 353, in set_module_tensor_to_device
new_value = value.to(device, non_blocking=non_blocking)
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\torch\cuda_init_.py", line 522, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions