This is the error I get
built empty Qwen3ForCausalLM via AutoModelForCausalLM (attn=sdpa)
Traceback (most recent call last):
File "c:\Users\acarr\Sviluppo\AirLLM\Chat.py", line 5, in
model = AutoModel.from_pretrained("Qwen/Qwen3-32B")
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\auto_model.py", line 63, in from_pretrained
return class_(pretrained_model_name_or_path, *inputs, **kwargs)
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\airllm_base.py", line 186, in init
self.init_model()
~~~~~~~~~~~~~~~^^
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\airllm_base.py", line 362, in init_model
set_module_tensor_to_device(self.model, buffer_name, self.running_device, value=buffer)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\accelerate\utils\modeling.py", line 353, in set_module_tensor_to_device
new_value = value.to(device, non_blocking=non_blocking)
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\torch\cuda_init_.py", line 522, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
This is the error I get
built empty Qwen3ForCausalLM via AutoModelForCausalLM (attn=sdpa)
Traceback (most recent call last):
File "c:\Users\acarr\Sviluppo\AirLLM\Chat.py", line 5, in
model = AutoModel.from_pretrained("Qwen/Qwen3-32B")
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\auto_model.py", line 63, in from_pretrained
return class_(pretrained_model_name_or_path, *inputs, **kwargs)
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\airllm_base.py", line 186, in init
self.init_model()
~~~~~~~~~~~~~~~^^
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\airllm\airllm_base.py", line 362, in init_model
set_module_tensor_to_device(self.model, buffer_name, self.running_device, value=buffer)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\accelerate\utils\modeling.py", line 353, in set_module_tensor_to_device
new_value = value.to(device, non_blocking=non_blocking)
File "C:\Users\acarr\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\torch\cuda_init_.py", line 522, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled