Converting from MLX to ONNX

[mitkox]

Are there plans for MLX to ONNX conversion?

[awni]

Great question, it's an open feature request #290. I don't have a timeline for that yet, but we are interested in supporting exporting to ONNX.

[l0d0v1c]

I tried to quantize (mlx_lm. convert), the use optimum. Optimum works on the finetune model by
optimum-cli export onnx --model ./dq4 --task text-generation ./model-onnx
but not on the quantized model

Traceback (most recent call last):
  File "/Users/pro/mambaforge/envs/torch/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/optimum/commands/optimum_cli.py", line 208, in main
    service.run()
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/optimum/commands/export/onnx.py", line 265, in run
    main_export(
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/optimum/exporters/onnx/__main__.py", line 293, in main_export
    model = TasksManager.get_model_from_task(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/optimum/exporters/tasks.py", line 1988, in get_model_from_task
    model = model_class.from_pretrained(model_name_or_path, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3838, in from_pretrained
    ) = cls._load_pretrained_model(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4349, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for Phi3ForCausalLM:

I try to find a solution. It is now interesting because webGPU works fine with Phi-3