Add Mistral4

[Goekdeniz-Guelmez]

No description provided.

[Goekdeniz-Guelmez]

So inference with a random tiny mistral4 model works.

python -m mlx_lm.generate --model onnx-internal-testing/tiny-random-Mistral4ForCausalLM --prompt "Hello" -m 12

<frozen runpy>:128: RuntimeWarning: 'mlx_lm.generate' found in sys.modules after import of package 'mlx_lm', but prior to execution of 'mlx_lm.generate'; this may result in unpredictable behaviour
Calling `python -m mlx_lm.generate...` directly is deprecated. Use `mlx_lm.generate...` or `python -m mlx_lm generate ...` instead.
Fetching 5 files: 100%|█████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 5917.47it/s]
Download complete: : 0.00B [00:00, ?B/s]                                                                      | 0/5 [00:00<?, ?it/s]
You are using a model of type `mistral4` to instantiate a model of type ``. This may be expected if you are loading a checkpoint that shares a subset of the architecture (e.g., loading a `sam2_video` checkpoint into `Sam2Model`), but is otherwise not supported and can yield errors. Please verify that the checkpoint is compatible with the model you are instantiating.
==========
projectsRemoteMYhistoire Committee posit notice cannotfortun déf traits optical
==========
Prompt: 2 tokens, 146.371 tokens-per-sec
Generation: 12 tokens, 1141.987 tokens-per-sec
Peak memory: 0.018 GB

I don't have the hardware to test out the real model, @angeloskath would you mind trying inference with the real model?

[Goekdeniz-Guelmez]

training works too

[angeloskath]

So, you can test whether the loading works by manually loading with lazy=True.

I would also propose that it is better to implement it the same way as Qwen3_5 so the text model is there under language_model instead of changing all keys.

The main issue is that the model type is mistral 3 according to the config so perhaps any incompatibility should be merged in mistral 3 instead of a new type of model.

[Goekdeniz-Guelmez]

even with lazy load it doesnt work:

<frozen runpy>:128: RuntimeWarning: 'mlx_lm.convert' found in sys.modules after import of package 'mlx_lm', but prior to execution of 'mlx_lm.convert'; this may result in unpredictable behaviour
Calling `python -m mlx_lm.convert ...` directly is deprecated. Use `mlx_lm.convert ...` or `python -m mlx_lm convert ...` instead.
[INFO] Loading
Fetching 12 files: 100%|███████████████████████████████████| 12/12 [00:00<00:00, 19901.80it/s]
Download complete: : 0.00B [00:00, ?B/s]                               | 0/12 [00:00<?, ?it/s]
You are using a model of type mistral3 to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
[INFO] Using dtype: bfloat16
[INFO] Quantizing
[INFO] Quantized model with 2.502 bits per weight.
libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Caused GPU Timeout Error (00000002:kIOGPUCommandBufferCallbackErrorTimeout)
[1]    37709 abort      python -m mlx_lm.convert --hf-path darkc0de/Mistral-Small-4-119B-2603-heretic
/opt/miniconda3/envs/mlx-lm-dev/lib/python3.12/multiprocessing/resource_tracker.py:279: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

what I ment is that I have a 24GB RAM Mac :D

[Goekdeniz-Guelmez]

the full model gets loaded just not quantized. @angeloskath would you mind running some inference and training?