MLX Support

[xSamiHK]

Great work Liquid AI team, I tried to run it locally, I guess its internally working with CUDA libraries which makes it tough to run on Macbook Pro M Chip machines.

tl:dr;
If there a way to run the audio models on MacOS?

[Prakhar-Bhartiya]

Hi @xSamiHK, it does work on Apple Silicon, just pass device="mps" in appropriate place for example here is how you will run ASR script. You will get some warning, but it should run.

import torch
import torchaudio
from liquid_audio import LFM2AudioModel, LFM2AudioProcessor, ChatState, LFMModality

# Load models
HF_REPO = "LiquidAI/LFM2-Audio-1.5B"

processor = LFM2AudioProcessor.from_pretrained(HF_REPO, device="mps").eval()
model = LFM2AudioModel.from_pretrained(HF_REPO, device="mps").eval()

# Set up inputs for the model
chat = ChatState(processor)

chat.new_turn("system")
chat.add_text("Perform ASR.")
chat.end_turn()

chat.new_turn("user")
wav, sampling_rate = torchaudio.load("assets/asr.wav")
chat.add_audio(wav, sampling_rate)
chat.end_turn()

chat.new_turn("assistant")

# Generate text
for t in model.generate_sequential(**chat, max_new_tokens=512):
    if t.numel() == 1:
        print(processor.text.decode(t), end="", flush=True)

# Output: The stale smell of old beer lingers. It takes heat to bring out the odor. A cold dip restores health and zest. A salt pickle tastes fine with ham. Tacos al pastor are my favorite. A zestful food is the hot cross bun.

Result

(liquid-audio) prakharbhartiya@Mac-1862 liquid-audio-main % python3 src/liquid_audio/demo/asr_demo.py
Fetching 10 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 16474.09it/s]
W1023 18:50:44.364000 6136 torch/distributed/elastic/multiprocessing/redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs.
/Users/prakharbhartiya/Downloads/liquid-audio-main/.venv/lib/python3.12/site-packages/torchaudio/_backend/utils.py:213: UserWarning: In 2.9, this function's implementation will be changed to use torchaudio.load_with_torchcodec` under the hood. Some parameters like ``normalize``, ``format``, ``buffer_size``, and ``backend`` will be ignored. We recommend that you port your code to rely directly on TorchCodec's decoder instead: https://docs.pytorch.org/torchcodec/stable/generated/torchcodec.decoders.AudioDecoder.html#torchcodec.decoders.AudioDecoder.
  warnings.warn(
The stale smell of old beer lingers. It takes heat to bring out the odor. A cold dip restores health and zest. A salt pickle tastes fine with ham. Tacos al pastor are my favorite. A zestful food is the hot cross bun.<|im_end|>%   

[xSamiHK]

Great, thanks for snippets.
Let me try this.