nativ cli - #228
nativ cli#228Lazarus-931 wants to merge 10 commits into
Conversation
The resulting api arg should be:
Where the tasks are the same as in mlx-audio. |
|
Regarding the agent, thanks for sharing the results, it does indeed provide great time savings! Could you share the inputs and traces as well? |
|
Reran as a clean A/B — the difference is baked into the binary, not the prompt. ReproductionTwo builds of the same commit; in one, the
So the only variable is whether Results
With n=1 per side (agent behavior is stochastic), but the direction is consistent across runs: a self-documenting |
|
The tests fail Also, have you updated to wrap the mlx-vlm CLI tools as we spoke or this is more of temporary solution till we build it in mlx-vlm? |
this is a temp solution, but i think i can push upstream pr so i'll do that before changing this to full supported wrap |
A standalone Swift package (cli/) that drives the local mlx-vlm server over its OpenAI-compatible API from the terminal: run/chat/serve/status/stop, models list/use/pull/rm, embed, image, transcribe. API-first and decoupled from the app internals via a small cli.json handshake; each command is one file in a registry. Builds with swift build. See cli/README.md.
… richer list/chat/run, tests Fold in the improvement pass: embedding/image/stt models resolved separately (env + cli.json, falling back to the chat model); a config command to read/write cli.json standalone; serve -d waits until the server answers; models list merges loaded with locally-cached (search path + HF hub cache); run --json for scripting; chat slash commands (/model, /system, /reset, /help) + multiline; an agent command that prints a full reference (markdown or JSON); a shared URLSession with an idle timeout; SSE + multipart extracted as unit-tested pure helpers.
/v1/audio/speech-backed `nativ speak`; ttsModel / NATIV_TTS_MODEL config with fallback to the chat model; config set --tts-model; agent + README + test.
Idle timeout resets per chunk, so it only fires on a silent server; large models with long prompts can take minutes to first token, so 120s was too tight.
TTS is one kind of audio-out; group it under `nativ audio` so other tasks (separation, enhancement, sfx) can land as thin reflections of future server endpoints. ttsModel config unchanged; agent reference + README updated.
…back) run reuses the running server if up, else cold-loads via the bundled engine (python3 -m mlx_vlm.generate --no-verbose), so it works with the app closed. EngineProcess locates the bundled python3 (NATIV_ENGINE_BIN override), sharing the app-bundle lookup with ServerProcess. --json still needs the server.
Per review: fold transcribe (stt) and speak (tts) into one `nativ audio` command driven by --task, matching mlx-audio's task taxonomy. tts and stt route to the server; sts/vad/lid are accepted names that error until the server exposes them. Removes the standalone transcribe command and the audio speak subcommand.
f27a735 to
7034637
Compare
mlx-vlm's server-aware generate treats --base-url as opt-in and errors if the URL is unreachable, so passing it unconditionally broke local fallback when the server is down. Gate it on isUp(): reuse the server when up, otherwise invoke the engine without --base-url so it loads the model locally.
2 participants
Commands
run— one-shot completion (streams;--system,--image,--json, stdin)chat— interactive REPL (/model,/system,/reset,"""multiline)serve/stop/status— manage the bundled server (serve -dwaits until ready)models list | use | pull | rm— list loaded + locally-cached models, set default, download, removeembed— text embeddingsimage— image generationtranscribe— speech-to-textaudio speak— text-to-speech (theaudiogroup grows into separation/enhancement/sfx as the engine exposes those endpoints)config show | set | path— the app writescli.json; this configures the CLI standaloneagent— a full reference for coding agents (markdown or--json)Design
cli.json→ default.cli.json, so Nativ internals can change without breaking the CLI, and the same binary works against a local or remote server.Direction (follow-ups)
Make the inference commands server-aware with a local fallback — reuse the running server when present, otherwise cold-load via the bundled engine CLI — so the CLI works whether or not the app is open. Offline tooling (
convert) will shell straight to the engine rather than being reimplemented.Note on the
agentcommand: it exists so a coding agent can learn the whole CLI in one cheap call instead of probing--helpor the repo. Quick measurement — two fresh agents asked to produce a full usage guide, one told aboutnativ agent, one left to figure it out:agentagent≈2× fewer tokens, one call instead of nine, and it avoided a mistaken finding. (n=1 per side — a broader multi-model run is planned.)