Skip to content

Construct orcarouter Flash-Next Uncensored-MLX via qwen4_exp shim - #1

Open
okwithit9-debug wants to merge 5 commits into
mainfrom
cursor/qwen38-flash-next-27b-ca05
Open

okwithit9-debug wants to merge 5 commits into
mainfrom
cursor/qwen38-flash-next-27b-ca05

Conversation

@okwithit9-debug

@okwithit9-debug okwithit9-debug commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Motivation

The required Flash-Next line is orcarouter only:

  • EXO MLX: orcarouter/Qwen3.8-Flash-Next-Uncensored-MLX
  • Spark vLLM companion (same family, not an EXO MLX card): orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4

Do not promote mlx-community generic Flash-Next, lychee888, or other quants as the default. Uncensored is a weight pack (abliteration), not a new training pipeline. EXO needs a working qwen4_exp / Qwen4Exp* construct path so orcarouter Uncensored-MLX can load without waiting solely on mlx-lm#1788.

Changes

Why It Works

orcarouter Uncensored-MLX is an mlx-vlm conversion (Qwen4ExpForConditionalGeneration / qwen4_exp) with no model_file. The EXO pin of mlx_lm already accepts get_model_classes; the shim supplies Model / ModelArgs that match load_model (from_dict → construct → sanitizeload_weights). Native mlx_lm wins when exo-explore#1788 ships; the vendor module is marked for deletion then.

Honest remaining gap: this unblocks construct/load, not a verified full MoE / hybrid GDN+QSA / PLE n-gram generate pass, and not typed Mac+Spark auto_parallel TP/PP. Those still need Apple-silicon (or CUDA MLX) runtime, the gated ~163 GiB pack, and Qwen4Exp layer handlers.

Test Plan

Manual Testing

  • pip install 'mlx-vlm>=0.6.17', accept the HuggingFace gate, set a Hub token, download on Mac, launch orcarouter/Qwen3.8-Flash-Next-Uncensored-MLX.
  • Confirm /models shows that id (not mlx-community Flash-Next). NVFP4 is Spark vLLM only.
  • After construct, treat generate / disagg as still unverified until run on Metal with the real pack.

Automated Testing

  • Registration tests: orcarouter card exists; mlx-community Flash-Next / NVFP4 cards do not; 27B qwen3_5 still registers.
  • Load-hook / shim tests (offline, no weight download): native mlx_lm wins; mlx-vlm fallback; documented error when shim missing; install does not overwrite native; adapter construct + logits + vision-key sanitize; EOS ids.
Open in Web Open in Cursor 

cursoragent and others added 3 commits September 15, 2026 08:45
Recognize official Qwen3.8-27B MLX packs (same Qwen3_5 architecture as
3.5/3.6) as first-class cards, allowlist Qwen4Exp* for Flash-Next, and
forward trust_remote_code / get_model_classes into mlx_lm with a
documented blocker until mlx-lm#1788 vendors qwen4_exp.

Co-authored-by: okwithit9-debug <okwithit9-debug@users.noreply.github.com>
Inject get_classes/load_model callables so registration and hook tests
stay offline, and tighten types for basedpyright.

Co-authored-by: okwithit9-debug <okwithit9-debug@users.noreply.github.com>
Co-authored-by: okwithit9-debug <okwithit9-debug@users.noreply.github.com>
@okwithit9-debug
okwithit9-debug marked this pull request as ready for review September 15, 2026 08:52
Drop the mlx-community Flash-Next card so EXO does not promote alternate
forks. Keep the qwen4_exp load path and point docs at the NVFP4 Spark
vLLM companion without adding it as an MLX card.

Co-authored-by: okwithit9-debug <okwithit9-debug@users.noreply.github.com>
@cursor cursor Bot changed the title Add Qwen3.8-27B catalog support and Flash-Next qwen4_exp hooks Load orcarouter Qwen3.8 Flash-Next Uncensored-MLX via qwen4_exp Sep 15, 2026
Stock mlx_lm still lacks qwen4_exp (ml-explore/mlx-lm#1788). Route
load_mlx_lm_model through a temporary mlx-vlm>=0.6.17 adapter so the
orcarouter Uncensored-MLX pack can be constructed, and document how to
remove the shim when native mlx_lm classes ship.

Co-authored-by: okwithit9-debug <okwithit9-debug@users.noreply.github.com>
@cursor cursor Bot changed the title Load orcarouter Qwen3.8 Flash-Next Uncensored-MLX via qwen4_exp Construct orcarouter Flash-Next Uncensored-MLX via qwen4_exp shim Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants