Skip to content

Default to trust_remote_code=False, fall back only when required (addresses #293) - #306

Open
Hotragn wants to merge 1 commit into
lyogavin:mainfrom
Hotragn:fix/trust-remote-code-default-293
Open

Hotragn wants to merge 1 commit into
lyogavin:mainfrom
Hotragn:fix/trust-remote-code-default-293

Conversation

@Hotragn

@Hotragn Hotragn commented Jul 20, 2026

Copy link
Copy Markdown

What

Stop hard-coding trust_remote_code=True at the config and tokenizer load boundaries. Instead prefer transformers' native implementation and only fall back to the repo's remote code when it's actually required.

Addresses #293. (Left as "addresses" rather than "fixes" — see Scope below.)

Why

AirLLMBaseModel already loads the model config/weights this way — it tries trust_remote_code=False and only falls back to True when transformers can't parse the model natively (base.py:113–120, 185/194). But two other load boundaries still passed trust_remote_code=True unconditionally:

  • AutoModel.get_module_class() — the config load, which runs first, before the model is even constructed (auto_model.py).
  • AirLLMBaseModel.get_tokenizer() and the MLX path's config + tokenizer loads.

Since AirLLM's advertised entry point is an arbitrary Hugging Face repo id (AutoModel.from_pretrained("some/repo")), those unconditional Trues mean a normal load of a standard architecture (Llama, Qwen3, Mistral, …) would still execute Python shipped in the repo, with no way for the caller to stay on transformers' default remote-code boundary.

This PR makes every load boundary consistent with the pattern the model loading already uses.

Changes

  • Add load_prefer_no_remote_code(loader, *args, **kwargs) in utils.py: try the loader with trust_remote_code=False, fall back to trust_remote_code=True only if the native load raises.
  • Use it for the config load in AutoModel.get_module_class, the tokenizer load in AirLLMBaseModel, and the config + tokenizer loads in AirLLMLlamaMlx.
  • Add offline unit tests.

Behavior: standard models no longer trigger remote-code execution during config/tokenizer loading. Models that genuinely ship custom code (ChatGLM, Baichuan, some Qwen) still work because the fallback re-loads with trust_remote_code=True. No API change, no new required arguments.

Scope / what this deliberately does not do

This mirrors your existing auto-fallback approach rather than adding a new public trust_remote_code= parameter. The auto-fallback is a strict improvement (the common case stops running remote code) but it does not give a hard opt-out: a repo that genuinely requires remote code will still run it via the fallback. A follow-up could add an explicit trust_remote_code: Optional[bool] = None kwarg (None = current auto behavior, False = hard refuse, True = force) for callers who want a guarantee. I left that out here to keep the change minimal and consistent with the current design — happy to add it if you'd prefer. That's also why this says "Addresses" rather than "Fixes" #293, so the issue stays open for that decision.

Testing

python -m pytest air_llm/tests/test_trust_remote_code.py — 5 offline tests (no network, no GPU), all pass:

  • helper uses trust_remote_code=False and does not retry when the native load succeeds
  • helper falls back to True when the False load raises
  • helper forwards extra kwargs (e.g. token)
  • get_module_class loads a standard arch's config with trust_remote_code=False only
  • get_module_class falls back to True for a custom arch (e.g. ChatGLM) and still resolves the right class

Note: the MLX path is changed identically but I couldn't run it locally (no Apple silicon) — it's a mechanical mirror of the generic path using the same tested helper.

@Hotragn

Hotragn commented Jul 29, 2026

Copy link
Copy Markdown
Author

Rebased onto v3.1.0 — the only conflict was the .utils import line in airllm_base.py (your new layer_tensor_names / load_layer_subset imports alongside this PR's load_prefer_no_remote_code), now resolved to keep both. Tests still pass (5/5 offline).

Still relevant on v3.1.0: AutoModel.get_module_class and AirLLMBaseModel.get_tokenizer both still pass trust_remote_code=True unconditionally, while the config/model load right below already prefers False and falls back. This just makes those two boundaries consistent with that.

AutoModel.get_module_class (config load) and the base/MLX tokenizer loads
hard-coded trust_remote_code=True, so a normal AirLLM load of an arbitrary
Hugging Face repo id would execute Python shipped in that repo -- even for
standard architectures that don't need it. AirLLMBaseModel already loads the
model config/weights with trust_remote_code=False and only falls back to True
when transformers can't parse the model natively; this makes every other load
boundary consistent with that.

- Add load_prefer_no_remote_code() helper: try trust_remote_code=False, fall
  back to True only if the native load raises.
- Use it for the config load in AutoModel.get_module_class, the tokenizer load
  in AirLLMBaseModel, and the config + tokenizer loads in the MLX path.
- Standard models never trigger remote code now; models that genuinely need it
  (ChatGLM, Baichuan, some Qwen) still work via the fallback. No API change.
- Add offline unit tests for the helper and the get_module_class behavior.

Addresses lyogavin#293.
@Hotragn
Hotragn force-pushed the fix/trust-remote-code-default-293 branch from 6553cf5 to 697aba6 Compare September 5, 2026 17:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant