Support DeepStack (Qwen3-VL) export

[Tachion (SanjayAMD)]

Summary

Adds DeepStack support to the Qwen3-VL-4B export recipe so the exported ONNX models carry the intermediate vision features the model needs for fine-grained visual understanding (OCR, documents, diagrams, grounding).

Qwen3-VL's vision encoder emits, in addition to the merged image embedding, a small set of intermediate hidden-state feature maps ("DeepStack" features — one per entry in vision_config.deepstack_visual_indexes, i.e. 3 for the 4B model, from ViT layers [5, 11, 17]). These are scattered onto the image-token positions and injected into the decoder residual stream after the first few decoder layers (feature i → after layer i), matching Hugging Face's Qwen3VLTextModel._deepstack_process.

The previous recipe dropped these features (vision returned only the merged embedding, update_genai_config wrote no DeepStack fields), so the exported model lost accuracy. This PR wires them through the vision export, the embedding scatter, and genai_config.json.

This is the export-side counterpart to the onnxruntime-genai runtime PR that consumes these tensors; both are required for end-to-end DeepStack.

How it works

The three sub-models exchange DeepStack tensors as follows:

vision.onnx      ->  deepstack_features_{i}   (per-token, image-token length)
embedding.onnx   ->  scatters them to full (batch, seq, hidden) -> deepstack_{i}
decoder.onnx     ->  adds deepstack_{i} into the residual after layer i

During generation there are no image tokens, so embedding.onnx emits all-zeros for deepstack_{i} and the decoder Adds become no-ops.

Changes (Qwen-Qwen3-VL-4B-Instruct/builtin, 5 files)

• codes/modeling_qwen3_vl.py

  • Qwen3VLVisionModel.forward: capture the hidden states at deepstack_visual_indexes, run them through their per-index patch mergers, and return them separately as (image_features, *deepstack_feats) instead of folding them into the merged features.
  • get_image_features: propagate the tuple.
  • get_fused_input_embeddings: accept deepstack_features_0/1/2, masked_scatter each into full-length (batch, seq, hidden) zero tensors at image-token positions, and return (inputs_embeds, deepstack_0, deepstack_1, deepstack_2).

• user_script.py

  • get_vision_io_config: add deepstack_features_0/1/2 to vision output names.
  • get_embedding_io_config: add the DeepStack inputs/outputs and dynamic axes.
  • get_embedding_dummy_inputs: add matching dummy DeepStack tensors for tracing.
  • _load_base_model: accept a local model directory (not only an HF hub id).

• cpu_and_mobile/vision.json, cuda/vision.json

  • RenameOutputDims for the 3 new vision outputs (dim-0 → num_logical_patches).

• optimize.py

  • update_genai_config: emit the DeepStack tensor-name arrays — vision.outputs.deepstack_features, embedding.inputs.deepstack_features, embedding.outputs.deepstack, and decoder.inputs.deepstack.

Validation

• Python byte-compiles; both vision.json files parse.
• Functional test: built the vision tower from the real HF config (random init) and confirmed it now returns image_features + one feature per deepstack_visual_index (3), all with matching shape; and that the embedding scatter fills only image-token positions (all other positions remain zero, so the decoder Adds are no-ops during generation).
• The runtime side has been validated end to end on an equivalently-exported Qwen3-VL-4B model (AI2D 81% ONNX INT4 vs 83% PyTorch FP32, within noise).

Scope / notes

• Change is confined to the Qwen3-VL-4B recipe; no shared code is touched.
• A full optimize.py export run was not included here (requires the ~8GB base checkpoint); the model changes are covered by the functional test and by the previously-exported, runtime-validated model produced by the equivalent code.

[Copilot]

Pull request overview

Adds DeepStack (Qwen3-VL) feature export support for the Qwen3-VL-4B recipe so intermediate vision hidden states are preserved through vision.onnx → embedding.onnx → decoder.onnx and surfaced in genai_config.json for onnxruntime-genai consumption.

Changes:

• Extend vision export to output deepstack_features_{0,1,2} alongside image_features.
• Update embedding export to accept DeepStack per-token features and scatter them into full-length (batch, seq, hidden) tensors (deepstack_{0,1,2}).
• Emit DeepStack tensor name arrays in genai_config.json and rename output dims in Olive vision configs.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
Qwen-Qwen3-VL-4B-Instruct/builtin/codes/modeling_qwen3_vl.py	Captures DeepStack vision features and returns/scatters them through the model’s embedding path.
Qwen-Qwen3-VL-4B-Instruct/builtin/user_script.py	Updates IO configs and dummy inputs for DeepStack tensors; adds local model dir support.
Qwen-Qwen3-VL-4B-Instruct/builtin/optimize.py	Writes DeepStack input/output tensor name arrays into genai_config.json.
Qwen-Qwen3-VL-4B-Instruct/builtin/cuda/vision.json	Renames dim-0 for added DeepStack vision outputs to num_logical_patches.
Qwen-Qwen3-VL-4B-Instruct/builtin/cpu_and_mobile/vision.json	Renames dim-0 for added DeepStack vision outputs to num_logical_patches.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.