Add mxbai reranking recipes and exact FP16 boundary exclusions

[ssss141414]

Summary

This contribution adds canonical CPU fp32/fp16 reranking support for mixedbread-ai/mxbai-rerank-base-v1, an English cross-encoder that scores query-document pairs with one raw relevance logit. The re-issued charter shipped at Effort L1 with planned Outcome L1: two task-canonical recipes plus a shared, opt-in exact-node FP16 conversion fix. Tester validation reached the full Goal ceiling, L3 PASS, on final candidate ec130eaaf71bce478ca0792afc0c19e9c1ffd614 with full coverage and no deferred tuples.

Depends on #1322. Closes #1353.

Model metadata

What the model does

An English cross-encoder reranker that jointly tokenizes one query-document pair and emits one raw relevance logit used to order candidate documents for search (verified). The pinned model card demonstrates CrossEncoder.rank(query, documents), Hub metadata declares pipeline_tag=text-ranking, and the pinned config declares DebertaV2ForSequenceClassification with num_labels=1.

Primary user stories

• A user supplies a search query and candidate documents to obtain relevance scores and a reordered candidate list for retrieval or RAG (verified).

Supported tasks

• reranking on checkpoint, Transformers, and WinML surfaces (verified): pinned Hub metadata reports text-ranking, the one-label Transformers head emits one logit, and dependency #1322 maps text-ranking to canonical reranking and provides the evaluator surface.
• text-classification on Transformers, Optimum ONNX, and WinML surfaces (verified): current-main inspection resolves the stock sequence-classification head and the Optimum vendor registry covers DeBERTa-v2 text classification.

Model architecture

DebertaV2ForSequenceClassification
|-- DebertaV2Model
|   |-- Token embeddings (vocab 128100 -> hidden 768; no token-type embeddings)
|   `-- Encoder layer x 12
|       |-- Disentangled self-attention (12 heads; c2p + p2c relative bias)
|       |-- Feed-forward (768 -> 3072 -> 768, GELU)
|       `-- Residual + LayerNorm
|-- ContextPooler (first token, 768 -> 768, GELU)
`-- Relevance head (768 -> 1 raw logit)

• Source/confidence: pinned checkpoint model card, pinned config, and DebertaV2ForSequenceClassification in Transformers 5.14.1 (verified).

Validation and support evidence

Baseline

• Pinned main: 774dd6e0811f6947eaf2bd8fb721d5f6e718b0c0. The baseline used the WinML CLI 0.3.0 source tree; the source-only runtime reported 0.0.1.dev0 because an editable install was skipped.
• Moved-main REUSE attestation: canonical main is now e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0; the measured baseline evidence remains honestly anchored to 774dd6e0811f6947eaf2bd8fb721d5f6e718b0c0. The complete old-to-current-main diff is one modified file, src/winml/modelkit/models/hf/vision_encoder_decoder.py. Its composite VisionEncoderDecoder-only change has no import, registration, dependency, recipe, workflow, or runtime reach into this DeBERTa-v2 encoder-only reranking contribution, shared quantization/FP16, CPU execution, parity, Eval, Analyze, or quality gates. Planner therefore sealed REUSE with an empty invalidation closure: config, build, perf, parity, eval, analyze, and quality-gate stages were reused and no stage was rerun. Virtual merge comparison proves candidate patch equivalence: all 11 candidate-owned source, recipe, and test blobs are unchanged, with no overlap with the moved-main file; the candidate remains exact SHA ec130eaaf71bce478ca0792afc0c19e9c1ffd614 on its authorized dependency base and no rebase or candidate change is required.
• Inspection passed and resolved text-classification with DebertaV2OnnxConfig. Auto-config passed with opset 17, batch 1, sequence length 512, input_ids:int32[1,512], attention_mask:int32[1,512], logits, and transformers_attention=eager.
• The fp32 baseline build passed in 90.6 s: 804 ONNX nodes, co-located external data of 627425280 bytes, and a loadable graph.
• CPU perf passed over 10 iterations after 2 warmups: mean 448.001 ms, p50 450.468 ms, p90 471.502 ms, throughput 2.23 samples/s, RSS total delta 326.29 MB, and RSS checkpoint peak 1187.61 MB.
• The eval floor remained EXPECTED_UNSUPPORTED_TASK: current main resolved requested reranking and rejected it because reranking was absent from supported evaluators. Source-only analyze exited 2 with EXACT-ENVIRONMENT-INCOMPATIBILITY because packaged rule parquet files were unavailable; component and ONNX evidence were captured independently.
• The Optimum probe passed as VENDOR-ONLY for DeBERTa-v2. Vendor tasks were feature extraction, fill mask, multiple choice, question answering, text classification, and token classification; WinML added no tasks, and alias deberta_v2 was unregistered.

Goal

• Original charter: Effort L0, Goal L3, planned Outcome L0 for cpu/cpu/fp32 and cpu/cpu/fp16.
• Re-issued charter: Effort L1, Goal L3, planned Outcome L1. Recipe-only repair was exhausted when the fp16 graph could not load; the bounded shared FP16 config/pass/converter capability and focused regressions required an L1 code fix. The Goal ceiling was preserved, not downgraded.
• Goal success definition: a fresh CPU fp32/fp16 L0-L3 march at the exact final SHA, including loadable builds, measured perf, raw-logit/ranking agreement, and a bounded final-SHA fp32 CPU reranking functional smoke.

Outcome

• Final shipped tier: L3.
• Highest Goal verdict: L3 PASS.
• Coverage: full.
• Deferred tuples: none ([]).
• Shipped recipes: reranking_fp32_config.json and reranking_fp16_config.json.
• Shipped shared code: quant/config.py, quant/fp16.py, and quant/passes/fp16.py, with focused config, converter, pass, and recipe regressions.
• Learner findings appended: deberta-003, deberta-004, deberta-005, deberta-006, deberta-007, deberta-008, and deberta-009; existing deberta-001 and deberta-002 were preserved. No skill/meta finding or Lane A PR was required.
• Learner methodology declaration: taxonomy result NO_STEP_4B_TRIGGER_1_THROUGH_8; local_friction_observed=true. Reason: "The run had overt PowerShell encoding, dependency hydration, artifact recovery, validator, and immutable-root harness friction, but none was a qualifying CLI-flag discovery, doc-code drift, successful-but-wrong output, genuinely missing verdict vocabulary, reviewer-identified contract gap, effort mis-estimate, PR-mined behavior, or Goal under-reach. The provisional ENVIRONMENT-BLOCKED-BEFORE-L0 row was already representable as HOST-BLOCKED and was not institutionalized as a new verdict."

Per-EP/device/precision results and Functional smoke Eval

Tier	EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM delta	VRAM delta
L0	CPUExecutionProvider / cpu	fp32	PASS	-	-	-	-	-
L0	CPUExecutionProvider / cpu	fp16	PASS	-	-	-	-	-
L1	CPUExecutionProvider / cpu	fp32	PASS	447.942 ms	449.866 ms	2.23 samples/s	326.39 MB	0.0 MB
L1	CPUExecutionProvider / cpu	fp16	PASS	476.833 ms	471.81 ms	2.1 samples/s	274.21 MB	0.0 MB

• L0 PASS: fresh override-free CPU fp32/fp16 builds passed checker, strict inference, CPU ORT load, external payload checks, and exact FP16-boundary checks. fp32 has 852 nodes, 627425280 payload bytes, and initializers FLOAT16=5, FLOAT=223, INT64=16. fp16 has 926 nodes, 437262336 payload bytes, initializers FLOAT16=228, INT64=16, and all 74 frozen boundaries.
• L1 PASS: fresh five-iteration CPU metrics are shown above.
• L2 PASS: named inputs were input_ids and attention_mask. fp32: cosine 0.9999999999999999, max absolute 0.0, mean absolute 0.0, PyTorch order [0,1], ONNX order [0,1], ranking agreement true. fp16: cosine 0.9999999153288317, max absolute 0.00390625, mean absolute 0.001953125, PyTorch order [0,1], ONNX order [0,1], ranking agreement true.
• L3 PASS: the bounded functional smoke below completed.

Functional smoke Eval: final candidate ec130eaaf71bce478ca0792afc0c19e9c1ffd614, CPUExecutionProvider / cpu, fp32 only. The tester-owned deterministic MS MARCO-style query/document group used split synthetic-bounded-functional-smoke: one English query, one relevant document, and one irrelevant document. It selected and processed 1 sample / query group and processed exactly 2 query-document pairs with seed 0. Fan-out caps were 2 candidate documents, 1 beam, and sequence length 512. Schema, label semantics, and prediction semantics were all verified; each prediction is one raw relevance logit sorted descending. The relevant document ranked first, producing MRR@2=1.0.

This is end-to-end operability evidence only. It is not representative accuracy or a benchmark-quality claim. Current-main reranking eval was formerly unsupported; dependency #1322 supplies canonical reranking task/evaluator semantics, while this bounded Tester smoke verifies final-candidate scalar-score fan-out and ranking behavior.

Delta

Recipes

• Added examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/reranking_fp32_config.json and reranking_fp16_config.json; deleted the parallel historical text-classification_fp32_config.json and text-classification_fp16_config.json so this checkpoint has one canonical user-facing task.
• Relative to the historical baseline recipe shape, both canonical recipes change /loader/task from "text-classification" to "reranking", /loader/model_class from "AutoModelForSequenceClassification" to "DebertaV2ForSequenceClassification", /optim from {"clamp_constant_values":true} to {}, and add /export/compatibility/transformers_attention = "eager"; /loader/model_type remains "deberta-v2".
• The fp32 recipe keeps /quant = null. The fp16 recipe changes /quant from null to an fp16 configuration with /quant/mode = "fp16", /quant/task = "reranking", /quant/model_id = "mixedbread-ai/mxbai-rerank-base-v1", /quant/model_type = "deberta-v2", /quant/fp16_keep_io_types = true, /quant/fp16_op_block_list = null, and /quant/fp16_nodes_to_exclude equal to the exact ordered 74-name graph-derived list.
• The exact set is 1 + (12 x 6) + 1 = 74: embeddings LayerNorm InsertedPrecisionFreeCast_/deberta/embeddings/LayerNorm/LayerNormalization_output_0; for every encoder layer 0..11, exactly Transpose_3, Reshape_1, Reshape_3, Transpose_8, Reshape_5, and Reshape_13 under InsertedPrecisionFreeCast_/deberta/encoder/layer.<layer>/attention/self/<boundary>_output_0; and pooler InsertedPrecisionFreeCast_/pooler/Gather_output_0. InsertedPrecisionFreeCast_/deberta/embeddings/Cast_output_0 is deliberately absent because it is INT32-to-FLOAT, not FP16-to-FLOAT.
• The production recipe README remains untouched.

Shared code and reducibility

• WinMLQuantizationConfig.fp16_nodes_to_exclude is an optional exact-name list. It validates a list of non-empty strings, de-duplicates stably in declaration order, serializes only for fp16 mode, and participates in cache identity for distinct non-empty lists; absent and null remain equivalent.
• FP16Pass.run forwards the list unchanged to convert_to_fp16(node_block_list=...). convert_to_fp16 forwards the same list to both normal and disable_shape_infer=True ORT conversion calls.
• _validate_converted_types(..., required=False) is the bounded input capability preflight. Strict-capable inputs require the same checker/strict-inference/checker sequence after conversion; only AttributeError, EncodeError, checker.ValidationError, and shape_inference.InferenceError may disable that generic converted-model sequence. Specialized external-data, sparse, container, initializer-output, topological, blocked-subgraph, and local-function safeguards remain active.
• Reducibility is consistent with the re-issued charter: the model recipe declares exact graph data, while the shared mechanism is opt-in and checkpoint-agnostic. Recipe-free/no-field controls preserve existing conversion behavior because absent/null produces no automatic exclusion, heuristic, or op-block addition.

Bug fix explanation

1. Symptom and minimal trigger: converting the optimized DeBERTa-v2 graph to fp16 rewrote an optimizer-generated Cast boundary to FLOAT16; CPU ORT load then rejected LayerNormalization because the emitted tensor(float16) did not match the retained tensor(float) contract.
2. Root cause: ORT already supports exact node_block_list, but WinML did not expose it through config or the FP16 pass. fp16_op_block_list=[LayerNormalization] cannot control the separate Cast, and broad Cast blocking would alter unrelated INT32-to-FLOAT and ordinary FP16 conversion semantics. Conditional strict validation also allowed an ordinary top-level type mismatch to survive until session creation.
3. Changed symbols and mechanism: WinMLQuantizationConfig.fp16_nodes_to_exclude, FP16Pass.run, convert_to_fp16, and _validate_converted_types add exact-list validation/serialization/cache identity, unchanged normal/fallback forwarding, and bounded strict pre/post validation. The fp16 recipe supplies the 74 exact ONNX node names.
4. General/data-driven rule: the recipe names came from inspecting the optimized fp32 graph and selecting only Cast nodes with FP16 input and FLOAT output. There is no model/checkpoint conditional, prefix/wildcard/substring match, runtime graph heuristic, synthesized fallback exclusion, or broad Cast block.
5. Compatibility and blast radius: the converter parameter is keyword-only and optional. Existing callers remain valid; absent/null configs preserve existing ORT calls and graph semantics; non-fp16 serialization does not gain the field; existing nodes_to_exclude, fp16_op_block_list, and fp16_keep_io_types semantics are unchanged. The intentional changes are exact-node preservation for recipes that opt in and earlier failure for incompatible strict-capable converted graphs.
6. Exact regression evidence: Tester L0-L3 all PASS on the final SHA. Ruff PASS (All checks passed!); mypy PASS (Success: no issues found in 438 source files). Five pytest partitions passed: analyze 1526 passed, 45 skipped; models 1534 passed, 6 skipped, 2 xfailed; optim 858 passed, 16 skipped, 1 xfailed; commands 3645 passed, 9 skipped; remaining 871 passed, 2 skipped, 1 deselected. Aggregate: 8434 passed, 78 skipped, 3 xfailed, 1 deselected. Exact selected/neighbor/unknown-name, normal/fallback forwarding, capability-exception, strict mismatch, no-mutation, recipe-list, cache, and specialized-safeguard regressions are included in the changed tests. Feature-gap tracking is #1353.

Analyze summary - component level and op level

Static analysis completed with ANALYZE-PARTIAL-SUCCESS: winml analyze --ep all exited 1 because some targets had no rule data, while seven EP rows and complete rule-backed classifications were parsed. This is static compatibility analysis, not runtime execution.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
fp32	embeddings; 12x DeBERTa attention/FFN; pooler; scalar classifier	666 mapped, 0 partial, 186 unmapped; confidence mapped-with-gaps	QNNExecutionProvider: partial GatherElements, Erf; unsupported none

The unresolved mapping gap is exactly 186 graph-root/generated nodes that lack a frozen semantic scope.

Op-level summary

Artifact	Graph	Dominant ops	EP roll-up
fp32	852 operators / 20 unique types	Reshape 240; Cast 138; Gemm 74; Transpose 72; Div 61; Add 61; MatMul 48; Mul 28	QNNExecutionProvider: partial GatherElements, Erf, unsupported none; no EP group was classified fully supported because rule-backed Cast remained unknown; targets without rule data produced no classification

Reproduce commands

$OUT='temp/mxbai-rerank-base-v1-tester-repro'
winml build -c examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/reranking_fp32_config.json -m mixedbread-ai/mxbai-rerank-base-v1 -o $OUT/fp32
winml build -c examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/reranking_fp16_config.json -m mixedbread-ai/mxbai-rerank-base-v1 -o $OUT/fp16 --precision fp16
winml analyze --model $OUT/fp32/model.onnx --ep all --output $OUT/analyze-all.json
winml perf -m $OUT/fp32/model.onnx --device cpu --ep cpu
winml perf -m $OUT/fp16/model.onnx --device cpu --ep cpu
mypy -p winml.modelkit

[ssss141414]

Independent post-PR review: REQUEST_CHANGES

Reviewed exact head ec130eaaf71bce478ca0792afc0c19e9c1ffd614 for mixedbread-ai/mxbai-rerank-base-v1 against dependency PR #1322 head producer/ms-marco-minilm-l6-v2-reranking.

Blocking findings

1. Planner -> Explainer: refresh or attest the moved-main baseline. The PR reports baseline main 774dd6e0811f6947eaf2bd8fb721d5f6e718b0c0, but canonical main is now e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0. The newer commit is a direct child and changes src/winml/modelkit/models/hf/vision_encoder_decoder.py; my sampled dependency review found no direct DeBERTa/quantization reach, but the frozen evidence contains no complete moved-main impact attestation required by _meta-052 / _meta-107. Planner must provide the old-to-current diff inventory, dependency-reachability rationale, candidate-patch equivalence, and reused/rerun provenance; Explainer must render the source-owned result in the PR body.
2. Orchestrator / Explainer shipment gate: obtain complete GitHub checks on this exact SHA. statusCheckRollup contains only license/cla = COMPLETED/SUCCESS; the WinML modelkit workflow checks are absent on this stacked-base PR. Branch-protection requirements were not readable through the current API identity. Do not treat local parity as a replacement for the required final-SHA GitHub check gate; arrange CI on ec130eaaf71bce478ca0792afc0c19e9c1ffd614 after the dependency/base workflow permits it, without changing state from this reviewer role.

Independently verified

• PR state/scope: OPEN + DRAFT, label model-scale-by-skill, exact head unchanged, dependency base exact, issue Add exact FP16 node exclusions and bounded strict-type validation #1353 linked. GitHub reports 10 files due rename detection; git diff --no-renames proves the 11-path truth. No README, skill, model-ID branch, prefix/wildcard, or unrelated scope leakage.
• Conversation gate: 0 issue comments, 0 line comments, 0 reviews, 0 paginated review threads, 0 open.
• Engineering review: optional fp16-only exact-name config, stable de-duplication, cache identity, unchanged absent/null defaults, normal/fallback ORT forwarding, and bounded strict validation all match the frozen L1 contract. Unknown names are inert; specialized sparse/external/container/local-function safeguards remain in place.
• Recipe/artifacts: exact ordered 74 Cast boundaries; excluded INT32 embedding Cast remains outside the list; fp32/fp16 payloads are 627,425,280 / 437,262,336 bytes; fp16 has 228 FLOAT16 initializers and no FLOAT initializers; checker, strict inference, exact input binding, and CPU load evidence agree.
• Goal evidence: CPU fp32/fp16 L1 means 447.942 / 476.833 ms; both precisions preserve ranking; fp32 cosine 0.9999999999999999, fp16 cosine 0.9999999153288317; bounded one-query/two-pair fp32 smoke reports MRR@2=1.0. Coverage is full with no deferred tuples.
• Local quality: exact-lock candidate environment; Ruff PASS; mypy PASS (438 files); focused changed-code tests PASS; all five non-hardware workflow partitions PASS. Standard uv run first attempted lock hydration and hit external TLS failure fetching optype; uv run --no-sync then exercised the already sealed exact-lock environment successfully and left the checkout clean.
• Source integrity/body: Planner, Producer, Tester (206dbb4e...), Learner, and Explainer (054a05c7...) seals independently rehashed. PR hierarchy, model profile/tree, full tuple metrics, six-part delta/bug explanation, analyze component+op summary, public commands, and methodology declaration are present without local paths. The stale baseline is the body defect identified above.

This is a fixable evidence/shipment return, not a rejection of the implementation. I made no review-state mutation, thread reply/resolution, body/title/label/readiness/base change, or branch push.

[ssss141414]

Independent post-PR review: APPROVE

Reviewed exact head ec130eaaf71bce478ca0792afc0c19e9c1ffd614 for mixedbread-ai/mxbai-rerank-base-v1 against dependency PR #1322 head 3708969b731425b0c6d4b97920d1b5e6519bb013.

Final gates

• Exact identity and shipment: PR is OPEN + DRAFT, retains model-scale-by-skill, exact dependency base producer/ms-marco-minilm-l6-v2-reranking, and the head remained ec130eaaf71bce478ca0792afc0c19e9c1ffd614. Scope is exactly the expected 11 recipe/source/test paths; no production README or unrelated path changed.
• Moved-main baseline: independently fetched canonical main at e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0. The complete 774dd6e0811f6947eaf2bd8fb721d5f6e718b0c0 to current-main history is one commit changing only src/winml/modelkit/models/hf/vision_encoder_decoder.py; content/import reach has no DeBERTa, reranking, quantization, FP16, runtime, Eval, Analyze, dependency, recipe, or workflow path. Candidate parent remains 3708969b731425b0c6d4b97920d1b5e6519bb013, all 11 candidate-owned paths are byte-equivalent in the old/current virtual merges, and the empty affected-stage closure supports Planner v5 REUSE with no rebase.
• PR body: required hierarchy and source-owned evidence are present, including the v5 moved-main attestation, complete CPU fp32/fp16 tuple evidence, full coverage with no deferred tuples, functional smoke, Delta, component/op Analyze summary, and reproduce commands.
• Conversation: enumerated 1 issue comments, 0 line comments, 0 reviews, and 0 fully paginated GraphQL review threads; 0 open threads. Prior REQUEST_CHANGES opinion 5400238484 is a normal issue comment, not a review/thread state; both cited blockers are resolved.
• Independent/local evidence: exact checkout remained clean at ec130eaaf71bce478ca0792afc0c19e9c1ffd614. Re-ran the four changed-code suites: 361 passed. The unchanged-source sealed review evidence still verifies Ruff, mypy (438 files), all five non-hardware partitions, engineering review, artifact structure, exact 74 fp16 boundaries, L1/L2/L3 results, and source identities for Tester, Learner, and Explainer.
• Coverage: full. Reachable and verified: cpu/cpu/fp32 and cpu/cpu/fp16; deferred: none. No applicable item is deferred.

Exact-SHA checks

• Analyze (Python): COMPLETED/SUCCESS
• CodeQL: COMPLETED/SUCCESS
• license/cla: COMPLETED/SUCCESS
• lint: COMPLETED/SUCCESS
• test (analyze): COMPLETED/SUCCESS
• test (commands): COMPLETED/SUCCESS
• test (models): COMPLETED/SUCCESS
• test (optim): COMPLETED/SUCCESS
• test (remaining): COMPLETED/SUCCESS

Workflow runs are exact-head successes: CodeQL 32771630721 (Analyze (Python)), Lint 32771630819 (lint), and WinML CLI CI 32771630820 (all five matrix jobs: models, analyze, optim, remaining, commands). Every enumerated run/job is COMPLETED/SUCCESS; none is missing, cancelled, skipped, queued, or in progress.

Opinion: APPROVE. Full reviewer checklist coverage is green with no deferred item. The PR intentionally remains DRAFT; this opinion does not authorize or perform a readiness change. I made no GitHub review-state mutation, metadata/body/base/branch change, thread reply/resolution, or other PR write beyond this one normal comment.