fix(layoutlm): resolve invoices question answering support

[拂柳残声 (fuliucansheng)]

Summary

This PR adds DmitrySpartak/layoutlm-invoices support for invoice and document question answering with CPU FP32 and FP16 recipes, metadata-driven automatic loader selection, and reusable bbox-aware document evaluation. It ships Effort L2 / Outcome L2 and reaches Goal L3 PASS through one bounded real-data FP32 CPU functional smoke; ANLS 0.0 is operability evidence only, not an accuracy claim. The checkpoint is licensed CC-BY-NC-SA-4.0 for noncommercial use.

Model metadata

What the model does

LayoutLMForQuestionAnswering consumes tokenized document text together with normalized 2D token bounding boxes and predicts start and end logits for extracting an answer span; this checkpoint is fine-tuned for questions about invoices and other documents.

• Evidence/confidence: pinned model card and config (verified).

Primary user stories

A user supplies an invoice or document question plus OCR-aligned words and boxes to obtain the document text span answering fields such as invoice number or purchase amount.

• Evidence/confidence: pinned model card (verified).

Supported tasks

• question-answering across the Transformers, Optimum ONNX, and WinML support surfaces (verified from the pinned checkpoint architecture, checked-in recipes, and Reproduce commands).
• document-question-answering as the checkpoint's document-domain task (verified from the pinned model card).

Model architecture

LayoutLMForQuestionAnswering
|-- LayoutLMModel
|   |-- Embeddings (token + 1D position + token type + 2D layout)
|   `-- Encoder layer x 12
`-- QA span head (start_logits + end_logits)

• Source/confidence: pinned checkpoint config and LayoutLMForQuestionAnswering source (verified).

Validation and support evidence

Baseline

WinML 0.2.0 baseline evidence was executed at commit 48bfd0f91478a8e281b205b0449222024729f3ea; pinned main is 2b9ec0e9e57bba8003d25eaff85f8c7c9e30d75a. Baseline automatic config selected next-sentence-prediction / AutoModelForNextSentencePrediction; build exited 2 because that class does not recognize LayoutLMConfig, so perf and Eval were not reached. The Optimum probe exposed feature extraction, fill mask, text classification, and token classification; WinML's override additionally exposed question answering.

The planner recorded REUSE because pinned main is unchanged from the prior current-main assessment. This retains baseline execution provenance and does not relabel it as newly executed on candidate head 61ffb5f5651573b22f47fed47571e163a1013335.

Goal

• Effort: L2, because the missing capability crosses task/loader resolution, evaluator schema, document preprocessing, bbox forwarding, span decoding, metric handling, optional OCR support, and metadata-derived export input constraints.
• Goal ceiling: L3, defined as CPU FP32/FP16 L0-L2 plus one bounded, document-aware FP32 CPU functional smoke on a licensed compatible dataset.
• Outcome: L2, shipping recipes and generalized code support with regression coverage and task-family findings.
• Canonicalization: automatic inspection surfaces document-question-answering; the concrete model is LayoutLMForQuestionAnswering, and the exporter boundary uses LayoutLMQAIOConfig. Checked-in recipes retain question-answering as the export/runtime identity while document-aware evaluation is selected from the declared bbox input and document columns.

Outcome

Overall verdict is PASS; highest reached Goal is L3 PASS and shipped Outcome is L2. Coverage is full for the declared CPUExecutionProvider / cpu FP32 and FP16 tuples, with no deferred tuples. The PR ships both CPU recipes, generalized architecture/config-driven resolution, LayoutLM export input derivation, bbox-aware QA evaluation and forwarding, ANLS, document schema support, an explicit native OCR contract, and regression tests. Model findings retain the contiguous start/end span limitation; no non-consecutive answer-decoding claim is made. No methodology finding was triggered.

Execution provenance is split without upgrading reused measurements:

• Exact-head evidence at 61ffb5f5651573b22f47fed47571e163a1013335: source/lock identity; exact checkpoint snapshot materialization; pinned-snapshot inspect/config/Eval; native Tesseract contract and runtime capture; fresh DocVQA L3 smoke; affected Eval/commands tests; OCR-focused tests; Ruff, mypy, license, lock, and docs validation.
• L0 FP32/FP16 artifact validation was executed at 2007e13ab59532f1a20a099babc432da9c1aa47d against artifacts originating at 5274a714997a8d03cc3b3048590aac2389419805. The final OCR-validation/docs-only repair does not change recipes, export, optimization, precision, graphs, or runtime forwarding, so their byte-identity evidence is retained.
• L1 perf, L2 parity, and Analyze were executed at 5274a714997a8d03cc3b3048590aac2389419805 and validated on 2007e13ab59532f1a20a099babc432da9c1aa47d through byte-identical explicit artifacts. They remain reused on 61ffb5f5651573b22f47fed47571e163a1013335 for the same path-invariance reason.
• Models and remaining test partitions were executed at 2007e13ab59532f1a20a099babc432da9c1aa47d and retained as path-invariant. All nine GitHub checks independently completed successfully on exact head 61ffb5f5651573b22f47fed47571e163a1013335.

The Lane A layoutlm-003 knowledge entry now names the two shipped recipe files exactly: examples/recipes/DmitrySpartak_layoutlm-invoices/cpu/cpu/question-answering_fp32_config.json and examples/recipes/DmitrySpartak_layoutlm-invoices/cpu/cpu/question-answering_fp16_config.json. That correction is preserved in a dedicated local Lane A commit and is intentionally unpushed; it is not mixed into this model-support PR.

Per-EP/device/precision results and Functional smoke Eval

Automatic recipe-free acceptance

At execution head 2007e13ab59532f1a20a099babc432da9c1aa47d, automatic inspection selected task document-question-answering, model class LayoutLMForQuestionAnswering, and exporter LayoutLMQAIOConfig without a recipe or task/model-class override. Metadata produced a usable sequence capacity of 512 from model_max_length=512, max_position_embeddings=514, and pad_token_id=1; paired question/document tokenization reserves four special tokens, leaving a 508-token content budget. FP32 and FP16 automatic configs consistently use [1, 512] for input_ids, attention_mask, and token_type_ids, [1, 512, 4] for bbox, and [0, 1) for token_type_ids from type_vocab_size=1.

The recipe-free CPU FP32 build passed. CPU ORT inference used all four int32 inputs, zero-only token_type_ids, bbox values within [0, 1000], and produced finite start_logits and end_logits, each shaped [1, 512]. Exact head 61ffb5f5651573b22f47fed47571e163a1013335 freshly materialized checkpoint revision ce2422049c250384731eccef90bf6d92e846b09c; inspect and config passed against that local snapshot with the same task, model class, exporter, and four-input schema.

Explicit tuple builds and L1 perf

Recipe-authoritative FP32 and FP16 builds passed at 2007e13ab59532f1a20a099babc432da9c1aa47d; the explicit artifacts were byte-identical to those measured at 5274a714997a8d03cc3b3048590aac2389419805. L1 used a synthetic invoice-like OCR fixture with named input_ids, bbox, attention_mask, and token_type_ids, 5 warmups, and 20 timed iterations. It is not presented as real-data performance.

Tier	EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM delta
L0/L1	CPUExecutionProvider / cpu	fp32	PASS	233.261 ms	231.604 ms	4.29 samples/s	+132.88 MB
L0/L1	CPUExecutionProvider / cpu	fp16	PASS	301.999 ms	298.991 ms	3.31 samples/s	+153.05 MB

FP32 produced an IR 8, opset 17 graph with 394 nodes and FLOAT initializers. FP16 produced an IR 8, opset 17 graph with 396 nodes and 206 FLOAT16 initializers. Both expose input_ids, bbox, attention_mask, and token_type_ids and produce start_logits / end_logits.

L2 PyTorch/ONNX parity

Parity used the same synthetic invoice-like OCR inputs, with 65 non-padding tokens and exact named input shapes: input_ids, attention_mask, and token_type_ids are [1, 512] int32; bbox is [1, 512, 4] int32.

Precision	Output	Cosine	Max abs	L2	Reference / ONNX argmax
fp32	start logits	0.9999999999999309	0.000034809112548828125	0.0001277129801970843	50 / 50
fp32	end logits	0.9999999999999252	0.000041961669921875	0.00014027023337799058	55 / 55
fp16	start logits	0.999999823476883	0.11089324951171875	0.20976535244230682	50 / 50
fp16	end logits	0.9999996397368223	0.11360549926757812	0.30969060293798095	55 / 55

Functional smoke Eval

L3 PASS: the FP32 CPU DocVQA smoke was freshly executed on exact head 61ffb5f5651573b22f47fed47571e163a1013335 against the locally materialized checkpoint revision. ANLS 0.0 proves end-to-end operability only; it is not representative accuracy or benchmark quality.

• Runtime: Python 3.11.16, ModelKit 0.3.0, Transformers 5.14.1, datasets 5.0.0, pytesseract 0.3.13, Pillow 12.3.0, JPEG decoder 8.0, zlib 1.3.1.zlib-ng, and native Tesseract 5.5.3.
• Dataset: public, ungated lmms-lab-encoder/DocVQA, config DocVQA, revision 539088ef8a8ada01ac8e2e6d4e372586748a265e, validation row index 2, questionId 57349; dataset card license Apache-2.0.
• Scope: 1 sample requested, 1 processed, 0 skipped. Question: "What is the name of the company?" References: itc limited, ITC Limited; prediction: ITC's.
• Input semantics: one real 1701x2386 grayscale PNG document decoded through Pillow; one native Tesseract 5.5.3 OCR pass produced 141 words and 141 normalized boxes in range [0, 997]; bbox forwarding was verified as [1, 512, 4] int64 alongside all four declared inputs.
• Bounds/accounting: 1 question/document, 1 OCR pass, 1 selected 512-token window, stride 128, top-k 1, maximum 64 answer words, 1 beam, 1 frame/crop, and 1 candidate label/prompt. Metrics: ANLS 0.0, n_samples=1, skipped_samples=0, windows_processed=1, ocr_samples=1, precomputed_samples=0, ocr_engine_version=5.5.3.
• The image-only path now validates native Tesseract 5.x before execution and records only its sanitized version. A separately verified precomputed words/boxes path forwards the same four named inputs without importing or probing pytesseract or native Tesseract; it reported ocr_samples=0 and precomputed_samples=1.
• Schema, answer-label semantics, contiguous document-word prediction semantics, OCR word/box normalization, bbox forwarding, and metric accounting were verified. The former blockers were missing document preprocessing/bbox-aware decoding and an undeclared native OCR runtime; this PR adds both contracts while preserving text-only QA behavior.

Delta

Baseline had no usable automatic recipe: it selected next-sentence prediction and failed before generating an authoritative starting recipe. The checked-in CPU recipes explicitly select question-answering / LayoutLMForQuestionAnswering, use 512-position inputs with normalized bbox and zero-only token types, enable constant clamping, and select FP32 or FP16 realization. The final OCR repair does not modify either recipe; retained explicit artifacts remain byte-identical. The production recipe README remains untouched.

The automatic-path repair replaces the prior raw position-table interpretation with metadata-derived usable capacity and exposes the valid token-type range at config generation. The prior automatic path's 514-position configuration and build failure are fixed-delta context only; accepted automatic shapes are 512 and recipe-free build/ORT pass.

Bug fix explanation: architecture/config-driven task and loader selection

1. Symptom and trigger: automatic resolution for the primary LayoutLMForQuestionAnswering architecture selected loader behavior that Transformers could not instantiate for LayoutLMConfig; the concrete recipe masked the automatic-path defect.
2. Root cause: the architecture-suffix fallback did not consult Transformers' per-config plain-QA and document-QA mappings, and surfacing document QA directly initially bypassed the existing Optimum QA exporter boundary.
3. Symbols and mechanism: loader resolution/task helpers consult MODEL_FOR_DOCUMENT_QUESTION_ANSWERING_MAPPING_NAMES and MODEL_FOR_QUESTION_ANSWERING_MAPPING_NAMES, choose document QA only for configurations supported exclusively there, and canonicalize the exporter boundary so LayoutLMQAIOConfig remains available.
4. General rule: selection is derived from config.architectures and Transformers config mappings, with no checkpoint ID or repository-name branch.
5. Compatibility/blast radius: TasksManager and explicit task/class choices remain authoritative; ordinary BertForQuestionAnswering stays plain question-answering, and only document-only configurations take the document path.
6. Regression evidence: automatic inspect/config/build/ORT pass; resolver, task-boundary, exporter, build-preflight, equality, evaluator, models/loader/export, commands, and remaining partitions pass.

Bug fix explanation: metadata-derived LayoutLM export inputs

1. Symptom and trigger: the prior recipe-free config used the raw LayoutLM position-table size and exposed an invalid token-type range, causing automatic CPU FP32 build to fail during hierarchy tracing.
2. Root cause: max_position_embeddings includes the RoBERTa-style offset, while post-processing generated token types to zero happened too late for serialized config reconstruction.
3. Symbols and mechanism: LayoutLMQAIOConfig applies the existing position-embedding adjustment before shape resolution; the LayoutLM input generator derives token_type_ids directly from type_vocab_size.
4. General rule: sequence capacity and token-type bounds come from checkpoint metadata, not checkpoint identity; explicit sequence_length remains authoritative.
5. Compatibility/blast radius: zero-offset LayoutLM remains at its full declared length, a requested 128-token budget applies consistently to all sequence-coupled inputs, and ordinary QA plus LayoutLMv3/RoBERTa behavior remains covered.
6. Regression evidence: automatic FP32/FP16 configs use length 512 and [0, 1) token types; recipe-free build and CPU ORT pass; explicit FP32/FP16 artifacts remain byte-identical; models/loader/datasets/export coverage reports 1,538 passed, 6 skipped, and 2 xfailed.

Bug fix explanation: bbox-aware document evaluation and native OCR validation

1. Symptom and trigger: the existing text-QA path could not construct or forward the graph's required bbox input, decode OCR-aligned document spans, or emit a document metric; the initial image-only implementation also depended on an undeclared native Tesseract executable.
2. Root cause: the evaluator had no document row schema, OCR/precomputed word-box preprocessing, subword box alignment, bbox-aware forwarding, ANLS path, or native executable/version validation.
3. Symbols and mechanism: the question-answering evaluator adds bounded document preprocessing and contiguous span decoding; the WinML QA model conditionally forwards declared bbox inputs; image-only validation requires native Tesseract 5.x and emits its sanitized version, while precomputed words/boxes bypass OCR entirely.
4. General rule: document mode is selected from a declared bbox model input plus document columns or an explicit request, not a checkpoint identifier. Precomputed words/boxes are preferred; image rows use one optional, version-validated OCR pass.
5. Compatibility/blast radius: models without a bbox input retain the existing SQuAD/SQuAD-v2 text-QA path and metrics; incidental bbox values are dropped; missing/unsupported OCR runtimes fail with actionable installation/version guidance; precomputed words/boxes require no native OCR.
6. Regression evidence: exact-head OCR contract tests report 5 passed/29 deselected; Eval reports 648 passed; accepted command shards report 1,427 passed/1 skipped; Ruff, mypy across 437 source files, license, lock, docs, and all nine exact-head GitHub checks pass. Models and remaining retained partitions report 1,538 passed/6 skipped/2 xfailed and 872 passed/2 skipped/1 deselected, respectively.

Analyze summary - component level and op level

Analyze completed with ANALYZE-PARTIAL-SUCCESS (exit 1) and seven parseable rows per artifact at execution SHA 5274a714997a8d03cc3b3048590aac2389419805; byte-identical artifact validation at 2007e13ab59532f1a20a099babc432da9c1aa47d remains reusable on exact head 61ffb5f5651573b22f47fed47571e163a1013335. This is static rule compatibility analysis, not runtime execution. There were no actionable partial or unsupported findings, and runtime_support=false is not runtime EP evidence.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
fp32	embeddings; 12x encoder attention; 12x encoder feed-forward; model plumbing; QA span head	394 mapped, 0 unmapped; mapped confidence	none
fp16	embeddings; 12x encoder attention; 12x encoder feed-forward; model plumbing; QA span head	396 mapped, 0 unmapped; mapped confidence	none

Op-level summary

Artifact	Graph	Dominant ops	EP roll-up
fp32	394 operators / 15 types	Reshape 122; Gemm 73; Transpose 48; Add 44	QNN/GPU: all 15 types statically supported; OpenVINO/GPU: 7 supported and 8 unknown; no actionable partial/unsupported types
fp16	396 operators / 15 types	Reshape 122; Gemm 73; Transpose 48; Add 44	QNN/GPU: all 15 types statically supported; OpenVINO/GPU: 7 supported and 8 unknown; no actionable partial/unsupported types

NvTensorRTRTX/GPU reports all 15 types unknown. CUDA/GPU, MIGraphX/GPU, TensorRT/GPU, and DML/GPU have no shipped classifications; unknown is not unsupported. There are no component mapping gaps.

Reproduce commands

$OUT = 'temp/layoutlm-invoices-repro'
$MODEL = Join-Path $OUT 'model_snapshot'
$REVISION = 'ce2422049c250384731eccef90bf6d92e846b09c'
$DATASET_REVISION = '539088ef8a8ada01ac8e2e6d4e372586748a265e'
$RECIPE = 'examples/recipes/DmitrySpartak_layoutlm-invoices/cpu/cpu/question-answering_fp32_config.json'
$ARTIFACT = Join-Path $OUT 'fp32/model.onnx'

# Materialize the reviewed checkpoint revision and use only this local snapshot.
python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='DmitrySpartak/layoutlm-invoices', revision='$REVISION', local_dir=r'$MODEL')"

# Image-only document QA requires supported native Tesseract 5.x on Windows.
winget install --id UB-Mannheim.TesseractOCR --version 5.4.0.20240606 --exact
tesseract --version
Get-Command tesseract | Select-Object Name, CommandType

# Record the relevant Python/runtime versions without machine-specific paths.
python --version
python -c "from importlib.metadata import version; from PIL import features; print('winml-cli='+version('winml-cli')); print('transformers='+version('transformers')); print('datasets='+version('datasets')); print('pytesseract='+version('pytesseract')); print('Pillow='+version('Pillow')); print('jpeg='+str(features.version('jpg')))"

# Automatic metadata-driven path and recipe-authoritative FP32 build use the snapshot.
winml inspect -m $MODEL --format json
winml config -m $MODEL --device cpu --ep cpu --precision fp32 --no-quant --no-compile -o (Join-Path $OUT 'fp32_config.json')
winml build -c $RECIPE -m $MODEL -o (Join-Path $OUT 'fp32')

# Pinned real-media FP32 CPU functional smoke; one sample/OCR pass/window, operability only.
winml eval -m $ARTIFACT --model-id $MODEL --task question-answering --dataset lmms-lab-encoder/DocVQA --dataset-name DocVQA --dataset-revision $DATASET_REVISION --split validation --samples 1 --no-shuffle --streaming --column question_column=question --column image_column=image --column label_column=answers --column document_mode=true --column ocr_engine=tesseract --column sample_index=2 --column max_windows=1 --column doc_stride=128 --column max_answer_words=64 --column top_k=1 --device cpu --ep cpu

# Precomputed words/boxes exercise document inputs without pytesseract or native OCR.
python -m pytest tests/unit/eval/test_question_answering_evaluator.py::TestDocumentCompute::test_precomputed_document_path_forwards_exact_named_inputs -q --no-cov

[ssss141414]

APPROVE

Reviewed head 3c58f1a6fa0cc7a50de465ba96350993e5beb5d5, base/merge-base 48bfd0f91478a8e281b205b0449222024729f3ea, and current origin/main 2b9ec0e9e57bba8003d25eaff85f8c7c9e30d75a. Main's intervening diff only changes winml sys memory reporting, docs, and tests; it does not affect resolution, config, build, runtime, Eval, or Analyze evidence.

Exact-head verification:

• Static: Ruff passed; mypy passed across 436 source files.
• Full non-hardware matrix: analyze 1,526 passed/45 skipped; models 1,533 passed/6 skipped/2 xfailed; optim 714 passed/16 skipped/1 xfailed; commands 3,611 passed/7 skipped; remaining 871 passed/2 skipped/1 deselected.
• L0: recipe-authoritative fp32/fp16 builds passed in 108.5/115.4 seconds; IR 8/opset 17; sidecars 508,803,072/254,401,536 bytes; fp16 contains FLOAT16 initializers.
• L1: fp32 284.861 ms, 3.510 samples/s, +462.160 MB; fp16 323.575 ms, 3.090 samples/s, +495.215 MB.
• L2: fp32 cosine effectively 1.0 with max-abs <=4.58e-5; fp16 cosine >=0.99999964; PyTorch/ONNX argmax agrees at 50/55 for both.
• L3: exactly one FP32 CPU Functional smoke Eval is CLI-BLOCKED before processing by the invalid hf://datasets/squad... URI; 1 requested/0 processed and no metric. No higher-tier PASS is claimed.
• Analyze: partial success with parseable static results; fp32 394 mapped nodes/15 op types and fp16 396/15, with zero unmapped or actionable rule-backed issues.

The final body has the required hierarchy, recipe-authoritative commands, exact CPU fp32/fp16 auto-config reproduction, and a self-contained real-invoice semantic harness that reproduces perf, peak-RSS memory measurement, and parity without random inputs or local scratch dependencies.

Lane A PR gim-home/ModelKitArtifacts#231 is draft/labeled at 135ddbe34e4f650f4f135f25818850a3d247ce50, with its check successful. Conversation gate: 0 issue comments, 0 inline comments, 0 reviews, 0 review threads, 0 open. GitHub gate: all 9 exact-head checks succeeded and the head remained unchanged.

Coverage is full for the declared CPUExecutionProvider/cpu fp32 and fp16 target; no target tuples are deferred. Residual risks are the L3 dataset-URI blocker, no accelerator runtime claims, and host-sensitive absolute timing.

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Two task-contract issues are noted inline.

[ssss141414]

REQUEST_CHANGES

Reviewed exact head 2007e13ab59532f1a20a099babc432da9c1aa47d against current main/base 2b9ec0e9e57bba8003d25eaff85f8c7c9e30d75a from a fresh isolated checkout. The PR is OPEN, DRAFT, labeled model-scale-by-skill, and MERGEABLE. All 9 exact-head GitHub checks are COMPLETED/SUCCESS. Conversation enumeration found 5 line comments, 1 conversation comment, 4 COMMENTED reviews, and 3/3 resolved review threads (0 open).

Blocking findings:

• Real-media L3 is not portably reproducible because the required native OCR runtime is undeclared and unversioned. pyproject.toml:89 adds only pytesseract==0.3.13, while src/winml/modelkit/eval/question_answering_evaluator.py:262-277 requires a separate native Tesseract executable. The sealed row-2 evidence records PngImageFile and ocr_source=tesseract, but no native Tesseract version or decoder/runtime version set. The PR's Reproduce block invokes OCR directly without provisioning or verifying native Tesseract. This leaves the reviewer contract's real-media runtime and complete-portable-command gates open.
• Public model reproduction is not revision-pinned. Acceptance is bound to checkpoint revision ce2422049c250384731eccef90bf6d92e846b09c, but the Reproduce block sets $MODEL='DmitrySpartak/layoutlm-invoices' and passes that floating Hub ID to inspect/config/build/eval. A later upstream checkpoint update would no longer reproduce the reviewed artifacts or semantics.
• Learner knowledge retains a nonexistent exact-recipe path. The authoritative learner-r3 layoutlm-003 finding was updated with final-head automatic-path facts, but its recipe_template still names examples/recipes/DmitrySpartak_layoutlm-invoices/cpu/cpu/document-question-answering_<fp32|fp16>_config.json. The shipped files are question-answering_fp32_config.json and question-answering_fp16_config.json. That stale guidance contradicts the canonical task decision and fails the exact-recipe knowledge audit.

Required owner actions:

• Producer: declare a supported native Tesseract prerequisite and reproducible provisioning path in the appropriate installation/eval documentation, or provide a fully portable precomputed-words/boxes reproduction path that exercises the same pinned real row without an undeclared native runtime.
• Tester: use a new immutable evidence root to capture exact tesseract --version, pytesseract, Pillow/image decoder, and datasets versions, then rerun or otherwise bind the pinned lmms-lab-encoder/DocVQA@539088ef8a8ada01ac8e2e6d4e372586748a265e validation row 2 smoke with the existing 1-sample/1-OCR/1-window fanout and ANLS accounting. Do not alter the sealed prior roots.
• Learner: correct layoutlm-003.recipe_template in the Lane A model-knowledge artifact to the two shipped question-answering_* recipe paths and refresh its provenance/hash without rewriting historical observations.
• Explainer: update the Reproduce commands to materialize checkpoint revision ce2422049c250384731eccef90bf6d92e846b09c into a local snapshot (or use an equivalent supported revision mechanism), add the declared OCR installation/version check (or portable precomputed-input path), and preserve the operability-only ANLS 0.0 claim.

Otherwise verified:

• Complete 21-file diff reviewed; no checkpoint-ID branch, no examples/recipes/README.md change, and no unrelated scope.
• Automatic resolver selects document-question-answering / LayoutLMForQuestionAnswering / LayoutLMQAIOConfig; metadata derives 512 tensor positions, four pair special tokens, a 508-token content budget, and zero-only [0,1) token types.
• Exact four-input input_ids/bbox/attention_mask/token_type_ids schemas, bbox-aware OCR/overflow forwarding, contiguous span decode, ANLS, equality behavior, and ordinary SQuAD/BERT sibling preservation are covered.
• Fresh final-head explicit fp32/fp16 artifacts are byte-identical to sealed 5274a714 artifacts; fp16 data is half-size with 206 FLOAT16 initializers. L1 perf, L2 parity, L3 row semantics, and Analyze reuse are therefore artifact-bound rather than relabeled.
• Analyze reconciles 394/394 fp32 and 396/396 fp16 optimized nodes with zero unmapped nodes and complete seven-row static classifications.
• Reviewer reran ANLS (8 passed), Ruff (PASS), mypy (437 source files), models (1,538 passed/6 skipped/2 xfailed), and commands (3,636 passed/9 skipped/1 warning). Sealed exact-head evidence additionally reports focused 248 passed, remaining 872 passed/2 skipped/1 deselected, and license PASS.
• Coverage is full for declared CPUExecutionProvider/cpu fp32 and fp16 tuples; no deferred tuples.

These are fixable evidence/reproduction and knowledge-capture gaps, so the verdict is REQUEST_CHANGES, not REJECT. Re-review must restart from the new exact PR head and freshly enumerate checks and threads.

[ssss141414]

Addressed all three Reviewer findings on exact head 61ffb5f5651573b22f47fed47571e163a1013335:

• Native OCR is now declared as Tesseract 5.x with a versioned Windows winget path and tesseract --version check. Fresh L3 evidence records Python 3.11.16, ModelKit 0.3.0, Transformers 5.14.1, datasets 5.0.0, pytesseract 0.3.13, Pillow 12.3.0/JPEG 8.0, and native Tesseract 5.5.3; pinned DocVQA validation row 2 again processed exactly 1 sample, 1 OCR pass, and 1 window with ANLS 0.0 as operability-only evidence. The precomputed words/boxes path is also documented and verified to avoid native OCR.
• Reproduce commands now materialize DmitrySpartak/layoutlm-invoices at exact revision ce2422049c250384731eccef90bf6d92e846b09c into a local snapshot and use that snapshot for inspect, config, build, and Eval. The DocVQA revision and deterministic row/fanout remain pinned.
• Lane A layoutlm-003 now names the shipped question-answering_fp32_config.json and question-answering_fp16_config.json recipe paths. The correction is in a dedicated local Lane A commit, intentionally unpushed and not mixed into this model PR.

The structured PR body has been refreshed with exact-head provenance while preserving the original execution SHAs for reused L0-L2 and Analyze evidence. The existing Reviewer verdict and resolved review threads were not edited.

[ssss141414]

APPROVE

Reviewed exact head 61ffb5f5651573b22f47fed47571e163a1013335 against current main/base 2b9ec0e9e57bba8003d25eaff85f8c7c9e30d75a from fresh isolated root reviewer-61ffb5f5-r2. I read the complete 23-file diff and current PR body, independently fetched PR/main, and treated all producer/tester/learner/explainer artifacts as claims to verify. The PR remains OPEN, DRAFT, labeled model-scale-by-skill, and MERGEABLE.

The three prior blockers are resolved:

• Native OCR is now a declared Tesseract 5.x prerequisite with versioned Windows provisioning, fail-fast missing/unsupported-version errors, and sanitized result metadata. A fresh reviewer DocVQA row-2 run used datasets 5.0.0, pytesseract 0.3.13, Pillow 12.3.0/JPEG 8.0, and native Tesseract 5.5.3; it decoded the real 1701x2386 image, made one OCR call, produced 141 matched words/boxes, forwarded bbox [1,512,4], processed one window, emitted ANLS 0.0 with n_samples=1/skipped_samples=0, bound ocr_engine_version=5.5.3, and exited 0. The precomputed words/boxes test proves no pytesseract/native runtime probe and reports ANLS 1.0 with ocr_samples=0.
• Public reproduction materializes checkpoint revision ce2422049c250384731eccef90bf6d92e846b09c into a local snapshot and uses that snapshot for inspect, config, build, and Eval. My fresh inspect/config rerun against the materialized snapshot exited 0 and resolved document-question-answering, LayoutLMForQuestionAnswering, LayoutLMQAIOConfig, four exact inputs at length 512, and [0,1) token types.
• Lane A commit 4924287c1a923cf86a99c66aeabc80006dd4a086 changes only model_knowledge/layoutlm.json; its exact blob is 837bc67933e97ea3da2bd533643825ea319e8031, matching learner provenance. layoutlm-003 names both shipped files exactly: question-answering_fp32_config.json and question-answering_fp16_config.json.

Independent review evidence:

• Engineering: task resolution is architecture/config-mapping driven with no checkpoint-ID branch; document mode requires declared bbox capability plus document columns or explicit selection; ordinary SQuAD/BERT behavior remains on the existing path. Recipes are schema-correct, the production recipes README is unchanged, and the final OCR repair touches only evaluator/tests/docs, leaving recipe/export/model artifact paths invariant.
• Local checks: tester seal 30/30 PASS; focused Eval/resolver/export/build suite 155 passed; fresh L3 semantic smoke PASS/exit 0; Ruff PASS/exit 0; mypy process exit 0 (437-file package gate; only the existing unused-config-section note); exact-head command JUnit 1,428 tests with 0 failures/errors and 1 skipped.
• Artifacts: fresh hash/structure verifier PASS. FP32 is IR 8/opset 17, 394 nodes, sidecar 508,803,072 bytes, SHA-256 1f716149...178e592; FP16 is IR 8/opset 17, 396 nodes, 206 FLOAT16 initializers, sidecar 254,401,536 bytes, SHA-256 54a00b19...b1065b9. Both expose the exact four named inputs and start/end logits at [1,512].
• Analyze: hierarchy plus topology reconciles 394/394 fp32 and 396/396 fp16 optimized nodes, zero unmapped, 15 operator types, and seven complete static EP rows per artifact. L1 perf and L2 parity retain their original execution SHA 5274a714997a8d03cc3b3048590aac2389419805; L0 byte identity was validated at 2007e13ab59532f1a20a099babc432da9c1aa47d; only invalidated OCR/L3 evidence was rerun at final head.
• GitHub checks: all 9 exact-head checks are COMPLETED/SUCCESS: Analyze (Python), lint, test (analyze), test (models), test (optim), test (commands), test (remaining), CodeQL, and license/cla.
• Conversation gate: 5 line comments, 3 conversation comments, 4 COMMENTED reviews, and 3/3 resolved review threads with 0 open; thread pagination reports no next page.

Coverage is full for the declared CPUExecutionProvider/cpu fp32 and fp16 tuples through L0-L2, plus one representative final-head FP32 CPU L3 functional smoke. No tuples are deferred. ANLS 0.0 is correctly presented as bounded operability evidence only, not representative accuracy or benchmark quality.

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Two issues block approval:

1. src/winml/modelkit/loader/resolution.py:205 now auto-resolves LayoutLM to document-question-answering, but eval/evaluate.py only registers question-answering; _resolve_task() therefore rejects the automatically inferred task before selecting the new evaluator. The documented smoke passes --task question-answering, so it bypasses this path. Please normalize at the eval/runtime boundary or register document-question-answering end-to-end, and add a regression test that evaluates without --task.

2. src/winml/modelkit/eval/question_answering_evaluator.py:64 treats box_coords=auto as absolute only when some coordinate exceeds 1000. Raw pixel boxes from common images at or below 1000 px are silently treated as already normalized, producing incorrectly scaled LayoutLM positions. Please make ambiguous coordinate input explicit (or otherwise avoid the heuristic) and cover sub-1000 absolute boxes in a test.

[ssss141414]

Qiong Wu (qiowu) (@DingmaomaoBJTU) Both blocking issues from the CHANGES_REQUESTED review are addressed in c463bfce8b5ae9182115036d6c18e749d204be64:

• document-question-answering is now registered through the Eval registry/schema, WinML wrapper map, and extractive compatibility pipeline. An independent real smoke called evaluate(config) with config.task=None; it inferred document-question-answering, executed WinMLModelForQuestionAnswering + _ExtractiveQuestionAnsweringPipeline + WinMLQuestionAnsweringEvaluator, and produced DORAL / ANLS 1.0 on one pinned DocVQA row.
• box_coords=auto now treats coordinates as absolute whenever image dimensions are available. Synthetic [160,120,80,60] at 800x600 normalized to [100,100,200,200]; the real DORAL box [305,345,416,379] at 2337x1762 normalized to [131,196,178,215].

Fresh independent validation: 8,558 ordinary CI tests passed; Ruff passed; mypy passed across 442 source files; sealed evidence reverified 108/108 files. Please re-review when convenient.