Refresh open-vakgyata CPU recipes

[ssss141414]

Summary

Refreshes the existing CPU FP32 and FP16 recipes for onecxi/open-vakgyata, a ten-class Indian speech-language identification checkpoint previously covered by #1131, so both recipes retain the current metadata-derived eager-attention export policy. This is an Effort/Goal/Outcome L0/L3/L0 recipe-only contribution stacked on the generic audio-classification evaluator in #1326. The tester reached the committed Goal L3 ceiling: both required CPU precision tuples passed build, performance, and PyTorch parity, and FP32 passed a bounded nine-class FLEURS functional smoke Eval.

Model metadata

What the model does

This speech-language identification checkpoint consumes a one-second 16 kHz waveform and emits ten logits for Indian English and nine Indian languages.

• Evidence/confidence: pinned checkpoint onecxi/open-vakgyata@f2754058e485dfc65cc62589b8d0e21c1d328399 and the verified ONNX contract input_values float32[1,16000] -> logits float32[1,10] (verified).

Primary user stories

• A speech application supplies an Indian-language utterance to identify its language or locale among the checkpoint's ten configured classes.
• Evidence/confidence: pinned task and exact id2label metadata (verified).

Supported tasks

• audio-classification across the checkpoint, Transformers, Optimum ONNX, and WinML surfaces.
• Evidence/confidence: Wav2Vec2ForSequenceClassification metadata and WinML inspect/config/build/perf success; on main, Eval stops before dataset loading because audio-classification is absent from the evaluator registry (verified). The generic capability is supplied by dependent Draft recipe(ast): use eager attention for AudioSet model #1326, not by this recipe-only change.

Model architecture

Wav2Vec2ForSequenceClassification
|- waveform convolutional feature encoder x7
|- feature projection 512 -> 1280
|- positional convolution
|- stable-LayerNorm Transformer + residual adapter x2
|- encoder final LayerNorm
|- projector 1280 -> 1024
|- temporal mean pooling
'- classifier 1024 -> 10

• Source/confidence: pinned checkpoint configuration (hidden_size=1280, intermediate_size=5120, two layers, 16 heads, classifier_proj_size=1024) plus complete optimized-graph component mapping (mapped).

Validation and support evidence

1. Baseline

The baseline is main commit 0876e5ae1c98a169a6137e092e0d7b30bf9cee33 with WinML 0.3.0, Python 3.11.9, Transformers 5.14.1, Optimum 2.1.0, Datasets 5.0.0, and SoundFile 0.14.0. The direct standalone Optimum probe produced no admissible result; independently, WinML resolved and executed the Wav2Vec2 audio-classification exporter.

Current-main auto-config already adds export.compatibility.transformers_attention=eager; otherwise its generated FP32 and FP16 configurations match the historical checked-in recipes, including fp16_keep_io_types=true. Both generated configurations built, passed ONNX checking, and ran on CPU with input_values float32[1,16000] -> logits float32[1,10]. FP32 had 129 nodes and 234,990,980 external-data bytes; FP16 had the same 129 semantic nodes plus two public-boundary Cast nodes and 117,495,490 external-data bytes.

Baseline tuple	Mean	p50	p90	Throughput	RSS delta
CPU / FP32	46.630 ms	47.107 ms	50.253 ms	21.45 samples/s	+107.16 MB
CPU / FP16	53.721 ms	54.108 ms	57.602 ms	18.61 samples/s	+43.54 MB

Baseline parity used the same deterministic one-second 16 kHz waveform for PyTorch and ONNX. FP32 had max/mean absolute error 0.00022792816162109375 / 0.00009121745824813843, cosine 1.0, and the same argmax. FP16 had max/mean absolute error 0.007291078567504883 / 0.0037896751891821623, cosine 0.9999998807907104, and the same argmax. All three paths predicted class 8, mr-IN, with finite logits.

Baseline Eval exited 1 before dataset loading because audio-classification was not supported. Draft #1326 at 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8 supplies the generic evaluator and raw-streaming SoundFile fix used by final validation.

2. Goal

• Effort: L0, configuration-only recipe refresh.
• Goal ceiling: L3, defined as override-free CPU FP32/FP16 recipe builds, runtime performance, named-input PyTorch parity, and one exact nine-class full-window FLEURS functional smoke.
• Outcome: L0, two refined recipes with no source, evaluator, registry, dependency, test, or README changes.
• Result: L0 PASS, L1 PASS, L2 PASS, L3 PASS; the ceiling was not downgraded.

3. Outcome

The shipped tier is L0 and the highest Goal result is L3 PASS. Coverage is full for the required tuples: CPU/FP32 and CPU/FP16 both passed, with no deferred tuples. The contribution changes only:

• examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp32_config.json
• examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp16_config.json

Historical context is #1131. Generic Eval support remains in dependent Draft #1326. The learner curated existing wav2vec2-003, wav2vec2-013, and wav2vec2-014 knowledge claims without adding duplicates. Lane A is gim-home/ModelKitArtifacts#254 at exact commit e06d2fe92bd699b08322792de726da5ada43fd06, which binds the _meta-111 command-closure contract used here. Methodology friction observed: _meta-111 added.

4. Per-EP/device/precision results and Functional smoke Eval

Tier	EP / Device	Precision	Verdict	Mean	p50	p90	Throughput	RAM delta
L0	CPUExecutionProvider / cpu	FP32	PASS	-	-	-	-	-
L0	CPUExecutionProvider / cpu	FP16	PASS	-	-	-	-	-
L1	CPUExecutionProvider / cpu	FP32	PASS	39.112 ms	38.295 ms	43.858 ms	25.57 samples/s	+107.19 MB
L1	CPUExecutionProvider / cpu	FP16	PASS	52.949 ms	53.341 ms	55.728 ms	18.89 samples/s	+43.53 MB

Both L0 builds preserved float32 public input/output. The FP16 public build row carries the contract-required explicit --precision fp16 and omits --no-quant. FP32 contains 129 semantic nodes with 86 FLOAT and 13 INT64 initializers; FP16 contains the same semantic graph, 86 FLOAT16 and 13 INT64 initializers, plus two public-boundary Cast nodes. FP16 halves external weight storage and lowers measured RAM delta, but its CPU p50 is slower than FP32; no FP16 speedup is claimed.

L2 parity tuple	Cosine	Max absolute error	Mean absolute error	Finite	Argmax
PyTorch vs FP32	0.999999999946133	0.00022792816162109375	0.00009134262800216674	yes	class 8 mr-IN, equal
PyTorch vs FP16	0.9999997509129488	0.009529590606689453	0.004625663161277771	yes	class 8 mr-IN, equal

Functional smoke Eval

FP32 on CPU passed against google/fleurs@70bb2e84b976b7e960aa89f1c648e09c59f894dd, validation split. Selection requested one deterministic first eligible row from each of nine exact locale configs: 9 requested, 9 eligible, 9 selected, 9 processed, and 0 rejected. The run processed every consecutive 16,000-sample window, right-padded only each final window, and averaged logits once per utterance: 133 windows total, 7-27 windows per utterance, and one prediction per utterance.

The exact checkpoint logit order is en-IN, hi-IN, or-IN, bn-IN, ta-IN, te-IN, kn-IN, ml-IN, mr-IN, gu-IN. Targets resolve from scalar FLEURS ClassLabel values through these exact semantics:

FLEURS config	Checkpoint class	Result
hi_in	1 hi-IN	predicted hi-IN
or_in	2 or-IN	predicted bn-IN
bn_in	3 bn-IN	predicted bn-IN
ta_in	4 ta-IN	predicted ta-IN
te_in	5 te-IN	predicted bn-IN
kn_in	6 kn-IN	predicted kn-IN
ml_in	7 ml-IN	predicted kn-IN
mr_in	8 mr-IN	predicted gu-IN
gu_in	9 gu-IN	predicted gu-IN

Schema, label semantics, and prediction semantics were verified. Accuracy was 0.5555555555555556; represented-class macro-F1 was 0.4259259259259259. This is 9/10 checkpoint-class coverage: FLEURS en_us was explicitly rejected because US English is not an authoritative mapping to checkpoint class en-IN. These bounded nine-utterance metrics prove end-to-end operability only; they are not representative accuracy, checkpoint quality, en-IN coverage, full language coverage, or a benchmark claim.

Metric provenance: tester successor evidence is bound to candidate 1a64af009e4737fb2a172da4171966a6c93d5e2a and dependency 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8 using pinned model onecxi/open-vakgyata@f2754058e485dfc65cc62589b8d0e21c1d328399 and google/fleurs@70bb2e84b976b7e960aa89f1c648e09c59f894dd validation data. Selection was the deterministic first eligible row for each of nine exact mapped configs; all 133 windows were evaluated, yielding exact 9/10 checkpoint-class coverage.

5. Delta

Relative to the exact #1326 base, each existing recipe adds one field and is otherwise structurally identical to current generated configuration:

Recipe	JSON pointer	Old value	New value
FP32	/export/compatibility/transformers_attention	missing	eager
FP16	/export/compatibility/transformers_attention	missing	eager

The loader task/class/model type, opset 17, float32 [1,16000] input, logits output, quantization choices, FP16 public-IO preservation, and null compile stage remain unchanged. The generator already derives eager attention, so no missing class-wide source rule exists to implement and no recipe-free acceptance run is required. There is no code bug fix in this change. examples/recipes/README.md remains untouched.

6. Analyze summary - component level and op level

Static analysis completed with ANALYZE-PARTIAL-SUCCESS for both artifacts (exit code 1 because some EPs have no rule data). This is static rule analysis, not runtime execution evidence.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
FP32	7-layer feature encoder; projection; positional convolution; 2 Transformer+adapter layers; normalization; projector; mean pool; classifier	129 mapped, 0 unmapped, 0 boundary Casts; confidence mapped	none
FP16	same semantic regions	129 mapped, 0 unmapped, 2 boundary Casts; confidence mapped	none

Op-level summary

Artifact	Graph	Dominant operators	Populated-rule roll-up
FP32	129 ops / 14 types	Reshape 32; Transpose 23; Gemm 19; LayerNormalization 15; Gelu 10; Add 9	all present types statically supported for NvTensorRTRTX GPU, QNN NPU/GPU, and OpenVINO NPU/GPU/CPU
FP16	131 ops / 15 types	same counts plus Cast 2	same populated-rule result

CUDA GPU, MIGraphX GPU, TensorRT GPU, DML GPU, CPUExecutionProvider CPU, and VitisAI NPU have no static rule data. No runtime support claim is inferred from these classifications.

7. Reproduce commands

Set $OUT to a new output root, then run rows 0-16 in ascending order in one PowerShell session. These are the tester-owned direct portable rows; prerequisite acquisition and all four exact helper-generation steps are included inline, and no scratch wrapper is required.

Public sequence closure

Tier	Tuple	Public command rows	Contract	Status
Prerequisites	repo/ref/dependency/environment/rules/model/dataset/mapping/providers	0, 1, 2, 3, 4	-	CLOSED
L0	CPUExecutionProvider/cpu/fp32 structure and realized precision	5, 7	-	CLOSED
L0	CPUExecutionProvider/cpu/fp16 structure and realized precision	6, 7	requires --precision fp16; forbids --no-quant	CLOSED
L1	CPUExecutionProvider/cpu/fp32	8	-	CLOSED
L1	CPUExecutionProvider/cpu/fp16	9	-	CLOSED
L2	identical named input/PyTorch/FP32/FP16	10	-	CLOSED
L3	CPUExecutionProvider/cpu/fp32 exact 9 classes and 133 windows	11	-	CLOSED
Analyze	fp32 artifact/all requested EPs	12	accepted exits 0, 1 with nonempty parsed JSON	CLOSED
Analyze	fp16 artifact/all requested EPs	13	accepted exits 0, 1 with nonempty parsed JSON	CLOSED
Quality	ruff/mypy/focused pytest/final clean checkout	14, 15, 16	-	CLOSED

17 tester-owned public command rows

`powershell

Row 0 - repo-ref-bootstrap

$ErrorActionPreference='Stop'; if (-not $OUT) { throw 'Set $OUT to a new output root' }; $OUT=[IO.Path]::GetFullPath($OUT); if (Test-Path -LiteralPath $OUT) { throw 'Immutable output root exists' }; New-Item -ItemType Directory -Path $OUT|Out-Null; @('public','assets','cache','artifacts','analysis','quality','parity','eval')|ForEach-Object{New-Item -ItemType Directory -Path (Join-Path $OUT $_)|Out-Null}; $PUBLIC=Join-Path $OUT 'public'; $CHECKOUT=Join-Path $OUT 'checkout'; $MODEL=Join-Path $OUT 'assets/model'; $RULESZIP=Join-Path $OUT 'assets/rules-v0.3.0.zip'; $RULES=Join-Path $OUT 'assets/rules'; $FP32=Join-Path $OUT 'artifacts/fp32'; $FP16=Join-Path $OUT 'artifacts/fp16'; $env:HF_HOME=Join-Path $OUT 'cache/huggingface'; $env:HF_DATASETS_CACHE=Join-Path $OUT 'cache/datasets'; $env:TRANSFORMERS_CACHE=Join-Path $OUT 'cache/transformers'; $env:WINMLCLI_RULES_DIR=$RULES; Remove-Item Env:PYTHONPATH -ErrorAction SilentlyContinue; git init $CHECKOUT; git -C $CHECKOUT remote add origin https://github.com/microsoft/winml-cli.git; git -C $CHECKOUT fetch --no-tags --depth 1 origin refs/pull/1326/head:refs/remotes/origin/pr-1326; git -C $CHECKOUT fetch --no-tags --depth 2 origin refs/pull/1332/head:refs/remotes/origin/pr-1332; git -C $CHECKOUT checkout --detach 1a64af0; if ((git -C $CHECKOUT rev-parse HEAD).Trim() -ne '1a64af009e4737fb2a172da4171966a6c93d5e2a') { throw 'Candidate mismatch' }; if ((git -C $CHECKOUT rev-parse HEAD^).Trim() -ne '1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8') { throw 'Parent mismatch' }; if ((git -C $CHECKOUT rev-parse refs/remotes/origin/pr-1326).Trim() -ne '1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8') { throw 'Dependency mismatch' }; $changed=@(git -C $CHECKOUT diff --name-only HEAD^..HEAD); $expected=@('examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp16_config.json','examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp32_config.json'); if ((Compare-Object $changed $expected).Count -ne 0) { throw 'Diff scope mismatch' }; if (git -C $CHECKOUT status --porcelain) { throw 'Checkout dirty' }

Row 1 - locked-environment

Push-Location $CHECKOUT; try { uv sync --locked --all-extras --all-groups; if ($LASTEXITCODE -ne 0) { throw 'uv sync failed' } } finally { Pop-Location }

Row 2 - rules-acquisition

Invoke-WebRequest -Uri 'https://github.com/microsoft/winml-cli/releases/download/v0.3.0/rules-v0.3.0.zip' -OutFile $RULESZIP; if ((Get-FileHash -LiteralPath $RULESZIP -Algorithm SHA256).Hash.ToLowerInvariant() -ne '32030e3169a837b84798d6e321e07f418416ec3d3070d9af049385bf619a8b1d') { throw 'Rules hash mismatch' }; Expand-Archive -LiteralPath $RULESZIP -DestinationPath $RULES; if (@(Get-ChildItem -LiteralPath $RULES -Recurse -Filter *.parquet).Count -ne 2110) { throw 'Rules count mismatch' }

Row 3 - model-dataset-mapping-acquisition

$helper=Join-Path $PUBLIC 'acquire_inputs.py'; [IO.File]::WriteAllBytes($helper,[Convert]::FromBase64String('aW1wb3J0IGFyZ3BhcnNlDQppbXBvcnQgaGFzaGxpYg0KaW1wb3J0IGpzb24NCmZyb20gcGF0aGxpYiBpbXBvcnQgUGF0aA0KDQpmcm9tIGRhdGFzZXRzIGltcG9ydCBBdWRpbywgQ2xhc3NMYWJlbCwgbG9hZF9kYXRhc2V0DQpmcm9tIGh1Z2dpbmdmYWNlX2h1YiBpbXBvcnQgSGZBcGksIHNuYXBzaG90X2Rvd25sb2FkDQpmcm9tIHRyYW5zZm9ybWVycyBpbXBvcnQgQXV0b0NvbmZpZw0KDQoNCk1PREVMX0lEID0gIm9uZWN4aS9vcGVuLXZha2d5YXRhIg0KREFUQVNFVF9JRCA9ICJnb29nbGUvZmxldXJzIg0KREFUQVNFVF9DT05GSUdfVE9fTU9ERUxfSUQgPSB7DQogICAgImhpX2luIjogMSwNCiAgICAib3JfaW4iOiAyLA0KICAgICJibl9pbiI6IDMsDQogICAgInRhX2luIjogNCwNCiAgICAidGVfaW4iOiA1LA0KICAgICJrbl9pbiI6IDYsDQogICAgIm1sX2luIjogNywNCiAgICAibXJfaW4iOiA4LA0KICAgICJndV9pbiI6IDksDQp9DQpFWFBFQ1RFRF9MQUJFTFMgPSB7DQogICAgMDogImVuLUlOIiwNCiAgICAxOiAiaGktSU4iLA0KICAgIDI6ICJvci1JTiIsDQogICAgMzogImJuLUlOIiwNCiAgICA0OiAidGEtSU4iLA0KICAgIDU6ICJ0ZS1JTiIsDQogICAgNjogImtuLUlOIiwNCiAgICA3OiAibWwtSU4iLA0KICAgIDg6ICJtci1JTiIsDQogICAgOTogImd1LUlOIiwNCn0NCg0KDQpkZWYgc2hhMjU2KHBhdGg6IFBhdGgpIC0+IHN0cjoNCiAgICBkaWdlc3QgPSBoYXNobGliLnNoYTI1NigpDQogICAgd2l0aCBwYXRoLm9wZW4oInJiIikgYXMgc3RyZWFtOg0KICAgICAgICBmb3IgY2h1bmsgaW4gaXRlcihsYW1iZGE6IHN0cmVhbS5yZWFkKDEwMjQgKiAxMDI0KSwgYiIiKToNCiAgICAgICAgICAgIGRpZ2VzdC51cGRhdGUoY2h1bmspDQogICAgcmV0dXJuIGRpZ2VzdC5oZXhkaWdlc3QoKQ0KDQoNCnBhcnNlciA9IGFyZ3BhcnNlLkFyZ3VtZW50UGFyc2VyKCkNCnBhcnNlci5hZGRfYXJndW1lbnQoIi0tbW9kZWwtZGlyIiwgdHlwZT1QYXRoLCByZXF1aXJlZD1UcnVlKQ0KcGFyc2VyLmFkZF9hcmd1bWVudCgiLS1tYW5pZmVzdCIsIHR5cGU9UGF0aCwgcmVxdWlyZWQ9VHJ1ZSkNCnBhcnNlci5hZGRfYXJndW1lbnQoIi0tbW9kZWwtcmV2aXNpb24iLCByZXF1aXJlZD1UcnVlKQ0KcGFyc2VyLmFkZF9hcmd1bWVudCgiLS1kYXRhc2V0LXJldmlzaW9uIiwgcmVxdWlyZWQ9VHJ1ZSkNCmFyZ3MgPSBwYXJzZXIucGFyc2VfYXJncygpDQppZiBhcmdzLm1hbmlmZXN0LmV4aXN0cygpIG9yIGFyZ3MubW9kZWxfZGlyLmV4aXN0cygpOg0KICAgIHJhaXNlIFN5c3RlbUV4aXQoImltbXV0YWJsZSBpbnB1dCBhY3F1aXNpdGlvbiBjb2xsaXNpb24iKQ0KDQphcGkgPSBIZkFwaSgpDQptb2RlbF9pbmZvID0gYXBpLm1vZGVsX2luZm8oTU9ERUxfSUQsIHJldmlzaW9uPWFyZ3MubW9kZWxfcmV2aXNpb24pDQpkYXRhc2V0X2luZm8gPSBhcGkuZGF0YXNldF9pbmZvKERBVEFTRVRfSUQsIHJldmlzaW9uPWFyZ3MuZGF0YXNldF9yZXZpc2lvbikNCmlmIG1vZGVsX2luZm8uc2hhICE9IGFyZ3MubW9kZWxfcmV2aXNpb246DQogICAgcmFpc2UgUnVudGltZUVycm9yKGYibW9kZWwgcmV2aXNpb24gbWlzbWF0Y2g6IHttb2RlbF9pbmZvLnNoYX0iKQ0KaWYgZGF0YXNldF9pbmZvLnNoYSAhPSBhcmdzLmRhdGFzZXRfcmV2aXNpb246DQogICAgcmFpc2UgUnVudGltZUVycm9yKGYiZGF0YXNldCByZXZpc2lvbiBtaXNtYXRjaDoge2RhdGFzZXRfaW5mby5zaGF9IikNCg0Kc25hcHNob3RfZG93bmxvYWQoDQogICAgcmVwb19pZD1NT0RFTF9JRCwNCiAgICByZXZpc2lvbj1hcmdzLm1vZGVsX3JldmlzaW9uLA0KICAgIGxvY2FsX2Rpcj1hcmdzLm1vZGVsX2RpciwNCiAgICBhbGxvd19wYXR0ZXJucz1bImNvbmZpZy5qc29uIiwgInByZXByb2Nlc3Nvcl9jb25maWcuanNvbiIsICJtb2RlbC5zYWZldGVuc29ycyIsICJSRUFETUUubWQiLCAiLmdpdGF0dHJpYnV0ZXMiXSwNCikNCmNvbmZpZyA9IEF1dG9Db25maWcuZnJvbV9wcmV0cmFpbmVkKGFyZ3MubW9kZWxfZGlyLCBsb2NhbF9maWxlc19vbmx5PVRydWUpDQppZDJsYWJlbCA9IHtpbnQoa2V5KTogc3RyKHZhbHVlKSBmb3Iga2V5LCB2YWx1ZSBpbiBjb25maWcuaWQybGFiZWwuaXRlbXMoKX0NCmlmIGlkMmxhYmVsICE9IEVYUEVDVEVEX0xBQkVMUzoNCiAgICByYWlzZSBSdW50aW1lRXJyb3IoZiJjaGVja3BvaW50IGxhYmVsIGRyaWZ0OiB7aWQybGFiZWx9IikNCg0KZGF0YXNldF9wcm9iZXMgPSB7fQ0KZm9yIGRhdGFzZXRfY29uZmlnLCBtb2RlbF9pZCBpbiBEQVRBU0VUX0NPTkZJR19UT19NT0RFTF9JRC5pdGVtcygpOg0KICAgIGRhdGFzZXQgPSBsb2FkX2RhdGFzZXQoDQogICAgICAgIERBVEFTRVRfSUQsDQogICAgICAgIG5hbWU9ZGF0YXNldF9jb25maWcsDQogICAgICAgIHNwbGl0PSJ2YWxpZGF0aW9uIiwNCiAgICAgICAgc3RyZWFtaW5nPVRydWUsDQogICAgICAgIHJldmlzaW9uPWFyZ3MuZGF0YXNldF9yZXZpc2lvbiwNCiAgICApDQogICAgbGFiZWxfZmVhdHVyZSA9IGRhdGFzZXQuZmVhdHVyZXNbImxhbmdfaWQiXQ0KICAgIGF1ZGlvX2ZlYXR1cmUgPSBkYXRhc2V0LmZlYXR1cmVzWyJhdWRpbyJdDQogICAgaWYgbm90IGlzaW5zdGFuY2UobGFiZWxfZmVhdHVyZSwgQ2xhc3NMYWJlbCkgb3Igbm90IGlzaW5zdGFuY2UoYXVkaW9fZmVhdHVyZSwgQXVkaW8pOg0KICAgICAgICByYWlzZSBSdW50aW1lRXJyb3IoZiJ1bmV4cGVjdGVkIGRhdGFzZXQgc2NoZW1hIGZvciB7ZGF0YXNldF9jb25maWd9IikNCiAgICByb3cgPSBuZXh0KGl0ZXIoZGF0YXNldC5jYXN0X2NvbHVtbigiYXVkaW8iLCBBdWRpbyhzYW1wbGluZ19yYXRlPWF1ZGlvX2ZlYXR1cmUuc2FtcGxpbmdfcmF0ZSwgZGVjb2RlPUZhbHNlKSkpKQ0KICAgIGRhdGFzZXRfbGFiZWwgPSBsYWJlbF9mZWF0dXJlLmludDJzdHIoaW50KHJvd1sibGFuZ19pZCJdKSkNCiAgICBpZiBkYXRhc2V0X2xhYmVsICE9IGRhdGFzZXRfY29uZmlnIG9yIGlkMmxhYmVsW21vZGVsX2lkXS5sb3dlcigpLnJlcGxhY2UoIi0iLCAiXyIpICE9IGRhdGFzZXRfY29uZmlnOg0KICAgICAgICByYWlzZSBSdW50aW1lRXJyb3IoZiJtYXBwaW5nIG1pc21hdGNoIGZvciB7ZGF0YXNldF9jb25maWd9IikNCiAgICByYXdfYXVkaW8gPSByb3dbImF1ZGlvIl0NCiAgICBpZiByYXdfYXVkaW8uZ2V0KCJieXRlcyIpIGlzIE5vbmUgYW5kIHJhd19hdWRpby5nZXQoInBhdGgiKSBpcyBOb25lOg0KICAgICAgICByYWlzZSBSdW50aW1lRXJyb3IoZiJubyByYXcgbWVkaWEgZm9yIHtkYXRhc2V0X2NvbmZpZ30iKQ0KICAgIGRhdGFzZXRfcHJvYmVzW2RhdGFzZXRfY29uZmlnXSA9IHsNCiAgICAgICAgInJvd19pZCI6IHJvdy5nZXQoImlkIiksDQogICAgICAgICJkYXRhc2V0X2xhYmVsIjogZGF0YXNldF9sYWJlbCwNCiAgICAgICAgImNoZWNrcG9pbnRfaWQiOiBtb2RlbF9pZCwNCiAgICAgICAgImNoZWNrcG9pbnRfbGFiZWwiOiBpZDJsYWJlbFttb2RlbF9pZF0sDQogICAgICAgICJlbmNvZGVkX2J5dGVzIjogbGVuKHJhd19hdWRpb1siYnl0ZXMiXSkgaWYgcmF3X2F1ZGlvLmdldCgiYnl0ZXMiKSBpcyBub3QgTm9uZSBlbHNlIE5vbmUsDQogICAgfQ0KDQppZiAiZW5fdXMiIGluIERBVEFTRVRfQ09ORklHX1RPX01PREVMX0lEOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiZW5fdXMgbXVzdCByZW1haW4gZXhjbHVkZWQiKQ0KDQpmaWxlcyA9IFtdDQpmb3IgcGF0aCBpbiBzb3J0ZWQoaXRlbSBmb3IgaXRlbSBpbiBhcmdzLm1vZGVsX2Rpci5yZ2xvYigiKiIpIGlmIGl0ZW0uaXNfZmlsZSgpIGFuZCAiLmNhY2hlIiBub3QgaW4gaXRlbS5wYXJ0cyk6DQogICAgZmlsZXMuYXBwZW5kKHsicGF0aCI6IHBhdGgucmVsYXRpdmVfdG8oYXJncy5tb2RlbF9kaXIpLmFzX3Bvc2l4KCksICJieXRlcyI6IHBhdGguc3RhdCgpLnN0X3NpemUsICJzaGEyNTYiOiBzaGEyNTYocGF0aCl9KQ0KDQpwYXlsb2FkID0gew0KICAgICJtb2RlbCI6IHsiaWQiOiBNT0RFTF9JRCwgInJldmlzaW9uIjogYXJncy5tb2RlbF9yZXZpc2lvbiwgInJlc29sdmVkX3NoYSI6IG1vZGVsX2luZm8uc2hhLCAiZmlsZXMiOiBmaWxlc30sDQogICAgImRhdGFzZXQiOiB7DQogICAgICAgICJpZCI6IERBVEFTRVRfSUQsDQogICAgICAgICJyZXZpc2lvbiI6IGFyZ3MuZGF0YXNldF9yZXZpc2lvbiwNCiAgICAgICAgInJlc29sdmVkX3NoYSI6IGRhdGFzZXRfaW5mby5zaGEsDQogICAgICAgICJzcGxpdCI6ICJ2YWxpZGF0aW9uIiwNCiAgICAgICAgInByb2JlcyI6IGRhdGFzZXRfcHJvYmVzLA0KICAgIH0sDQogICAgIm1hcHBpbmciOiB7c3RyKGtleSk6IHZhbHVlIGZvciBrZXksIHZhbHVlIGluIERBVEFTRVRfQ09ORklHX1RPX01PREVMX0lELml0ZW1zKCl9LA0KICAgICJjaGVja3BvaW50X2lkMmxhYmVsIjoge3N0cihrZXkpOiB2YWx1ZSBmb3Iga2V5LCB2YWx1ZSBpbiBpZDJsYWJlbC5pdGVtcygpfSwNCiAgICAiZXhjbHVkZWRfZGF0YXNldF9jb25maWdzIjogWyJlbl91cyJdLA0KICAgICJleGNsdWRlZF9jaGVja3BvaW50X2xhYmVscyI6IFsiZW4tSU4iXSwNCn0NCmFyZ3MubWFuaWZlc3QucGFyZW50Lm1rZGlyKHBhcmVudHM9VHJ1ZSwgZXhpc3Rfb2s9VHJ1ZSkNCmFyZ3MubWFuaWZlc3Qud3JpdGVfdGV4dChqc29uLmR1bXBzKHBheWxvYWQsIGluZGVudD0yKSArICJcbiIsIGVuY29kaW5nPSJ1dGYtOCIpDQpwcmludChqc29uLmR1bXBzKHBheWxvYWQsIGluZGVudD0yKSk=')); if ((Get-FileHash -LiteralPath $helper -Algorithm SHA256).Hash.ToLowerInvariant() -ne '7ed895e6e3056b5a8d59645d13975e6b2020161e8db43f46027ad22dd6244f39') { throw 'Helper hash mismatch: acquire_inputs.py' }; & (Join-Path $CHECKOUT '.venv/Scripts/python.exe') $helper --model-dir $MODEL --manifest (Join-Path $OUT 'assets/input_manifest.json') --model-revision 'f2754058e485dfc65cc62589b8d0e21c1d328399' --dataset-revision '70bb2e84b976b7e960aa89f1c648e09c59f894dd'; if ($LASTEXITCODE -ne 0) { throw 'Input acquisition failed' }

Row 4 - provider-snapshot

& (Join-Path $CHECKOUT '.venv/Scripts/python.exe') -c "import json,onnxruntime as ort;print(json.dumps(ort.get_available_providers()))"; if ($LASTEXITCODE -ne 0) { throw 'Provider probe failed' }

Row 5 - l0-fp32-build

& (Join-Path $CHECKOUT '.venv/Scripts/winml.exe') build -c (Join-Path $CHECKOUT 'examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp32_config.json') -m $MODEL -o $FP32 --rebuild --no-color; if ($LASTEXITCODE -ne 0) { throw 'FP32 build failed' }

Row 6 - l0-fp16-build

& (Join-Path $CHECKOUT '.venv/Scripts/winml.exe') build -c (Join-Path $CHECKOUT 'examples/recipes/onecxi_open-vakgyata/cpu/cpu/audio-classification_fp16_config.json') -m $MODEL -o $FP16 --rebuild --precision fp16 --no-color; if ($LASTEXITCODE -ne 0) { throw 'FP16 build failed' }

Row 7 - l0-structure-realized-precision

$helper=Join-Path $PUBLIC 'verify_artifact.py'; [IO.File]::WriteAllBytes($helper,[Convert]::FromBase64String('aW1wb3J0IGFyZ3BhcnNlDQppbXBvcnQgaGFzaGxpYg0KaW1wb3J0IGpzb24NCmZyb20gY29sbGVjdGlvbnMgaW1wb3J0IENvdW50ZXINCmZyb20gcGF0aGxpYiBpbXBvcnQgUGF0aA0KDQppbXBvcnQgb25ueA0KDQoNCmRlZiBzaGEyNTYocGF0aDogUGF0aCkgLT4gc3RyOg0KICAgIGRpZ2VzdCA9IGhhc2hsaWIuc2hhMjU2KCkNCiAgICB3aXRoIHBhdGgub3BlbigicmIiKSBhcyBzdHJlYW06DQogICAgICAgIGZvciBjaHVuayBpbiBpdGVyKGxhbWJkYTogc3RyZWFtLnJlYWQoMTAyNCAqIDEwMjQpLCBiIiIpOg0KICAgICAgICAgICAgZGlnZXN0LnVwZGF0ZShjaHVuaykNCiAgICByZXR1cm4gZGlnZXN0LmhleGRpZ2VzdCgpDQoNCg0KZGVmIHNoYXBlKHZhbHVlX2luZm86IG9ubnguVmFsdWVJbmZvUHJvdG8pIC0+IGxpc3RbaW50IHwgc3RyXToNCiAgICByZXR1cm4gW2RpbWVuc2lvbi5kaW1fdmFsdWUgb3IgZGltZW5zaW9uLmRpbV9wYXJhbSBmb3IgZGltZW5zaW9uIGluIHZhbHVlX2luZm8udHlwZS50ZW5zb3JfdHlwZS5zaGFwZS5kaW1dDQoNCg0KZGVmIGluc3BlY3QocGF0aDogUGF0aCkgLT4gZGljdFtzdHIsIG9iamVjdF06DQogICAgbW9kZWwgPSBvbm54LmxvYWQocGF0aCwgbG9hZF9leHRlcm5hbF9kYXRhPUZhbHNlKQ0KICAgIGV4dGVybmFsX2xvY2F0aW9ucyA9IHNvcnRlZCgNCiAgICAgICAgew0KICAgICAgICAgICAgaXRlbS52YWx1ZS5kZWNvZGUoKSBpZiBpc2luc3RhbmNlKGl0ZW0udmFsdWUsIGJ5dGVzKSBlbHNlIGl0ZW0udmFsdWUNCiAgICAgICAgICAgIGZvciBpbml0aWFsaXplciBpbiBtb2RlbC5ncmFwaC5pbml0aWFsaXplcg0KICAgICAgICAgICAgZm9yIGl0ZW0gaW4gaW5pdGlhbGl6ZXIuZXh0ZXJuYWxfZGF0YQ0KICAgICAgICAgICAgaWYgaXRlbS5rZXkgPT0gImxvY2F0aW9uIg0KICAgICAgICB9DQogICAgKQ0KICAgIGZvciBsb2NhdGlvbiBpbiBleHRlcm5hbF9sb2NhdGlvbnM6DQogICAgICAgIGlmIG5vdCAocGF0aC5wYXJlbnQgLyBsb2NhdGlvbikuaXNfZmlsZSgpOg0KICAgICAgICAgICAgcmFpc2UgUnVudGltZUVycm9yKGYibWlzc2luZyBleHRlcm5hbCBkYXRhIGJlc2lkZSB7cGF0aH06IHtsb2NhdGlvbn0iKQ0KICAgIHJldHVybiB7DQogICAgICAgICJwYXRoIjogc3RyKHBhdGgpLA0KICAgICAgICAiYnl0ZXMiOiBwYXRoLnN0YXQoKS5zdF9zaXplLA0KICAgICAgICAic2hhMjU2Ijogc2hhMjU2KHBhdGgpLA0KICAgICAgICAiZXh0ZXJuYWxfZGF0YSI6IFsNCiAgICAgICAgICAgIHsibmFtZSI6IGxvY2F0aW9uLCAiYnl0ZXMiOiAocGF0aC5wYXJlbnQgLyBsb2NhdGlvbikuc3RhdCgpLnN0X3NpemUsICJzaGEyNTYiOiBzaGEyNTYocGF0aC5wYXJlbnQgLyBsb2NhdGlvbil9DQogICAgICAgICAgICBmb3IgbG9jYXRpb24gaW4gZXh0ZXJuYWxfbG9jYXRpb25zDQogICAgICAgIF0sDQogICAgICAgICJpcl92ZXJzaW9uIjogbW9kZWwuaXJfdmVyc2lvbiwNCiAgICAgICAgIm9wc2V0IjogbWF4KGl0ZW0udmVyc2lvbiBmb3IgaXRlbSBpbiBtb2RlbC5vcHNldF9pbXBvcnQpLA0KICAgICAgICAiaW5wdXRzIjogW3sibmFtZSI6IGl0ZW0ubmFtZSwgInNoYXBlIjogc2hhcGUoaXRlbSksICJkdHlwZSI6IG9ubnguVGVuc29yUHJvdG8uRGF0YVR5cGUuTmFtZShpdGVtLnR5cGUudGVuc29yX3R5cGUuZWxlbV90eXBlKX0gZm9yIGl0ZW0gaW4gbW9kZWwuZ3JhcGguaW5wdXRdLA0KICAgICAgICAib3V0cHV0cyI6IFt7Im5hbWUiOiBpdGVtLm5hbWUsICJzaGFwZSI6IHNoYXBlKGl0ZW0pLCAiZHR5cGUiOiBvbm54LlRlbnNvclByb3RvLkRhdGFUeXBlLk5hbWUoaXRlbS50eXBlLnRlbnNvcl90eXBlLmVsZW1fdHlwZSl9IGZvciBpdGVtIGluIG1vZGVsLmdyYXBoLm91dHB1dF0sDQogICAgICAgICJpbml0aWFsaXplcl90eXBlcyI6IGRpY3QoQ291bnRlcihvbm54LlRlbnNvclByb3RvLkRhdGFUeXBlLk5hbWUoaXRlbS5kYXRhX3R5cGUpIGZvciBpdGVtIGluIG1vZGVsLmdyYXBoLmluaXRpYWxpemVyKSksDQogICAgICAgICJub2RlX2NvdW50IjogbGVuKG1vZGVsLmdyYXBoLm5vZGUpLA0KICAgICAgICAib3BlcmF0b3JfY291bnRzIjogZGljdChDb3VudGVyKGl0ZW0ub3BfdHlwZSBmb3IgaXRlbSBpbiBtb2RlbC5ncmFwaC5ub2RlKSksDQogICAgICAgICJjYXN0cyI6IFtpdGVtLm5hbWUgZm9yIGl0ZW0gaW4gbW9kZWwuZ3JhcGgubm9kZSBpZiBpdGVtLm9wX3R5cGUgPT0gIkNhc3QiXSwNCiAgICB9DQoNCg0KcGFyc2VyID0gYXJncGFyc2UuQXJndW1lbnRQYXJzZXIoKQ0KcGFyc2VyLmFkZF9hcmd1bWVudCgiLS1mcDMyIiwgdHlwZT1QYXRoLCByZXF1aXJlZD1UcnVlKQ0KcGFyc2VyLmFkZF9hcmd1bWVudCgiLS1mcDE2IiwgdHlwZT1QYXRoLCByZXF1aXJlZD1UcnVlKQ0KcGFyc2VyLmFkZF9hcmd1bWVudCgiLS1vdXRwdXQiLCB0eXBlPVBhdGgsIHJlcXVpcmVkPVRydWUpDQphcmdzID0gcGFyc2VyLnBhcnNlX2FyZ3MoKQ0KaWYgYXJncy5vdXRwdXQuZXhpc3RzKCk6DQogICAgcmFpc2UgU3lzdGVtRXhpdCgiaW1tdXRhYmxlIHN0cnVjdHVyZSBldmlkZW5jZSBjb2xsaXNpb24iKQ0KDQpwYXlsb2FkID0geyJmcDMyIjogaW5zcGVjdChhcmdzLmZwMzIpLCAiZnAxNiI6IGluc3BlY3QoYXJncy5mcDE2KX0NCmV4cGVjdGVkX2lucHV0cyA9IFt7Im5hbWUiOiAiaW5wdXRfdmFsdWVzIiwgInNoYXBlIjogWzEsIDE2MDAwXSwgImR0eXBlIjogIkZMT0FUIn1dDQpleHBlY3RlZF9vdXRwdXRzID0gW3sibmFtZSI6ICJsb2dpdHMiLCAic2hhcGUiOiBbMSwgMTBdLCAiZHR5cGUiOiAiRkxPQVQifV0NCmZvciBwcmVjaXNpb24sIGFydGlmYWN0IGluIHBheWxvYWQuaXRlbXMoKToNCiAgICBpZiBhcnRpZmFjdFsiaXJfdmVyc2lvbiJdICE9IDggb3IgYXJ0aWZhY3RbIm9wc2V0Il0gIT0gMTc6DQogICAgICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmInVuZXhwZWN0ZWQge3ByZWNpc2lvbn0gSVIvb3BzZXQiKQ0KICAgIGlmIGFydGlmYWN0WyJpbnB1dHMiXSAhPSBleHBlY3RlZF9pbnB1dHMgb3IgYXJ0aWZhY3RbIm91dHB1dHMiXSAhPSBleHBlY3RlZF9vdXRwdXRzOg0KICAgICAgICByYWlzZSBSdW50aW1lRXJyb3IoZiJ1bmV4cGVjdGVkIHtwcmVjaXNpb259IHB1YmxpYyBJTyIpDQppZiBwYXlsb2FkWyJmcDMyIl1bImluaXRpYWxpemVyX3R5cGVzIl0uZ2V0KCJGTE9BVCIpICE9IDg2Og0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiRlAzMiBpbml0aWFsaXplciByZWFsaXphdGlvbiBtaXNtYXRjaCIpDQppZiBwYXlsb2FkWyJmcDE2Il1bImluaXRpYWxpemVyX3R5cGVzIl0uZ2V0KCJGTE9BVDE2IikgIT0gODYgb3IgcGF5bG9hZFsiZnAxNiJdWyJpbml0aWFsaXplcl90eXBlcyJdLmdldCgiRkxPQVQiLCAwKSAhPSAwOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiRlAxNiBpbml0aWFsaXplciByZWFsaXphdGlvbiBtaXNtYXRjaCIpDQpmcDMyX2V4dGVybmFsID0gc3VtKGl0ZW1bImJ5dGVzIl0gZm9yIGl0ZW0gaW4gcGF5bG9hZFsiZnAzMiJdWyJleHRlcm5hbF9kYXRhIl0pDQpmcDE2X2V4dGVybmFsID0gc3VtKGl0ZW1bImJ5dGVzIl0gZm9yIGl0ZW0gaW4gcGF5bG9hZFsiZnAxNiJdWyJleHRlcm5hbF9kYXRhIl0pDQppZiBub3QgKDAuNDkgPD0gZnAxNl9leHRlcm5hbCAvIGZwMzJfZXh0ZXJuYWwgPD0gMC41MSk6DQogICAgcmFpc2UgUnVudGltZUVycm9yKCJGUDE2IGV4dGVybmFsIGRhdGEgaXMgbm90IGFwcHJveGltYXRlbHkgaGFsZiBGUDMyIikNCmlmIHBheWxvYWRbImZwMTYiXVsibm9kZV9jb3VudCJdICE9IHBheWxvYWRbImZwMzIiXVsibm9kZV9jb3VudCJdICsgMiBvciBsZW4ocGF5bG9hZFsiZnAxNiJdWyJjYXN0cyJdKSAhPSAyOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiRlAxNiBib3VuZGFyeS1jYXN0IHN0cnVjdHVyZSBtaXNtYXRjaCIpDQpwYXlsb2FkWyJ2ZXJkaWN0Il0gPSAiUEFTUyINCmFyZ3Mub3V0cHV0LnBhcmVudC5ta2RpcihwYXJlbnRzPVRydWUsIGV4aXN0X29rPVRydWUpDQphcmdzLm91dHB1dC53cml0ZV90ZXh0KGpzb24uZHVtcHMocGF5bG9hZCwgaW5kZW50PTIpICsgIlxuIiwgZW5jb2Rpbmc9InV0Zi04IikNCnByaW50KGpzb24uZHVtcHMocGF5bG9hZCwgaW5kZW50PTIpKQ==')); if ((Get-FileHash -LiteralPath $helper -Algorithm SHA256).Hash.ToLowerInvariant() -ne '0b2dd3f8f209e30fbad760aa44399477691f6aa545763788e22fd214ec93b4c9') { throw 'Helper hash mismatch: verify_artifact.py' }; & (Join-Path $CHECKOUT '.venv/Scripts/python.exe') $helper --fp32 (Join-Path $FP32 'model.onnx') --fp16 (Join-Path $FP16 'model.onnx') --output (Join-Path $OUT 'analysis/structure.json'); if ($LASTEXITCODE -ne 0) { throw 'Structure verification failed' }

Row 8 - l1-fp32-perf

& (Join-Path $CHECKOUT '.venv/Scripts/winml.exe') perf -m (Join-Path $FP32 'model.onnx') --device cpu --ep cpu --warmup 5 --iterations 20 --memory --format json --output (Join-Path $OUT 'quality/fp32-perf.json') --no-color; if ($LASTEXITCODE -ne 0) { throw 'FP32 perf failed' }

Row 9 - l1-fp16-perf

& (Join-Path $CHECKOUT '.venv/Scripts/winml.exe') perf -m (Join-Path $FP16 'model.onnx') --device cpu --ep cpu --warmup 5 --iterations 20 --memory --format json --output (Join-Path $OUT 'quality/fp16-perf.json') --no-color; if ($LASTEXITCODE -ne 0) { throw 'FP16 perf failed' }

Row 10 - l2-identical-input-parity

$helper=Join-Path $PUBLIC 'run_parity.py'; [IO.File]::WriteAllBytes($helper,[Convert]::FromBase64String('aW1wb3J0IGFyZ3BhcnNlDQppbXBvcnQgaGFzaGxpYg0KaW1wb3J0IGpzb24NCmZyb20gcGF0aGxpYiBpbXBvcnQgUGF0aA0KDQppbXBvcnQgbnVtcHkgYXMgbnANCmltcG9ydCBvbm54cnVudGltZSBhcyBvcnQNCmltcG9ydCB0b3JjaA0KZnJvbSB0cmFuc2Zvcm1lcnMgaW1wb3J0IEF1dG9Nb2RlbEZvckF1ZGlvQ2xhc3NpZmljYXRpb24NCg0KDQpFWFBFQ1RFRF9MQUJFTFMgPSB7MDogImVuLUlOIiwgMTogImhpLUlOIiwgMjogIm9yLUlOIiwgMzogImJuLUlOIiwgNDogInRhLUlOIiwgNTogInRlLUlOIiwgNjogImtuLUlOIiwgNzogIm1sLUlOIiwgODogIm1yLUlOIiwgOTogImd1LUlOIn0NCg0KDQpkZWYgbWV0cmljcyhyZWZlcmVuY2U6IG5wLm5kYXJyYXksIGNhbmRpZGF0ZTogbnAubmRhcnJheSkgLT4gZGljdFtzdHIsIG9iamVjdF06DQogICAgcmVmZXJlbmNlNjQgPSByZWZlcmVuY2UuYXN0eXBlKG5wLmZsb2F0NjQpLnJhdmVsKCkNCiAgICBjYW5kaWRhdGU2NCA9IGNhbmRpZGF0ZS5hc3R5cGUobnAuZmxvYXQ2NCkucmF2ZWwoKQ0KICAgIHJldHVybiB7DQogICAgICAgICJyZWZlcmVuY2VfbG9naXRzIjogcmVmZXJlbmNlLnRvbGlzdCgpLA0KICAgICAgICAiY2FuZGlkYXRlX2xvZ2l0cyI6IGNhbmRpZGF0ZS50b2xpc3QoKSwNCiAgICAgICAgImNvc2luZSI6IGZsb2F0KG5wLmRvdChyZWZlcmVuY2U2NCwgY2FuZGlkYXRlNjQpIC8gKG5wLmxpbmFsZy5ub3JtKHJlZmVyZW5jZTY0KSAqIG5wLmxpbmFsZy5ub3JtKGNhbmRpZGF0ZTY0KSkpLA0KICAgICAgICAibWF4X2FicyI6IGZsb2F0KG5wLm1heChucC5hYnMocmVmZXJlbmNlNjQgLSBjYW5kaWRhdGU2NCkpKSwNCiAgICAgICAgIm1lYW5fYWJzIjogZmxvYXQobnAubWVhbihucC5hYnMocmVmZXJlbmNlNjQgLSBjYW5kaWRhdGU2NCkpKSwNCiAgICAgICAgImFyZ21heF9yZWZlcmVuY2UiOiBpbnQobnAuYXJnbWF4KHJlZmVyZW5jZTY0KSksDQogICAgICAgICJhcmdtYXhfY2FuZGlkYXRlIjogaW50KG5wLmFyZ21heChjYW5kaWRhdGU2NCkpLA0KICAgICAgICAiYXJnbWF4X2VxdWFsIjogaW50KG5wLmFyZ21heChyZWZlcmVuY2U2NCkpID09IGludChucC5hcmdtYXgoY2FuZGlkYXRlNjQpKSwNCiAgICAgICAgImZpbml0ZSI6IGJvb2wobnAuaXNmaW5pdGUoY2FuZGlkYXRlNjQpLmFsbCgpKSwNCiAgICB9DQoNCg0KcGFyc2VyID0gYXJncGFyc2UuQXJndW1lbnRQYXJzZXIoKQ0KcGFyc2VyLmFkZF9hcmd1bWVudCgiLS1tb2RlbC1kaXIiLCB0eXBlPVBhdGgsIHJlcXVpcmVkPVRydWUpDQpwYXJzZXIuYWRkX2FyZ3VtZW50KCItLWZwMzIiLCB0eXBlPVBhdGgsIHJlcXVpcmVkPVRydWUpDQpwYXJzZXIuYWRkX2FyZ3VtZW50KCItLWZwMTYiLCB0eXBlPVBhdGgsIHJlcXVpcmVkPVRydWUpDQpwYXJzZXIuYWRkX2FyZ3VtZW50KCItLW91dHB1dCIsIHR5cGU9UGF0aCwgcmVxdWlyZWQ9VHJ1ZSkNCmFyZ3MgPSBwYXJzZXIucGFyc2VfYXJncygpDQppZiBhcmdzLm91dHB1dC5leGlzdHMoKToNCiAgICByYWlzZSBTeXN0ZW1FeGl0KCJpbW11dGFibGUgcGFyaXR5IGV2aWRlbmNlIGNvbGxpc2lvbiIpDQoNCnNhbXBsZV9pbmRleCA9IG5wLmFyYW5nZSgxNjAwMCwgZHR5cGU9bnAuZmxvYXQzMikNCm5hbWVkX2lucHV0ID0gKG5wLmZsb2F0MzIoMC4yKSAqIG5wLnNpbihucC5mbG9hdDMyKDIgKiBucC5waSAqIDQ0MCAvIDE2MDAwKSAqIHNhbXBsZV9pbmRleCkgKyBucC5mbG9hdDMyKDAuMDUpICogbnAuc2luKG5wLmZsb2F0MzIoMiAqIG5wLnBpICogODgwIC8gMTYwMDApICogc2FtcGxlX2luZGV4KSkuYXN0eXBlKG5wLmZsb2F0MzIpLnJlc2hhcGUoMSwgMTYwMDApDQptb2RlbCA9IEF1dG9Nb2RlbEZvckF1ZGlvQ2xhc3NpZmljYXRpb24uZnJvbV9wcmV0cmFpbmVkKGFyZ3MubW9kZWxfZGlyLCBsb2NhbF9maWxlc19vbmx5PVRydWUsIGF0dG5faW1wbGVtZW50YXRpb249ImVhZ2VyIikuZXZhbCgpDQppZDJsYWJlbCA9IHtpbnQoa2V5KTogc3RyKHZhbHVlKSBmb3Iga2V5LCB2YWx1ZSBpbiBtb2RlbC5jb25maWcuaWQybGFiZWwuaXRlbXMoKX0NCmlmIGlkMmxhYmVsICE9IEVYUEVDVEVEX0xBQkVMUzoNCiAgICByYWlzZSBSdW50aW1lRXJyb3IoZiJjaGVja3BvaW50IGxhYmVsIGRyaWZ0OiB7aWQybGFiZWx9IikNCndpdGggdG9yY2gubm9fZ3JhZCgpOg0KICAgIHJlZmVyZW5jZSA9IG1vZGVsKGlucHV0X3ZhbHVlcz10b3JjaC5mcm9tX251bXB5KG5hbWVkX2lucHV0KSkubG9naXRzLmNwdSgpLm51bXB5KCkNCmlmIGxpc3QocmVmZXJlbmNlLnNoYXBlKSAhPSBbMSwgMTBdIG9yIG5vdCBucC5pc2Zpbml0ZShyZWZlcmVuY2UpLmFsbCgpOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiaW52YWxpZCBQeVRvcmNoIGxvZ2l0cyIpDQoNCnJlc3VsdHMgPSB7fQ0KZm9yIHByZWNpc2lvbiwgbW9kZWxfcGF0aCBpbiAoKCJmcDMyIiwgYXJncy5mcDMyKSwgKCJmcDE2IiwgYXJncy5mcDE2KSk6DQogICAgc2Vzc2lvbiA9IG9ydC5JbmZlcmVuY2VTZXNzaW9uKHN0cihtb2RlbF9wYXRoKSwgcHJvdmlkZXJzPVsiQ1BVRXhlY3V0aW9uUHJvdmlkZXIiXSkNCiAgICBpZiBbKGl0ZW0ubmFtZSwgaXRlbS5zaGFwZSwgaXRlbS50eXBlKSBmb3IgaXRlbSBpbiBzZXNzaW9uLmdldF9pbnB1dHMoKV0gIT0gWygiaW5wdXRfdmFsdWVzIiwgWzEsIDE2MDAwXSwgInRlbnNvcihmbG9hdCkiKV06DQogICAgICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmInVuZXhwZWN0ZWQge3ByZWNpc2lvbn0gaW5wdXQgY29udHJhY3QiKQ0KICAgIHJlc3VsdHNbcHJlY2lzaW9uXSA9IG1ldHJpY3MocmVmZXJlbmNlLCBzZXNzaW9uLnJ1bihbImxvZ2l0cyJdLCB7ImlucHV0X3ZhbHVlcyI6IG5hbWVkX2lucHV0fSlbMF0pDQoNCnBheWxvYWQgPSB7DQogICAgIm1vZGVsX3NvdXJjZSI6IHN0cihhcmdzLm1vZGVsX2RpciksDQogICAgImNoZWNrcG9pbnRfaWQybGFiZWwiOiB7c3RyKGtleSk6IHZhbHVlIGZvciBrZXksIHZhbHVlIGluIGlkMmxhYmVsLml0ZW1zKCl9LA0KICAgICJpbnB1dF9uYW1lIjogImlucHV0X3ZhbHVlcyIsDQogICAgImlucHV0X3NoYXBlIjogbGlzdChuYW1lZF9pbnB1dC5zaGFwZSksDQogICAgImlucHV0X2R0eXBlIjogc3RyKG5hbWVkX2lucHV0LmR0eXBlKSwNCiAgICAiaW5wdXRfZ2VuZXJhdGlvbiI6ICIwLjIqc2luKDIqcGkqNDQwKnQpKzAuMDUqc2luKDIqcGkqODgwKnQpLCAxNjAwMCBzYW1wbGVzIGF0IDE2IGtIejsgaWRlbnRpY2FsIG5hbWVkIGlucHV0IGJ5dGVzIGZvciBQeVRvcmNoIGFuZCBib3RoIE9OTlggc2Vzc2lvbnMiLA0KICAgICJpbnB1dF9zaGEyNTYiOiBoYXNobGliLnNoYTI1NihuYW1lZF9pbnB1dC50b2J5dGVzKCkpLmhleGRpZ2VzdCgpLA0KICAgICJyZXN1bHRzIjogcmVzdWx0cywNCiAgICAidmVyZGljdCI6ICJQQVNTIiBpZiBhbGwoaXRlbVsiZmluaXRlIl0gYW5kIGl0ZW1bImFyZ21heF9lcXVhbCJdIGZvciBpdGVtIGluIHJlc3VsdHMudmFsdWVzKCkpIGVsc2UgIkZBSUwiLA0KfQ0KYXJncy5vdXRwdXQucGFyZW50Lm1rZGlyKHBhcmVudHM9VHJ1ZSwgZXhpc3Rfb2s9VHJ1ZSkNCmFyZ3Mub3V0cHV0LndyaXRlX3RleHQoanNvbi5kdW1wcyhwYXlsb2FkLCBpbmRlbnQ9MikgKyAiXG4iLCBlbmNvZGluZz0idXRmLTgiKQ0KcHJpbnQoanNvbi5kdW1wcyhwYXlsb2FkLCBpbmRlbnQ9MikpDQpyYWlzZSBTeXN0ZW1FeGl0KDAgaWYgcGF5bG9hZFsidmVyZGljdCJdID09ICJQQVNTIiBlbHNlIDEp')); if ((Get-FileHash -LiteralPath $helper -Algorithm SHA256).Hash.ToLowerInvariant() -ne 'fa8e6da8010279646391c629568d8974a59fafbaa895e55c67c7d16af2e27e51') { throw 'Helper hash mismatch: run_parity.py' }; & (Join-Path $CHECKOUT '.venv/Scripts/python.exe') $helper --model-dir $MODEL --fp32 (Join-Path $FP32 'model.onnx') --fp16 (Join-Path $FP16 'model.onnx') --output (Join-Path $OUT 'parity/parity.json'); if ($LASTEXITCODE -ne 0) { throw 'Parity failed' }

Row 11 - l3-exact-pinned-smoke

$helper=Join-Path $PUBLIC 'run_pinned_eval.py'; [IO.File]::WriteAllBytes($helper,[Convert]::FromBase64String('aW1wb3J0IGFyZ3BhcnNlDQppbXBvcnQganNvbg0KaW1wb3J0IG1hdGgNCmZyb20gY29sbGVjdGlvbnMgaW1wb3J0IENvdW50ZXINCmZyb20gaW8gaW1wb3J0IEJ5dGVzSU8NCmZyb20gcGF0aGxpYiBpbXBvcnQgUGF0aA0KDQppbXBvcnQgbnVtcHkgYXMgbnANCmltcG9ydCBvbm54cnVudGltZSBhcyBvcnQNCmltcG9ydCBzb3VuZGZpbGUgYXMgc2YNCmZyb20gZGF0YXNldHMgaW1wb3J0IEF1ZGlvLCBDbGFzc0xhYmVsLCBsb2FkX2RhdGFzZXQNCmZyb20gc2NpcHkuc2lnbmFsIGltcG9ydCByZXNhbXBsZV9wb2x5DQpmcm9tIHNrbGVhcm4ubWV0cmljcyBpbXBvcnQgYWNjdXJhY3lfc2NvcmUsIGNvbmZ1c2lvbl9tYXRyaXgsIGYxX3Njb3JlDQpmcm9tIHRyYW5zZm9ybWVycyBpbXBvcnQgQXV0b0NvbmZpZywgQXV0b0ZlYXR1cmVFeHRyYWN0b3INCg0KDQpDQU5ESURBVEVfU0hBID0gIjFhNjRhZjAwOWU0NzM3ZmIyYTE3MmRhNDE3MTk2NmE2YzkzZDVlMmEiDQpEQVRBU0VUX0lEID0gImdvb2dsZS9mbGV1cnMiDQpEQVRBU0VUX1JFVklTSU9OID0gIjcwYmIyZTg0Yjk3NmI3ZTk2MGFhODlmMWM2NDhlMDljNTlmODk0ZGQiDQpEQVRBU0VUX0NPTkZJR19UT19NT0RFTF9JRCA9IHsNCiAgICAiaGlfaW4iOiAxLA0KICAgICJvcl9pbiI6IDIsDQogICAgImJuX2luIjogMywNCiAgICAidGFfaW4iOiA0LA0KICAgICJ0ZV9pbiI6IDUsDQogICAgImtuX2luIjogNiwNCiAgICAibWxfaW4iOiA3LA0KICAgICJtcl9pbiI6IDgsDQogICAgImd1X2luIjogOSwNCn0NCkVYUEVDVEVEX0xBQkVMUyA9IHsNCiAgICAwOiAiZW4tSU4iLA0KICAgIDE6ICJoaS1JTiIsDQogICAgMjogIm9yLUlOIiwNCiAgICAzOiAiYm4tSU4iLA0KICAgIDQ6ICJ0YS1JTiIsDQogICAgNTogInRlLUlOIiwNCiAgICA2OiAia24tSU4iLA0KICAgIDc6ICJtbC1JTiIsDQogICAgODogIm1yLUlOIiwNCiAgICA5OiAiZ3UtSU4iLA0KfQ0KDQoNCmRlZiBkZWNvZGUocmF3OiBkaWN0KSAtPiB0dXBsZVtucC5uZGFycmF5LCBpbnQsIGludCB8IE5vbmVdOg0KICAgIGVuY29kZWQgPSByYXcuZ2V0KCJieXRlcyIpDQogICAgc291cmNlID0gQnl0ZXNJTyhlbmNvZGVkKSBpZiBlbmNvZGVkIGlzIG5vdCBOb25lIGVsc2UgcmF3LmdldCgicGF0aCIpDQogICAgaWYgc291cmNlIGlzIE5vbmU6DQogICAgICAgIHJhaXNlIFZhbHVlRXJyb3IoImF1ZGlvIHJvdyBoYXMgbmVpdGhlciBieXRlcyBub3IgcGF0aCIpDQogICAgd2F2ZWZvcm0sIHNhbXBsZV9yYXRlID0gc2YucmVhZChzb3VyY2UsIGR0eXBlPSJmbG9hdDMyIiwgYWx3YXlzXzJkPUZhbHNlKQ0KICAgIHdhdmVmb3JtID0gbnAuYXNhcnJheSh3YXZlZm9ybSwgZHR5cGU9bnAuZmxvYXQzMikNCiAgICBpZiB3YXZlZm9ybS5uZGltID09IDI6DQogICAgICAgIHdhdmVmb3JtID0gd2F2ZWZvcm0ubWVhbihheGlzPTEpDQogICAgaWYgd2F2ZWZvcm0ubmRpbSAhPSAxIG9yIHdhdmVmb3JtLnNpemUgPT0gMCBvciBub3QgbnAuaXNmaW5pdGUod2F2ZWZvcm0pLmFsbCgpOg0KICAgICAgICByYWlzZSBWYWx1ZUVycm9yKGYiaW52YWxpZCB3YXZlZm9ybSBzaGFwZS92YWx1ZXM6IHt3YXZlZm9ybS5zaGFwZX0iKQ0KICAgIHJldHVybiB3YXZlZm9ybSwgaW50KHNhbXBsZV9yYXRlKSwgbGVuKGVuY29kZWQpIGlmIGVuY29kZWQgaXMgbm90IE5vbmUgZWxzZSBOb25lDQoNCg0KZGVmIHJlc2FtcGxlKHdhdmVmb3JtOiBucC5uZGFycmF5LCBzb3VyY2VfcmF0ZTogaW50KSAtPiBucC5uZGFycmF5Og0KICAgIGlmIHNvdXJjZV9yYXRlIDw9IDA6DQogICAgICAgIHJhaXNlIFZhbHVlRXJyb3IoZiJpbnZhbGlkIHNvdXJjZSByYXRlOiB7c291cmNlX3JhdGV9IikNCiAgICBpZiBzb3VyY2VfcmF0ZSA9PSAxNjAwMDoNCiAgICAgICAgcmV0dXJuIHdhdmVmb3JtLmFzdHlwZShucC5mbG9hdDMyLCBjb3B5PUZhbHNlKQ0KICAgIGRpdmlzb3IgPSBtYXRoLmdjZChzb3VyY2VfcmF0ZSwgMTYwMDApDQogICAgcmV0dXJuIG5wLmFzYXJyYXkocmVzYW1wbGVfcG9seSh3YXZlZm9ybSwgMTYwMDAgLy8gZGl2aXNvciwgc291cmNlX3JhdGUgLy8gZGl2aXNvciksIGR0eXBlPW5wLmZsb2F0MzIpDQoNCg0KZGVmIHdpbmRvd3Mod2F2ZWZvcm06IG5wLm5kYXJyYXkpIC0+IGxpc3RbdHVwbGVbbnAubmRhcnJheSwgaW50XV06DQogICAgcmVzdWx0ID0gW10NCiAgICBmb3Igc3RhcnQgaW4gcmFuZ2UoMCwgd2F2ZWZvcm0uc2l6ZSwgMTYwMDApOg0KICAgICAgICB2YWx1ZSA9IHdhdmVmb3JtW3N0YXJ0IDogc3RhcnQgKyAxNjAwMF0NCiAgICAgICAgdmFsaWRfc2FtcGxlcyA9IGludCh2YWx1ZS5zaXplKQ0KICAgICAgICBpZiB2YWxpZF9zYW1wbGVzIDwgMTYwMDA6DQogICAgICAgICAgICB2YWx1ZSA9IG5wLnBhZCh2YWx1ZSwgKDAsIDE2MDAwIC0gdmFsaWRfc2FtcGxlcykpDQogICAgICAgIHJlc3VsdC5hcHBlbmQoKG5wLmFzYXJyYXkodmFsdWUsIGR0eXBlPW5wLmZsb2F0MzIpLCB2YWxpZF9zYW1wbGVzKSkNCiAgICBpZiBub3QgcmVzdWx0Og0KICAgICAgICByYWlzZSBWYWx1ZUVycm9yKCJlbXB0eSBhdWRpbyBhZnRlciByZXNhbXBsaW5nIikNCiAgICByZXR1cm4gcmVzdWx0DQoNCg0KZGVmIHJlc29sdmVfZGF0YXNldF9sYWJlbChkYXRhc2V0X2xhYmVsOiBzdHIpIC0+IGludDoNCiAgICBpZiBkYXRhc2V0X2xhYmVsIG5vdCBpbiBEQVRBU0VUX0NPTkZJR19UT19NT0RFTF9JRDoNCiAgICAgICAgcmFpc2UgVmFsdWVFcnJvcihmImRhdGFzZXQgbGFiZWwge2RhdGFzZXRfbGFiZWwhcn0gaXMgbm90IGFuIGV4YWN0IGF1dGhvcml0YXRpdmUgY2hlY2twb2ludCBtYXBwaW5nIikNCiAgICByZXR1cm4gREFUQVNFVF9DT05GSUdfVE9fTU9ERUxfSURbZGF0YXNldF9sYWJlbF0NCg0KDQpwYXJzZXIgPSBhcmdwYXJzZS5Bcmd1bWVudFBhcnNlcigpDQpwYXJzZXIuYWRkX2FyZ3VtZW50KCItLW1vZGVsLWRpciIsIHR5cGU9UGF0aCwgcmVxdWlyZWQ9VHJ1ZSkNCnBhcnNlci5hZGRfYXJndW1lbnQoIi0tbW9kZWwiLCB0eXBlPVBhdGgsIHJlcXVpcmVkPVRydWUpDQpwYXJzZXIuYWRkX2FyZ3VtZW50KCItLW91dHB1dCIsIHR5cGU9UGF0aCwgcmVxdWlyZWQ9VHJ1ZSkNCmFyZ3MgPSBwYXJzZXIucGFyc2VfYXJncygpDQppZiBhcmdzLm91dHB1dC5leGlzdHMoKToNCiAgICByYWlzZSBTeXN0ZW1FeGl0KCJpbW11dGFibGUgZXZhbCBldmlkZW5jZSBjb2xsaXNpb24iKQ0KDQpjb25maWcgPSBBdXRvQ29uZmlnLmZyb21fcHJldHJhaW5lZChhcmdzLm1vZGVsX2RpciwgbG9jYWxfZmlsZXNfb25seT1UcnVlKQ0KaWQybGFiZWwgPSB7aW50KGtleSk6IHN0cih2YWx1ZSkgZm9yIGtleSwgdmFsdWUgaW4gY29uZmlnLmlkMmxhYmVsLml0ZW1zKCl9DQppZiBpZDJsYWJlbCAhPSBFWFBFQ1RFRF9MQUJFTFM6DQogICAgcmFpc2UgUnVudGltZUVycm9yKGYiY2hlY2twb2ludCBsYWJlbCBkcmlmdDoge2lkMmxhYmVsfSIpDQppZiAiZW5fdXMiIGluIERBVEFTRVRfQ09ORklHX1RPX01PREVMX0lEOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiZW5fdXMgbXVzdCByZW1haW4gZXhjbHVkZWQiKQ0KdHJ5Og0KICAgIHJlc29sdmVfZGF0YXNldF9sYWJlbCgiZW5fdXMiKQ0KZXhjZXB0IFZhbHVlRXJyb3IgYXMgZXJyb3I6DQogICAgZW5fdXNfZXhjbHVzaW9uID0geyJzdGF0dXMiOiAiUEFTUyIsICJlcnJvciI6IHN0cihlcnJvcil9DQplbHNlOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiZW5fdXMgZWxpZ2liaWxpdHkgdmFsaWRhdGlvbiBkaWQgbm90IGZhaWwgY2xvc2VkIikNCg0KZXh0cmFjdG9yID0gQXV0b0ZlYXR1cmVFeHRyYWN0b3IuZnJvbV9wcmV0cmFpbmVkKGFyZ3MubW9kZWxfZGlyLCBsb2NhbF9maWxlc19vbmx5PVRydWUpDQppZiBpbnQoZXh0cmFjdG9yLnNhbXBsaW5nX3JhdGUpICE9IDE2MDAwOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmImNoZWNrcG9pbnQgc2FtcGxpbmcgcmF0ZSBkcmlmdDoge2V4dHJhY3Rvci5zYW1wbGluZ19yYXRlfSIpDQpzZXNzaW9uID0gb3J0LkluZmVyZW5jZVNlc3Npb24oc3RyKGFyZ3MubW9kZWwpLCBwcm92aWRlcnM9WyJDUFVFeGVjdXRpb25Qcm92aWRlciJdKQ0KaWYgWyhpdGVtLm5hbWUsIGl0ZW0uc2hhcGUsIGl0ZW0udHlwZSkgZm9yIGl0ZW0gaW4gc2Vzc2lvbi5nZXRfaW5wdXRzKCldICE9IFsoImlucHV0X3ZhbHVlcyIsIFsxLCAxNjAwMF0sICJ0ZW5zb3IoZmxvYXQpIildOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiY2FuZGlkYXRlIGlucHV0IGNvbnRyYWN0IGlzIG5vdCBuYW1lZCBpbnB1dF92YWx1ZXMgZmxvYXQzMlsxLDE2MDAwXSIpDQppZiBbKGl0ZW0ubmFtZSwgaXRlbS5zaGFwZSwgaXRlbS50eXBlKSBmb3IgaXRlbSBpbiBzZXNzaW9uLmdldF9vdXRwdXRzKCldICE9IFsoImxvZ2l0cyIsIFsxLCAxMF0sICJ0ZW5zb3IoZmxvYXQpIildOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcigiY2FuZGlkYXRlIG91dHB1dCBjb250cmFjdCBpcyBub3QgbG9naXRzIGZsb2F0MzJbMSwxMF0iKQ0KDQpyZWZlcmVuY2VzID0gW10NCnByZWRpY3Rpb25zID0gW10NCnJvd3MgPSBbXQ0KcmVqZWN0ZWQgPSBDb3VudGVyKCkNCnJvd3NfZXhhbWluZWQgPSB7fQ0KZGF0YXNldF9maW5nZXJwcmludHMgPSB7fQ0Kc2NoZW1hX2J5X2NvbmZpZyA9IHt9DQpmb3IgZGF0YXNldF9jb25maWcsIGV4cGVjdGVkX21vZGVsX2lkIGluIERBVEFTRVRfQ09ORklHX1RPX01PREVMX0lELml0ZW1zKCk6DQogICAgZGF0YXNldCA9IGxvYWRfZGF0YXNldCgNCiAgICAgICAgREFUQVNFVF9JRCwNCiAgICAgICAgbmFtZT1kYXRhc2V0X2NvbmZpZywNCiAgICAgICAgc3BsaXQ9InZhbGlkYXRpb24iLA0KICAgICAgICBzdHJlYW1pbmc9VHJ1ZSwNCiAgICAgICAgcmV2aXNpb249REFUQVNFVF9SRVZJU0lPTiwNCiAgICApDQogICAgbGFiZWxfZmVhdHVyZSA9IGRhdGFzZXQuZmVhdHVyZXNbImxhbmdfaWQiXQ0KICAgIGF1ZGlvX2ZlYXR1cmUgPSBkYXRhc2V0LmZlYXR1cmVzWyJhdWRpbyJdDQogICAgaWYgbm90IGlzaW5zdGFuY2UobGFiZWxfZmVhdHVyZSwgQ2xhc3NMYWJlbCk6DQogICAgICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmIntkYXRhc2V0X2NvbmZpZ30gbGFuZ19pZCBpcyBub3Qgc2NhbGFyIENsYXNzTGFiZWw6IHtsYWJlbF9mZWF0dXJlIXJ9IikNCiAgICBpZiBub3QgaXNpbnN0YW5jZShhdWRpb19mZWF0dXJlLCBBdWRpbyk6DQogICAgICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmIntkYXRhc2V0X2NvbmZpZ30gYXVkaW8gaXMgbm90IEF1ZGlvOiB7YXVkaW9fZmVhdHVyZSFyfSIpDQogICAgc2NoZW1hX2J5X2NvbmZpZ1tkYXRhc2V0X2NvbmZpZ10gPSB7DQogICAgICAgICJhdWRpbyI6IHJlcHIoYXVkaW9fZmVhdHVyZSksDQogICAgICAgICJsYW5nX2lkIjogcmVwcihsYWJlbF9mZWF0dXJlKSwNCiAgICAgICAgImxhbmd1YWdlIjogcmVwcihkYXRhc2V0LmZlYXR1cmVzLmdldCgibGFuZ3VhZ2UiKSksDQogICAgfQ0KICAgIGRhdGFzZXQgPSBkYXRhc2V0LmNhc3RfY29sdW1uKCJhdWRpbyIsIEF1ZGlvKHNhbXBsaW5nX3JhdGU9YXVkaW9fZmVhdHVyZS5zYW1wbGluZ19yYXRlLCBkZWNvZGU9RmFsc2UpKQ0KICAgIHNlbGVjdGVkID0gTm9uZQ0KICAgIGV4YW1pbmVkID0gMA0KICAgIGZvciByb3dfaW5kZXgsIHJvdyBpbiBlbnVtZXJhdGUoZGF0YXNldCk6DQogICAgICAgIGV4YW1pbmVkICs9IDENCiAgICAgICAgZGF0YXNldF9sYWJlbCA9IGxhYmVsX2ZlYXR1cmUuaW50MnN0cihpbnQocm93WyJsYW5nX2lkIl0pKQ0KICAgICAgICBpZiBkYXRhc2V0X2xhYmVsID09IGRhdGFzZXRfY29uZmlnOg0KICAgICAgICAgICAgc2VsZWN0ZWQgPSAocm93X2luZGV4LCByb3csIGRhdGFzZXRfbGFiZWwpDQogICAgICAgICAgICBicmVhaw0KICAgIHJvd3NfZXhhbWluZWRbZGF0YXNldF9jb25maWddID0gZXhhbWluZWQNCiAgICBkYXRhc2V0X2ZpbmdlcnByaW50c1tkYXRhc2V0X2NvbmZpZ10gPSBnZXRhdHRyKGRhdGFzZXQsICJfZmluZ2VycHJpbnQiLCBOb25lKQ0KICAgIGlmIHNlbGVjdGVkIGlzIE5vbmU6DQogICAgICAgIHJlamVjdGVkWyJOb0V4YWN0Q2xhc3NSb3ciXSArPSAxDQogICAgICAgIGNvbnRpbnVlDQogICAgcm93X2luZGV4LCByb3csIGRhdGFzZXRfbGFiZWwgPSBzZWxlY3RlZA0KICAgIHRhcmdldF9pZCA9IHJlc29sdmVfZGF0YXNldF9sYWJlbChkYXRhc2V0X2xhYmVsKQ0KICAgIGlmIHRhcmdldF9pZCAhPSBleHBlY3RlZF9tb2RlbF9pZCBvciBpZDJsYWJlbFt0YXJnZXRfaWRdLmxvd2VyKCkucmVwbGFjZSgiLSIsICJfIikgIT0gZGF0YXNldF9jb25maWc6DQogICAgICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmIm1hcHBpbmcgbWlzbWF0Y2ggZm9yIHtkYXRhc2V0X2NvbmZpZ306IHRhcmdldD17dGFyZ2V0X2lkfSwgY2hlY2twb2ludD17aWQybGFiZWxbdGFyZ2V0X2lkXX0iKQ0KICAgIHRyeToNCiAgICAgICAgd2F2ZWZvcm0sIHNvdXJjZV9yYXRlLCBlbmNvZGVkX2J5dGVzID0gZGVjb2RlKHJvd1siYXVkaW8iXSkNCiAgICAgICAgc291cmNlX2ZyYW1lcyA9IGludCh3YXZlZm9ybS5zaXplKQ0KICAgICAgICB3YXZlZm9ybSA9IHJlc2FtcGxlKHdhdmVmb3JtLCBzb3VyY2VfcmF0ZSkNCiAgICAgICAgcHJlcGFyZWRfd2luZG93cyA9IHdpbmRvd3Mod2F2ZWZvcm0pDQogICAgICAgIGxvZ2l0c19ieV93aW5kb3cgPSBbXQ0KICAgICAgICB2YWxpZF9zYW1wbGVzX2J5X3dpbmRvdyA9IFtdDQogICAgICAgIGZvciB3aW5kb3csIHZhbGlkX3NhbXBsZXMgaW4gcHJlcGFyZWRfd2luZG93czoNCiAgICAgICAgICAgIGVuY29kZWQgPSBleHRyYWN0b3IoDQogICAgICAgICAgICAgICAgd2luZG93LA0KICAgICAgICAgICAgICAgIHNhbXBsaW5nX3JhdGU9MTYwMDAsDQogICAgICAgICAgICAgICAgcmV0dXJuX3RlbnNvcnM9Im5wIiwNCiAgICAgICAgICAgICAgICBwYWRkaW5nPSJtYXhfbGVuZ3RoIiwNCiAgICAgICAgICAgICAgICB0cnVuY2F0aW9uPVRydWUsDQogICAgICAgICAgICAgICAgbWF4X2xlbmd0aD0xNjAwMCwNCiAgICAgICAgICAgICkNCiAgICAgICAgICAgIGlucHV0X3ZhbHVlcyA9IG5wLmFzYXJyYXkoZW5jb2RlZFsiaW5wdXRfdmFsdWVzIl0sIGR0eXBlPW5wLmZsb2F0MzIpDQogICAgICAgICAgICBpZiBsaXN0KGlucHV0X3ZhbHVlcy5zaGFwZSkgIT0gWzEsIDE2MDAwXToNCiAgICAgICAgICAgICAgICByYWlzZSBWYWx1ZUVycm9yKGYiZmVhdHVyZSBleHRyYWN0b3IgZW1pdHRlZCB7aW5wdXRfdmFsdWVzLnNoYXBlfSIpDQogICAgICAgICAgICBsb2dpdHMgPSBzZXNzaW9uLnJ1bihbImxvZ2l0cyJdLCB7ImlucHV0X3ZhbHVlcyI6IGlucHV0X3ZhbHVlc30pWzBdWzBdDQogICAgICAgICAgICBpZiBsb2dpdHMuc2hhcGUgIT0gKDEwLCkgb3Igbm90IG5wLmlzZmluaXRlKGxvZ2l0cykuYWxsKCk6DQogICAgICAgICAgICAgICAgcmFpc2UgVmFsdWVFcnJvcihmImludmFsaWQgbG9naXRzIHNoYXBlL3ZhbHVlczoge2xvZ2l0c30iKQ0KICAgICAgICAgICAgbG9naXRzX2J5X3dpbmRvdy5hcHBlbmQobG9naXRzKQ0KICAgICAgICAgICAgdmFsaWRfc2FtcGxlc19ieV93aW5kb3cuYXBwZW5kKHZhbGlkX3NhbXBsZXMpDQogICAgICAgIG1lYW5fbG9naXRzID0gbnAubWVhbihucC5zdGFjayhsb2dpdHNfYnlfd2luZG93KSwgYXhpcz0wKQ0KICAgICAgICBwcmVkaWN0ZWRfaWQgPSBpbnQobnAuYXJnbWF4KG1lYW5fbG9naXRzKSkNCiAgICAgICAgcmVmZXJlbmNlcy5hcHBlbmQodGFyZ2V0X2lkKQ0KICAgICAgICBwcmVkaWN0aW9ucy5hcHBlbmQocHJlZGljdGVkX2lkKQ0KICAgICAgICByb3dzLmFwcGVuZCgNCiAgICAgICAgICAgIHsNCiAgICAgICAgICAgICAgICAiZGF0YXNldF9jb25maWciOiBkYXRhc2V0X2NvbmZpZywNCiAgICAgICAgICAgICAgICAiZGF0YXNldF9yb3dfaW5kZXgiOiByb3dfaW5kZXgsDQogICAgICAgICAgICAgICAgInJvd19pZCI6IHJvdy5nZXQoImlkIiksDQogICAgICAgICAgICAgICAgImRhdGFzZXRfbGFiZWwiOiBkYXRhc2V0X2xhYmVsLA0KICAgICAgICAgICAgICAgICJ0YXJnZXRfaWQiOiB0YXJnZXRfaWQsDQogICAgICAgICAgICAgICAgInRhcmdldF9sYWJlbCI6IGlkMmxhYmVsW3RhcmdldF9pZF0sDQogICAgICAgICAgICAgICAgInByZWRpY3RlZF9pZCI6IHByZWRpY3RlZF9pZCwNCiAgICAgICAgICAgICAgICAicHJlZGljdGVkX2xhYmVsIjogaWQybGFiZWxbcHJlZGljdGVkX2lkXSwNCiAgICAgICAgICAgICAgICAic291cmNlX3JhdGUiOiBzb3VyY2VfcmF0ZSwNCiAgICAgICAgICAgICAgICAic291cmNlX2ZyYW1lcyI6IHNvdXJjZV9mcmFtZXMsDQogICAgICAgICAgICAgICAgInJlc2FtcGxlZF9mcmFtZXMiOiBpbnQod2F2ZWZvcm0uc2l6ZSksDQogICAgICAgICAgICAgICAgImR1cmF0aW9uX3NlY29uZHMiOiBmbG9hdCh3YXZlZm9ybS5zaXplIC8gMTYwMDApLA0KICAgICAgICAgICAgICAgICJlbmNvZGVkX2J5dGVzIjogZW5jb2RlZF9ieXRlcywNCiAgICAgICAgICAgICAgICAid2luZG93X2NvdW50IjogbGVuKHByZXBhcmVkX3dpbmRvd3MpLA0KICAgICAgICAgICAgICAgICJ2YWxpZF9zYW1wbGVzX2J5X3dpbmRvdyI6IHZhbGlkX3NhbXBsZXNfYnlfd2luZG93LA0KICAgICAgICAgICAgICAgICJmaW5hbF93aW5kb3dfcGFkZGVkX3NhbXBsZXMiOiAxNjAwMCAtIHZhbGlkX3NhbXBsZXNfYnlfd2luZG93Wy0xXSwNCiAgICAgICAgICAgICAgICAibWVhbl9sb2dpdHMiOiBtZWFuX2xvZ2l0cy50b2xpc3QoKSwNCiAgICAgICAgICAgICAgICAic3RhdHVzIjogInByb2Nlc3NlZCIsDQogICAgICAgICAgICB9DQogICAgICAgICkNCiAgICBleGNlcHQgRXhjZXB0aW9uIGFzIGVycm9yOg0KICAgICAgICByZWplY3RlZFt0eXBlKGVycm9yKS5fX25hbWVfX10gKz0gMQ0KICAgICAgICByb3dzLmFwcGVuZCgNCiAgICAgICAgICAgIHsNCiAgICAgICAgICAgICAgICAiZGF0YXNldF9jb25maWciOiBkYXRhc2V0X2NvbmZpZywNCiAgICAgICAgICAgICAgICAiZGF0YXNldF9yb3dfaW5kZXgiOiByb3dfaW5kZXgsDQogICAgICAgICAgICAgICAgInJvd19pZCI6IHJvdy5nZXQoImlkIiksDQogICAgICAgICAgICAgICAgImRhdGFzZXRfbGFiZWwiOiBkYXRhc2V0X2xhYmVsLA0KICAgICAgICAgICAgICAgICJ0YXJnZXRfaWQiOiB0YXJnZXRfaWQsDQogICAgICAgICAgICAgICAgInRhcmdldF9sYWJlbCI6IGlkMmxhYmVsW3RhcmdldF9pZF0sDQogICAgICAgICAgICAgICAgInN0YXR1cyI6ICJyZWplY3RlZCIsDQogICAgICAgICAgICAgICAgInJlYXNvbiI6IGYie3R5cGUoZXJyb3IpLl9fbmFtZV9ffToge2Vycm9yfSIsDQogICAgICAgICAgICB9DQogICAgICAgICkNCg0KZXhwZWN0ZWRfdGFyZ2V0cyA9IGxpc3QocmFuZ2UoMSwgMTApKQ0KaWYgcmVmZXJlbmNlcyAhPSBleHBlY3RlZF90YXJnZXRzOg0KICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihmImV4cGVjdGVkIGV4YWN0bHkgb25lIHByb2Nlc3NlZCB1dHRlcmFuY2UgZm9yIG1vZGVsIElEcyAxLTksIGdvdCB7cmVmZXJlbmNlc30iKQ0KaWYgcmVqZWN0ZWQ6DQogICAgcmFpc2UgUnVudGltZUVycm9yKGYicHJvY2Vzc2VkPXtsZW4ocmVmZXJlbmNlcyl9LCByZWplY3RlZD17ZGljdChyZWplY3RlZCl9IikNCnJlcHJlc2VudGVkID0gc29ydGVkKHNldChyZWZlcmVuY2VzKSkNCndpbmRvd19jb3VudHMgPSBbcm93WyJ3aW5kb3dfY291bnQiXSBmb3Igcm93IGluIHJvd3NdDQpwYXlsb2FkID0gew0KICAgICJjYW5kaWRhdGVfc2hhIjogQ0FORElEQVRFX1NIQSwNCiAgICAibW9kZWxfYXJ0aWZhY3QiOiBzdHIoYXJncy5tb2RlbCksDQogICAgIm1vZGVsX3NvdXJjZSI6IHN0cihhcmdzLm1vZGVsX2RpciksDQogICAgImRhdGFzZXQiOiB7DQogICAgICAgICJwYXRoIjogREFUQVNFVF9JRCwNCiAgICAgICAgInJldmlzaW9uIjogREFUQVNFVF9SRVZJU0lPTiwNCiAgICAgICAgImNvbmZpZ3MiOiBsaXN0KERBVEFTRVRfQ09ORklHX1RPX01PREVMX0lEKSwNCiAgICAgICAgInNwbGl0IjogInZhbGlkYXRpb24iLA0KICAgICAgICAic3RyZWFtaW5nIjogVHJ1ZSwNCiAgICAgICAgImZpbmdlcnByaW50cyI6IGRhdGFzZXRfZmluZ2VycHJpbnRzLA0KICAgIH0sDQogICAgImNoZWNrcG9pbnRfaWQybGFiZWwiOiB7c3RyKGtleSk6IHZhbHVlIGZvciBrZXksIHZhbHVlIGluIGlkMmxhYmVsLml0ZW1zKCl9LA0KICAgICJzY2hlbWFfYnlfY29uZmlnIjogc2NoZW1hX2J5X2NvbmZpZywNCiAgICAiZGF0YXNldF90b19jaGVja3BvaW50X21hcHBpbmciOiBEQVRBU0VUX0NPTkZJR19UT19NT0RFTF9JRCwNCiAgICAiZXhjbHVkZWRfY2hlY2twb2ludF9sYWJlbHMiOiBbImVuLUlOIl0sDQogICAgImV4Y2x1ZGVkX2RhdGFzZXRfY29uZmlncyI6IFsiZW5fdXMiXSwNCiAgICAiZW5fdXNfZXhjbHVzaW9uX3Byb2JlIjogZW5fdXNfZXhjbHVzaW9uLA0KICAgICJzZWxlY3Rpb24iOiAiZmlyc3QgdmFsaWRhdGlvbiByb3cgaW4gZGV0ZXJtaW5pc3RpYyBkYXRhc2V0IG9yZGVyIHdob3NlIHNjYWxhciBDbGFzc0xhYmVsIHJlc29sdmVzIGV4YWN0bHkgdG8gZWFjaCBhdXRob3JpdGF0aXZlIGNvbmZpZzsgbm8gc2h1ZmZsZSwgc2VlZCwgb3IgZmFsbGJhY2sgYWxpYXMiLA0KICAgICJyb3dzX2V4YW1pbmVkX2J5X2NvbmZpZyI6IHJvd3NfZXhhbWluZWQsDQogICAgImRlY29kZV9jb250cmFjdCI6ICJBdWRpbyhkZWNvZGU9RmFsc2UpOyBTb3VuZEZpbGUgZmxvYXQzMjsgZG93bm1peCBjaGFubmVscyB0byBtb25vOyByZWplY3QgZW1wdHkvbm9uLWZpbml0ZSBhdWRpbzsgcmF0aW9uYWwgcmVzYW1wbGUgdG8gMTYga0h6IiwNCiAgICAiaW5mZXJlbmNlX2NvbnRyYWN0IjogImFsbCBjb25zZWN1dGl2ZSAxNjAwMC1zYW1wbGUgd2luZG93czsgcmlnaHQtcGFkIG9ubHkgZmluYWwgd2luZG93OyBuYW1lZCBpbnB1dF92YWx1ZXMgZmxvYXQzMlsxLDE2MDAwXTsgb25lIE9SVCBmb3J3YXJkIHBlciB3aW5kb3c7IGFyaXRobWV0aWMgbWVhbiBsb2dpdHMgb25jZSBwZXIgdXR0ZXJhbmNlOyB0ZW4tY2xhc3MgYXJnbWF4IiwNCiAgICAicmVxdWVzdGVkX3NhbXBsZXMiOiA5LA0KICAgICJlbGlnaWJsZV9zYW1wbGVzIjogOSwNCiAgICAic2VsZWN0ZWRfc2FtcGxlcyI6IDksDQogICAgInByb2Nlc3NlZF9zYW1wbGVzIjogbGVuKHJlZmVyZW5jZXMpLA0KICAgICJyZWplY3RlZF9zYW1wbGVzIjogc3VtKHJlamVjdGVkLnZhbHVlcygpKSwNCiAgICAicmVqZWN0ZWRfYnlfcmVhc29uIjogZGljdChyZWplY3RlZCksDQogICAgInJlcXVlc3RlZF9jbGFzc2VzIjogbGlzdChEQVRBU0VUX0NPTkZJR19UT19NT0RFTF9JRCksDQogICAgInJlcHJlc2VudGVkX21vZGVsX2lkcyI6IHJlcHJlc2VudGVkLA0KICAgICJyZXByZXNlbnRlZF9jbGFzc2VzIjogW2lkMmxhYmVsW3ZhbHVlXSBmb3IgdmFsdWUgaW4gcmVwcmVzZW50ZWRdLA0KICAgICJjaGVja3BvaW50X2NsYXNzX2NvdmVyYWdlIjogIjkvMTAiLA0KICAgICJhY2N1cmFjeSI6IGZsb2F0KGFjY3VyYWN5X3Njb3JlKHJlZmVyZW5jZXMsIHByZWRpY3Rpb25zKSksDQogICAgInJlcHJlc2VudGVkX2NsYXNzX21hY3JvX2YxIjogZmxvYXQoZjFfc2NvcmUocmVmZXJlbmNlcywgcHJlZGljdGlvbnMsIGxhYmVscz1yZXByZXNlbnRlZCwgYXZlcmFnZT0ibWFjcm8iLCB6ZXJvX2RpdmlzaW9uPTApKSwNCiAgICAiY29uZnVzaW9uX21hdHJpeF9sYWJlbHMiOiBbaWQybGFiZWxbdmFsdWVdIGZvciB2YWx1ZSBpbiByYW5nZSgxMCldLA0KICAgICJjb25mdXNpb25fbWF0cml4IjogY29uZnVzaW9uX21hdHJpeChyZWZlcmVuY2VzLCBwcmVkaWN0aW9ucywgbGFiZWxzPWxpc3QocmFuZ2UoMTApKSkudG9saXN0KCksDQogICAgInRvdGFsX3dpbmRvd3MiOiBzdW0od2luZG93X2NvdW50cyksDQogICAgIm1pbmltdW1fd2luZG93c19wZXJfdXR0ZXJhbmNlIjogbWluKHdpbmRvd19jb3VudHMpLA0KICAgICJtYXhpbXVtX3dpbmRvd3NfcGVyX3V0dGVyYW5jZSI6IG1heCh3aW5kb3dfY291bnRzKSwNCiAgICAiZmFub3V0X2NhcHMiOiB7DQogICAgICAgICJyZXByZXNlbnRlZF9jbGFzc2VzIjogOSwNCiAgICAgICAgInJvd3NfcGVyX2NsYXNzIjogMSwNCiAgICAgICAgIm1heGltdW1fdXR0ZXJhbmNlcyI6IDksDQogICAgICAgICJhbGxfd2luZG93c19yZXF1aXJlZCI6IFRydWUsDQogICAgICAgICJzYW1wbGVzX3Blcl93aW5kb3ciOiAxNjAwMCwNCiAgICAgICAgInByZWRpY3Rpb25zX3Blcl91dHRlcmFuY2UiOiAxLA0KICAgICAgICAiY2FuZGlkYXRlX2xhYmVsc19vcl9wcm9tcHRzIjogTm9uZSwNCiAgICAgICAgImJlYW1zIjogTm9uZSwNCiAgICB9LA0KICAgICJjbGFpbSI6ICJCb3VuZGVkIGZ1bmN0aW9uYWwgc21va2Ugb3ZlciBuaW5lIGV4YWN0IHJlcHJlc2VudGVkIHB1YmxpYyBjbGFzc2VzIG9ubHk7IG5vIGNoZWNrcG9pbnQgYmVuY2htYXJrLCBtb2RlbC1xdWFsaXR5LCByZXByZXNlbnRhdGl2ZS1hY2N1cmFjeSwgZW4tSU4sIG9yIGZ1bGwtbGFuZ3VhZ2UtY292ZXJhZ2UgY2xhaW0uIiwNCiAgICAicm93cyI6IHJvd3MsDQp9DQphcmdzLm91dHB1dC5wYXJlbnQubWtkaXIocGFyZW50cz1UcnVlLCBleGlzdF9vaz1UcnVlKQ0KYXJncy5vdXRwdXQud3JpdGVfdGV4dChqc29uLmR1bXBzKHBheWxvYWQsIGluZGVudD0yKSArICJcbiIsIGVuY29kaW5nPSJ1dGYtOCIpDQpwcmludChqc29uLmR1bXBzKHtrZXk6IHZhbHVlIGZvciBrZXksIHZhbHVlIGluIHBheWxvYWQuaXRlbXMoKSBpZiBrZXkgbm90IGluIHsicm93cyIsICJzY2hlbWFfYnlfY29uZmlnIiwgImNvbmZ1c2lvbl9tYXRyaXgifX0sIGluZGVudD0yKSk=')); if ((Get-FileHash -LiteralPath $helper -Algorithm SHA256).Hash.ToLowerInvariant() -ne 'f5e02c4680b957582cb24c8b84e46208f6777dc21b71ab511ac9726c58bea4d9') { throw 'Helper hash mismatch: run_pinned_eval.py' }; & (Join-Path $CHECKOUT '.venv/Scripts/python.exe') $helper --model-dir $MODEL --model (Join-Path $FP32 'model.onnx') --output (Join-Path $OUT 'eval/pinned-fleurs.json'); if ($LASTEXITCODE -ne 0) { throw 'Pinned eval failed' }

Row 12 - analyze-fp32

$env:WINMLCLI_RULES_DIR=$RULES; & (Join-Path $CHECKOUT '.venv/Scripts/winml.exe') analyze --model (Join-Path $FP32 'model.onnx') --ep all --device all --output (Join-Path $OUT 'analysis/fp32-all.json') --no-color; $code=$LASTEXITCODE; if ($code -notin @(0,1)) { throw 'FP32 Analyze exit policy failed' }; $value=Get-Content (Join-Path $OUT 'analysis/fp32-all.json') -Raw|ConvertFrom-Json; if (-not $value.metadata -or @($value.results).Count -eq 0) { throw 'FP32 Analyze JSON incomplete' }

Row 13 - analyze-fp16

$env:WINMLCLI_RULES_DIR=$RULES; & (Join-Path $CHECKOUT '.venv/Scripts/winml.exe') analyze --model (Join-Path $FP16 'model.onnx') --ep all --device all --output (Join-Path $OUT 'analysis/fp16-all.json') --no-color; $code=$LASTEXITCODE; if ($code -notin @(0,1)) { throw 'FP16 Analyze exit policy failed' }; $value=Get-Content (Join-Path $OUT 'analysis/fp16-all.json') -Raw|ConvertFrom-Json; if (-not $value.metadata -or @($value.results).Count -eq 0) { throw 'FP16 Analyze JSON incomplete' }

Row 14 - quality-ruff

Push-Location $CHECKOUT; try { uv run --no-sync ruff check src/ tests/; if ($LASTEXITCODE -ne 0) { throw 'Ruff failed' } } finally { Pop-Location }

Row 15 - quality-mypy

Push-Location $CHECKOUT; try { uv run --no-sync mypy -p winml.modelkit; if ($LASTEXITCODE -ne 0) { throw 'Mypy failed' } } finally { Pop-Location }

Row 16 - quality-focused-tests

Push-Location $CHECKOUT; try { uv run --no-sync pytest -q tests/unit/eval/test_audio_classification_evaluator.py; if ($LASTEXITCODE -ne 0) { throw 'Focused tests failed' } } finally { Pop-Location }; if (git -C $CHECKOUT status --porcelain) { throw 'Checkout became dirty' }

`

## Quality gates

• pre-commit run insert-license --all-files: PASS.
• ruff check src/ tests/: PASS.
• mypy -p winml.modelkit: PASS, no issues in 439 source files.
• Commands/config/build/compiler/session/eval unit partition: 3,640 passed, 9 skipped.
• Models/loader/datasets/export unit partition: 1,538 passed, 6 skipped, 2 xfailed.
• WinML provider auto-detection installed OpenVINOExecutionProvider 1.8.15.0 during the CPU flow; this was a nonsemantic environment side effect and did not alter candidate JSON, repository state, recipe authority, CPU provider selection, artifact validity, or the technical verdict.

[ssss141414]

REQUEST_CHANGES

Reviewed exact PR head 1a64af009e4737fb2a172da4171966a6c93d5e2a against dependency head 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8 and base 0876e5ae1c98a169a6137e092e0d7b30bf9cee33.

Blocking findings

1. The FP16 proof and public build command omit the currently required --precision fp16. The current reviewer contract (copilot-skills/dev_skill/adding-model-support/agents/reviewer.md, fp16 artifact and Goal-L0 checks) explicitly makes omission blocking and requires --precision fp16 without --no-quant; the producer and tester contracts carry the same requirement. However, the sealed tester reproducible_commands FP16 build and the PR's Reproduce commands FP16 build both omit the flag. The emitted artifact is genuinely FP16 (86 FLOAT16 initializers, 117,495,490-byte external data versus 234,990,980 bytes for FP32, two public-boundary Casts, and Model Precision: fp16), so this is a command/contract discrepancy rather than an artifact-quality failure. Resolve it fail-closed: either rebuild and reseal the exact FP16 tuple with --precision fp16 and no --no-quant, or first land an evidence-backed Lane A correction to all conflicting role contracts and then regenerate the handoffs/publication under that corrected contract.

2. The required tester-owned portable reproduction field is absent, and the published replacement does not reproduce the claimed ceiling. agents/tester.md requires public_reproducible_commands with portable counterparts to the exact commands, and agents/explainer.md requires rendering that field unchanged or bouncing the handoff. The sealed verdict_table-v2.json has no public_reproducible_commands property; its public_reproduction is only a pending script reference. The PR instead embeds a rewritten script that builds/perfs and checks the dataset revision but does not execute the claimed L2 parity or L3 pinned FLEURS smoke. Have the tester emit and seal the complete portable command set, including the claimed L2/L3 paths, and have the explainer render it without semantic rewriting.

3. No methodology friction observed is contradicted by the evidence. The producer handoff says the FP16 flag was intentionally omitted because precision is recipe-owned, while the current producer/tester/reviewer contracts say the opposite; the successful override-free FP16 artifact further shows the documented behavior needs reconciliation for the tested WinML version. This is at least doc-code drift and a reviewer-found methodology gap under agents/learner.md Step 4b. Route it through the learner, add the required _meta-NNN plus consistent role-file edit in a pushed Lane A commit/PR, and replace the no-friction declaration before re-review.

Independently verified

• Candidate-isolated diff is exactly the two recipe JSON files; each adds only export.compatibility.transformers_attention = "eager".
• Pinned checkpoint revision and exact ten-label order, authoritative 9/10 FLEURS mapping, and explicit en_us exclusion are correct.
• Reviewer-owned FLEURS rerun matched all 33 substantive sealed fields: 9 selected/processed, 0 rejected, 133 windows, accuracy 0.5555555555555556, represented-class macro-F1 0.4259259259259259, and identical row/confusion data.
• Reviewer-owned parity rerun passed for FP32 and FP16 with finite logits and matching argmax; no FP16 speedup is claimed.
• Ruff passed, mypy found no issues in 439 source files, and the focused audio-classification evaluator suite passed 10/10.
• Immediately before this comment: PR is OPEN/DRAFT with model-scale-by-skill, exact head unchanged, all 9 checks are COMPLETED/SUCCESS, and there are 0 open review threads.

[ssss141414]

_meta-111 successor closure

All three blockers in #1332 (comment) are resolved at the unchanged candidate head 1a64af009e4737fb2a172da4171966a6c93d5e2a; no candidate files or branch refs changed.

1. FP16 command contract: tester public row 6 now carries explicit --precision fp16 and excludes --no-quant. The sealed structure remains the accepted FP16 artifact: 86 FLOAT16 initializers, 117,495,490-byte external weights, and two public-boundary Casts.
2. Portable reproduction: the canonical body now transcribes all 17 tester-owned direct command rows, including repository/dependency/environment/rules/model/dataset/mapping prerequisites, exact inline generation of the four sealed helpers, L0 FP32/FP16, L1 FP32/FP16, L2 identical-input parity, L3 pinned FLEURS smoke, both Analyze artifacts, and Ruff/mypy/focused-pytest quality closure. The 10-row public_sequence_closure table is CLOSED throughout. L3 remains 9 processed, 0 rejected, 133 windows, accuracy 0.5555555555555556, represented-class macro-F1 0.4259259259259259, and exact 9/10 checkpoint-class coverage.
3. Methodology declaration: Lane A gim-home/ModelKitArtifacts#254 is bound at e06d2fe92bd699b08322792de726da5ada43fd06 and the body now states exactly: Methodology friction observed: _meta-111 added.

The tester successor preserves L0/L1/L2/L3 PASS, both Analyze results under the accepted exit 0/1 policy, and the original metric/model/dataset provenance. Please re-review this unchanged head against the published closure.

[ssss141414]

APPROVE

Independently reviewed exact PR head 1a64af009e4737fb2a172da4171966a6c93d5e2a against dependency parent 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8, the updated _meta-111 reviewer contract at Lane A commit e06d2fe92bd699b08322792de726da5ada43fd06, the prior REQUEST_CHANGES, tester CLOSED successor evidence, learner handoff, and explainer successor shipment/current body/reply.

All three prior blockers are closed:

1. The tester-owned FP16 public build row explicitly includes --precision fp16, excludes --no-quant, and remains bound to the accepted artifact. Independent ONNX inspection confirmed 86 FLOAT16 initializers, 117,495,490 external-data bytes versus 234,990,980 for FP32, float32 public I/O, and two boundary Casts.
2. public_reproducible_commands contains 17 direct portable PowerShell rows. All rows parse, contain no tester/scratch wrapper dependency, and close prerequisites, L0 FP32/FP16, L1 FP32/FP16, L2 identical-input parity, L3 pinned FLEURS smoke, both Analyze artifacts, and quality through 10/10 CLOSED closure rows. The four inline helpers are accessible, compile, and are byte-identical to their sealed r11 authorities.
3. Lane A PR refactor: move remaining network-dependent tests to integration #254 is pinned at e06d2fe92bd699b08322792de726da5ada43fd06, binds _meta-111, and the current PR body states exactly Methodology friction observed: _meta-111 added.

Independent ref, mapping, metric, and boundedness guards passed: a fresh detached checkout resolved the exact candidate and parent with only the two expected recipe files changed; checkpoint labels and the authoritative nine-class FLEURS mapping match; en_us remains excluded; the pinned model/dataset revisions and validation split match; and the sealed L3 result remains 9 processed, 0 rejected, 133 windows, accuracy 0.5555555555555556, and represented-class macro-F1 0.4259259259259259. The current remote body is byte-identical to the canonical successor body.

Immediately before this verdict, PR #1332 was OPEN/DRAFT with model-scale-by-skill, exact head unchanged, all 9 checks COMPLETED/SUCCESS, and 0 unresolved review threads. No blocking findings remain.