recipe(wav2vec-vm): refresh CPU audio-classification recipes

[ssss141414]

Summary

This refreshes the existing CPU FP32 and FP16 recipes for jakeBland/wav2vec-vm-finetune, an English Wav2Vec2 human-versus-voicemail audio classifier originally contributed in #1114. Effort L0 refines two existing recipe files on the generic audio-classification evaluator from dependency #1326; Outcome L0 ships both CPU precisions, while the committed Goal reaches L3 PASS with full bounded coverage.

Model metadata

What the model does

This English Wav2Vec2 classifier consumes the first two seconds of an isolated call response at 16 kHz and emits two logits that distinguish a live human response from a voicemail greeting.

• Evidence: the pinned checkpoint card describes voicemail detection from the first two seconds of a call response; its config declares Wav2Vec2ForSequenceClassification and id2label={0: human, 1: voicemail}; the ONNX contract is input_values float32 [1,16000] to logits float32 [1,2] (verified).

Primary user stories

• A call assistant supplies the isolated first two seconds of a callee response to classify whether a live person or a voicemail greeting answered. Evidence: pinned checkpoint revision 663b7dfbe8e08d615c6a83353a119e35d47903bd (verified).

Supported tasks

• audio-classification through the checkpoint, Transformers, Optimum ONNX, and WinML surfaces. The pinned checkpoint pipeline tag and architecture identify the task; WinML resolves AutoModelForAudioClassification and Wav2Vec2OnnxConfig (verified).

Model architecture

Wav2Vec2ForSequenceClassification
|- wav2vec2.feature_extractor: convolutional layers x7
|- wav2vec2.feature_projection: 512 -> 1024
|- wav2vec2.encoder.pos_conv_embed
|- wav2vec2.encoder.layers[]: stable-LayerNorm Transformer x24
|- projector: 1024 -> 256
|- temporal mean pooling
'- classifier: 256 -> 2 logits

• Source/confidence: pinned checkpoint config, Transformers 5.14.1 source, and mapped ONNX graph (mapped).

Validation and support evidence

1. Baseline

Baseline authority is microsoft/winml-cli main commit 0876e5ae1c98a169a6137e092e0d7b30bf9cee33, WinML 0.3.0.

• Auto-config: FP32 and FP16 both generated successfully with export.compatibility.transformers_attention=eager and without obsolete loader.trust_remote_code; FP16 selected quant.mode=fp16 and fp16_keep_io_types=true.
• Build floor: recipe-free FP32 CPU build PASS in 161.3 s (86.2 s export, 34.2 s optimize), opset 17, complete 897/897 component mapping, input_values float32 [1,16000], and logits float32 [1,2].
• Perf floor: CPUExecutionProvider mean 135.879 ms, p50 135.832 ms, p90 139.92 ms, 7.36 samples/s, and +91.33 MB RSS.
• Eval floor: audio-classification was rejected before dataset loading because main had no registered evaluator. Dependency recipe(ast): use eager attention for AudioSet model #1326 provides that generic evaluator and raw-streaming media fix.
• Optimum probe: DIRECT-PROBE-BLOCKED because locked Optimum imports a removed private symbol from Transformers 5.14.1. Independent WinML inspect/config/build resolution confirms the vendor Wav2Vec2 exporter; no WinML-specific task registration is added here.

2. Goal

Dimension	Commitment
Effort	L0, configuration-only refinement of two existing CPU recipes
Goal	L3
Outcome	L0

Goal success means CPU FP32/FP16 recipe-authority build, structure, realized precision, named-input runtime/perf/PyTorch parity, and final-SHA-pinned 42-row FP32 functional evaluation. The branch is a direct child of approved dependency #1326 at 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8; dependent #1329 is intentionally not included because this dataset uses scalar ClassLabel.

3. Outcome

• Shipped tier: L0.
• Highest Goal verdict: L3 PASS.
• Coverage: full; deferred tuples: none; ceiling was not downgraded.
• Candidate: d15ac73b633d5bb0209a0b1a2e2f4e468df49e92, tree c5b0034b60c3c0f06f46d11ec4dfe0bb96260a66.
• Shipped paths: examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_fp32_config.json and examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_fp16_config.json.
• Lane A knowledge: gim-home/ModelKitArtifacts#254 at 90770910fcf0d2ddb0a0d76189b220fb89bdb726, wav2vec2.json SHA-256 1496004226d438662b55f4db05ed8dbafec347111c9994a8ccdac13f21c8d443, findings wav2vec2-019 and wav2vec2-020.
• Methodology declaration: No methodology friction observed.

4. Per-EP/device/precision results and Functional smoke Eval

Tier	EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM delta
L0	CPUExecutionProvider / cpu	fp32	PASS	-	-	-	-
L0	CPUExecutionProvider / cpu	fp16	PASS	-	-	-	-
L1	CPUExecutionProvider / cpu	fp32	PASS	133.304 ms	130.776 ms	7.50 samples/s	+91.00 MB
L1	CPUExecutionProvider / cpu	fp16	PASS	155.551 ms	155.420 ms	6.43 samples/s	+43.55 MB

L0 build and realized precision

• FP32: build PASS in 192.040 s; 771 nodes / 13 operator types; 426 FLOAT and 11 INT64 initializers; model bytes 299,942 plus 1,262,811,524 external-data bytes; finite named-input runtime.
• FP16: build PASS in 195.179 s; 773 nodes / 14 operator types; 426 FLOAT16 and 11 INT64 initializers; two public-boundary Casts; model bytes 300,248 plus 631,405,250 external-data bytes; finite named-input runtime.
• Both expose input_values float32 [1,16000] and logits float32 [1,2]. FP16 realizes half-size external weights while preserving float32 public IO.

L2 PyTorch parity

The identical named input sin(arange(16000) * 0.003) as float32 [1,16000] was supplied byte-for-byte to pinned PyTorch revision 663b7dfbe8e08d615c6a83353a119e35d47903bd and both ONNX sessions.

Precision	Verdict	Cosine	Max absolute delta	Reference / candidate argmax
fp32	PASS	0.9999999999995335	0.000008299946784973145	1 / 1
fp16	PASS	0.9999999998848055	0.000028654932975769043	1 / 1

L3 Functional smoke Eval

FP32 CPU only. This is end-to-end operability evidence on one exact pinned public dataset, not a checkpoint benchmark, representative accuracy result, or general voicemail-accuracy claim. No Eval accuracy was measured for FP16 or another EP.

• Candidate SHA: d15ac73b633d5bb0209a0b1a2e2f4e468df49e92.
• Dataset: herbiel/id-voicemail-dataset-v2 revision 3761e739c2b499461739f61f826ad4208467c7f7, config default, split test, fingerprint b1fc4518bf13edee.
• Accounting: requested 42, eligible 42, selected 42, processed 42, rejected 0; true counts human 20 / voicemail 22; prediction counts human 20 / voicemail 22.
• Semantics: schema, label semantics, and prediction semantics verified; checkpoint labels are 0=human, 1=voicemail.
• Fan-out caps: 2 classes, maximum 42 rows, exactly two 16,000-sample windows per row, maximum 84 inference windows.
• Result: accuracy 0.9523809523809523; represented-class macro-F1 0.9522727272727273; labels [human, voicemail]; confusion matrix [[19,1],[1,21]].
• Former blocker/capability: baseline main had no audio-classification evaluator; generic scalar-ClassLabel evaluation and raw-streaming media support come from recipe(ast): use eager attention for AudioSet model #1326.

5. Delta

The two checked-in recipes refine existing historical files and are structurally identical to the planner-generated current-main FP32/FP16 configurations. No product source, tests, shared evaluator, dependency file, or production recipe README changes are included.

Relative to the approved dependency-base historical recipes:

Recipe	JSON pointer	Old	New	Reason
fp32, fp16	/export/compatibility/transformers_attention	absent	eager	Current-main export compatibility for standard Wav2Vec2
fp32, fp16	/loader/trust_remote_code	true	absent	Standard AutoModelForAudioClassification needs no remote code
fp16	/quant/task	absent	audio-classification	Current-main auto-config precision metadata
fp16	/quant/model_id	absent	jakeBland/wav2vec-vm-finetune	Current-main checkpoint identity
fp16	/quant/model_type	absent	wav2vec2	Current-main architecture identity

Reducibility is consistent with the charter: no shipped field differs from generated baseline, no model-ID conditional or evaluator duplication is introduced, and recipe-free acceptance is not required for this recipe-only coverage refresh.

6. Analyze summary - component level and op level

ANALYZE-PARTIAL-SUCCESS: static rule analysis emitted complete JSON for both artifacts, but OpenVINO plugin registration failed on this host because onnxruntime_providers_shared.dll was missing (Windows error 126). This is a host packaging caveat, not model incompatibility, an operator failure, or runtime execution evidence.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
fp32	feature extractor; feature projection; positional convolution; encoder layers x24; projector; temporal mean pooling; classifier	517 scope-preserved mapped nodes; 254 optimizer-transition gaps	None in rule-covered EP groups
fp16	same seven regions plus two public-boundary Casts	517 scope-preserved mapped nodes; 256 optimizer-transition gaps	None in rule-covered EP groups

Op-level summary

Artifact	Graph	Dominant operators	EP roll-up
fp32	771 operators / 13 types	Reshape 244; Gemm 147; Transpose 111; Add 73; LayerNormalization 57	NvTensorRTRTX, QNN, and OpenVINO rule rows fully supported
fp16	773 operators / 14 types	same semantic graph plus two boundary Casts	NvTensorRTRTX, QNN, and OpenVINO rule rows fully supported

No partial or unsupported operator types were reported. CUDA, MIGraphX, TensorRT, and DML have no shipped rule data, so their operator classifications remain unknown.

7. Reproduce commands

These guards acquire the exact public dependency and candidate refs, verify both advertised SHAs, and fail if the candidate is not the direct child tested above. $OUT is a disposable relative output directory.

$REPO='https://github.com/microsoft/winml-cli.git'
$DEP_REF='refs/pull/1326/head'
$DEP_SHA='1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8'
$CANDIDATE_REF='refs/heads/ssss141414/add-jakeBland-wav2vec-vm-finetune-recipe'
$CANDIDATE_SHA='d15ac73b633d5bb0209a0b1a2e2f4e468df49e92'

$depRemote=(git ls-remote --exit-code $REPO $DEP_REF).Split()[0]
if ($depRemote -ne $DEP_SHA) { throw "dependency ref mismatch: $depRemote" }
$candidateRemote=(git ls-remote --exit-code $REPO $CANDIDATE_REF).Split()[0]
if ($candidateRemote -ne $CANDIDATE_SHA) { throw "candidate ref mismatch: $candidateRemote" }

git clone $REPO winml-cli-1114
Set-Location winml-cli-1114
git fetch origin $DEP_REF`:$DEP_REF $CANDIDATE_REF`:$CANDIDATE_REF
git checkout --detach $CANDIDATE_SHA
if ((git rev-parse HEAD) -ne $CANDIDATE_SHA) { throw 'candidate checkout mismatch' }
if ((git rev-parse HEAD^) -ne $DEP_SHA) { throw 'dependency mismatch' }
git merge-base --is-ancestor $DEP_SHA $CANDIDATE_SHA
if ($LASTEXITCODE -ne 0) { throw 'dependency ancestry mismatch' }

uv python install 3.11
uv sync --locked --all-extras --all-groups
$OUT='temp/wav2vec-vm-repro'

uv run --no-sync winml build -c examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_fp32_config.json -m jakeBland/wav2vec-vm-finetune -o $OUT/artifacts/fp32
uv run --no-sync winml build -c examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_fp16_config.json -m jakeBland/wav2vec-vm-finetune -o $OUT/artifacts/fp16
uv run --no-sync winml perf -m $OUT/artifacts/fp32/model.onnx --iterations 20 --warmup 5 -d cpu --no-color --memory -o $OUT/perf-fp32.json
uv run --no-sync winml perf -m $OUT/artifacts/fp16/model.onnx --iterations 20 --warmup 5 -d cpu --no-color --memory -o $OUT/perf-fp16.json

$release=Invoke-RestMethod 'https://api.github.com/repos/microsoft/winml-cli/releases/latest'
$asset=$release.assets | Where-Object name -Like 'rules-v*.zip' | Select-Object -First 1
if(-not $asset){throw 'released rules asset missing'}
Invoke-WebRequest $asset.browser_download_url -OutFile $OUT/rules.zip
Expand-Archive $OUT/rules.zip -DestinationPath $OUT/rules -Force
$env:WINMLCLI_RULES_DIR=(Resolve-Path $OUT/rules)
uv run --no-sync winml analyze --model $OUT/artifacts/fp32/model.onnx --ep all --output $OUT/analyze-fp32.json
uv run --no-sync winml analyze --model $OUT/artifacts/fp16/model.onnx --ep all --output $OUT/analyze-fp16.json
Remove-Item Env:WINMLCLI_RULES_DIR

uv run --no-sync pre-commit run insert-license --all-files
uv run --no-sync ruff check src/ tests/
uv run --no-sync mypy -p winml.modelkit

The bounded L2 parity and L3 dataset harnesses are tester evidence rather than checked-in product files; their exact results and limits are transcribed above without inventing a public command surface.

Quality gates

Gate	Result
License insertion check	PASS
Ruff src/ tests/	PASS
mypy winml.modelkit	PASS, 439 source files
Models/loader/datasets/export unit partition	PASS, 1538 passed, 6 skipped, 2 xfailed
Commands/config/build/compiler/session/eval unit partition	PASS, 3640 passed, 9 skipped

This is a dependent Draft PR against main: #1326 must land first, and its exact tested head is verified above. Historical context is #1114. Lane A knowledge is tracked separately in gim-home/ModelKitArtifacts#254 at exact commit 90770910fcf0d2ddb0a0d76189b220fb89bdb726; no Lane A files are present in this model PR.

[ssss141414]

APPROVE

Independent review completed against exact head d15ac73b633d5bb0209a0b1a2e2f4e468df49e92 from a fresh detached checkout.

• Identity and scope: HEAD^ is exact dependency recipe(ast): use eager attention for AudioSet model #1326 head 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8; the dependency is both the direct parent and an ancestor. The model delta contains exactly the two existing CPU recipe JSON files. There are no src/, tests/, dependency, skill, or production examples/recipes/README.md changes.
• Recipe authority: both checked-in recipes validate through _load_config and WinMLBuildConfig.validate, structurally equal the planner-generated current-main configurations, retain AutoModelForAudioClassification / wav2vec2 / opset 17 / input_values float32 [1,16000] / logits float32 [1,2], use eager attention, and require no remote-code trust. FP16 uses quant.mode=fp16 with float32 public IO.
• Evidence integrity: independently rehashed all 68 sealed tester files plus 4 external inputs; 0 missing or mismatched files. Recipe SHA-256 values match the producer handoff.
• L0 artifacts: independent ONNX checker and named-input CPU runtime passed for both. FP32 has 426 FLOAT initializers and 1,262,811,524 external-data bytes; FP16 has 426 FLOAT16 initializers, two boundary Casts, and 631,405,250 external-data bytes.
• L1 perf: fresh 20-iteration CPU reruns passed and stayed within 20% of sealed means. FP32 reports precision fp32, mean 138.908 ms, p50 139.463 ms, 7.20 samples/s, +90.91 MB RSS. FP16 reports precision fp16, mean 166.601 ms, p50 166.019 ms, 6.00 samples/s, +43.64 MB RSS.
• L2 parity: fresh pinned-PyTorch rerun passed. FP32 cosine 0.9999999999995335, max-abs 8.299946784973145e-06; FP16 cosine 0.9999999998848055, max-abs 2.8654932975769043e-05; reference/candidate argmax is 1 for both.
• L3 semantics: fresh pinned Eval rerun confirms checkpoint 0=human, 1=voicemail and dataset scalar ClassLabel(names=['human','voicemail']). Raw Audio(decode=False) media was decoded with SoundFile, normalized to 16 kHz, capped to the first 32,000 samples, and evaluated as exactly two 16,000-sample windows per row. Requested/eligible/selected/processed/rejected = 42/42/42/42/0; true and predicted counts are 20 human / 22 voicemail; confusion is [[19,1],[1,21]]; accuracy 0.9523809523809523; represented-class macro-F1 0.9522727272727273; maximum fan-out 84 windows. This is correctly labeled exact pinned-dataset functional validation, not a benchmark or general accuracy claim.
• Analyze: direct artifacts contain complete component and op evidence. FP32 is 771 ops / 13 types; FP16 is 773 / 14. The nonzero Analyze exit is correctly limited to the host OpenVINO registration failure for missing onnxruntime_providers_shared.dll (Error 126), not presented as model incompatibility or runtime support evidence.
• Knowledge: Lane A refactor: move remaining network-dependent tests to integration #254 is exact head 90770910fcf0d2ddb0a0d76189b220fb89bdb726; wav2vec2.json SHA-256 is 1496004226d438662b55f4db05ed8dbafec347111c9994a8ccdac13f21c8d443; findings wav2vec2-019 and wav2vec2-020 preserve HTP counts (315,701,634 parameters; 118/409 traced modules), tuple evidence, label semantics, accounting, caveats, and the no-benchmark boundary.
• Quality and remote gate: reviewer-run ruff check src/ tests/ and mypy -p winml.modelkit passed. Exact-head GitHub status contains 9/9 COMPLETED/SUCCESS checks. PR remains Draft with model-scale-by-skill. Enumerated 0 issue comments, 0 review comments, 0 review threads, and 0 open threads.

Coverage is full for the chartered CPU FP32 and FP16 tuples; deferred tuples: none. Dependency #1326 must land before merge, and this approval does not change Draft state or authorize readiness/merge.