Skip to content

recipe(hubert): refresh Russian emotion CPU configs - #1333

Draft
ssss141414 wants to merge 1 commit into
microsoft:ssss141414/add-common-voice-gender-dependentfrom
ssss141414:ssss141414/refresh-xbgoose-hubert-russian-emotion
Draft

recipe(hubert): refresh Russian emotion CPU configs#1333
ssss141414 wants to merge 1 commit into
microsoft:ssss141414/add-common-voice-gender-dependentfrom
ssss141414:ssss141414/refresh-xbgoose-hubert-russian-emotion

Conversation

@ssss141414

@ssss141414 ssss141414 commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Refreshes the existing CPU FP32 and FP16 recipes for xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned, a five-class Russian speech-emotion classifier, for the current Transformers 5 export path. This is an L0 recipe-only outcome stacked on the audio-classification evaluator work in Draft PR #1329, which contains Draft PR #1326. The exact candidate reached Goal L3 PASS with full CPU FP32/FP16 coverage; its five-row DUSHA result is functional-smoke evidence only, not representative accuracy or benchmark quality.

Model metadata

What the model does

Classifies Russian speech audio into exactly five checkpoint classes: neutral, angry, positive, sad, and other. The pinned processor normalizes mono waveform samples at 16 kHz; the exported graph consumes one FP32 [1,16000] window and emits FP32 [1,5] logits.

  • Evidence: pinned checkpoint and feature-extractor configuration, current-main inspection and exported ONNX graph, and checkpoint model card. Confidence: verified.

Primary user stories

  • A user supplies Russian speech audio to obtain one of five checkpoint emotion labels for speech-emotion classification. Evidence: checkpoint model card and checkpoint id2label. Confidence: verified.
  • A Windows application runs fixed one-second waveform windows locally on CPU in FP32 or FP16 and consumes five logits. Evidence: current-main FP32 and FP16 builds and CPU performance runs. Confidence: verified.

Supported tasks

  • audio-classification: checkpoint, Transformers, and WinML surfaces. Evidence: HubertForSequenceClassification, WinML HubertOnnxConfig registration, and successful FP32/FP16 builds. Confidence: verified.
  • feature-extraction: Transformers and WinML surfaces. Evidence: WinML-registered HuBERT task lookup. Confidence: mapped.
  • automatic-speech-recognition: Transformers and WinML surfaces. Evidence: WinML-registered HuBERT task lookup. Confidence: mapped.

Model architecture

HubertForSequenceClassification
|-- hubert: HubertModel
|   |-- feature_extractor: HubertFeatureEncoder
|   |   `-- conv_layers[0..6]: 512 channels, kernels [10,3,3,3,3,2,2], strides [5,2,2,2,2,2,2]
|   |-- feature_projection: HubertFeatureProjection -> hidden size 1024
|   `-- encoder: HubertEncoderStableLayerNorm
|       |-- pos_conv_embed: HubertPositionalConvEmbedding
|       `-- layers[0..23]: HubertEncoderLayerStableLayerNorm
|           |-- attention: HubertAttention, 16 heads
|           `-- feed_forward: 1024 -> 4096 -> 1024, GELU
|-- mean pooling over encoded sequence
|-- projector: 1024 -> 256
`-- classifier: 256 -> 5 logits
  • Source/confidence: pinned checkpoint config, exporter hierarchy metadata, and exported ONNX topology (verified).

Validation and support evidence

1. Baseline

The frozen baseline is microsoft/winml-cli main commit 0876e5ae1c98a169a6137e092e0d7b30bf9cee33, verified again as the current origin/main before shipment. The locked environment used WinML CLI 0.3.0, Transformers 5.14.1, datasets 5.0.0, SoundFile 0.14.0, PyTorch 2.13.0, and onnxruntime-windowsml 1.24.5.202604171637.

Inspection passed for audio-classification with AutoModelForAudioClassification, HubertOnnxConfig, opset 17, input [1,16000], and logits output. Native Optimum vendor lookup had no HuBERT task registrations; WinML registration supplied feature extraction, automatic speech recognition, and audio classification. On exact main, winml eval --schema --task audio-classification exited 1 because audio classification was not yet supported without the dependency stack.

Recipe-free FP32 build passed in 134.7 s (80.1 s export, 36.5 s optimize). Its 100-iteration CPU run after 10 warmups measured mean 138.84 ms, p50 136.49 ms, p90 146.535 ms, 7.2 samples/s, peak RSS 1782.46 MB, and inference RSS delta 90.53 MB. The historical FP16 recipe also built on current main in 198.4 s (78.6 s export, 79.9 s optimize, 37.2 s FP16 conversion), producing 773 nodes, 420 FLOAT16 initializers, FP32 input/output, and a 631,400,642-byte external-data file. Its CPU run measured mean 159.563 ms, p50 160.209 ms, p90 165.296 ms, 6.27 samples/s, and total peak RSS delta 43.51 MB. FP16 was slower than FP32 on this CPU baseline and is retained as required supported precision, not as a performance recommendation.

Historical PR #1211 originally added the two CPU recipes and merged at 2cb8b003e8af5b7354aa9118e6e399e41da6dea2; this follow-up does not rewrite or reuse that historical branch.

2. Goal

  • Effort: L0. The existing-model change is a deterministic recipe refresh, not new HuBERT source.
  • Goal ceiling: L3. Success required exact CPU FP32/FP16 recipe builds, performance, named-input PyTorch parity, five-row functional smoke, standalone Analyze, compatibility, and CI parity.
  • Outcome: L0 recipe-only.
  • Baseline floor: L1 because recipe-free current-main FP32 export/build and CPU performance already passed.

No ceiling change or re-issued charter event occurred.

3. Outcome

Candidate e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3 (tree fef6e2bc539ef39360c737fab31c83d5cc92bc01) reached L3 PASS. Coverage is full for the chartered CPU FP32 and FP16 tuples, with no deferred tuple or blocker. The shipped L0 paths are:

  • examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu/audio-classification_fp32_config.json
  • examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu/audio-classification_fp16_config.json

The candidate is stacked directly on Draft PR #1329 exact head 8010667eb20d0eb08e916157a6cc5080c91d385d, which contains Draft PR #1326 exact head 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8. #1329 supplies strict scalar-string-to-checkpoint-ID evaluation needed by the authoritative DUSHA dataset. Its guards require a non-empty mapping, exact-string keys, integer checkpoint IDs, unique destinations, known and complete checkpoint-ID coverage, and rejection of unmapped observed values before inference. Both dependency heads had 9/9 successful checks and an exact-head structured APPROVE verdict at charter issuance.

Lane A knowledge is separately bound to Draft PR gim-home/ModelKitArtifacts#254, exact commit 01021859c4bfbbc1f200cad2bde4259dbd1fa88b, with its check successful and model-scale-by-skill present. It adds hubert-002 through hubert-005 while preserving hubert-001; no skill or knowledge file is included in this product PR.

No methodology friction observed. No new methodology finding was required because existing _meta-111 already governs the public-command closure exercised by this run.

4. Per-EP/device/precision results and Functional smoke Eval

Tier EP / Device Precision Verdict Mean p50 p90 Throughput RAM delta
L0 CPUExecutionProvider / cpu fp32 PASS - - - - -
L0 CPUExecutionProvider / cpu fp16 PASS - - - - -
L1 CPUExecutionProvider / cpu fp32 PASS 137.563 ms 135.837 ms 144.559 ms 7.27 samples/s inference +90.66 MB; total +90.94 MB
L1 CPUExecutionProvider / cpu fp16 PASS 167.434 ms 165.904 ms 179.107 ms 5.97 samples/s inference +43.36 MB; total +43.66 MB

Both L0 builds preserved FP32 input_values [1,16000] and FP32 logits [1,5]. FP32 produced 771 nodes and 420 FLOAT initializers. Explicit FP16 produced 773 nodes and 420 FLOAT16 initializers; its external-data size ratio to FP32 was 0.4999995945525307.

L2 named-input PyTorch parity passed. For deterministic FP32 [1,16000] input_values, FP32 cosine was 1.0 with max absolute difference 1.0132789611816406e-05 and mean absolute difference 6.186962127685547e-06; FP16 cosine was 0.9999966025352478 with max absolute difference 0.01157236099243164 and mean absolute difference 0.006389713287353516. Reference, FP32, and FP16 prediction ID was 4.

Functional smoke Eval: L3 PASS on candidate e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3, FP32 CPU, using xbgoose/dusha@e3e309de7723f4dc4a0230b4b9d725061d561044, config default, split test. Selection used the first authoritative crowd_test row per exact checkpoint class: rows 0, 1, 6, 7, and 20. Requested/selected/processed was 5/5/5 with 0 skipped or rejected; all 5/5 classes were represented. Schema, label semantics, and prediction semantics were verified.

The exact checkpoint mapping was neutral=0, angry=1, positive=2, sad=3, other=4; no synonym, translation, case-folding, positional, or inferred mapping was allowed. Each selected row used one forward over one 16,000-sample window, for five total forwards, with no multi-window fan-out. Long clips were truncated to the first 16,000 samples and short clips were right-padded.

All five selected WAVs were byte-identical to exact official crowd_test/wavs/ archive members and had zero Podcast overlap. Source attribution is DUSHA Crowd from xbgoose/dusha at the pinned revision, with the original archive at https://cdn.chatwm.opensmodel.sberdevices.ru/dusha/crowd.tar and license/provenance authority at https://github.com/salute-developers/golos/tree/5c5c5f87044803fcccdf7e149ef5384c95cff107. No audio was modified; no endorsement is implied; no additional restrictive terms are applied. The official terms permit the bounded local functional evaluation and scratch retention subject to retaining creator identification, copyright/license/disclaimer notices, and supplied URI or hyperlink.

Accuracy was 0.2; represented-class macro-F1 was 0.08. These are explicitly non-representative functional-smoke outputs only, proving end-to-end evaluator/model-path operability. They are not checkpoint-quality, population-accuracy, or benchmark-quality claims.

5. Delta

Relative to parent 8010667eb20d0eb08e916157a6cc5080c91d385d, each shipped recipe adds exactly one JSON pointer:

Recipe JSON pointer Old value New value
FP32 CPU /export/compatibility/transformers_attention absent / null eager
FP16 CPU /export/compatibility/transformers_attention absent / null eager

The FP32 recipe is otherwise identical to the planner's canonical generated baseline; FP16 is otherwise identical outside its retained recipe-owned quant.mode=fp16 block. Reducibility is consistent with the L0 charter. No source code changed, recipe-free acceptance was not required, and examples/recipes/README.md remains untouched.

The user-visible symptom was that the historical PR #1211 recipes predated Transformers 5 and omitted the generated eager-attention compatibility override required by the current export path. The root cause was stale checkpoint-specific recipe metadata, while current auto-config normalized the effective configuration and injected export.compatibility.transformers_attention=eager. The fix changes only that field in the two existing CPU recipes; it is generated compatibility metadata rather than checkpoint-conditional source logic. The blast radius is limited to this checkpoint's recipes and preserves task, loader, model type, opset 17, fixed FP32 I/O shapes, and FP16 weights with FP32 I/O. Exact FP32/FP16 recipe builds, graph verification, performance, parity, functional smoke, compatibility tests, Analyze, quality partitions, and git diff --check all passed.

6. Analyze summary - component level and op level

ANALYZE-PARTIAL-SUCCESS: both standalone --ep all --device all runs emitted complete 12-row JSON. Exit code 1 is retained because some provider groups were unavailable or unruled. This is static rule analysis, not runtime execution evidence.

Component-level summary

Artifact Architecture regions Mapping Actionable EP findings
fp32 7x waveform convolution; feature projection; positional convolution; 24x attention/feed-forward encoder; mean pool; projector; five-class head 517 mapped; 254 partial/unmapped; partial after optimization, 100% tagged in exporter metadata None
fp16 7x waveform convolution; feature projection; positional convolution; 24x attention/feed-forward encoder; mean pool; projector; five-class head 517 mapped; 256 partial/unmapped; partial after optimization, 100% tagged in exporter metadata None

Optimization rewrote 252 FP32 and 254 FP16 nodes to root-level generated names; these remain explicitly unmapped rather than inferred.

Op-level summary

Artifact Graph Dominant operators EP roll-up
fp32 771 operators / 13 types Reshape 244; Gemm 147; Transpose 111; Add 73; LayerNormalization 57 NvTensorRTRTX GPU, OpenVINO CPU/GPU/NPU, and QNN GPU/NPU classify all operators supported; no actionable partial/unsupported type
fp16 773 operators / 14 types Reshape 244; Gemm 147; Transpose 111; Add 73; LayerNormalization 57 NvTensorRTRTX GPU, OpenVINO CPU/GPU/NPU, and QNN GPU/NPU classify all operators supported; no actionable partial/unsupported type

CPU, CUDA, DirectML, MIGraphX, TensorRT, and VitisAI groups were runtime-unavailable or unruled in this static analysis and carry no support claim here.

7. Reproduce commands

The complete tester-owned public runner is embedded below, so it is available directly from this PR body rather than an uncommitted local file. It self-materializes the exact candidate, locked environment, pinned model and five-row dataset, exact mapping and attribution, validation helpers, and official pinned rules archive. Its exact UTF-8 source is 24,099 bytes with SHA-256 f3b8bde064da98b6ab7f7ffef5950bfcef389f95a50ec61b81da2c18d35aca56.

Save the following block as public-reproducible-commands.ps1, then invoke it from a clean public environment with a caller-selected path that does not yet exist:

param(
    [Parameter(Mandatory = $true)]
    [string]$Out
)

$ErrorActionPreference = 'Stop'
$env:PYTHONUTF8 = '1'
$candidate = 'e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3'
$candidateTree = 'fef6e2bc539ef39360c737fab31c83d5cc92bc01'
$dependency = '8010667eb20d0eb08e916157a6cc5080c91d385d'
$containedDependency = '1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8'
$rulesSha256 = '32030e3169a837b84798d6e321e07f418416ec3d3070d9af049385bf619a8b1d'
$modelId = 'xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned'
$commandRecords = [Collections.Generic.List[object]]::new()
$runStartedAt = [DateTime]::UtcNow

if (Test-Path $Out) {
    throw "Output root must not already exist: $Out"
}
$Out = [IO.Path]::GetFullPath($Out)
$repo = Join-Path $Out 'winml-cli'
$rulesRoot = Join-Path $Out 'rules'
$assets = Join-Path $Out 'assets'
$results = Join-Path $Out 'results'
$helpers = Join-Path $Out 'helpers'
$logs = Join-Path $Out 'logs'
New-Item -ItemType Directory -Path $Out, $results, $helpers, $logs | Out-Null
$publishedScript = Join-Path $Out 'public-reproducible-commands.ps1'
[IO.File]::WriteAllBytes($publishedScript, [IO.File]::ReadAllBytes($PSCommandPath))

function Invoke-Checked {
    param(
        [Parameter(Mandatory = $true)][string]$Name,
        [Parameter(Mandatory = $true)][string]$File,
        [Parameter(Mandatory = $true)][string[]]$Arguments,
        [int[]]$AcceptedExitCodes = @(0)
    )
    $log = Join-Path $logs "$Name.log"
    $startedAt = [DateTime]::UtcNow
    $stopwatch = [Diagnostics.Stopwatch]::StartNew()
    $previousErrorActionPreference = $ErrorActionPreference
    try {
        $ErrorActionPreference = 'Continue'
        & $File @Arguments 2>&1 | Tee-Object -FilePath $log
        $exitCode = $LASTEXITCODE
    } finally {
        $stopwatch.Stop()
        $ErrorActionPreference = $previousErrorActionPreference
    }
    $commandRecords.Add([ordered]@{
        name = $Name
        file = $File
        arguments = $Arguments
        command = (($File, $Arguments) -join ' ')
        started_at_utc = $startedAt.ToString('o')
        ended_at_utc = [DateTime]::UtcNow.ToString('o')
        elapsed_seconds = [Math]::Round($stopwatch.Elapsed.TotalSeconds, 3)
        exit_code = $exitCode
        accepted_exit_codes = $AcceptedExitCodes
        log = $log
        log_sha256 = if (Test-Path $log) { (Get-FileHash $log -Algorithm SHA256).Hash.ToLowerInvariant() } else { $null }
    }) | Out-Null
    if ($AcceptedExitCodes -notcontains $exitCode) {
        throw "$Name failed with exit code $exitCode; see $log"
    }
    return $exitCode
}

Invoke-Checked -Name 'clone-candidate' -File 'git' -Arguments @(
    'clone', '--no-checkout', 'https://github.com/microsoft/winml-cli.git', $repo
)
Invoke-Checked -Name 'fetch-candidate' -File 'git' -Arguments @(
    '-C', $repo, 'fetch', '--depth', '200', 'origin', 'pull/1333/head:refs/remotes/origin/pr-1333'
)
if ((git -C $repo rev-parse refs/remotes/origin/pr-1333).Trim() -ne $candidate) {
    throw 'PR #1333 head identity mismatch'
}
Invoke-Checked -Name 'checkout-candidate' -File 'git' -Arguments @('-C', $repo, 'checkout', '--detach', $candidate)
if ((git -C $repo rev-parse 'HEAD^{tree}').Trim() -ne $candidateTree) { throw 'Candidate tree mismatch' }
if ((git -C $repo rev-parse 'HEAD^').Trim() -ne $dependency) { throw 'Candidate parent mismatch' }
if ((git -C $repo status --porcelain) -ne $null) { throw 'Candidate checkout is dirty' }
Invoke-Checked -Name 'fetch-dependency' -File 'git' -Arguments @(
    '-C', $repo, 'fetch', '--depth', '200', 'origin', 'pull/1329/head:refs/remotes/origin/pr-1329'
)
if ((git -C $repo rev-parse refs/remotes/origin/pr-1329).Trim() -ne $dependency) {
    throw 'PR #1329 head identity mismatch'
}
Invoke-Checked -Name 'fetch-contained-dependency' -File 'git' -Arguments @(
    '-C', $repo, 'fetch', '--depth', '200', 'origin', 'pull/1326/head:refs/remotes/origin/pr-1326'
)
if ((git -C $repo rev-parse refs/remotes/origin/pr-1326).Trim() -ne $containedDependency) {
    throw 'PR #1326 head identity mismatch'
}
Invoke-Checked -Name 'dependency-ancestry' -File 'git' -Arguments @(
    '-C', $repo, 'merge-base', '--is-ancestor', $containedDependency, $dependency
)
$recipeRoot = Join-Path $repo 'examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu'
$fp32Recipe = Join-Path $recipeRoot 'audio-classification_fp32_config.json'
$fp16Recipe = Join-Path $recipeRoot 'audio-classification_fp16_config.json'
if ((Get-FileHash $fp32Recipe -Algorithm SHA256).Hash.ToLowerInvariant() -ne '498dfd4f5d6fd0fd1e18eb0c6be8a48edb97c566756696d61de7af34691cf2ac') { throw 'FP32 recipe hash mismatch' }
if ((Get-FileHash $fp16Recipe -Algorithm SHA256).Hash.ToLowerInvariant() -ne '8872d0c71f5911847fab19b9f3e0de712779e5fc4becff2eab9f45cc2acc3ae4') { throw 'FP16 recipe hash mismatch' }

Invoke-Checked -Name 'sync' -File 'uv' -Arguments @('--directory', $repo, 'sync', '--locked', '--all-extras', '--all-groups')
Invoke-Checked -Name 'ort-repair' -File 'uv' -Arguments @(
    '--directory', $repo, 'pip', 'install', '--python', '.venv/Scripts/python.exe', '--reinstall', '--no-deps', 'onnxruntime-windowsml==1.24.5.202604171637'
)
$python = Join-Path $repo '.venv/Scripts/python.exe'
$winml = Join-Path $repo '.venv/Scripts/winml.exe'

$acquire = @'
from __future__ import annotations
import argparse, hashlib, json
from pathlib import Path
from datasets import Audio, Dataset, load_dataset
from huggingface_hub import snapshot_download
MODEL_ID = "xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned"
MODEL_REVISION = "2eaa20433d7e6d5be7587b5f9f0057527ed274ba"
DATASET_ID = "xbgoose/dusha"
DATASET_REVISION = "e3e309de7723f4dc4a0230b4b9d725061d561044"
ROWS = {0:("neutral","9e9961c53ca6eeb440b217e539fbf46c.wav","ada344669196aa8b9af02695f032c75804cbfbcee53b270253e92715529939d9"),1:("sad","0166f65a30354db8282682b1a280e64c.wav","85adceb46c2615ee737d4ab451f85738fc46e8cb509f870df640e5787705823e"),6:("angry","b6392f1564e3e807c8fa9aecb04a3434.wav","15d1dc08596911609578a2e5ed1b7170f51acdb4eddd7de7c6efaef28e05655b"),7:("positive","edde29c071343653eced8877747ed54b.wav","36588ebcc27a35acf1fb0af3b0e4a8fc2238214285b40f856d4b2221f37dd8aa"),20:("other","fa61e64e78f350b31832866854a8276b.wav","e7f360a4ee07d28453ad8420d39ebb5520f43970ab2a716465de2b0797a21e3c")}
MAPPING={"neutral":0,"angry":1,"positive":2,"sad":3,"other":4}
parser=argparse.ArgumentParser(); parser.add_argument("--assets",type=Path,required=True); args=parser.parse_args()
args.assets.mkdir(parents=True); model=args.assets/"model"; snapshot_download(repo_id=MODEL_ID,revision=MODEL_REVISION,local_dir=model)
stream=load_dataset(DATASET_ID,"default",split="test",revision=DATASET_REVISION,streaming=True).cast_column("audio",Audio(decode=False))
audio_dir=args.assets/"audio"; audio_dir.mkdir(); paths=[]; labels=[]; rows=[]
for index,row in enumerate(stream):
    if index not in ROWS:
        if index>=max(ROWS): break
        continue
    label,name,want=ROWS[index]; assert row["emotion"]==label; payload=row["audio"].get("bytes")
    if payload is None: payload=Path(row["audio"]["path"]).read_bytes()
    assert hashlib.sha256(payload).hexdigest()==want; path=audio_dir/name; path.write_bytes(payload)
    paths.append(str(path)); labels.append(label); rows.append({"source_row":index,"label":label,"filename":name,"bytes":len(payload),"sha256":want,"official_member":f"crowd_test/wavs/{name}"})
assert [r["source_row"] for r in rows]==list(ROWS)
dataset=Dataset.from_dict({"audio":paths,"emotion":labels}).cast_column("audio",Audio(sampling_rate=16000,decode=False)); dataset_path=args.assets/"dusha-five-class"; dataset.save_to_disk(dataset_path)
(args.assets/"label-mapping.json").write_text(json.dumps(MAPPING,indent=2)+"\n",encoding="utf-8")
(args.assets/"DUSHA-ATTRIBUTION.md").write_text("# DUSHA functional-smoke inputs\n\nSource: xbgoose/dusha at revision e3e309de7723f4dc4a0230b4b9d725061d561044.\nOriginal Crowd archive: https://cdn.chatwm.opensmodel.sberdevices.ru/dusha/crowd.tar\nLicense/provenance authority: https://github.com/salute-developers/golos/tree/5c5c5f87044803fcccdf7e149ef5384c95cff107\nNo audio was modified; rows are retained solely for bounded local functional evaluation.\nNo endorsement is implied and no additional restrictive terms are applied.\n",encoding="utf-8")
manifest={"model":{"id":MODEL_ID,"revision":MODEL_REVISION,"path":str(model)},"dataset":{"id":DATASET_ID,"revision":DATASET_REVISION,"config":"default","split":"test","rows":rows,"path":str(dataset_path)},"label_mapping":MAPPING,"requested":5,"selected":5}
(args.assets/"input-manifest.json").write_text(json.dumps(manifest,indent=2)+"\n",encoding="utf-8"); print(json.dumps(manifest,indent=2))
'@
[IO.File]::WriteAllText((Join-Path $helpers 'acquire.py'), $acquire.Replace("`r`n", "`n"), [Text.UTF8Encoding]::new($false))

$verify = @'
from __future__ import annotations
import argparse, hashlib, json
from collections import Counter
from pathlib import Path
import onnx
from onnx import TensorProto
def digest(path):
    h=hashlib.sha256()
    with path.open("rb") as stream:
        for chunk in iter(lambda:stream.read(1024*1024),b""): h.update(chunk)
    return h.hexdigest()
def inspect(path,precision):
    model=onnx.load(path,load_external_data=False)
    shape=lambda value:[dim.dim_value if dim.HasField("dim_value") else dim.dim_param for dim in value.type.tensor_type.shape.dim]
    assert [(v.name,TensorProto.DataType.Name(v.type.tensor_type.elem_type),shape(v)) for v in model.graph.input]==[("input_values","FLOAT",[1,16000])]
    assert [(v.name,TensorProto.DataType.Name(v.type.tensor_type.elem_type),shape(v)) for v in model.graph.output]==[("logits","FLOAT",[1,5])]
    assert {item.domain:item.version for item in model.opset_import}[""]==17
    counts=Counter(TensorProto.DataType.Name(item.data_type) for item in model.graph.initializer)
    assert counts["FLOAT16" if precision=="fp16" else "FLOAT"]==420
    data=path.with_name("model.onnx.data"); assert data.is_file()
    return {"path":str(path),"sha256":digest(path),"data_bytes":data.stat().st_size,"data_sha256":digest(data),"nodes":len(model.graph.node),"initializer_dtype_counts":dict(counts)}
parser=argparse.ArgumentParser(); parser.add_argument("--fp32",type=Path,required=True); parser.add_argument("--fp16",type=Path,required=True); parser.add_argument("--output",type=Path,required=True); args=parser.parse_args()
result={"fp32":inspect(args.fp32,"fp32"),"fp16":inspect(args.fp16,"fp16")}; result["external_data_size_ratio_fp16_to_fp32"]=result["fp16"]["data_bytes"]/result["fp32"]["data_bytes"]; assert .45<result["external_data_size_ratio_fp16_to_fp32"]<.55
args.output.write_text(json.dumps(result,indent=2)+"\n",encoding="utf-8"); print(json.dumps(result,indent=2))
'@
[IO.File]::WriteAllText((Join-Path $helpers 'verify.py'), $verify.Replace("`r`n", "`n"), [Text.UTF8Encoding]::new($false))

$parity = @'
from __future__ import annotations
import argparse, hashlib, json
from pathlib import Path
import numpy as np, onnxruntime as ort, torch
from transformers import AutoModelForAudioClassification
parser=argparse.ArgumentParser(); parser.add_argument("--reference",type=Path,required=True); parser.add_argument("--fp32",type=Path,required=True); parser.add_argument("--fp16",type=Path,required=True); parser.add_argument("--output",type=Path,required=True); args=parser.parse_args()
axis=np.arange(16000,dtype=np.float32)/np.float32(16000); values=(np.sin(2*np.pi*np.float32(220)*axis)+np.float32(.25)*np.sin(2*np.pi*np.float32(440)*axis)).reshape(1,16000).astype(np.float32)
np.random.seed(0); torch.manual_seed(0); model=AutoModelForAudioClassification.from_pretrained(args.reference,attn_implementation="eager").eval()
with torch.inference_mode(): reference=model(input_values=torch.from_numpy(values)).logits.float().cpu().numpy()
artifacts={}
for precision,path in (("fp32",args.fp32),("fp16",args.fp16)):
    session=ort.InferenceSession(str(path.resolve()),providers=["CPUExecutionProvider"]); candidate=session.run(["logits"],{"input_values":values})[0]; delta=np.abs(reference.astype(np.float64).ravel()-candidate.astype(np.float64).ravel()); artifacts[precision]={"providers":session.get_providers(),"cosine":float(np.dot(reference.ravel(),candidate.ravel())/(np.linalg.norm(reference)*np.linalg.norm(candidate))),"max_abs":float(delta.max()),"mean_abs":float(delta.mean()),"prediction_id":int(np.argmax(candidate))}
result={"input_name":"input_values","input_shape":[1,16000],"input_dtype":"float32","input_sha256":hashlib.sha256(values.tobytes()).hexdigest(),"reference_prediction_id":int(np.argmax(reference)),"artifacts":artifacts}; args.output.write_text(json.dumps(result,indent=2)+"\n",encoding="utf-8"); print(json.dumps(result,indent=2))
'@
[IO.File]::WriteAllText((Join-Path $helpers 'parity.py'), $parity.Replace("`r`n", "`n"), [Text.UTF8Encoding]::new($false))

$helperManifest = Get-ChildItem -LiteralPath $helpers -File | Sort-Object Name | ForEach-Object {
    [ordered]@{
        name = $_.Name
        bytes = $_.Length
        sha256 = (Get-FileHash $_.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
        path = $_.FullName
        materialized_by_public_sequence = $true
    }
}
$helperManifest | ConvertTo-Json -Depth 5 | Set-Content -Encoding utf8 (Join-Path $Out 'helper-manifest.json')

Invoke-Checked -Name 'acquire-inputs' -File $python -Arguments @((Join-Path $helpers 'acquire.py'), '--assets', $assets)
Invoke-Checked -Name 'providers' -File $python -Arguments @('-c', 'import json, onnxruntime as ort; print(json.dumps(ort.get_available_providers()))')
$audioEvaluatorTests = Join-Path $repo 'tests/unit/eval/test_audio_classification_evaluator.py'
Invoke-Checked -Name 'compatibility-scalar-labels' -File $python -Arguments @('-m','pytest','-q',$audioEvaluatorTests,'-k','scalar_string or scalar_binary')
Invoke-Checked -Name 'compatibility-prevalidation' -File $python -Arguments @('-m','pytest','-q',($audioEvaluatorTests + '::test_scalar_string_label_mapping_rejects_unmapped_observed_value'))

$fp32Out = Join-Path $results 'fp32'
$fp16Out = Join-Path $results 'fp16'
Invoke-Checked -Name 'l0-fp32' -File $winml -Arguments @('build','-c',$fp32Recipe,'-m',(Join-Path $assets 'model'),'-o',$fp32Out,'--rebuild','--no-color')
Invoke-Checked -Name 'l0-fp16' -File $winml -Arguments @('build','-c',$fp16Recipe,'-m',(Join-Path $assets 'model'),'-o',$fp16Out,'--precision','fp16','--rebuild','--no-color')
Invoke-Checked -Name 'l0-structure' -File $python -Arguments @((Join-Path $helpers 'verify.py'),'--fp32',(Join-Path $fp32Out 'model.onnx'),'--fp16',(Join-Path $fp16Out 'model.onnx'),'--output',(Join-Path $results 'structure.json'))
Invoke-Checked -Name 'l1-fp32' -File $winml -Arguments @('perf','-m',(Join-Path $fp32Out 'model.onnx'),'--device','cpu','--ep','cpu','--warmup','10','--iterations','100','--memory','--format','json','--output',(Join-Path $results 'perf-fp32.json'),'--no-color')
Invoke-Checked -Name 'l1-fp16' -File $winml -Arguments @('perf','-m',(Join-Path $fp16Out 'model.onnx'),'--device','cpu','--ep','cpu','--warmup','10','--iterations','100','--memory','--format','json','--output',(Join-Path $results 'perf-fp16.json'),'--no-color')
Invoke-Checked -Name 'l2' -File $python -Arguments @((Join-Path $helpers 'parity.py'),'--reference',(Join-Path $assets 'model'),'--fp32',(Join-Path $fp32Out 'model.onnx'),'--fp16',(Join-Path $fp16Out 'model.onnx'),'--output',(Join-Path $results 'parity.json'))
Invoke-Checked -Name 'l3-schema' -File $winml -Arguments @('eval','--schema','--task','audio-classification','--no-color')
Invoke-Checked -Name 'l3' -File $winml -Arguments @('eval','-m',(Join-Path $fp32Out 'model.onnx'),'--model-id',(Join-Path $assets 'model'),'--task','audio-classification','--dataset',(Join-Path $assets 'dusha-five-class'),'--split','test','--samples','5','--no-shuffle','--column','input_column=audio','--column','label_column=emotion','--label-mapping',(Join-Path $assets 'label-mapping.json'),'--device','cpu','--ep','cpu','--format','json','--output',(Join-Path $results 'eval.json'),'--no-color')

$rulesArchive = Join-Path $Out 'rules-v0.3.0.zip'
$rulesStartedAt = [DateTime]::UtcNow
$rulesStopwatch = [Diagnostics.Stopwatch]::StartNew()
Invoke-WebRequest -Uri 'https://github.com/microsoft/winml-cli/releases/download/v0.3.0/rules-v0.3.0.zip' -OutFile $rulesArchive
$rulesStopwatch.Stop()
$commandRecords.Add([ordered]@{
    name = 'acquire-rules'
    file = 'Invoke-WebRequest'
    arguments = @('-Uri', 'https://github.com/microsoft/winml-cli/releases/download/v0.3.0/rules-v0.3.0.zip', '-OutFile', $rulesArchive)
    command = "Invoke-WebRequest -Uri https://github.com/microsoft/winml-cli/releases/download/v0.3.0/rules-v0.3.0.zip -OutFile $rulesArchive"
    started_at_utc = $rulesStartedAt.ToString('o')
    ended_at_utc = [DateTime]::UtcNow.ToString('o')
    elapsed_seconds = [Math]::Round($rulesStopwatch.Elapsed.TotalSeconds, 3)
    exit_code = 0
    accepted_exit_codes = @(0)
    log = $null
    log_sha256 = $null
}) | Out-Null
if ((Get-FileHash $rulesArchive -Algorithm SHA256).Hash.ToLowerInvariant() -ne $rulesSha256) { throw 'Official rules archive hash mismatch' }
Expand-Archive -Path $rulesArchive -DestinationPath $rulesRoot
$env:WINMLCLI_RULES_DIR = $rulesRoot
Invoke-Checked -Name 'analyze-fp32' -File $winml -Arguments @('analyze','--model',(Join-Path $fp32Out 'model.onnx'),'--ep','all','--device','all','--output',(Join-Path $results 'analyze-fp32.json'),'--no-color') -AcceptedExitCodes @(0,1)
Invoke-Checked -Name 'analyze-fp16' -File $winml -Arguments @('analyze','--model',(Join-Path $fp16Out 'model.onnx'),'--ep','all','--device','all','--output',(Join-Path $results 'analyze-fp16.json'),'--no-color') -AcceptedExitCodes @(0,1)
foreach ($analysisPath in @((Join-Path $results 'analyze-fp32.json'),(Join-Path $results 'analyze-fp16.json'))) {
    $analysis = Get-Content $analysisPath -Raw | ConvertFrom-Json
    if ($analysis.Count -ne 12 -or $analysis[0].metadata.total_operators -lt 1) { throw "Incomplete analyze result: $analysisPath" }
}

Push-Location $repo
try {
    Invoke-Checked -Name 'quality-license' -File 'uv' -Arguments @('run','--no-sync','pre-commit','run','insert-license','--all-files')
    Invoke-Checked -Name 'quality-ruff' -File 'uv' -Arguments @('run','--no-sync','ruff','check','src/','tests/')
    Invoke-Checked -Name 'quality-mypy' -File 'uv' -Arguments @('run','--no-sync','mypy','-p','winml.modelkit')
    Invoke-Checked -Name 'quality-format' -File 'git' -Arguments @('diff','--check',($dependency + '..' + $candidate))
    $marker = 'not e2e and not npu and not gpu'
    Invoke-Checked -Name 'quality-analyze' -File $python -Arguments @('-m','pytest','tests/unit/analyze','--tb=short','--no-cov','-m',$marker)
    Invoke-Checked -Name 'quality-models' -File $python -Arguments @('-m','pytest','tests/unit/models','tests/unit/loader','tests/unit/datasets','tests/unit/export','--tb=short','--no-cov','-m',$marker)
    Invoke-Checked -Name 'quality-optim' -File $python -Arguments @('-m','pytest','tests/unit/optim','--tb=short','--no-cov','-m',$marker)
    Invoke-Checked -Name 'quality-commands' -File $python -Arguments @('-m','pytest','tests/unit/commands','tests/unit/config','tests/unit/build','tests/unit/compiler','tests/unit/session','tests/unit/eval','--tb=short','--no-cov','-m',$marker)
    Invoke-Checked -Name 'quality-remaining' -File $python -Arguments @('-m','pytest','tests/unit/core','tests/unit/onnx','tests/unit/cache','tests/unit/utils','tests/unit/test_helpers','tests/unit/sysinfo','tests/unit/inspect','tests/unit/optracing','tests/unit/serve','tests/regression','tests/cli','--tb=short','--no-cov','-m',$marker)
} finally {
    Pop-Location
}
if ((git -C $repo rev-parse HEAD).Trim() -ne $candidate) { throw 'Candidate HEAD changed during validation' }
if ((git -C $repo status --porcelain) -ne $null) { throw 'Public candidate checkout was modified by validation' }

$closures = @(
    [ordered]@{ name = 'Prerequisites'; state = 'CLOSED'; command_names = @('clone-candidate','fetch-candidate','checkout-candidate','fetch-dependency','fetch-contained-dependency','dependency-ancestry','sync','ort-repair','acquire-inputs','providers','compatibility-scalar-labels','compatibility-prevalidation') }
    [ordered]@{ name = 'L0 FP32'; state = 'CLOSED'; command_names = @('l0-fp32','l0-structure') }
    [ordered]@{ name = 'L0 FP16'; state = 'CLOSED'; command_names = @('l0-fp16','l0-structure'); explicit_precision = '--precision fp16' }
    [ordered]@{ name = 'L1 FP32'; state = 'CLOSED'; command_names = @('l1-fp32') }
    [ordered]@{ name = 'L1 FP16'; state = 'CLOSED'; command_names = @('l1-fp16') }
    [ordered]@{ name = 'L2'; state = 'CLOSED'; command_names = @('l2'); named_input = 'input_values' }
    [ordered]@{ name = 'L3'; state = 'CLOSED'; command_names = @('l3-schema','l3'); exact_samples = 5 }
    [ordered]@{ name = 'Analyze FP32'; state = 'CLOSED'; command_names = @('acquire-rules','analyze-fp32') }
    [ordered]@{ name = 'Analyze FP16'; state = 'CLOSED'; command_names = @('acquire-rules','analyze-fp16') }
    [ordered]@{ name = 'Quality'; state = 'CLOSED'; command_names = @('quality-license','quality-ruff','quality-mypy','quality-format','quality-analyze','quality-models','quality-optim','quality-commands','quality-remaining') }
)
$selectedAssetFiles = @(
    (Join-Path $assets 'input-manifest.json'),
    (Join-Path $assets 'label-mapping.json'),
    (Join-Path $assets 'DUSHA-ATTRIBUTION.md')
) + @(Get-ChildItem -LiteralPath (Join-Path $assets 'audio') -File | Select-Object -ExpandProperty FullName)
$outputFiles = @(
    Get-ChildItem -LiteralPath $results -Recurse -File
    Get-ChildItem -LiteralPath $helpers -File
    Get-ChildItem -LiteralPath $logs -File
    Get-Item -LiteralPath $rulesArchive
    Get-Item -LiteralPath $publishedScript
    Get-Item -LiteralPath $selectedAssetFiles
)
$outputInventory = $outputFiles | Sort-Object FullName -Unique | ForEach-Object {
    [ordered]@{
        path = $_.FullName
        bytes = $_.Length
        sha256 = (Get-FileHash $_.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
    }
}
$commandRecords | ConvertTo-Json -Depth 8 | Set-Content -Encoding utf8 (Join-Path $Out 'command-records.json')
$outputInventory | ConvertTo-Json -Depth 5 | Set-Content -Encoding utf8 (Join-Path $Out 'output-inventory.json')
$summary = [ordered]@{
    schema_version = '1.0'
    role = 'TESTER'
    repair = 'PR #1333 public-command closure'
    candidate_sha = $candidate
    candidate_tree = $candidateTree
    dependency_sha = $dependency
    contained_dependency_sha = $containedDependency
    checkout_path = $repo
    checkout_head = (git -C $repo rev-parse HEAD).Trim()
    checkout_tree = (git -C $repo rev-parse 'HEAD^{tree}').Trim()
    clean_checkout = $true
    public_sequence_self_materializes_helpers = $true
    public_sequence_uses_private_paths = $false
    public_sequence_path = $publishedScript
    public_sequence_bytes = (Get-Item $publishedScript).Length
    public_sequence_sha256 = (Get-FileHash $publishedScript -Algorithm SHA256).Hash.ToLowerInvariant()
    helper_manifest = 'helper-manifest.json'
    command_records = 'command-records.json'
    output_inventory = 'output-inventory.json'
    started_at_utc = $runStartedAt.ToString('o')
    ended_at_utc = [DateTime]::UtcNow.ToString('o')
    elapsed_seconds = [Math]::Round(([DateTime]::UtcNow - $runStartedAt).TotalSeconds, 3)
    closure_count = $closures.Count
    closed_count = @($closures | Where-Object { $_.state -eq 'CLOSED' }).Count
    closure_rows = $closures
}
$summary | ConvertTo-Json -Depth 8 | Set-Content -Encoding utf8 (Join-Path $Out 'public-execution-summary.json')
$OUT = 'C:\path\to\new-empty-successor-root'
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\public-reproducible-commands.ps1 -Out $OUT

The tester executed this exact 32-command sequence from a new empty root in 2,112.807 seconds with zero unaccepted exits. The fresh run closed all ten tester-owned rows:

Closure row State
Prerequisites CLOSED
L0 FP32 CLOSED
L0 FP16 CLOSED
L1 FP32 CLOSED
L1 FP16 CLOSED
L2 CLOSED
L3 CLOSED
Analyze FP32 CLOSED
Analyze FP16 CLOSED
Quality CLOSED

Fresh successor outcomes: FP32 model SHA-256 62802015e81c64710829d681107aa56c350e3b2aa4388ea51c99e49022a707be; FP16 model SHA-256 8de091be816b5913646745158cf4b709d5b298bd25ab5199f55f56e4c4ee75cb; FP16/FP32 external-data ratio 0.4999995945525307. FP32 mean/p50/p90 was 143.148/140.517/155.327 ms at 6.99 samples/s; FP16 was 172.447/171.031/184.086 ms at 5.8 samples/s. L2 cosine was FP32 1.0 and FP16 0.9999966025352478, with reference, FP32, and FP16 prediction ID 4. L3 requested/selected/processed 5/5/5, rejected 0, represented all five classes, and reported accuracy 0.2 and macro-F1 0.08 as functional-smoke evidence only. Both Analyze runs emitted metadata and all 12 rows; their expected partial-success exit code 1 was accepted.

Quality evidence: license hook PASS; Ruff PASS; mypy found no issues in 439 source files; Analyze partition 1,526 passed/45 skipped; models partition 1,538 passed/6 skipped/2 xfailed; optimization partition 848 passed/16 skipped/1 xfailed; commands partition 3,648 passed/9 skipped; remaining partition 926 passed/2 skipped/1 deselected; candidate git diff --check PASS. The non-workflow diagnostic ruff format --check src/ tests/ reported 92 pre-existing files that would be reformatted, changed no files, and is not a current PR-triggered workflow gate.

@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Aug 22, 2026
@ssss141414

Copy link
Copy Markdown
Contributor Author

REQUEST_CHANGES - independent reviewer verdict

PR: #1333
Reviewed head: e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3
Stacked base / PR #1329 head: 8010667eb20d0eb08e916157a6cc5080c91d385d
Contained PR #1326 head: 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8
Current main: 0876e5ae1c98a169a6137e092e0d7b30bf9cee33

Blocking finding

  1. [explainer + tester] The advertised public reproduction command is not accessible from a clean checkout. The live PR body instructs a user to run:

    $OUT='temp/russian-emotion-support-repro'
    powershell -NoProfile -ExecutionPolicy Bypass -File public-reproducible-commands.ps1 -Out $OUT

    At the exact reviewed head, git ls-tree -r HEAD --name-only contains no public-reproducible-commands.ps1; both root and recursive Test-Path probes return false. Running the advertised command from a fresh detached checkout exits -196608 with The argument 'public-reproducible-commands.ps1' to the -File parameter does not exist, before creating $OUT. The tester's successful script exists only in the internal scratch handoff; its internal existence and SHA-256 do not make the PR's public instructions runnable.

    Required fix: the explainer must replace the inaccessible entry point with a complete command sequence that is genuinely available to a clean public checkout, or reference a permitted durable public location. The tester must then execute exactly that publicly advertised sequence from a new clean checkout, close all ten public-reproduction rows from that run, and refresh the sealed tester handoff and PR evidence. Keep internal scratch paths out of the public body.

Independently verified passes

  • Diff and scope: the PR-owned delta is exactly two recipe JSON files, six inserted lines total: each recipe adds only /export/compatibility/transformers_attention = eager. FP16 retains quant.mode = fp16 and FP32 I/O. examples/recipes/README.md is unchanged; git diff --check passes; no Python source is changed.
  • Model and semantics: pinned model/revision/class/task, FP32 [1,16000] input, FP32 [1,5] logits, and exact mapping neutral=0, angry=1, positive=2, sad=3, other=4 agree across charter, model snapshot, recipes, evaluator evidence, and body. The five WAV hashes match the authoritative DUSHA Crowd archive members and have zero Podcast overlap.
  • L0/L1: independently rehashed artifacts match the sealed evidence. FP32 has 420 FLOAT initializers and 1,262,802,308-byte external data; FP16 has 420 FLOAT16 initializers and 631,400,642-byte external data (ratio 0.4999995946). CPU perf is FP32 mean/p50/p90 137.563/135.837/144.559 ms, 7.27 samples/s; FP16 167.434/165.904/179.107 ms, 5.97 samples/s, with memory fields present.
  • L2/L3 and Analyze: parity evidence reports FP32 cosine 1.0, max-abs 1.0133e-05, and FP16 cosine 0.9999966, max-abs 0.0115724, with prediction ID 4 throughout. Final-SHA FP32 CPU functional smoke processes all 5/5 pinned rows, covers all five classes, rejects none, and reports accuracy 0.2 / macro-F1 0.08 as non-representative operability evidence. Both Analyze artifacts retain component mapping gaps and complete 12-group op classifications; no runtime support is inferred from static rules.
  • Quality: license hook and full Ruff passed; mypy found no issues in 439 source files; focused audio evaluator passed 18/18; commands/eval passed 3,648 with 9 skipped; models/export passed 1,538 with 6 skipped and 2 xfailed. All ran independently on the exact detached head, which remained unchanged and clean.
  • GitHub and handoffs: PR recipe(hubert): refresh Russian emotion CPU configs #1333 remains OPEN/DRAFT with model-scale-by-skill; its sole license/cla check is COMPLETED/SUCCESS. Stacked PRs feat(eval): support Common Voice gender classification #1329 and recipe(ast): use eager attention for AudioSet model #1326 remain at the exact reviewed heads with 9/9 successful checks each. Lane A PR refactor: move remaining network-dependent tests to integration #254 remains OPEN/DRAFT at 01021859c4bfbbc1f200cad2bde4259dbd1fa88b, labeled, with its check successful. Planner, producer, tester, learner, and explainer handoff hashes revalidated.
  • Conversation gate: enumerated 0 issue comments, 0 line comments, 0 submitted reviews, and all 0 review threads; open threads = 0.

Ownership routing

  • Explainer: repair the PR body's Reproduce commands using only clean-checkout-accessible or durable public inputs; keep the PR draft and retain the label.
  • Tester: rerun all ten closure rows from the exact repaired public sequence in a fresh checkout and reseal the verdict/report/integrity evidence.
  • Reviewer: re-enter from a fresh exact-head checkout after the refreshed handoffs and body are published; re-enumerate checks and all review threads before the next verdict.

@ssss141414

Copy link
Copy Markdown
Contributor Author

Repair response for issue comment #5381789642

The canonical PR body has been updated without changing code. It remains bound to candidate e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3, tree fef6e2bc539ef39360c737fab31c83d5cc92bc01, stacked base / PR #1329 head 8010667eb20d0eb08e916157a6cc5080c91d385d, and contained PR #1326 head 1bd919b583aa8ccd974e35fa6a1a132ac7e1c9f8.

The repaired body no longer points to C:\temp or merely names an uncommitted file. Section 7 now embeds the complete tester-owned self-materializing public-reproducible-commands.ps1 source in a copy-pasteable PowerShell block, followed by its invocation with a caller-selected new directory. The embedded source is content-identical to the tester authority after line-ending normalization; the tester source is 24,099 bytes with SHA-256 f3b8bde064da98b6ab7f7ffef5950bfcef389f95a50ec61b81da2c18d35aca56. The canonical live PR body SHA-256 is 79c6345b7b26757fb003e662c8b1c1f8ec05e7f35ac86b7b302bdebd922f28bd.

Tester successor root authority bindings under 20260823-e2667d9-successor-r2:

  • PUBLIC_REPRODUCTION_HANDOFF.md: 59a7eb69151e99dbe2f2f939b6c756eb7033af332ae7bef105a1d82de712030c
  • successor-verdict.json: ca4284b00da1d2b5d055abce11c1328b36f16197399710ba6d00d6597299e1c5
  • integrity-manifest.json: 8c7a082ab8bc6bc3f483ea56f62a0d6017d9c0818540c9da9cf0d2b7650f493f
  • public-execution-summary.json: 0b681114c6400f8425f2961b154453c1beb4e53595d010291b7a82cf43205a1e
  • command-records.json: 27295dab1c3ac21b5b2cce3a437533aabff42292651d91e64c0a4edb3314ba9b
  • helper-manifest.json: 9851ab6889b605d6e6cec0fd5fcd4cedfbfdf7387549a36f1c1a0dd22aaa8dbb
  • output-inventory.json: cecdbf3ed7a25a075ca9e5fb5e5062431bd0d2fc5b87269a8f904b0f6bcef891

The tester executed that exact public sequence from a new empty root for 2,112.807 seconds: 32 recorded commands, zero unaccepted exits, no prior measurement reuse, and 10/10 rows CLOSED: Prerequisites, L0 FP32, L0 FP16, L1 FP32, L1 FP16, L2, L3, Analyze FP32, Analyze FP16, and Quality. The refreshed body transcribes the successor metrics, DUSHA source/license statements, stack guards, Lane A binding, and No methodology friction observed while preserving the earlier frozen evidence.

GitHub state after publication: PR remains OPEN/DRAFT with model-scale-by-skill; exact head and stacked base are unchanged. The current license/cla check is completed/success on exact head e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3. No terminal reviewer opinion is posted by this repair response.

@ssss141414

Copy link
Copy Markdown
Contributor Author

APPROVE - successor independent reviewer verdict

PR: #1333
Reviewed head: e2667d973ca9f55437f8a5604d3ee2b4b42ba8f3
Reviewed tree: fef6e2bc539ef39360c737fab31c83d5cc92bc01
Stacked base / direct parent: 8010667eb20d0eb08e916157a6cc5080c91d385d

Prior blocker closure

  • CLOSED: public-reproduction-entrypoint-unavailable from issue comment 5381789642. I extracted the complete PowerShell runner from the live canonical PR body (79c6345b7b26757fb003e662c8b1c1f8ec05e7f35ac86b7b302bdebd922f28bd). Its normalized SHA-256 is b55c4c096409013d4c7e4e6d67ed7d822f3455addbeb8039d9cc4a40f368b0b4, exactly matching the normalized tester successor runner. The tester source retains raw SHA-256 f3b8bde064da98b6ab7f7ffef5950bfcef389f95a50ec61b81da2c18d35aca56; the difference is CRLF normalization only.
  • The body-extracted runner has zero PowerShell parse errors and no C:\temp, tester-root, successor-root, or other local scratch dependency. From the fresh exact-head checkout, a bounded invocation reached the runner's own existing-output guard and returned its expected failure before clone, sync, build, perf, Eval, Analyze, or tests. This proves the body source is accessible and executable without repeating the already-equivalent expensive stages.
  • Rehashed successor authority: verdict ca4284b00da1d2b5d055abce11c1328b36f16197399710ba6d00d6597299e1c5; runner f3b8bde064da98b6ab7f7ffef5950bfcef389f95a50ec61b81da2c18d35aca56; integrity manifest 8c7a082ab8bc6bc3f483ea56f62a0d6017d9c0818540c9da9cf0d2b7650f493f. All compact manifest entries rehashed successfully.
  • The successor records contain exactly 32 commands and zero unaccepted exits. The only nonzero exits are analyze-fp32 and analyze-fp16, both exit 1 explicitly accepted after each emitted metadata and all 12 requested rows. Every named command resolves into the 10/10 CLOSED rows: Prerequisites, L0 FP32, L0 FP16, L1 FP32, L1 FP16, L2, L3, Analyze FP32, Analyze FP16, and Quality. The live body contains the same sequence, closure table, and successor metrics.

Candidate and evidence

  • Fresh independent checkout was detached at the exact head/tree above; HEAD^ is the exact stacked base, status was clean, and it was removed before this verdict. The PR-owned diff is only the two CPU recipe JSON files, six inserted lines total, adding export.compatibility.transformers_attention = eager. examples/recipes/README.md is unchanged and git diff --check passes.
  • Focused recipe validation passed: both JSON files parse, retain audio-classification, input_values FP32 [1,16000], and eager attention; FP32 retains quant: null; FP16 retains quant.mode: fp16 and fp16_keep_io_types: true.
  • Successor FP32/FP16 model and external-data hashes equal the artifacts independently reviewed before the body-only repair. Therefore the prior product, model identity, DUSHA source/license, five-class mapping, L0/L1 structure and perf, L2 parity, L3 functional-smoke limitation/metrics, Analyze, and quality findings remain valid. No expensive model stage was rerun because candidate identity and evidence equivalence passed.
  • Successor quality evidence remains complete on this exact head: license and Ruff PASS; mypy clean across 439 files; non-hardware partitions report 1,526, 1,538, 848, 3,648, and 926 passing tests respectively, with the recorded skips/xfails. This recipe-only successor re-entry added dependency-free JSON/schema and diff checks, both passing.

GitHub gate

  • PR is OPEN/DRAFT with model-scale-by-skill; head and base are exact and unchanged.
  • Exact-head check matrix: license/cla = COMPLETED/SUCCESS.
  • Conversation enumeration immediately before posting: 2 issue comments (the prior verdict and repair reply), 0 line comments, 0 submitted reviews, 0 review threads, 0 open/actionable threads.
  • This is a normal issue comment only. It does not create GitHub Review state, mark ready, edit the body or labels, resolve threads, merge, or mutate Lane A.

Terminal skill verdict: APPROVE. The prior blocker is closed and no actionable blocker remains. The PR remains draft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant