fix(download): interrupted first download no longer bricks streaming ASR managers (#819); Unified int8 A16 compat docs (#828) - #829
Merged
Conversation
…ASR managers (#819) StreamingUnifiedAsrManager, UnifiedAsrManager, StreamingNemotronAsrManager, and StreamingEouAsrManager gated their cache-or-download decision on bare file/directory existence and then called MLModel.load directly. An interrupted first download (app backgrounded mid-way through the ~550 MB encoder fetch) leaves the encoder .mlmodelc directory present but containing only weights/weight.bin.partial and no root coremldata.bin — the gate reports "cached", the load fails, nothing ever re-downloads, and the app stays broken until its data is deleted. The hardened path (ModelHub.loadModels: layout validation + purge-and-retry) never ran for these managers because they need per-model MLModelConfigurations (encoder on ANE, decoder/joint on CPU) that loadModels' single computeUnits cannot express. Add ModelHub.loadWithRecovery, a recovery wrapper for self-loading managers with the same guarantees as loadModels: - Cache validity is judged per required file via the new ModelCache.incompleteFiles: every .mlmodelc must have its root coremldata.bin and contain no *.partial staging file; plain files must exist. An interrupted download therefore re-enters ModelHub.download, which resumes the .partial via HTTP Range instead of restarting. - On load failure the repo cache is purged, re-downloaded, and the load retried once — except in offline mode, on cancellation, and on transient network errors, where the cache is preserved and the error rethrown (mirroring loadModels' guards). The partial-file scan is scoped to required bundles so a leftover partial from a bundle the caller does not need (e.g. the fp16 encoder when int8 is requested) does not force a listing round-trip on every load. Nemotron's decoder_joint.mlmodelc and metadata.json stay out of the validity set because loadModels(from:) treats them as optional; the download registry still fetches them. SenseVoiceModels/ParaformerModels have the same existence-only gate but a different public download()/load(from:) API split; left for a follow-up. Fixes #819
PocketTTS Smoke Test ✅
Runtime: 0m6s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality and performance may differ from Apple Silicon. |
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m8s • 07/31/2026, 11:34 PM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 106.2s processing • Test runtime: 2m 0s • 07/31/2026, 11:58 PM EST |
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
Supertonic3 Smoke Test ✅
Runtime: 0m35s Note: CI VMs lack a physical Neural Engine; the ANE-bucketed VectorEstimator falls back to CPU here. This validates download + variant resolution + synthesis, not ANE residency/perf. |
✅ Nemotron Multilingual Benchmark — FLEURSFLEURS
Logs (tail) |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 44.1s diarization time • Test runtime: 2m 26s • 07/31/2026, 11:46 PM EST |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 8m59s • 08/01/2026, 12:02 AM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
…vs-incompatible load diagnosis Extends the loadWithRecovery hardening to the two remaining existence-only gates and adds the size diagnosis discussed in #819: - SenseVoiceModels/ParaformerModels: modelsExist now judges load-readiness (root coremldata.bin present, no *.partial staging files) instead of bare existence, and downloadAndLoad routes through ModelHub.loadWithRecovery for completeness-checked download plus purge-and-retry. The public download()/load(from:)/modelsExist API shapes are unchanged. - ModelHub.logLoadFailureSizeDiagnosis: when the purge-and-redownload retry ALSO fails (both in loadModels and loadWithRecovery), compare each required file's on-disk bytes against the published HuggingFace tree sizes and log which failure class this is: short files mean a truncated cache (clear + re-download helps), all-full-size means the model cannot run on this hardware/OS (#828 — re-download cannot help). CoreML's "Unable to load model" reads identically in both cases, which cost the #819 reporters days. Best-effort: needs the network for the listing, silent in offline mode, skipped on cancellation, never throws. The pure size comparison lives in ModelCache.undersizedFiles for testability. Also fixes a CI-only test-isolation bug in ModelCacheCompletenessTests: the loadWithRecovery tests placed a silero-vad cache directly in the shared temp directory, where parallel test processes from other suites create and purge the same repo path; each test now uses a UUID-unique models root. Verified live against the real HF VAD repo: both diagnosis branches log correctly (intact -> incompatibility message; truncated weight file -> truncated message with byte counts), and the SenseVoice/Paraformer gates reject the interrupted-download shape while accepting complete caches.
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 2m 42s • 2026-08-01T03:35:05.868Z |
…ailure hint Issue #828: the int8 Unified 0.6B encoder fails to load on A16 (iPhone 14 Pro) on every compute unit — including .cpuOnly — with CoreML 'Failed to build the model execution plan' (-14), even from an intact download. fp16 loads and transcribes on the same device. The failure reads like a corrupt download, which sent the reporter into #819's cache loop for days. - Documentation/Models.md: Parakeet Unified 0.6B was absent from the model catalog entirely. Add batch + streaming entries and a Model Sources row, with the hardware note the reporter asked for: int8 verified on M-series, known not to load on A16 (any compute unit), other A-series unverified, use fp16 on iOS. - Documentation/Benchmarks.md: same note in the Unified benchmark section, next to the int8-is-default recommendation. - Unified{,Streaming}AsrManager.loadModels(from:): when the int8 encoder load throws, log the A-series caveat and the encoderPrecision: .fp16 escape hatch before rethrowing, so the failure is diagnosable at the load site instead of only after loadWithRecovery's purge-and-retry exhausts itself. The purge-and-retry flow is intentionally unchanged: a full-size-but- corrupt cache is still a real failure class that purging fixes, and gating the purge on the best-effort HF size check would regress that recovery; the post-retry size diagnosis (d9116ce) already labels the incompatibility case.
✅ Nemotron Multilingual Benchmark — FLEURSFLEURS
Logs (tail) |
Alex-Wengg
changed the title
✅ Nemotron Multilingual Benchmark — FLEURSFLEURS
Logs (tail) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #819.
Problem
StreamingUnifiedAsrManager,UnifiedAsrManager,StreamingNemotronAsrManager, andStreamingEouAsrManagergated their cache-or-download decision on bare file/directory existence and then calledMLModel.loaddirectly. An interrupted first download (e.g. app backgrounded midway through the ~550 MB encoder fetch) leaves the encoder.mlmodelcdirectory present but containing onlyweights/weight.bin.partialand no rootcoremldata.bin— the gate reports "cached", the load fails identically on every launch, nothing ever re-downloads, and the app stays broken until its data is deleted.The hardened path (
ModelHub.loadModels: layout validation + purge-and-retry) never ran for these managers because they need per-modelMLModelConfigurations (encoder on ANE, decoder/joint on CPU) thatloadModels' singlecomputeUnitsparameter cannot express.Fix
1.
ModelHub.loadWithRecovery— a recovery wrapper for self-loading managers with the same guarantees asloadModels:ModelCache.incompleteFiles: every required.mlmodelcmust have its rootcoremldata.binand contain no*.partialstaging file; plain files must exist. An interrupted download therefore re-entersModelHub.download, which resumes the.partialvia HTTP Range instead of restarting from zero.loadModels' guards).All four managers now declare their required file set and load through the wrapper. The partial-file scan is scoped to required bundles so a leftover partial from an unneeded bundle (e.g. fp16 encoder when int8 is requested) doesn't force a listing round-trip on every load. Nemotron's
decoder_joint.mlmodelc/metadata.jsonstay out of the validity set because its loader treats them as optional (the download registry still fetches them).2. SenseVoice / Paraformer — the same existence-only gates, hardened the same way:
modelsExistnow judges load-readiness instead of bare existence, anddownloadAndLoadroutes throughloadWithRecovery. Public API shapes (download()/load(from:)/modelsExist) unchanged.3. Truncated vs. hardware-incompatible diagnosis (per the #819 discussion / #828): when the purge-and-redownload retry also fails — in both
loadModelsandloadWithRecovery—ModelHub.logLoadFailureSizeDiagnosiscompares each required file's on-disk bytes against the published HuggingFace tree sizes and logs which look-alike failure class this is:local/remotebyte counts)Best-effort diagnostics: needs the network for the listing, silent in offline mode, skipped on cancellation, never throws.
4. #828: Parakeet Unified hardware compatibility docs + int8 load-failure hint — the int8 Unified 0.6B encoder fails to load on A16 (iPhone 14 Pro) on every compute unit, including
.cpuOnly(CoreML "Failed to build the model execution plan", -14), even from an intact download; fp16 loads on the same device. The failure reads like a corrupt download, which sent the #828 reporter into #819's cache loop for days.Documentation/Models.md: Parakeet Unified 0.6B was absent from the model catalog entirely. Added batch + streaming entries and a Model Sources row with the compatibility note the reporter asked for: int8 verified on M-series, known not to load on A16 (any compute unit), other A-series unverified — useencoderPrecision: .fp16on iOS.Documentation/Benchmarks.md: same iOS note in the Unified benchmark section, next to the int8-is-default recommendation.Unified{,Streaming}AsrManager.loadModels(from:): when the int8 encoder load throws, log the A-series caveat and the.fp16escape hatch before rethrowing, so the diagnosis appears at the load site instead of only after the recovery path's purge-and-retry exhausts itself.The purge-and-retry flow is intentionally unchanged: a full-size-but-corrupt cache is still a real failure class that purging fixes, and gating the purge on the best-effort HF size check would regress that recovery; the post-retry size diagnosis (point 3) already labels the incompatibility case.
Verification
swift buildgreen,swift format lintclean on touched files.ModelCacheCompletenessTests, 15 tests): the exact StreamingUnifiedAsrManager: interrupted first download permanently bricks model loading (cache check is directory-existence only) #819 on-disk shape, partial-with-valid-layout, nested Nemotron bundle paths, out-of-scope partials, the SenseVoice/Paraformer gates,undersizedFiles(truncated / matching / subPath-stripped), and the wrapper's offline/cancellation paths.CI note
The first
tests.ymlrun failed ontestLoadWithRecoveryOfflineLoadFailureDoesNotPurgeCache— a test-isolation bug, not a library bug: the test placed its fixture cache directly in the shared temp directory, where parallel test processes from other suites create/purge the samesilero-vadrepo path. Fixed by giving each test a UUID-unique models root.🤖 Generated with Claude Code