Skip to content

docs(kokoro): add bilingual demo and complete training handoff - #94

Open
Alex-Wengg wants to merge 2 commits into
mainfrom
docs/kokoro-bilingual-demo
Open

Alex-Wengg wants to merge 2 commits into
mainfrom
docs/kokoro-bilingual-demo

Conversation

@Alex-Wengg

@Alex-Wengg Alex-Wengg commented Sep 15, 2026

Copy link
Copy Markdown
Member

Summary

Add a complete cross-environment project handoff plus the existing small,
exploratory English–Mandarin Kokoro demo package.

Start here: project-handoff.md.

Shared project handoff

  • Settled scope: one female voice, English + Mandarin + code-switching, compact Kokoro v1.1-zh-compatible adaptation, FluidAudio/Core ML deployment direction, source Mac for demos/preparation only.
  • Clear inventory of what this PR includes, what remains local-only, and what has not been built. Instructions to recover the receiving environment's existing setup, data, and approvals without asking the user to repeat settled goals.
  • Training-data/consent/speaker/split audit, frontend contract and known bugs, strict checkpoint mapping and untouched-output parity, gradient/target checks, tiny real-data overfit/resume test, and bounded bilingual pilot.
  • Proposed freeze/loss/style strategy explicitly separated from proven implementation and user-approved requirements; no invented dataset readiness, training budget, or quality thresholds.
  • Automated evaluation protocol, judge limitations/calibration, frontend-vs-weight comparisons, trained-checkpoint export requirements, designated-device acceptance, and versioned release/rollback requirements.
  • Artifact-transfer and unresolved-input ledgers, expected evidence outputs, an ordered receiver checklist, and a copyable receiving-environment task.
  • Model-level AGENTS.md plus README links make the shared handoff discoverable. Internal .mobius execution plans remain outside Git.

Saved demo evidence

Under models/tts/kokoro-v1.1-zh/coreml/bilingual-demo:

  • Fixed selection of 12 existing clips: 4 English, 4 Mandarin, 4 mixed; 44.625 seconds total, all zf_001.
  • Listening guide, playlist, exact phonemes/token IDs, original per-clip diagnostic records, and redacted run/scorer provenance.
  • Evidence-backed issue list: numeral 二 accepted by the erhua merge, word-like API pronunciation, GitHub consonant sequence, ASR language-mode sensitivity, and unresolved acoustic questions.
  • A dependency-free artifact-integrity checker and nine unit tests. No model loading or network calls.
  • A scoped AGENTS.md that keeps this Mac for demos and lightweight checks, not official evaluation or training.

Validation

  • uv run python -m unittest discover -s tests -v: 9 passed.
  • uv run python verify-demo.py: all 12 bundled metadata/evidence records validated; explicitly reports audio not checked.
  • uv run python verify-demo.py --audio-dir <existing-local-demo-audio>: 12/12 original WAV hashes verified.
  • Same checker also validated the assembled local demo and playlist.
  • Relevant inspected FluidAudio frontend source hashes match those recorded by the original render run.
  • Local documentation link targets checked; no private machine paths or credentials added to the handoff.
  • git diff --check passes.

Scope and exclusions

This PR documents the project contract and packages prior exploratory evidence;
it does not implement a trainer, fix frontend behavior,
train a model, run conversion, prove checkpoint parity, or establish an official
quality baseline. No synthesis, ASR, learned scoring, or training was rerun to
assemble the demo. Existing ASR and predicted-MOS values remain diagnostic
proxies, not certified pronunciation/tone accuracy or human listening scores.

The source run used a modified FluidAudio working tree, identified by recorded
file hashes. The current Swift/Core ML and MLX tools are not a CUDA training stack.

The handoff specifically calls out that the existing converter loads the upstream
checkpoint by default; exporting a future trained candidate needs explicit
candidate inputs and hash-bound validation. It also distinguishes the older
converter's noise stage from the inspected runtime's KokoroNoise_v2 asset.

WAVs stay local/ignored in accordance with the repository's existing asset policy;
playback links work after supplying the matching local files. No pretrained
weights, private recordings, machine-specific paths, internal .mobius planning
documents, or unrelated model work are included. All selected text prompts were
authored for this project; no MiniMax prompt text is redistributed.

The first receiving-environment actions are artifact/setup/data inventory,
frontend regressions, strict checkpoint-parity implementation, and data/target
audit. Use existing valid setup/run approvals; request only genuinely missing
access or authority before consequential execution. This PR does not authorize
provisioning, paid jobs, data uploads, publishing weights, or production rollout.
Git delivery does not imply the other environment has already pulled the branch
or received local-only source/audio/data; it must verify its own inventory.

@Alex-Wengg Alex-Wengg changed the title docs(kokoro): add bilingual demo evidence and scope docs(kokoro): add bilingual demo and complete training handoff Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant