Skip to content

feat(audio-lab): add wavekat-zh Smart Turn fine-tune as a second turn backend - #44

Merged
wavekat-eason merged 3 commits into
mainfrom
feat/audio-lab-wavekat-zh-turn
May 14, 2026
Merged

feat(audio-lab): add wavekat-zh Smart Turn fine-tune as a second turn backend#44
wavekat-eason merged 3 commits into
mainfrom
feat/audio-lab-wavekat-zh-turn

Conversation

@wavekat-eason

Copy link
Copy Markdown
Contributor

Summary

  • Bumps wavekat-turn to 0.0.8 and enables the wavekat-smart-turn feature so audio-lab can load the Mandarin fine-tune from wavekat/smart-turn-ONNX.
  • Registers a new wavekat-zh turn backend next to pipecat; both share the same ONNX contract, only the weights differ.
  • Frontend now seeds one default turn config per known backend (pipecat first, then wavekat-zh, then any future fine-tunes alphabetically), so the two models appear side-by-side out of the box for direct comparison.
  • README updated with the HF download note and the WAVEKAT_TURN_MODEL_DIR offline override.

The wavekat-zh ONNX is fetched from HuggingFace on first selection and cached under \$HF_HOME/hub/.

Test plan

  • `make -C tools/audio-lab dev-backend` and `make -C tools/audio-lab dev-frontend`, confirm both `pipecat` and `wavekat-zh` configs appear in the Turn Detection panel by default.
  • Trigger `wavekat-zh` for the first time with internet access; confirm model is fetched + cached and predictions stream alongside `pipecat`.
  • Run on a Mandarin clip; confirm `wavekat-zh` probabilities differ from `pipecat` and behave sensibly at EOU.
  • Set `WAVEKAT_TURN_MODEL_DIR` to a pre-populated dir and confirm offline startup works.
  • `cargo fmt --all -- --check`, `cargo clippy --workspace -- -D warnings`, `npm run build` all green.

🤖 Generated with Claude Code

wavekat-eason and others added 3 commits May 11, 2026 22:16
… backend

Bumps wavekat-turn to 0.0.8 and enables the wavekat-smart-turn feature so
audio-lab can load the Mandarin fine-tune from wavekat/smart-turn-ONNX
alongside upstream Pipecat Smart Turn v3. Backend list now exposes both
"pipecat" and "wavekat-zh"; the frontend seeds one default config per
known backend so the two models show up side-by-side without manual
setup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The .nvmrc lives in frontend/, but nvm use was running from tools/audio-lab/
before the cd, so it could not pick up the pinned Node version and the make
targets failed with exit 127.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ckend

Pipeline Mode previously started empty and required manual setup, which
hid the new wavekat-zh comparison behind extra clicks. Auto-seeds one
pipeline per turn backend (paired with silero-vad, or the first
available VAD config) on first load and whenever the saved list is
empty, so the upstream Pipecat vs WaveKat fine-tune comparison shows
up in Pipeline Mode by default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wavekat-eason
wavekat-eason merged commit 07f92c8 into main May 14, 2026
9 checks passed
@wavekat-eason
wavekat-eason deleted the feat/audio-lab-wavekat-zh-turn branch May 14, 2026 23:30
wavekat-eason pushed a commit that referenced this pull request May 15, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.0.15](v0.0.14...v0.0.15)
(2026-05-14)


### Features

* **audio-lab:** add wavekat-zh Smart Turn fine-tune as a second turn
backend ([#44](#44))
([07f92c8](07f92c8))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant