Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ A research repo for the [WaveKat](https://github.com/wavekat) project — intera
```
wavekat-lab/
├── tools/
│ ├── audio-lab/ Real-time VAD + Turn Detection comparison app (Rust + React)
│ ├── audio-lab/ Real-time VAD + Turn Detection + ASR comparison app (Rust + React)
│ └── cv-explorer/ Mozilla Common Voice dataset browser (Cloudflare Workers + React)
├── notebooks/ Jupyter notebooks (training, validation, dataset splits)
└── docs/ Plans and design docs
Expand All @@ -30,9 +30,9 @@ Each tool is self-contained — its own Makefile, lockfiles, and build setup liv

### [Audio Lab](tools/audio-lab/) — `tools/audio-lab/`

Web app for testing and comparing WaveKat library backends side by side in real time. Live mic capture, WAV upload, multi-config fan-out, VAD-gated pipeline mode, waveform + spectrogram + probability timelines.
Web app for testing and comparing WaveKat library backends side by side in real time. Live mic capture, WAV upload, multi-config fan-out, VAD-gated pipeline mode, live ASR transcripts, waveform + spectrogram + probability timelines.

Backends: webrtc-vad, silero-vad, ten-vad, firered-vad, pipecat smart-turn. [Details →](tools/audio-lab/README.md)
Backends: webrtc-vad, silero-vad, ten-vad, firered-vad, pipecat smart-turn, sherpa-onnx ASR. [Details →](tools/audio-lab/README.md)

### [Common Voice Explorer](tools/cv-explorer/) — `tools/cv-explorer/`

Expand Down
6 changes: 3 additions & 3 deletions tools/audio-lab/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
SHELL := /bin/bash
NVM := source $$NVM_DIR/nvm.sh && nvm use --silent &&
NVM := source $$NVM_DIR/nvm.sh &&

.PHONY: help \
install install-backend install-frontend \
Expand Down Expand Up @@ -38,15 +38,15 @@ install-backend:
cargo install cargo-watch

install-frontend:
$(NVM) cd frontend && npm install
$(NVM) cd frontend && nvm use --silent && npm install

# ─── Development ──────────────────────────────────────────────────────────────

dev-backend:
cargo watch -x 'run -p audio-lab'

dev-frontend:
$(NVM) cd frontend && npm run dev
$(NVM) cd frontend && nvm use --silent && npm run dev

dev:
@echo "Run 'make dev-backend' and 'make dev-frontend' in separate terminals"
Expand Down
14 changes: 13 additions & 1 deletion tools/audio-lab/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Audio Lab

A web-based experimentation tool for testing and comparing [WaveKat](https://github.com/wavekat) library backends — VAD, turn detection, and more — side by side in real time.
A web-based experimentation tool for testing and comparing [WaveKat](https://github.com/wavekat) library backends — VAD, turn detection, ASR, and more — side by side in real time.

> [!WARNING]
> Early development. Things may change.
Expand All @@ -10,6 +10,7 @@ A web-based experimentation tool for testing and comparing [WaveKat](https://git
- **Live recording** — capture audio from your microphone server-side, stream results to the browser in real time
- **File analysis** — upload a WAV file and run multiple configs against it at full speed
- **Side-by-side comparison** — fan out audio to N configurations simultaneously and compare outputs
- **Live transcripts** — stream partial + final ASR transcripts per config as audio plays
- **Preprocessing exploration** — apply high-pass filters, RNNoise denoising, or normalization per-config
- **Interactive visualization** — waveform, spectrogram, and probability timelines with synchronized zoom, pan, and hover

Expand Down Expand Up @@ -57,6 +58,17 @@ Each config can also enable per-config preprocessing: high-pass filter, RNNoise
|---------|-------------|-------|
| **pipecat** | Pipecat Smart Turn v3 — audio-based EOU detection | 16 kHz PCM audio |

### ASR

Streaming speech-to-text via [`wavekat-asr`](https://github.com/wavekat/wavekat-asr). Each ASR config gets a per-config transcript card stacked under the timelines: committed finals with `[mm:ss.s–mm:ss.s]` timestamps, a dimmed trailing line for the live partial, and footer stats (last confidence, count, average segment duration).

| Backend | Description | Preset (`preset` param) |
|---------|-------------|-------------------------|
| **sherpa-onnx** | Local streaming Zipformer / Paraformer via [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) | `bilingual` (default, ZH+EN) · `en` · `zh` · `paraformer-zh-en` |

> [!NOTE]
> The first time you record or load a file with an ASR config enabled, sherpa-onnx downloads the chosen model from HuggingFace (~75 MB for `bilingual`) into `$HF_HOME` (default `~/.cache/huggingface/hub/`). The transcript card shows `loading model…` until the model is ready; subsequent runs are instant.

## Architecture

The Rust backend handles all audio capture and processing; the React frontend is embedded in the binary and handles visualization only.
Expand Down
Loading