Feat/tts benchmark - #281

Merged

robdmac merged 69 commits into

mainfrom

feat/tts-benchmark

Aug 21, 2026

Commits on Aug 14, 2026

Add the open-weight TTS benchmark behind benchmark tabs

robdmac
and
claude
committed
1ca1159
Use each vendor's own capitalisation for TTS engine names

robdmac
and
claude
committed
9d09c04
Add blind preference ballots for the TTS benchmark

robdmac
and
claude
committed
41e4b14
Update TTS data, restrict to real-time engines, add two providers

robdmac
and
claude
committed
fd6b70e
Move the real-time cutoff to 2.1x so Chatterbox-q4 stays in

robdmac
and
claude
committed
9275a9c
Pool the NeuTTS device builds and widen the results table

robdmac
and
claude
committed
f360105
Add the blind ranking dialog and the live Human column

robdmac
and
claude
committed
7eed09d
Add the speed-against-accuracy chart and a social card

robdmac
and
claude
committed
d14972e
Rank the chart on human preference, not word error

robdmac
and
claude
committed
77cc622
Send /benchmarks straight to the default benchmark

robdmac
and
claude
committed
a6e333b
Drop the Passed and Frame rate columns from the TTS table

robdmac
and
claude
committed
7d27175
Rebuild benchmark content after the intro rewrite

robdmac
and
claude
committed
664a607

Commits on Aug 15, 2026

Import the 15 Aug export: all 27 configurations, real time as a filter

robdmac
and
claude
committed
6ec1350
Return the sort cycle to the default instead of unsorted

robdmac
and
claude
committed
c5c08dd
Import the 33-configuration export, and stop unmeasured cells sorting

robdmac
and
claude
committed
bd772b7
Trim the TTS table: drop Libs and Lead-in, shorten Disk and the licence

robdmac
and
claude
committed
f5784fe
Capitalise the licence abbreviation as <$5M

robdmac
and
claude
committed
7fcbc3e
Stop highlighting the NeuTTS rows

robdmac
and
claude
committed
0ce6589
Play samples through the iOS media audio session, not ambient

robdmac
and
claude
committed
6f21f0b

Commits on Aug 16, 2026

Animate the ranking swap and add a confirmation step

robdmac
and
claude
committed
6166fe3

Commits on Aug 17, 2026

Rank by dragging clips into labelled slots

robdmac
and
claude
committed
deade71
Remove the analysis, keep the methodology

robdmac
and
claude
committed
c3d5742
Import the 17 Aug export and fingerprint the audio URLs

robdmac
and
claude
committed
128c897
Pick up the re-cut export: all 34 configurations now have audio

robdmac
and
claude
committed
3a17faf
Show one word error rate instead of two

robdmac
and
claude
committed
b99ae93
Drop the NeuTTS CPU rows and the device suffixes

robdmac
and
claude
committed
0ce6171
Fix four samples, and bring back the word-error chart

robdmac
and
claude
committed
eebf636
Import the export with more PESQ and Peak RSS coverage

robdmac
and
claude
committed
f9f4edd
Import the export that fills the NeuTTS-2E Q8 row

robdmac
and
claude
committed
9c9059e
Swap clips when one is dragged onto another

robdmac
and
claude
committed
8360b84
Remove the exclusions section and the device paragraph

robdmac
and
claude
committed
d29cbfc
Import more RSS, and link the licence names

robdmac
and
claude
committed
f6cac4a
Link model names, and add a vendor strip

robdmac
and
claude
committed
e775439

Commits on Aug 18, 2026

Import the 18 Aug export: PESQ is complete

robdmac
and
claude
committed
92a22de
Static vendor rows, and make the side index opt-out per post

robdmac
and
claude
committed
6b75c82
Alphabetise the vendors, and widen the column without the index

robdmac
and
claude
committed
2bea0c7
Link licences to the project's own licence, not the licence text

robdmac
and
claude
committed
d605765
Split KittenTTS into its four checkpoints

robdmac
and
claude
committed
a9b7c0d
Collapse Notes into an icon with a popup

robdmac
and
claude
committed
8551a30
Cap the licence column and tighten the cell padding

robdmac
and
claude
committed
e935948
Shorten the class codes, drop Avg audio, and explain the classes on hover

robdmac
and
claude
committed
4f79ed8
Move the real-time cutoff to 2x and draw the line in the table

robdmac
and
claude
committed
11a2677
Restyle the cutoff band as structure, not a warning

robdmac
and
claude
committed
77f1b6d
Import the 18 Aug 11:47 export

robdmac
and
claude
committed
1cfc1c3
Rename Peak RSS to RSS

robdmac
and
claude
committed
1e64e68
Remove the empty strip under the table, and flip the band's direction word

robdmac
and
claude
committed
4b05cfa
Let readers rank up to three times, never the same clip twice

robdmac
and
claude
committed
09a8433
Fill in TaDA's licence: Llama 3.2 Community

robdmac
and
claude
committed
d8ca8c7
Resolve the last two unknown licences: OmniVoice and Dots-TTS

robdmac
and
claude
committed
f43944b
Fill in OmniVoice's size and release date from the paper

robdmac
and
claude
committed
2b20796
Complete BananaMind, and fit twenty vendors back into two rows

robdmac
and
claude
committed
85bdd97
Zonos and Dots-TTS are full runs now, and the chart axis follows the data

robdmac
and
claude
committed
627d360
Import the corrected RSS figures

robdmac
and
claude
committed
416e4a9
Filter the table by architecture class

robdmac
and
claude
committed
333e14b
Share one class filter across the page, and sync the ballot pool on import

robdmac
and
claude
committed
89735e0
Keep clip playback alive when Safari suspends the audio context

robdmac
and
claude
committed
4a59373
Stop awaiting resume(), and fall back to an element when Web Audio stays down

robdmac
and
claude
committed
fa9c72a
Import the export that drops Parler-TTS and rewrites eight notes
robdmac
committed
af2c61a
Let the reader diagnose silence, since I cannot reproduce it

robdmac
and
claude
committed
9afebce
Make the sound diagnostics a button someone can find

robdmac
and
claude
committed
0e53bfa
Play clips through an element, because Safari will not play Web Audio here

robdmac
and
claude
committed
163cd74
Remove the sound diagnostics now that they have done their job

robdmac
and
claude
committed
4252d1f
Take the gesture before the download, and stop losing ballots quietly

robdmac
and
claude
committed
cecc754
Claim playback from the gesture that closes the ballot dialog

robdmac
and
claude
committed
db476a7

Commits on Aug 19, 2026

Add Supertonic 3, which is the most accurate engine in the table

robdmac
and
claude
committed
0eab97d
Date Supertonic 3 to April, the release it actually is

robdmac
and
claude
committed
863fabf
Add Qwen3-TTS VoiceDesign and Orpheus 3B Q4

robdmac
and
claude
committed
e2f22ca
Show human scores from three ballots, as whole points

robdmac
and
claude
committed
db83ca3
Shrink the vendor chips back into two rows

robdmac
and
claude
committed
8bafb38