server: accept base64-encoded voice reference in speech requests - #226
server: accept base64-encoded voice reference in speech requests#226IIIIIllllIIIIIlllll wants to merge 3 commits into
Conversation
Add voice_ref_b64 to /v1/audio/speech so cloning clients can inline a
base64 WAV payload (data URI accepted) instead of staging a file on the
server first. Mutually exclusive with voice_ref and shares its
precedence over presets, voice_dir, and default_voice_preset.
Move base64 helpers into app/server/base64.{h,cpp}, add a strict
decoder, and cover it with a server_base64_test unittest.
|
@IIIIIllllIIIIIlllll Thanks! I like the feature. For reference voice, models usually recommands 5 to 15s. We can assume a safe max is 30s (base64 < 5MB). I wonder if we can avoid adding a separate top-level |
OK, I think we can distinguish between a file path and base64-encoded data by checking the beginning of the string. Edited: |
|
@IIIIIllllIIIIIlllll Sorry, just for clarification: And for Base64, the user must explicitly send it. So you don't need to distinguish between them based on prefix. PS: In fact, we can’t prevent users from sending random stuff. Comprehensive checks are better handled in the frontend or gateway before requests reach the HTTP server so the server remains lightweight. |
voice_ref now accepts either a plain path string (unchanged behavior)
or an object: {"type": "path", "path": ...} or {"type":
"base64", "data": ...}, replacing the separate voice_ref_b64 field.
Pre-check the encoded length (4/3 expansion plus slack for a data URI prefix) and verify the decoded size, so an oversized inline reference is rejected before it can amplify through decode + f32 expansion. Path references and the global max_request_body_bytes default are unchanged.
Add base64 support to the speech API. During use, I noticed that every time I clone a voice, I have to upload the audio file to the server first, and then use the file path for audiocpp_server to read it—which is quite inflexible. Therefore, I plan to add this feature so that clients can directly upload audio files encoded in base64.
Move base64 helpers into app/server/base64.{h,cpp}, add a strict decoder, and cover it with a server_base64_test unittest.
The maximum file size needs to be considered—I'm concerned that users might inadvertently upload extremely large audio files, causing machines with limited memory to run out of memory (OOM) directly.
What do you think?