Skip to content

server: accept base64-encoded voice reference in speech requests - #226

Open
IIIIIllllIIIIIlllll wants to merge 3 commits into
0xShug0:mainfrom
IIIIIllllIIIIIlllll:feature/server-voice-ref-base64
Open

server: accept base64-encoded voice reference in speech requests#226
IIIIIllllIIIIIlllll wants to merge 3 commits into
0xShug0:mainfrom
IIIIIllllIIIIIlllll:feature/server-voice-ref-base64

Conversation

@IIIIIllllIIIIIlllll

Copy link
Copy Markdown
Contributor

Add base64 support to the speech API. During use, I noticed that every time I clone a voice, I have to upload the audio file to the server first, and then use the file path for audiocpp_server to read it—which is quite inflexible. Therefore, I plan to add this feature so that clients can directly upload audio files encoded in base64.

Move base64 helpers into app/server/base64.{h,cpp}, add a strict decoder, and cover it with a server_base64_test unittest.

The maximum file size needs to be considered—I'm concerned that users might inadvertently upload extremely large audio files, causing machines with limited memory to run out of memory (OOM) directly.

What do you think?

Add voice_ref_b64 to /v1/audio/speech so cloning clients can inline a
base64 WAV payload (data URI accepted) instead of staging a file on the
server first. Mutually exclusive with voice_ref and shares its
precedence over presets, voice_dir, and default_voice_preset.

Move base64 helpers into app/server/base64.{h,cpp}, add a strict
decoder, and cover it with a server_base64_test unittest.
@0xShug0

0xShug0 commented Aug 13, 2026

Copy link
Copy Markdown
Owner

@IIIIIllllIIIIIlllll Thanks! I like the feature. For reference voice, models usually recommands 5 to 15s. We can assume a safe max is 30s (base64 < 5MB).

I wonder if we can avoid adding a separate top-level voice_ref_b64 option by overloading the voice_ref option?

"voice_ref": {
  "type": "base64",
  "data": "UklGRh..."
}

"voice_ref": {
  "type": "path",
  "path": "voices/alice.wav"
}

"voice_ref": "voices/alice.wav"

@IIIIIllllIIIIIlllll

IIIIIllllIIIIIlllll commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

I wonder if we can avoid adding a separate top-level voice_ref_b64 option by overloading the voice_ref option?

OK, I think we can distinguish between a file path and base64-encoded data by checking the beginning of the string.

Edited:
Your method is more reliable after all.

@0xShug0

0xShug0 commented Aug 14, 2026

Copy link
Copy Markdown
Owner

@IIIIIllllIIIIIlllll Sorry, just for clarification: "voice_ref": "voices/alice.wav" is backward-compatible, so existing users can continue using it, or they can use:

"voice_ref": {
  "type": "path",
  "path": "voices/alice.wav"
}

And for Base64, the user must explicitly send it.

"voice_ref": {
  "type": "base64",
  "data": "UklGRh..."
}

So you don't need to distinguish between them based on prefix.

PS: In fact, we can’t prevent users from sending random stuff. Comprehensive checks are better handled in the frontend or gateway before requests reach the HTTP server so the server remains lightweight.

voice_ref now accepts either a plain path string (unchanged behavior)
or an object: {"type": "path", "path": ...} or {"type":
"base64", "data": ...}, replacing the separate voice_ref_b64 field.
Pre-check the encoded length (4/3 expansion plus slack for a data URI
prefix) and verify the decoded size, so an oversized inline reference
is rejected before it can amplify through decode + f32 expansion. Path
references and the global max_request_body_bytes default are unchanged.
@IIIIIllllIIIIIlllll
IIIIIllllIIIIIlllll marked this pull request as ready for review August 14, 2026 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants