Skip to content

feat: concurrent chat panes and on-device SenseVoice input - #623

Open
zephyr7zhang wants to merge 1 commit into
agegr:mainfrom
zephyr7zhang:feat/concurrent-chats-and-voice
Open

feat: concurrent chat panes and on-device SenseVoice input#623
zephyr7zhang wants to merge 1 commit into
agegr:mainfrom
zephyr7zhang:feat/concurrent-chats-and-voice

Conversation

@zephyr7zhang

Copy link
Copy Markdown

Summary

This PR adds two local-only features:

  1. Concurrent conversations — switching sessions no longer unmounts the previous chat. Open conversations stay mounted in tabbed panes (up to 8), so an in-flight agent run keeps going in the background. Esc / abort only affects the visible pane.
  2. On-device voice input — a microphone button next to the composer records in the browser and transcribes with local SenseVoice (sherpa-onnx-node). Audio never leaves the machine, and no cloud STT API keys are used or committed.

Also included:

  • Do not pass empty toolNames: [] when creating a session (avoids merging/creating a tools-off session).
  • New composers no longer inherit a stored "none" (chat-only) tool preset, so coding tools stay available by default.
  • i18n strings for en / zh-CN / zh-TW.

Why

Today, selecting another session tears down ChatWindow / useAgentSession. That stops the running agent and makes it impossible to work on two chats at once.

Voice input is fully local (Hugging Face / hf-mirror model files under ~/.pi/voice/). Nothing in this PR reads ~/.pi/agent/auth.json, env API keys, or other credentials.

How it works

Panes

  • lib/chat-panes.ts tracks open panes.
  • AppShell keeps every open ChatWindow mounted and toggles isActive.
  • Inactive panes keep their SSE/agent session alive, but do not steal the shared input ref, Esc handler, or top-bar stats.
  • Idle panes are evicted first when the cap is hit; running panes are kept.

Voice

  • POST /api/voice/prepare downloads the SenseVoice int8 model on first use.
  • POST /api/voice/transcribe accepts WAV bytes and returns text.
  • Right-click the mic button to cycle language: auto / zh / en.

Tests

  • lib/chat-panes.test.mjs
  • lib/wav-pcm.test.mjs
  • components/AppShell.workspace-memory.test.mjs (mounted-pane assertion)
  • lib/i18n/registry.test.mjs key parity

Out of scope (not in this PR)

Local Windows launchers and a machine-specific fs.readlink patch under scripts/ were left untracked on purpose.

中文摘要

  • 多对话页签:切换会话时不再拆掉正在跑的 agent,后台对话可以继续跑。
  • 本地语音输入:用 SenseVoice 在本机转写,不上传音频,也不需要/不会提交任何 API 密钥。

Keep multiple conversations mounted in tabbed panes so switching
sessions no longer tears down in-flight agent runs. Add a local
SenseVoice microphone button that transcribes in-process without
sending audio or API keys to a cloud STT service.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant