fix: eliminate ticking sound and filter erroneous STT responses (#3) - #4
Open
TrivCodez wants to merge 4 commits into
Open
fix: eliminate ticking sound and filter erroneous STT responses (#3)#4TrivCodez wants to merge 4 commits into
TrivCodez wants to merge 4 commits into
Conversation
Root cause — The STT engine's SILENCE_THRESHOLD (800) was far too sensitive. Ambient room noise and incidental sounds (tapping, breathing, fan hum) routinely exceeded this threshold. When even ~0.19s of noise reached Whisper, it frequently hallucinated short words or syllables. These were then translated by MT and synthesised by TTS, producing audible "tick" bursts and non-speech responses. Changes (stt_engine.py): - Calibrate an adaptive noise floor on startup: sample 1s of ambient audio, compute mean RMS, and set SILENCE_THRESHOLD to max(mean × 3.5, 800). This adapts to the user's actual environment rather than using a fixed value. - Raise MIN_SPEECH_CHUNKS from 3 to 8, requiring ~0.5s of continuous audio above threshold before sending to Whisper. This eliminates clicks, pops, and short bursts of noise from being transcribed. - Replace the simplistic 4-phrase hallucinations filter with a comprehensive filter that catches: common Whisper noise hallucinations, single-character utterances, phrases under 3 characters, and known filler/hallucination patterns. - Add a minimum-confidence gate: transcribed text must contain a real word (alphabetic content) to be accepted — pure numbers, punctuation, or symbols from noise are discarded.
|
Warning Review limit reached
Next review available in: 31 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. 📝 WalkthroughWalkthroughThe STT engine now calibrates ambient noise, uses RMS-based speech gating, requires minimum speech content before enqueueing audio, filters hallucinated transcripts centrally, and changes the thread started by ChangesSTT Engine Noise Calibration and Filtering
Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/utils/stt_engine.py`:
- Line 67: The RMS threshold comment in stt_engine is using the non-ASCII
multiplication symbol, which Ruff flags. Update the wording in the comment near
the threshold calculation to use plain ASCII "x" instead of "×", keeping the
meaning the same and preserving the reference to the mean_rms threshold
description.
- Around line 119-122: The hallucination check in the STT engine is too
aggressive because `self.HALLUCINATIONS` is matched with `startswith` and
`endswith`, causing common short words to flag valid speech. Update the matching
logic in the relevant loop inside the STT engine method so it only treats a
phrase as a hallucination when it is an exact standalone match or otherwise
clearly bounded, and avoid prefix/suffix matching for entries like “a”, “to”,
“in”, and “is”. Keep the existing `lower` normalization and
`self.HALLUCINATIONS` iteration, but tighten the condition to prevent valid
sentences from being rejected.
- Around line 155-179: The utterance-length check in the speech capture loop is
using len(frames), which counts trailing silence and can let noise bursts pass
as real speech. Update the buffering logic in the chunk-processing loop to track
only speech chunks (for example, a separate counter in the recording_started
path of the STT engine) and use that count for the MIN_SPEECH_CHUNKS gate before
audio_queue.put. Keep the existing silence reset behavior, but ensure only
non-silent speech frames contribute to the threshold.
- Around line 83-102: The calibration stream in the STT noise calibration flow
is not guaranteed to be closed if an exception occurs during `read()` or
`_calculate_rms()`, which can leave the input device busy. Update the
calibration logic in the noise calibration method that opens `cal_stream` so
`stop_stream()` and `close()` are always executed even on failure, using a
cleanup path such as a guarded/finally-style close around the loop. Keep the
existing threshold calculation and logging behavior intact, but ensure
`cal_stream` is released before the exception handler logs the fallback message.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
1. Replace ambiguous non-ASCII × with plain x in docstring (Ruff RUF002) 2. Guard calibration stream with try/finally to ensure device is released on failure paths — prevents the input device staying busy 3. Only match hallucination phrases as exact matches (not startswith/endswith) to avoid false positives on valid speech like "a quick test" or "to be" 4. Track speech_chunks_count separately so trailing silence frames don't inflate the counter — a single noise spike followed by silence can no longer satisfy MIN_SPEECH_CHUNKS
Author
|
Thanks for the review, CodeRabbit — all 4 suggestions have been addressed in the latest commit (c8cbff3):
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/utils/stt_engine.py`:
- Around line 124-128: Strip surrounding punctuation before the exact-match
hallucination check in the stt_engine logic. In the loop over
self.HALLUCINATIONS, normalize the lower text so phrases like “thank you.” and
“bye bye!” match the same exact values as “thank you” and “bye bye” without
broadening the filter. Keep the existing exact-match behavior in this matching
block, just update the normalization used before comparing phrase == lower.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Whisper often appends punctuation like "." or "!" to transcriptions, so "thank you." and "bye bye!" miss the exact-match check against HALLUCINATIONS entries and leak through as valid speech. Strip common trailing punctuation before comparison to catch these variants without broadening the filter to startswith/endswith. Addresses finityfly#4 (comment)
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/utils/stt_engine.py`:
- Around line 265-269: Fix start() so it starts the initialized
self._stream_thread instead of the undefined self._thread, then start
self._process_thread as currently intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Author
|
Thanks for the thorough review — all feedback has been addressed. Changes madeCommit
Commit
The PR should be ready to merge now. Let me know if there's anything else you'd like adjusted. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes issue #3 — two related bugs in the STS pipeline that made it produce audible "ticking" sounds and respond with non-speech content.
Root Cause
The STT engine (
stt_engine.py) used a fixed silence threshold of 800 that was far too sensitive. Ambient room noise — tapping, breathing, fan hum, clicks — routinely exceeded this threshold. When even ~0.19s (3 chunks) of noise reached Groq Whisper, the model frequently hallucinated short words or syllables. These hallucinated fragments were then translated by the MT processor and synthesised by TTS, producing:Additionally, the old hallucination filter only blocked 4 hard-coded phrases, leaving dozens of common Whisper noise hallucinations uncaught.
Changes Made (
src/utils/stt_engine.py)SILENCE_THRESHOLD = 800max(mean_rms × 3.5, 800)MIN_SPEECH_CHUNKS = 3(~0.19s)MIN_SPEECH_CHUNKS = 8(~0.5s)MAX_SILENT_CHUNKS = 12(~0.77s)MAX_SILENT_CHUNKS = 15(~0.96s)"thank you","thanks for watching","subtitle","bye bye")_is_hallucination()method: ~30 known hallucination phrases + structural checks (min length, alphabetic content, repeated chars)[STT] Filtered hallucination: '...'logged for every discarded transcriptionTesting Notes
start()call, so re-running the script recalibrates.Closes #3
Summary by CodeRabbit