Why
A read-only analysis of the SaaS onboarding data (2026-07-06) found that the
day-2 retention story is not what the dashboard suggested, and it points a
finger squarely at the extension's first-run experience:
- The scary "3% return" was a measurement artifact (right-censoring). True
D2+ retention among activated users is ~39%.
- Among activated users who never return, the churn decomposes into: ~9% hit the
free-quota wall, ~28% one-touch tourists, and ~63% who genuinely engaged
(multiple turns, no wall) and simply never came back.
- That 63% is the churn worth fixing — and it is completely invisible to the
SaaS backend, because the first conversation happens client-side against
third-party AI. The SaaS can size the hole but cannot explain it.
The quota wall was exonerated as a cause (users who hit it return at 59% vs
36% — it's a heavy-engagement marker). So the lever isn't pricing; it's whatever
happens in that first conversation. To learn what, the extension needs to emit a
small set of first-run experience signals.
What
Emit best-effort first-conversation signals through the existing feedback
sink we already use for post-win surveys (FEEDBACK_URL,
src/onboarding/feedbackPayload.ts / feedbackClient.ts). No new endpoint
needed — the SaaS /api/feedback sink is generic and already accepts these
(Bearer JWT; body { source, rating?, message?, metadata?, app?, version? }).
Proposed canonical vocabulary (final naming is yours; SaaS will aggregate on it):
source |
when |
metadata |
first_conversation_issue |
a friction event during the first run |
{ issue: "mic_denied" | "mic_no_device" | "audio_playback_failed" | "transcription_rerecord" | "page_incompatible" | "extension_error" } |
first_conversation_completed |
first conversation ends |
{ turns, durationMs, surface } |
first_conversation_rating |
optional one-tap "was that useful?" |
rating (0–10 or a thumbs→score mapping) + optional message |
Always include app and version (the sink stores them).
Constraints (from how the sink already works)
- Best-effort only — a feedback failure must never disrupt the conversation
(same swallow-errors discipline as submitPostWinFeedback).
- Scope to first-run / first N conversations — this is onboarding telemetry,
not steady-state; don't emit for every conversation forever.
- Bounded volume (the sink has a per-IP + per-identity rate limit).
Acceptance criteria
- New-user first conversations produce rows in the SaaS
Feedback table with the
agreed source values and structured metadata.
- Emission is best-effort and scoped to first-run.
- The vocabulary is documented alongside
feedbackPayload.ts.
Notes / non-goals
- No conversation content — events + ratings only, never transcripts
(third-party AI, privacy).
- Detection mechanics (how you know a mic was denied, a re-record happened, etc.)
are left to your judgement — this issue defines the contract, not the impl.
- Complements a companion saypi-api issue to cohort-label the STT/TTS quality
metrics; together they cover the technical + experiential sides of the first
conversation.
Why
A read-only analysis of the SaaS onboarding data (2026-07-06) found that the
day-2 retention story is not what the dashboard suggested, and it points a
finger squarely at the extension's first-run experience:
D2+ retention among activated users is ~39%.
free-quota wall, ~28% one-touch tourists, and ~63% who genuinely engaged
(multiple turns, no wall) and simply never came back.
SaaS backend, because the first conversation happens client-side against
third-party AI. The SaaS can size the hole but cannot explain it.
The quota wall was exonerated as a cause (users who hit it return at 59% vs
36% — it's a heavy-engagement marker). So the lever isn't pricing; it's whatever
happens in that first conversation. To learn what, the extension needs to emit a
small set of first-run experience signals.
What
Emit best-effort first-conversation signals through the existing feedback
sink we already use for post-win surveys (
FEEDBACK_URL,src/onboarding/feedbackPayload.ts/feedbackClient.ts). No new endpointneeded — the SaaS
/api/feedbacksink is generic and already accepts these(Bearer JWT; body
{ source, rating?, message?, metadata?, app?, version? }).Proposed canonical vocabulary (final naming is yours; SaaS will aggregate on it):
sourcemetadatafirst_conversation_issue{ issue: "mic_denied" | "mic_no_device" | "audio_playback_failed" | "transcription_rerecord" | "page_incompatible" | "extension_error" }first_conversation_completed{ turns, durationMs, surface }first_conversation_ratingrating(0–10 or a thumbs→score mapping) + optionalmessageAlways include
appandversion(the sink stores them).Constraints (from how the sink already works)
(same swallow-errors discipline as
submitPostWinFeedback).not steady-state; don't emit for every conversation forever.
Acceptance criteria
Feedbacktable with theagreed
sourcevalues and structuredmetadata.feedbackPayload.ts.Notes / non-goals
(third-party AI, privacy).
are left to your judgement — this issue defines the contract, not the impl.
metrics; together they cover the technical + experiential sides of the first
conversation.