Skip to content

Emit first-conversation experience signals to the feedback sink #518

Description

@rosscado

Why

A read-only analysis of the SaaS onboarding data (2026-07-06) found that the
day-2 retention story is not what the dashboard suggested, and it points a
finger squarely at the extension's first-run experience:

  • The scary "3% return" was a measurement artifact (right-censoring). True
    D2+ retention among activated users is ~39%.
  • Among activated users who never return, the churn decomposes into: ~9% hit the
    free-quota wall, ~28% one-touch tourists, and ~63% who genuinely engaged
    (multiple turns, no wall) and simply never came back.
  • That 63% is the churn worth fixing — and it is completely invisible to the
    SaaS backend
    , because the first conversation happens client-side against
    third-party AI. The SaaS can size the hole but cannot explain it.

The quota wall was exonerated as a cause (users who hit it return at 59% vs
36% — it's a heavy-engagement marker). So the lever isn't pricing; it's whatever
happens in that first conversation. To learn what, the extension needs to emit a
small set of first-run experience signals.

What

Emit best-effort first-conversation signals through the existing feedback
sink
we already use for post-win surveys (FEEDBACK_URL,
src/onboarding/feedbackPayload.ts / feedbackClient.ts). No new endpoint
needed — the SaaS /api/feedback sink is generic and already accepts these
(Bearer JWT; body { source, rating?, message?, metadata?, app?, version? }).

Proposed canonical vocabulary (final naming is yours; SaaS will aggregate on it):

source when metadata
first_conversation_issue a friction event during the first run { issue: "mic_denied" | "mic_no_device" | "audio_playback_failed" | "transcription_rerecord" | "page_incompatible" | "extension_error" }
first_conversation_completed first conversation ends { turns, durationMs, surface }
first_conversation_rating optional one-tap "was that useful?" rating (0–10 or a thumbs→score mapping) + optional message

Always include app and version (the sink stores them).

Constraints (from how the sink already works)

  • Best-effort only — a feedback failure must never disrupt the conversation
    (same swallow-errors discipline as submitPostWinFeedback).
  • Scope to first-run / first N conversations — this is onboarding telemetry,
    not steady-state; don't emit for every conversation forever.
  • Bounded volume (the sink has a per-IP + per-identity rate limit).

Acceptance criteria

  • New-user first conversations produce rows in the SaaS Feedback table with the
    agreed source values and structured metadata.
  • Emission is best-effort and scoped to first-run.
  • The vocabulary is documented alongside feedbackPayload.ts.

Notes / non-goals

  • No conversation content — events + ratings only, never transcripts
    (third-party AI, privacy).
  • Detection mechanics (how you know a mic was denied, a re-record happened, etc.)
    are left to your judgement — this issue defines the contract, not the impl.
  • Complements a companion saypi-api issue to cohort-label the STT/TTS quality
    metrics; together they cover the technical + experiential sides of the first
    conversation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions