Skip to content

CI: feed Sentry breadcrumbs into the RCA handler - #4334

Open
agalin920 wants to merge 2 commits into
devfrom
maintenance/sentry-handler-breadcrumbs
Open

agalin920 wants to merge 2 commits into
devfrom
maintenance/sentry-handler-breadcrumbs

Conversation

@agalin920

@agalin920 agalin920 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Prompt-only change to .github/workflows/claude-sentry-handler.yml. No issue.

Makes the Sentry RCA handler read breadcrumbs, which it never asked for before.

  • A.2 (new) — fetches the trail with get_issue_breadcrumbs and greps the data-cy to locate the component. Non-fatal on failure. Steps renumbered A.2A.4A.3A.5.
  • C.2 — adds a Last user actions field to the RCA comment.

The handler fetched stacktrace, culprit, tags, frequency and affected-user
count, then read source around each non-vendor frame. It never asked for
breadcrumbs, even though src/utility/sentry.js enriches ui.click and ui.input
crumbs with data-cy, id, name and button text specifically to make that trail
useful.

Prompt-only. get_issue_breadcrumbs is already covered by the mcp__sentry prefix
in --allowedTools, needs the same event:read scope as the existing issue fetch,
and resolves org + issue id from SENTRY_PERMALINK alone, so no new secret,
tool grant or YAML structure.

- A.2 fetches the trail and greps the data-cy to locate the component. Pins
  eventId to step 1's event where available, since two independent "latest"
  resolutions can diverge on a high-frequency error.
- A.2 states explicitly that breadcrumbs localize but do not establish
  certainty: a trace bottoming out in vendor internals still FAILs
  "Root-cause certainty". sentry-rca-classification.md is untouched and remains
  the single source of truth for the auto-fix bar.
- C.2 gains a "Last user actions" field so the trail reaches a human. Scoped to
  ui.click/ui.input/navigation crumbs with console and HTTP payloads excluded,
  since this repo is public; breadcrumb text is stripped of backticks and
  leading @ or # before quoting, as it is untrusted production data rendered as
  live markdown in an issue comment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

QA Review — ✅ PASS

No linked issue found — change-only QA.

  1. ✅ Step renumbering (A.2–A.4 → A.3–A.5) is applied consistently — every in-prompt cross-reference ("step 1", "step 4", "A.2", "A.5", C.1/C.2/C.3/C.4/C.5) matches the new sequence with no stale reference left behind.
  2. ✅ The data-cy enrichment claim is accurate — src/utility/sentry.js's beforeBreadcrumb hook tags ui.click/ui.input breadcrumbs with data-cy, matching the prompt's description of what Grep should look for.
  3. ✅ Breadcrumb-fetch failure is correctly scoped as non-fatal and separate from the existing fetch-failed path — only a failed issue fetch in step 1 routes to C.3; a failed or absent breadcrumb call falls through to "carry on" / "No breadcrumb trail" / "Breadcrumb fetch failed" text instead.
  4. ✅ The new Last user actions field addresses injection risk from untrusted production breadcrumb data landing in a public PR comment — it forbids quoting console/HTTP payloads, and requires stripping backticks before wrapping each crumb in its own code span specifically to stop @/# autolinking and comment-fence escapes.
  5. ⚠️ Whether mcp__sentry__get_issue_breadcrumbs and its issueUrl/eventId parameters match the real Sentry MCP server's tool schema — can't confirm against an external MCP server from the diff alone.
Suggested Cypress coverage

This PR only edits .github/workflows/claude-sentry-handler.yml (a prompt fed to an AI agent in CI) — there is no application code change and no corresponding cypress/e2e/ area to cover. The existing Cypress suite has no bearing on this change; verification for prompt behavior would happen via the workflow's own runs against real Sentry issues, not Cypress.

Comment on lines +317 to +320
payload. Breadcrumb text is untrusted production data rendered as live
markdown here, so strip backticks and any leading `@` or `#` from it
before quoting, or it will ping real accounts, cross-link unrelated
issues, or open a code fence that swallows the rest of this comment. If

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Sanitization only strips leading @/#, not all occurrences
GitHub autolinks #1234 and @user anywhere in plain comment text, not just at the start of the string — e.g. a ui.click breadcrumb whose button text is dynamic content like "Notify @security-team about ticket #4321" would still ping/cross-link once quoted, since only a leading occurrence gets stripped. This defeats the stated goal ("or it will ping real accounts, cross-link unrelated issues"). Fix: instruct stripping/escaping every @ and # occurrence in the breadcrumb text, not just a leading one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ecfda3e8f. Enumerating characters is the wrong shape of fix, so rather than also stripping non-leading occurrences, each crumb is now wrapped in single backticks — GitHub does not autolink inside a code span, which covers @ and # wherever they appear plus any other markdown in the string. Stripping backticks from the crumb first (already required) is what keeps it from escaping its own span.

@github-actions

Copy link
Copy Markdown
Contributor

Code Review — 🔴 1 blocker(s) — see inline comments

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Negative QA

No reproducible edge-case failures found on the surfaces this PR changes.

Also checked and working correctly
  • Confirmed the diff is scoped entirely to .github/workflows/claude-sentry-handler.yml (a CI automation prompt for the Sentry RCA handler) with no changes under src/, public/, or any other application code — there is no rendered UI surface for this PR to probe.

The auto-reviewer is right that stripping only a leading @ or # misses the
case that matters: GitHub autolinks @user and #1234 anywhere in a line, and
button text is dynamic content, so "Notify @some-team about #4321" would still
ping and cross-link once quoted.

Enumerating characters is the wrong shape of fix. Wrap each crumb in single
backticks instead — GitHub does not autolink inside a code span, which covers
@ and # wherever they appear plus any other markdown in the string. Stripping
backticks from the crumb first is what keeps it from escaping its own span.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants