Skip to content

[CI] - Stop pre-checking Sentry auto-fix PR reviewers via collaborators endpoint - #4313

Merged
finnar-bin merged 7 commits into
devfrom
fix/4298-resolve-reviewer-false-flag-checks
Sep 17, 2026
Merged

finnar-bin merged 7 commits into
devfrom
fix/4298-resolve-reviewer-false-flag-checks

Conversation

@finnar-bin

@finnar-bin finnar-bin commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #4298

Summary

  • The "Open auto-fix PR" step in .github/workflows/claude-sentry-handler.yml used to pre-check each candidate reviewer (agalin920, geodem127, finnar-bin) via gh api repos/{owner}/{repo}/collaborators/{username} before calling gh pr create --reviewer.

  • That endpoint can't see access granted via org team membership under the default GITHUB_TOKEN, so it 404'd for all three despite them having real write access through team membership — the pre-check treated them as having no access and silently skipped every reviewer request on every auto-fix PR.

  • Fix: create the PR first without --reviewer, then request each candidate individually by POSTing directly to repos/{owner}/{repo}/pulls/{n}/requested_reviewers.

    • Not gh pr edit --add-reviewer: confirmed live against this PR that its GraphQL query pulls the deprecated Projects (classic) projectCards field, which errors out on every invocation in this repo regardless of the reviewer's validity — using it would have warned on every single reviewer, every time, masking valid ones just like the original bug.
    • The REST endpoint has its own quirk: it returns HTTP 200 and silently drops an unknown/nonexistent login instead of erroring, so exit code alone can't detect a bad login.
    • The warning now surfaces the actual reason instead of a generic message: GitHub's .message for a real access error (e.g. "not a collaborator"), or an explicit note when the login was silently dropped from a 200 response with no reviewer added.
  • Verified against three real response shapes by testing live against this PR (cleaning up each test review request immediately after):

    • valid login, has access → added, appears in requested_reviewers, no warning
    • nonexistent login → HTTP 200, silently absent from requested_reviewers → warning: "the login may not exist"
    • real user, not a collaborator → HTTP 422 with a message → warning includes that message verbatim

    All three are handled correctly, and set -euo pipefail does not abort the step on either failure case (the failing command sits inside resp=$(...) || true, so it isn't part of any &&/pipeline chain that -e would catch).

  • Single file changed: .github/workflows/claude-sentry-handler.yml.

Test plan

  • Validated YAML syntax locally: python3 -c "import yaml; yaml.safe_load(open('.github/workflows/claude-sentry-handler.yml'))"
  • Live-tested the REST reviewer-request call against this PR itself ([CI] - Stop pre-checking Sentry auto-fix PR reviewers via collaborators endpoint #4313) for all three outcome shapes (real access, nonexistent login, real user without access), then removed the test review requests immediately after
  • Confirmed gh pr edit --add-reviewer (and gh pr edit in general) fails unconditionally in this repo due to an unrelated Projects (classic) deprecation, which is why the fix avoids it entirely
  • Full end-to-end verification still requires a real sentry-rca workflow run that produces a simple verdict (per the issue's acceptance criteria), since org team-membership resolution under the Actions GITHUB_TOKEN can only be exercised from within Actions

… endpoint

GET repos/{owner}/{repo}/collaborators/{username} can't see access granted
via org team membership under GITHUB_TOKEN, so it 404s for team-granted
reviewers and silently drops all of them. Create the PR first, then request
each candidate reviewer individually so a genuinely stale login just warns
instead of masking valid ones.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@finnar-bin finnar-bin added bug Something isn't working severity: low Minor impact — cosmetic, rare edge case, or easy workaround labels Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

QA Review — ✅ PASS

Validates #4298: CI - Sentry auto-fix PR reviewer check false-flags team-access reviewers as having no access

  1. ✅ No longer gates reviewer requests on GET repos/{owner}/{repo}/collaborators/{username} — that pre-check loop is deleted entirely; the PR is now created with no --reviewer args at all.
  2. ✅ Auto-fix PR is created first, then each candidate is requested individually via POST repos/$REPO/pulls/$pr_number/requested_reviewers.
  3. ✅ A single invalid/stale login degrades gracefully — the gh api call is followed by || true and the loop continues to the next candidate rather than aborting the step.
  4. ✅ An un-addable reviewer emits a visible ::warning:: with a reason, using the response .message when present and a fallback explanation when GitHub silently drops an unknown login from a 200 response.
  5. ⚠️ set -euo pipefail / exit-code semantics preserved — the unguarded reason=$(jq -r '.message // empty' <<<"$resp") assignment isn't wrapped in || true; if resp were ever empty/non-JSON (e.g. a total gh api network failure rather than a JSON 404 body), that jq call would return nonzero and set -e would abort the step, which is a different failure mode than the intended "one bad login just warns." Can't confirm from code alone whether this path is ever hit in practice — ordinary stale-login 404s return a JSON body, per the comment's stated assumption.
  6. ✅ No change to the job permissions: block — untouched by this diff.
Suggested Cypress coverage

This change touches only .github/workflows/claude-sentry-handler.yml (a CI/automation workflow), not application UI code, so there is no corresponding app feature under cypress/e2e/ and no spec should be added for it. Coverage for this behavior belongs in a workflow-level check (e.g. exercising the sentry-rca job via workflow_dispatch or an act/dry-run harness) rather than Cypress.

Comment thread .github/workflows/claude-sentry-handler.yml Outdated
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Negative QA

No reproducible edge-case failures found on the surfaces this PR changes.

Also checked and working correctly
  • Confirmed the diff is limited to .github/workflows/claude-sentry-handler.yml (a CI automation workflow for auto-generated Sentry fix PRs) with no changes to any application source under src/ — there is no UI route or component surface introduced or modified by this PR to attack.

@finnar-bin
finnar-bin requested review from agalin920 and removed request for agalin920 September 1, 2026 01:25
finnar-bin and others added 2 commits September 1, 2026 09:27
…body

gh pr edit --add-reviewer fails on every call in this repo: its GraphQL
query pulls the deprecated Projects (classic) projectCards field, which
errors out regardless of the reviewer. Confirmed by testing directly
against PR #4313 with both a real accessible user and a nonexistent one —
both failed identically on the projectCards error, meaning every reviewer
request would have warned even for valid logins.

Switch to POSTing repos/{owner}/{repo}/pulls/{n}/requested_reviewers
directly. That endpoint also has its own quirk: it returns 200 and
silently drops an unknown login instead of erroring, so success can't be
read from the exit code alone. Verified against three real response
shapes (valid login added, unknown login silently dropped, real user
without access returns 422) — the fix now checks the response body for
the login rather than trusting exit status.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dded

The prior warning just said "no longer has access" for every failure
case, without saying why. Extract the real reason from the response: the
GitHub-provided .message for an actual access error (e.g. not a
collaborator), or an explicit note that the login was silently dropped
when GitHub returns 200 without adding it (the observed behavior for an
unknown/misspelled login).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@finnar-bin

Copy link
Copy Markdown
Contributor Author

Thanks for confirming — matches expectations, since this PR only touches .github/workflows/claude-sentry-handler.yml (the Sentry auto-fix PR's reviewer-request step) and has no src/ or UI surface for the negative-QA browser pass to exercise.

For anyone following along: this change was also verified live against this PR itself while developing it — the reviewer-request REST call was exercised against three real response shapes (valid reviewer added, unknown login silently dropped by GitHub, and a real non-collaborator returning 422), with each test review request removed immediately after. Full end-to-end coverage of the actual reviewer-assignment behavior still needs a real sentry-rca run with a simple verdict, since it depends on org team-membership resolution under the Actions GITHUB_TOKEN that can only be exercised from within Actions.

Comment thread .github/workflows/claude-sentry-handler.yml
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Overall Coverage

Lines Statements Branches Functions
61.59% 61.21% 55.94% 56.15%

Changed Files Coverage

File Lines Statements Branches Functions
No changed files found - - - -

geodem127
geodem127 previously approved these changes Sep 1, 2026
@finnar-bin finnar-bin self-assigned this Sep 1, 2026
agalin920
agalin920 previously approved these changes Sep 2, 2026
Comment thread .github/workflows/claude-sentry-handler.yml
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

# Conflicts:
#	.github/workflows/claude-sentry-handler.yml
@finnar-bin
finnar-bin dismissed stale reviews from agalin920 and geodem127 via c241916 September 4, 2026 00:13
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers

@finnar-bin
finnar-bin enabled auto-merge (squash) September 4, 2026 02:46
Comment thread .github/workflows/claude-sentry-handler.yml
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

@github-actions

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers

@finnar-bin
finnar-bin merged commit ff3b52b into dev Sep 17, 2026
19 of 24 checks passed
@finnar-bin
finnar-bin deleted the fix/4298-resolve-reviewer-false-flag-checks branch September 17, 2026 05:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working severity: low Minor impact — cosmetic, rare edge case, or easy workaround

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI - Sentry auto-fix PR reviewer check false-flags team-access reviewers as having no access

3 participants