Skip to content

docs(pr-fixup): Step 3 加入 PR 顶层 comment 和 description 检查 - #244

Merged
lishuceo merged 4 commits into
mainfrom
feat/pr-fixup-issue-comments
May 24, 2026
Merged

lishuceo merged 4 commits into
mainfrom
feat/pr-fixup-issue-comments

Conversation

@lishuceo

Copy link
Copy Markdown
Owner

Summary

  • pr-fixup skill 之前只查 GraphQL reviewThreads,漏了 review bot 留在 PR 主时间线或 PR description summary 里的反馈
  • Step 3 现在拆成 3a/3b/3c 三个数据源:inline review、顶层 issue comment、PR description summary
  • 用最后一次 push 的时间过滤 issue comment,避免重复响应历史评论
  • Step 4 的 reply/resolve 代码也分情况处理:inline 走 GraphQL resolve;issue comment / summary 没有 thread,直接 gh pr comment 回复

Background

实际遇到的场景:review bot 在 PR description 里贴了"🟡 遗留 nit"标记,pr-fixup 没看到,下一轮 review 又把同样的问题提了一次。

Test plan

  • 下次 pr-fixup 运行时确认会拉取 gh pr view --json comments 和 PR body
  • 验证 gh pr comment 回复能正确关联到原评论

之前 pr-fixup 只查 reviewThreads(inline review 评论),漏了 review bot
留在 PR 主时间线或 PR description 里的整体评论,导致 nit 在下一轮 review
又会被重新提一次。现在 Step 3 分 3a/3b/3c 三个数据源拉取,并用最后一次
push 时间过滤掉历史评论。
@github-actions

github-actions Bot commented May 24, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @lishuceo's task in 3m 48s —— View job


Claude Code Review

  • Check previous review comments (none from claude[bot])
  • Read CLAUDE.md and PR context
  • Read full SKILL.md and diff
  • Analyze for bugs/security/architecture issues
  • Post review summary

⚠️ Issues Found

PR 的方向是对的:现有 skill 确实只看 reviewThreads,会漏掉 review bot 留在 PR 主时间线和 description 里的反馈,补这一块有价值。但当前实现里有几个会让 skill 实际跑起来表现不符合预期的问题。

🟡 1. #ISSUE_COMMENT_ID 占位符不能产生有效的 GitHub 跨引用(confidence 85)

Step 4 的"真实问题"模板:

gh pr comment PR_NUMBER --body "Fixed — <简述修改内容>(回复 #ISSUE_COMMENT_ID)"

GitHub 的 #NUMBER 自动链接是 issue/PR 的快捷引用,没有 "comment 引用" 这种语法。如果 agent 把 #ISSUE_COMMENT_ID 替换成实际数字(比如 #1779607495),GitHub 会:

  • 找到该编号的 issue/PR → 链到一个完全不相关的对象
  • 找不到 → 渲染成灰字"#1779607495",没有跳转

PR description 里也写了 "验证 gh pr comment 回复能正确关联到原评论" —— 当前模板按字面替换是关联不上的。

要真正关联到 issue comment,需要完整 URL 或 markdown 链接:

gh pr comment PR_NUMBER --body "Fixed — <简述修改内容>([回复原评论](https://github.com/OWNER/REPO/pull/PR_NUMBER#issuecomment-COMMENT_ID))"

或者干脆引用评论正文片段而不是 ID。

Fix this →

🟡 2. 3c(PR description)在同一轮 fixup 内会被重复处理(confidence 80)

3a 用 isResolved == false 去重,3b 用 createdAt > LAST_PUSH 去重,但 3c 完全没有去重机制。流程:

  1. 第 N 轮:3c 命中 "🟡 nit: foo" → Claude 改代码 → push(LAST_PUSH 前进)
  2. 第 N+1 轮:3a 没东西,3b 被 LAST_PUSH 过滤掉了,但 3c 仍命中同一条(description 没变)
  3. Claude 再读代码 → 已经是修好的 → 既不是"真实问题"也不是"误报",会陷入分类困境

至少需要其中一条机制:

  • agent 在当前 run 内记录已处理过的 3c 条目(in-memory),下一轮跳过
  • 用代码 diff 判断该问题是否已经在最近 commit 中处理(hash/grep)
  • 把 3c 条目复制到 PR 顶层 comment 进行 ack,让后续轮次靠 LAST_PUSH 过滤

否则同一个 /pr-fixup 循环里同一条 nit 会被反复"处理",每次都耗一次 LLM 调用。

🟡 3. Step 5 退出条件没跟上新的 3b/3c 分支(confidence 75)

Step 5 现有判定:

如果只有误报被 resolve(无代码修复)且 CI 全部通过: 结束循环

3b/3c 的"误报"路径只回复 gh pr comment,没有 thread 可以 resolve(Step 4 自己也明确说了"仅 inline review thread 适用")。如果本轮全是 3b/3c 反馈、Claude 全判为误报、没有代码修复,按字面意思读这个分支:被 resolve 的数 = 0,既不满足"有代码修复 → 回到 Step 1"也不满足"误报被 resolve → 结束",逻辑落入未定义状态。

建议把退出条件改为按"已处理(reply + 可选 resolve)"判断,比如:

如果本轮所有 review 反馈都已 ack(inline 已 resolve + issue comment/description 已回复)且无代码修复且 CI 全过 → 结束循环

🟡 4. 3c 关键字集合容易在普通 PR description 上误判(confidence 75)

检查 PR body 中是否包含 review summary 关键字(如 ## Review Summary、### Issues Found、🟡、🔴、nit、confidence、Suggested Action)

🟡/🔴 是 PR 作者写描述时也常用的视觉标记;nit/confidence 在自然语句里也会出现。例如本 PR description 没用这些标记,但其他 PR 作者会用类似 emoji 给自己列待办,会被 skill 误当成 review feedback 处理。

建议把判定收紧:要么要求 ## Review Summary / ### Code Review 这种结构化标题同时出现,要么要求 reviewer 标识(如 PR description 末尾的 bot signature),单凭单个 emoji 不应触发。


其他观察(不构成阻塞)

  • LAST_PUSH=$(git log -1 --format=%cI HEAD) 取的是本地 committer date,不是真正的 GitHub push 时间。在 amend/rebase 场景下可能略偏,但对此 skill 的语义影响不大。
  • Step 3 开头 gh pr view ... -q '{author, body, sha}' 只是打印 JSON,没演示如何拆到 shell 变量。Agent 应能自行处理,但不如直接给出 PR_AUTHOR=$(...) PR_BODY=$(...) 的写法明确。

整体评价

补 issue comment 和 description summary 这个方向是对的,遇到的具体场景(bot 把"遗留 nit"写在 description 里)也是真实痛点。主要的问题是新分支的去重和退出逻辑没和原有 3a 对齐 —— 这会让 skill 在同一个 fixup 循环里要么死活转不完一轮(3c),要么在 Step 5 卡在未定义状态(3b/3c 误报)。修掉上面 #1–#3 之后应该可以合并。
· branch feat/pr-fixup-issue-comments

@github-actions

github-actions Bot commented May 24, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @lishuceo's task in 2m 9s —— View job


Claude Code Review — Re-review after sync

  • Read CLAUDE.md and PR context
  • Read current SKILL.md and diff vs origin/main
  • Check previous review feedback (4 issues raised)
  • Analyze for any new issues
  • Post review summary

⚠️ Issues Still Outstanding — 0/4 fixed

The only change since my previous review is the merge commit 038b0b2 Merge branch 'main' into feat/pr-fixup-issue-comments. SKILL.md content is unchanged from 560116b, so all 4 issues from the prior review remain. I've now anchored them as inline comments for one-click [Fix this →] workflow:

# Issue Confidence Inline
🟡 1 #ISSUE_COMMENT_ID placeholder can't cross-reference a comment via #NUMBER syntax (line 233) 90 link
🟡 2 3c (PR description summary) has no dedup → same nit gets re-processed every loop iteration (line 194) 85 link
🟡 3 Step 5 exit condition doesn't handle 3b/3c reply-only paths (no thread to resolve) → undefined state when round is all-bogus 3b/3c (line 249, not in diff so listed here) 80 —
🟡 4 3c keyword set (🟡, 🔴, nit, confidence) over-matches normal PR descriptions (line 192) 78 link

For #3 (Step 5), the fix is to change the exit condition from "only false positives were resolved" to "all review feedback handled (inline resolved + issue comment/description replied) and no code fixes and CI green". Fix this →

Verdict

⚠️ Changes still requested — direction of the PR is correct (real pain point: review bot writes summary into PR description, current skill misses it), but the dedup/exit-condition gaps introduced by 3b/3c will make the skill misbehave in practice. Fix #1–#3 before merging; #4 is a follow-up tightening.

· branch feat/pr-fixup-issue-comments ·

Comment thread .claude/skills/pr-fixup/SKILL.md
Comment thread .claude/skills/pr-fixup/SKILL.md
Comment thread .claude/skills/pr-fixup/SKILL.md
- LAST_PUSH 改用 gh api 拿 head commit 的 committer date,避免本地
  rebase/amend 后时间不一致
- 顶层 comment 回复链接改用完整 URL,GitHub 不会把 #COMMENT_ID 解析成
  comment 跳转
- 3c PR description summary 检测改成强/弱信号双层规则,避免把普通 PR
  说明里偶尔出现的 🟡 误判为 review
@github-actions

github-actions Bot commented May 24, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @lishuceo's task in 3m 33s —— View job


Claude Code Review — Re-review after 4e1909d

  • Check previous review comments status (3 inline)
  • Read CLAUDE.md and PR context
  • Read current SKILL.md (committed HEAD) and diff vs origin/main
  • Analyze each prior issue against current state
  • Reply to all 3 inline comments with verdicts
  • Resolve fixed threads — ❌ blocked by Resource not accessible by integration
  • Post review summary

⚠️ Status — 2/3 inline issues fixed, 1 still outstanding

The new commit 4e1909d addresses Issues #1 and #4 from the prior review. Issue #2 (3c dedup) has been replied-to but the underlying gap remains. Issue #3 (Step 5 exit condition, never inline) is also still present but isn't on a changed line so can't be anchored as inline.

# Issue Conf Verdict
🟢 1 #ISSUE_COMMENT_ID placeholder → full URL with #issuecomment- anchor 90 Fixed (thread)
🟡 2 3c (PR description summary) has no dedup mechanism 85 Still outstanding (thread) — detection rules tightened, but a 3c entry will still re-match on every loop iteration since PR body doesn't change after Claude addresses it
🟡 3 Step 5 exit condition doesn't cover 3b/3c reply-only rounds (no thread to resolve) 80 Still outstanding — see below; can't anchor inline since line 298 isn't in the diff
🟢 4 3c keywords over-match → strong/weak signal split 78 Fixed (thread)

🟡 Issue #3 detail (carried forward, not inline-anchorable)

Step 4 explicitly states resolve only applies to inline review threads (line 251, 274): "仅 inline review thread 适用,issue comment 和 description summary 无 thread 可 resolve". But Step 5's only "no code fix" exit branch requires "误报被 resolve":

Failure case: a round where all review feedback comes from 3b/3c and all classified as "Not an issue" → Claude posts gh pr comment replies, no thread is resolved, no code is changed. Neither Step 5 branch matches → undefined behavior.

Suggested rewording (uses "handled" instead of "resolved", uniformly covers 3a/3b/3c):

如果本轮所有 review 反馈都已处理(inline 已 reply+resolve、3b/3c 已 reply)且无代码修复且 CI 全部通过 → 结束循环

Fix this →

Notes

  • I couldn't auto-resolve the two fixed threads (飞书 Claude 集成 #1, test: 集成 Vitest 测试框架 #4) — the GitHub Actions bot integration lacks resolveReviewThread permission. You'll need to mark them resolved manually in the GitHub UI.
  • Other observations: LAST_PUSH now uses GitHub's committer date via gh api (a good fix for the rebase/amend drift I'd noted earlier).

Verdict

⚠️ Changes still requested — direction & most fixes look good, but #2 (3c dedup) and #3 (Step 5 exit logic) still cause the skill to misbehave in practice for 3c-only rounds. Address both before merging.

· branch feat/pr-fixup-issue-comments ·

review bot 在 4e1909d 后再次指出:
- 3c (PR description summary) 没有去重机制,PR body 静态不变,会让
  同一条目在每轮循环里都被重新匹配
- Step 5 的"只有误报被 resolve"退出分支假设了 thread 存在,3b/3c
  的反驳路径只 reply 不 resolve,会让循环卡在未定义状态

修复:
- 3c 加 "本轮内存记录" 去重要求,处理过的条目下一轮跳过
- Step 5 退出条件改为按"已处理"判断(inline reply+resolve、
  3b/3c reply 视为已处理),不再要求 resolve 动作
@github-actions

github-actions Bot commented May 24, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @lishuceo's task in 2m 32s —— View job


Claude Code Review — Re-review after 4f0c5d5


✅ Approved — 4/4 prior issues fixed

The latest commit 4f0c5d5 closes out the remaining gaps from the previous round. All 4 inline threads are now resolved.

# Issue Conf Status
🟢 1 #ISSUE_COMMENT_ID placeholder → full URL with #issuecomment- anchor 90 Fixed in 4e1909d (line 252)
🟢 2 3c (PR description summary) dedup mechanism 85 Fixed in 4f0c5d5 (lines 210-215)
🟢 3 Step 5 exit condition covers 3b/3c reply-only rounds 80 Fixed in 4f0c5d5 (lines 304-308)
🟢 4 3c keyword detection split into strong/weak signals 78 Fixed in 4e1909d (lines 203-206)

Issue #2 verification

Lines 210-215 now explicitly mandate dedup for 3c entries, with two strategies offered and the in-memory approach recommended:

去重(重要):…为避免同一 /pr-fixup 调用内同一条 3c 条目被反复处理,必须做以下两件事之一:

  • 本轮内存记录:…维护一个集合…处理过的 3c 条目下一轮直接跳过
  • 镜像到顶层 comment:…让后续轮次靠 3b 的 LAST_PUSH 过滤自动跳过

推荐第一种(更便宜,不污染 PR 时间线)。

Because the skill is run by an LLM agent that retains conversation history across loop iterations, in-memory tracking is realistic and matches the original suggestion in the prior review.

Issue #3 verification

Lines 304-308 rewrite Step 5's "no code fix" exit branch:

如果本轮所有 review 反馈都已处理(inline 已 reply+resolve、3b/3c 已 reply)且无代码修复且 CI 全部通过:

  • 输出 "✅ 第 N 轮:处理 Y 个 review 反馈(含 Z 个反驳),所有 CI checks 通过"
  • 结束循环

注意:3b/3c 没有 thread 可以 resolve,"已处理"的标准是已经发出 gh pr comment 回复。不要因为"没有 resolve 动作"就误判为未处理而陷入死循环。

This closes the undefined-state hole when an entire round is 3b/3c reply-only. Combined with the existing Step 3 short-circuit (no CI fails + no 3a/3b/3c feedback → exit), the loop now has complete coverage for every termination scenario:

  • Mixed feedback with code fixes → "有代码修复" branch → push → loop
  • All-reply-only round (no code fix) → "都已处理" branch → exit ✅ (new)
  • Nothing pending → Step 3 short-circuit → exit

Other observations (non-blocking)

  • Line 153's gh pr view ... -q '{author, body, sha}' is just an exploratory JSON dump; the subsequent HEAD_SHA=$(...) / LAST_PUSH=$(...) lines do the actual variable capture. Agents should handle this fine.
  • LAST_PUSH now correctly uses GitHub's commit.committer.date via gh api, avoiding local rebase/amend drift — a good improvement from the original git log %cI approach noted in an earlier review.

Verdict

✅ Approved — direction is right (real pain point: review bot summary in PR description), and all 4 prior issues are now resolved. Ready to merge.

· branch feat/pr-fixup-issue-comments ·

@lishuceo

Copy link
Copy Markdown
Owner Author

已在 4f0c5d5 处理 review bot 指出的剩余两个问题:

  • Issue feat: SQLite 持久化会话,重启不丢状态 #2 (3c dedup): Step 3c 新增"本轮内存记录"去重要求 — Claude 在当前 /pr-fixup 调用内维护已处理 3c 条目的集合,下一轮跳过;并给出了备选方案(镜像到 PR 顶层 comment 让 LAST_PUSH 接管)。
  • Issue feat: Claude Code PR review & @claude comment Actions #3 (Step 5 exit condition): 退出分支改为按"已处理"判断:inline reply+resolve、3b/3c reply 都算已处理,不再要求 resolve 动作。新增 note 明确"3b/3c 没有 thread 可以 resolve,不要因此误判为未处理"。

3 个 inline thread 已 reply + resolve。

@lishuceo
lishuceo merged commit 7464114 into main May 24, 2026
10 checks passed
@lishuceo
lishuceo deleted the feat/pr-fixup-issue-comments branch May 24, 2026 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant