release: automate deferred remote gates - #171
Conversation
|
Warning Review limit reached
Next review available in: 39 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (19)
📝 WalkthroughWalkthroughThe PR defines a deferred-work wakeup policy and applies it to release, deployment, provider, and project guidance. Repository release automation now uses bounded ChangesDeferred continuation and release workflow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Agent
participant ReleaseScript
participant GitHub
participant HostScheduler
Agent->>ReleaseScript: run prepare
ReleaseScript->>GitHub: create or update release PR
ReleaseScript->>HostScheduler: emit delayed continuation
HostScheduler->>ReleaseScript: run inspect
ReleaseScript->>GitHub: read checks and review state
ReleaseScript->>GitHub: run advance when gates and identities match
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01746199bb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| CHECK_STATE="$(gh pr checks "$pr_number" --json name,state,workflow --jq 'map(.name + ":" + .state) | join(", ")' 2>/dev/null || printf 'unreadable')" | ||
| check_count="$(gh pr checks "$pr_number" --json state --jq 'length' 2>/dev/null || printf '0')" | ||
| pending_count="$(gh pr checks "$pr_number" --json state --jq '[.[].state | select(. == "PENDING" or . == "QUEUED" or . == "IN_PROGRESS" or . == "WAITING")] | length' 2>/dev/null || printf '1')" | ||
| failing_count="$(gh pr checks "$pr_number" --json state --jq '[.[].state | select(. == "FAILURE" or . == "ERROR" or . == "CANCELLED" or . == "TIMED_OUT" or . == "ACTION_REQUIRED")] | length' 2>/dev/null || printf '1')" |
There was a problem hiding this comment.
Whenever any check is still pending, gh pr checks intentionally exits with status 8 (documented by its help as “Checks still pending”) even though it emits valid JSON. The || printf '1' therefore appends a fallback to the valid zero result, making failing_count something like 0\n1; the subsequent comparison classifies healthy in-progress CI as failed-checks instead of scheduling the required continuation. Capture the output and exit status separately, treating status 8 as a valid pending snapshot.
AGENTS.md reference: AGENTS.md:L31-L31
Useful? React with 👍 / 👎.
| pr_number="$(gh pr list --head "$branch_name" --base "$base_branch" --json number --jq '.[0].number // empty' --limit 1)" | ||
| [ -n "$pr_number" ] || die "No release PR exists for branch $branch_name into $base_branch. Run --operation prepare first." |
There was a problem hiding this comment.
If the PR branch tip changes after prepare—for example, another actor pushes a commit—the advance path finds the PR only by branch name, skips local validation, and merges the new tip without comparing it to the continuation packet's head_commit. Neither command accepts the expected SHA, and inspect does not read the remote PR head OID, so an agent cannot enforce the documented identity gate; persist or pass the prepared SHA and reject advancement unless the local branch and PR head still match it.
AGENTS.md reference: AGENTS.md:L62-L62
Useful? React with 👍 / 👎.
| printf '%s\n' "{\"schema\":\"repo-maintenance-continuation/v1\",\"operation\":\"standard-release\",\"repository\":\"$repo_name\",\"release_tag\":\"$RELEASE_TAG\",\"branch\":\"$branch_name\",\"head_commit\":\"$head_sha\",\"pr_number\":\"$pr_number\",\"phase\":\"$phase\",\"minimum_delay_minutes\":5,\"resume_command\":\"scripts/repo-maintenance/release.sh --mode standard --version $RELEASE_TAG --operation inspect\",\"advance_command\":\"scripts/repo-maintenance/release.sh --mode standard --version $RELEASE_TAG --operation advance\"}" | ||
| log "Schedule exactly one host-native continuation no sooner than five minutes. On wakeup run inspect first; run advance only if this branch, commit, PR, and tag still match." |
There was a problem hiding this comment.
The packet always resumes with the PR-only inspect/advance commands, but that is unusable for two phases emitted by this script: after awaiting-branch-visibility no PR has been created yet, while after awaiting-tag-visibility or awaiting-github-release-visibility the PR has been merged and fast_forward_base_branch has switched to main. The former dies for a missing PR; the latter is rejected by ensure_branch_release_context (and the default PR listing would not find the closed PR anyway), so transient visibility—the exact condition this feature handles—strands the release before publication or cleanup.
AGENTS.md reference: AGENTS.md:L31-L31
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6641a12b7c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| phase="$3" | ||
| repo_name="$(gh repo view --json nameWithOwner --jq '.nameWithOwner' 2>/dev/null || printf 'unknown')" | ||
| head_sha="$(git -C "$REPO_ROOT" rev-parse HEAD)" | ||
| printf '%s\n' "{\"schema\":\"repo-maintenance-continuation/v1\",\"operation\":\"standard-release\",\"repository\":\"$repo_name\",\"release_tag\":\"$RELEASE_TAG\",\"branch\":\"$branch_name\",\"head_commit\":\"$head_sha\",\"pr_number\":\"$pr_number\",\"phase\":\"$phase\",\"minimum_delay_minutes\":5,\"resume_command\":\"scripts/repo-maintenance/release.sh --mode standard --version $RELEASE_TAG --operation inspect\",\"advance_command\":\"scripts/repo-maintenance/release.sh --mode standard --version $RELEASE_TAG --operation advance\"}" |
There was a problem hiding this comment.
The newly documented packet contract in docs/maintainers/deferred-work-wakeup-policy-plan.md requires observed_at and not_before, with the latter validated as at least five minutes after observation, but every packet emitted here contains only minimum_delay_minutes. A delayed or fresh-session consumer therefore cannot determine when the snapshot was taken or validate the required scheduling floor from the packet itself. Include the two absolute timestamps so the script matches the continuation contract.
AGENTS.md reference: AGENTS.md:L62-L62
Useful? React with 👍 / 👎.
| phase="$3" | ||
| repo_name="$(gh repo view --json nameWithOwner --jq '.nameWithOwner' 2>/dev/null || printf 'unknown')" | ||
| head_sha="$(git -C "$REPO_ROOT" rev-parse HEAD)" | ||
| printf '%s\n' "{\"schema\":\"repo-maintenance-continuation/v1\",\"operation\":\"standard-release\",\"repository\":\"$repo_name\",\"release_tag\":\"$RELEASE_TAG\",\"branch\":\"$branch_name\",\"head_commit\":\"$head_sha\",\"pr_number\":\"$pr_number\",\"phase\":\"$phase\",\"minimum_delay_minutes\":5,\"resume_command\":\"scripts/repo-maintenance/release.sh --mode standard --version $RELEASE_TAG --operation inspect\",\"advance_command\":\"scripts/repo-maintenance/release.sh --mode standard --version $RELEASE_TAG --operation advance\"}" |
There was a problem hiding this comment.
Because the JSON object is assembled by direct shell interpolation, a valid Git ref containing a double quote—for example, a branch accepted by git check-ref-format 'refs/heads/release/"quoted"'—makes the emitted continuation packet invalid JSON. The same applies to permitted prerelease-tag characters embedded in release_tag and the command strings. Serialize these values with a JSON-aware tool rather than interpolating them into a literal.
AGENTS.md reference: AGENTS.md:L31-L31
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@plugins/cloud-deployment-skills/skills/dockerized-service-release-deployment-workflow/SKILL.md`:
- Line 60: Require identity equality before resumed work: at
plugins/cloud-deployment-skills/skills/dockerized-service-release-deployment-workflow/SKILL.md
lines 60 and 118, skills/dockerized-service-release-deployment-workflow/SKILL.md
lines 60 and 118, plugins/cloud-inference-skills/AGENTS.md line 21, and
plugins/server-side-swift/skills/fly-io-deployment-workflow/SKILL.md line 215,
compare all recorded identities with the freshly inspected values before
continuing. Require tag, digest, run, environment, target, release, deployment
URL, health target, Runpod state, or Fly app/release/image identities as
applicable; on any mismatch, stop and create a new continuation packet rather
than resuming, advancing, diagnosing, rolling back, or deploying.
In
`@plugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/release.sh`:
- Around line 268-276: Update emit_continuation_packet to choose prepare rather
than inspect for phases before PR creation, while retaining inspect for
resumable post-PR phases. In the inspect flow around ensure_clean_worktree and
PR validation, avoid die() when the worktree is dirty or no PR exists; return
the normalized “not started” state instead, preserving inspect’s no-wait,
no-mutation behavior.
- Around line 268-276: Align the documented continuation-packet contract with
the shipped implementation: in
plugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/release.sh
lines 268-276, keep emit_continuation_packet unchanged; in
docs/maintainers/deferred-work-wakeup-policy-plan.md lines 169-201, document
minimum_delay_minutes as the sole delay field and remove or revise references to
observed_at, not_before, and blocked_gate, including the related acceptance
criterion at lines 270-272.
- Around line 246-266: The inspect_pr_gate function should derive check outcomes
from gh pr checks bucket values rather than manually matching raw states, using
pending, fail, pass, skipping, and cancel to drive the gate phase. Replace the
unconditional check_count == 0 block with an explicit minimum required-check
setting or advance opt-in, allowing zero-check repositories to proceed only when
that signal is configured.
In
`@plugins/repository-skills/skills/maintain-project-repo/references/automation-prompts.md`:
- Line 9: Update the instruction near the “watch CI” wording in the automation
prompt to describe bounded inspection or snapshot checks instead of ongoing CI
watching or polling. Keep it consistent with the bounded prepare/inspect/advance
continuation flow and the delayed host-native wakeup described alongside it.
In
`@plugins/repository-skills/skills/maintain-project-repo/references/release-modes.md`:
- Around line 56-58: Update the submodule-mode release instructions around
branch, tag, and GitHub release visibility checks to require exactly one
host-native continuation at least five minutes later when any visibility check
is missing, followed by inspection before proceeding with additional release
actions; remove the generic continuation wording while preserving the no-polling
behavior.
- Line 13: Update the version bump instruction in release-modes.md to resolve
the hook path from the selected profile’s root, preserving profile-specific
casing such as Scripts/repo-maintenance for xcode-app; alternatively document
both supported paths.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: c54e7120-5c1a-4214-936b-fd3edfdcadbf
📒 Files selected for processing (23)
AGENTS.mdROADMAP.mddocs/maintainers/deferred-work-wakeup-policy-plan.mdplugins/agent-engineering-skills/skills/schedule-agent-work/SKILL.mdplugins/apple-dev-skills/skills/bootstrap-swift-package/assets/AGENTS.mdplugins/apple-dev-skills/skills/bootstrap-xcode-app-project/assets/AGENTS.mdplugins/apple-dev-skills/skills/sync-swift-package-guidance/assets/AGENTS.mdplugins/apple-dev-skills/skills/sync-swift-package-guidance/assets/append-section.mdplugins/apple-dev-skills/skills/sync-xcode-project-guidance/assets/AGENTS.mdplugins/apple-dev-skills/skills/sync-xcode-project-guidance/assets/append-section.mdplugins/cloud-deployment-skills/skills/dockerized-service-release-deployment-workflow/SKILL.mdplugins/cloud-inference-skills/AGENTS.mdplugins/repository-skills/skills/maintain-project-repo/SKILL.mdplugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/config/release.envplugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/lib/common.shplugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/release.shplugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/release/30-push-release.shplugins/repository-skills/skills/maintain-project-repo/assets/repo-maintenance/release/40-github-release.shplugins/repository-skills/skills/maintain-project-repo/references/automation-prompts.mdplugins/repository-skills/skills/maintain-project-repo/references/release-modes.mdplugins/repository-skills/skills/maintain-project-repo/tests/test_maintain_project_repo_workflow.pyplugins/server-side-swift/skills/fly-io-deployment-workflow/SKILL.mdskills/dockerized-service-release-deployment-workflow/SKILL.md
|
Addressed the valid review findings in 3a504ec: pending-check JSON is preserved and interpreted via |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a504ec1fa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| elif [ "$REVIEW_DECISION" = "CHANGES_REQUESTED" ]; then | ||
| GATE_PHASE="changes-requested" |
There was a problem hiding this comment.
When branch protection requires an approval and none has been submitted, GitHub reports reviewDecision=REVIEW_REQUIRED; this condition falls through to ready-to-advance, so advance invokes gh pr merge, which is rejected by branch protection, and set -e exits without the continuation packet required for remote review gates. Classify REVIEW_REQUIRED as a pending review phase and emit a continuation instead.
AGENTS.md reference: AGENTS.md:L31-L31
Useful? React with 👍 / 👎.
| Planned. This document is an audit and implementation plan only; it does not | ||
| change any released skill, release script, CI workflow, or deployment adapter. |
There was a problem hiding this comment.
This new document says the policy is only planned and does not change released skills or scripts, but the same commit changes the shipped maintain-project-repo release script and multiple skills to enforce the policy, while ROADMAP.md also leaves implementation unchecked. Maintainers therefore cannot tell whether this is an active contract or a future proposal; update the status and roadmap entry to match the shipped behavior.
AGENTS.md reference: AGENTS.md:L62-L62
Useful? React with 👍 / 👎.
1 participant
Summary\n\n- replace release-script polling with bounded prepare, inspect, and advance transitions\n- require one host-native continuation at least five minutes later for remote gates\n- align Apple, deployment, Hermes, and upstream-overlay guidance\n\n## Verification\n\n- ============================= test session starts ==============================
platform darwin -- Python 3.14.3, pytest-9.0.3, pluggy-1.6.0
rootdir: /Users/galew/.codex/worktrees/af25/socket
configfile: pyproject.toml
testpaths: tests, plugins/agent-portability-skills/skills/bootstrap-skills-plugin-repo/tests, plugins/agent-portability-skills/skills/sync-skills-repo-guidance/tests, plugins/repository-skills/skills/maintain-project-repo/tests, plugins/model-lab-skills/skills/design-model-experiment/scripts, plugins/model-lab-skills/skills/evaluate-language-model/scripts, plugins/model-lab-skills/skills/compare-model-checkpoints/scripts
collected 138 items
tests/test_audit_skill_surfaces.py ....... [ 5%]
tests/test_cleanup_legacy_socket_installs.py ..... [ 8%]
tests/test_cybersecurity_skill_contracts.py .............. [ 18%]
tests/test_deployment_build_safety_contracts.py ... [ 21%]
tests/test_macos_platform_security_forward_scenarios.py ........ [ 26%]
tests/test_macos_virtualization_forward_scenarios.py .......... [ 34%]
tests/test_macos_virtualization_skill_contracts.py .. [ 35%]
tests/test_model_lab_skill_contracts.py ......... [ 42%]
tests/test_release_version.py ............. [ 51%]
tests/test_spi_add_package.py .......... [ 58%]
tests/test_swiftasb_skills_install.py .. [ 60%]
tests/test_validate_claude_compatibility.py ... [ 62%]
tests/test_validate_hermes_compatibility.py ....... [ 67%]
tests/test_validate_socket_metadata.py .................. [ 80%]
plugins/agent-portability-skills/skills/bootstrap-skills-plugin-repo/tests/test_bootstrap_skills_plugin_repo.py . [ 81%]
... [ 83%]
plugins/agent-portability-skills/skills/sync-skills-repo-guidance/tests/test_sync_skills_repo_guidance.py . [ 84%]
..... [ 87%]
plugins/repository-skills/skills/maintain-project-repo/tests/test_maintain_project_repo_workflow.py . [ 88%]
..............s. [100%]
======================== 137 passed, 1 skipped in 1.26s ======================== (137 passed, 1 skipped)\n- Socket Hermes compatibility validation passed.\n- Validating root marketplace presence...
Validating marketplace entries...
Socket marketplace validation passed.
Summary by CodeRabbit
New Features
Documentation
Tests