Rewrite skills README and harden release guidance

.agents/skills/release/SKILL.md

@@ -14,9 +14,10 @@ public side effects, not one.
 ## Establish the release state
 
 1. Confirm the repository root, clean working tree, current branch, and remotes.
-2. Resolve the canonical `OWNER/REPO` and its default branch with `gh repo view`.
-   Identify the local remotes for that repository and the contributor fork by
-   their URLs; do not assume particular remote names.
+2. Resolve the canonical `OWNER/REPO`, its default branch, and its permitted
+   merge methods with `gh repo view`. Identify the local remotes for that
+   repository and the contributor fork by their URLs; do not assume particular
+   remote names or merge settings.
 3. Inspect the release contract in:
    - `.github/workflows/pypi-release.yml`
    - `.github/workflows/docs.yml`
@@ -25,7 +26,10 @@ public side effects, not one.
    - `pyproject.toml`
    - `agentlightning/__init__.py`
 4. Confirm the canonical default branch is already green before branching from
-   it. A release branch inherits every failure that main is carrying.
+   it. Resolve its current commit with `gh api repos/OWNER/REPO/commits/BRANCH`
+   and inspect that commit's check runs; a general recent-run listing can omit
+   or mix commits. A release branch inherits every failure that main is
+   carrying.
 5. Query the canonical repository's tags and compare them with the versions
    published at `https://pypi.org/pypi/agentlightning/json`. Confirm the target
    version exists in neither place, and stop for an explicit release decision
@@ -52,6 +56,12 @@ git switch -c chore/release-vX.Y.Z <canonical-remote>/<default-branch>
 scripts/bump_version.sh patch  # or minor / major
 ```
 
+The bump updates `pyproject.toml` before uv finishes resolving and writing the
+lockfile. If resolution or network access fails, the command can exit after a
+partial bump. Inspect `git diff` after any failure and restore or reconcile all
+three version files before retrying; blindly rerunning a partial patch bump can
+advance the version twice.
+
 The bump rewrites exactly three files. Confirm that with `git diff --stat`:
 
 - `pyproject.toml`
@@ -63,10 +73,11 @@ Other version strings in the tree, such as the FastAPI `version` in
 alone; changing them is a separate pull request, not release work.
 
 Review the version diff, but do not run the release tests or package build
-locally as a matter of course. `tests.yml` runs the same test set and package
-build on the pull request that `pypi-release.yml` will run on the tag, so the
-pull request's GitHub checks are the verification gate. Reproduce a single
-failure locally only when the workflow logs are not enough to fix it.
+locally as a matter of course. `tests.yml` runs a broader test suite and the
+same package build on the pull request, covering the narrower tests and build
+that `pypi-release.yml` will run on the tag. The pull request's GitHub checks
+are therefore the verification gate. Reproduce a single failure locally only
+when the workflow logs are not enough to fix it.
 
 Commit the version change, push it to the fork, and open the pull request with
 the GitHub CLI when those external actions are authorized:
@@ -89,10 +100,18 @@ Follow the pull request through its required checks with
 `gh pr checks <pr> --repo OWNER/REPO --watch`. If a check fails, take the run id
 from that output, inspect it with
 `gh run view <run-id> --repo OWNER/REPO --log-failed`, correct the source on the
-same branch, and resume watching. Once every required check has succeeded, merge
-the pull request with `gh pr merge <pr> --repo OWNER/REPO` using a merge method
-the repository permits. Committing, pushing, opening the pull request, and
-merging are each distinct external actions and each requires authorization.
+same branch, and resume watching. Once every required check has succeeded,
+resolve the reviewed head with
+`gh pr view <pr> --repo OWNER/REPO --json headRefOid` and pass both an explicit
+permitted method and `--match-head-commit` to `gh pr merge`. The canonical
+repository currently permits only squash merges, for example:
+
+```bash
+gh pr merge <pr> --repo OWNER/REPO --squash --match-head-commit <head-sha>
+```
+
+Committing, pushing, opening the pull request, and merging are each distinct
+external actions and each requires authorization.
 
 ## Tag and publish the merged release
 
@@ -120,11 +139,11 @@ grep '^__version__' agentlightning/__init__.py
 ```
 
 The workflow itself reads the runtime value as
-`python -c 'from agentlightning import __version__; print(__version__)'`, after
-`uv sync` has installed the checkout. Locally that import can resolve to some
-other installed copy of the package instead of the tree being tagged, so read
-the file directly here; `agentlightning/__init__.py` assigns `__version__` as a
-single literal, so the two agree by construction.
+`python -c 'from agentlightning import __version__; print(__version__)'` from
+the repository root before its dependency-sync step, so Python resolves the
+checkout through the current working directory. Read the file directly for the
+local pre-tag check; `agentlightning/__init__.py` assigns `__version__` as a
+single literal, making that check independent of the active Python environment.
 
 GitHub reads workflow files as they exist **at the tagged commit**, not at the
 tip of the default branch. Confirm that the commit being tagged actually
@@ -155,13 +174,23 @@ One tag push starts two workflows, and both belong to the release:
   versioned documentation and repoints the public `stable` alias at this
   release.
 
-Follow both to a terminal result with `gh run list --repo OWNER/REPO` and
-`gh run watch <run-id> --repo OWNER/REPO --exit-status`. After `PyPI Release`
-succeeds, verify that PyPI exposes the exact version with both the expected
-wheel and source distribution. After `Deploy Documentation` succeeds, verify
-that the published site serves `X.Y.Z` and that `stable` resolves to it. A green
-PyPI job with a failed documentation job is a half-finished release: report both
-workflow URLs and both outcomes.
+Look up each run by workflow and tag rather than selecting from an unfiltered
+recent-run list:
+
+```bash
+gh run list --repo OWNER/REPO --workflow pypi-release.yml \
+  --branch vX.Y.Z --event push --limit 1
+gh run list --repo OWNER/REPO --workflow docs.yml \
+  --branch vX.Y.Z --event push --limit 1
+```
+
+Confirm both runs have the expected tag commit, then follow them to a terminal
+result with `gh run watch <run-id> --repo OWNER/REPO --exit-status`. After
+`PyPI Release` succeeds, verify that PyPI exposes the exact version with both
+the expected wheel and source distribution. After `Deploy Documentation`
+succeeds, verify that the published site serves `X.Y.Z` and that `stable`
+resolves to it. A green PyPI job with a failed documentation job is a
+half-finished release: report both workflow URLs and both outcomes.
 
 For a transient workflow failure, rerun only with authorization. For a source
 or workflow defect, do not move the public tag; prepare a corrective release

skills/README.md

@@ -1,12 +1,6 @@
-# Agent Skills
+## Agent Lightning Skill
 
-Skills in the [Agent Skills](https://agentskills.io) format (`<name>/SKILL.md`), installable into any compatible agent.
-
-## Agent Lightning
-
-Turns your coding agent into an **agent optimizer**: given an editable agent and a benchmark to hillclimb on, it improves the agent's accuracy, cost, and latency through focused, individually-measured edits — keeping only what moves the frontier. It was measured against a no-skill control under a fair, leakage-free protocol.
-
-You provide the environment; the skill does the optimizing. Before invoking it, have ready: a working copy of the agent (keep the original pristine), labeled examples, a frozen eval command, and an objective + budget.
+**Teaches your coding agent how to write better agent code.** Given an editable agent and a benchmark to hillclimb on, it improves the agent's accuracy, cost, and latency through edits of prompts, skills, tools, workflows, configurations and pre-/post-processings.
 
 ### Installation
 
@@ -18,13 +12,17 @@ gh skill install microsoft/agent-lightning agent-lightning --agent codex
 gh skill install microsoft/agent-lightning agent-lightning --agent github-copilot
 ```
 
-The `skills/agent-lightning/` directory is both the canonical Agent Skills package and the Claude Code plugin root, so both publication paths use the same `SKILL.md` without a copied or symlinked wrapper.
+The core files of the skill is in [`agent-lightning`][agent-lightning] directory.
+
+### How It Works
+
+A large portion of the skill's power comes from the model and the coding-agent harness itself. As a matter of fact, coding-agent harnesses (like Claude Code, Codex, GitHub Copilot) are already strong optimizers. They can improve an agent's performance simply by using the following prompt:
 
-### Results
+> I've got an agent in this workspace — and it's underperforming on our benchmark. Can you raise its benchmark score while keeping any increase in per-run cost minimal — buy score cheaply, and only pay more when it clearly earns its keep?
 
-**Main finding:** Coding-agent harnesses are already strong optimizers. The clearest opportunity is improving consistency while preserving their high average performance, rather than expecting large score gains.
+The improvement can be further boosted when the coding agent is armed with our skill, which makes the optimization more powerful and robust.
 
-SkillOpt and the other non-agentic results are taken from the [SkillOpt paper](https://github.com/microsoft/SkillOpt) (Table 1); our agentic rows use the same splits and average all optimizers, budgets, and replicates.
+We've challenged Claude Code, Codex, GitHub Copilot to optimize three poorly-written agents on three benchmarks. The model to to drive these agents being optimized are GPT-5.4-mini; The models that are used by the optimizer coding agents are Opus 4.8 for Claude Code, and GPT-5.6-Sol for Codex and GitHub Copilot respectively. We compared against other methods that are non-coding-agent-based (all other results are taken from the [SkillOpt paper](https://github.com/microsoft/SkillOpt)). The results are shown below.
 
 | Method | SpreadsheetBench accuracy (%) | OfficeQA correctness (%) | ALFWorld success (%) |
 | :--- | ---: | ---: | ---: |
@@ -35,24 +33,24 @@ SkillOpt and the other non-agentic results are taken from the [SkillOpt paper](h
 | TextGrad | 38.2 | 30.0 | 70.9 |
 | GEPA | 42.5 | 45.3 | 81.3 |
 | SkillOpt | 47.5 | 48.8 | 85.8 |
-| Agentic optimizer average, no skill | 62.9 | 54.1 | 88.6 |
-| **Agentic optimizer average, Agent Lightning** | **66.7** | **54.5** | **94.9** |
+| Coding Agent (Avg. of CC+Codex+GHCP) | 62.9 | 54.1 | 88.6 |
+| **Coding Agent (with Agent Lightning Skill)** | **66.7** | **54.5** | **94.9** |
 
-#### Performance versus overall cost
+### Performance Breakdowns
 
-Each benchmark includes the \$5, \$10, and \$25 nominal-budget groups with three runs per treatment cell. Every point averages the three held-out finale runs for one harness, treatment, and budget: the x-axis is average overall cost on a log scale, and the y-axis is average SpreadsheetBench accuracy, OfficeQA correctness, or ALFWorld success. Color and shape identify the optimizer; filled markers use Agent Lightning and hollow markers are no-skill controls. Budget is not encoded in the legend. Overall cost includes optimizer LLM calls, train/self-evaluation, and held-out finale deployment; it excludes the pristine-baseline evaluations.
+To further measure how the coding agent responds to a limited API budget, we control the API credit balance they can use during the optimization. Note that every API call, including those calls made by the coding agent itself, and those calls made by the agent being optimized, are billed into the credit. We experimented with three groups, each with \$5, \$10, and \$25 budget, and we performed three runs per group, per coding-agent harness.
 
-Claude Code uses Claude Opus 4.8; Codex and GitHub Copilot use GPT 5.6 Sol as their optimizer models.
+The results are shown below, every point on the chart averages the three held-out finale runs for one harness, treatment, and budget: the x-axis is average overall cost on a log scale, and the y-axis is average SpreadsheetBench accuracy, OfficeQA correctness, or ALFWorld success. Color and shape identify the optimizer; filled markers use Agent Lightning and hollow markers run without. Budget is not encoded in the legend. Overall cost includes optimizer LLM calls, train/self-evaluation, and held-out finale deployment; it excludes the pristine-baseline evaluations.
 
 ![SpreadsheetBench accuracy versus overall cost](assets/agent-lightning-spreadsheetbench-accuracy-overall-cost.svg)
 
 ![OfficeQA correctness versus overall cost](assets/agent-lightning-officeqa-correctness-overall-cost.svg)
 
 ![ALFWorld success versus overall cost](assets/agent-lightning-alfworld-success-overall-cost.svg)
 
-#### Performance versus finale cost
+As the coding agent can change anything in the agent code, it can sometimes change some hard-coded settings in the code being optimized (e.g., tweaking the reasoning effort, or using an more expensive model). It's valuable to see whether the performance improvements are actually bought with a more expensive API cost, which we call "finale evaluation cost".
 
-The selected-budget views use the groups with the strongest aggregate skill-over-control lift: \$5 for SpreadsheetBench and \$10 for OfficeQA and ALFWorld. Every harness/treatment point is one of three runs; the x-axis is that run's finale cost, and the y-axis is held-out SpreadsheetBench accuracy, OfficeQA correctness, or ALFWorld success. Finale cost measures LLM gateway spend, so an ALFWorld deterministic controller can have exactly \$0 finale cost while still executing and scoring real environment steps; coincident zero-cost ALFWorld results are offset slightly along the x-axis so each replicate remains visible. SpreadsheetBench and OfficeQA show their aggregate pristine-baseline results as single reference points. The dotted ALFWorld baseline is a score-only reference: the corrected records do not include baseline deployment cost, so assigning it an x-coordinate would invent data.
+We chose a slice from the pervious experiment, \$5 optimizer budget for SpreadsheetBench, and \$10 budget for OfficeQA and ALFWorld as the datasets are larger. Every point in the chart is an average of three runs; the x-axis is that run's finale cost, and the y-axis is held-out SpreadsheetBench accuracy, OfficeQA correctness, or ALFWorld success. As shown in the chart, the overall performance increases much more compared to the smaller increase in API cost.
 
 ![SpreadsheetBench accuracy versus finale cost](assets/agent-lightning-spreadsheetbench-accuracy-finale-cost.svg)