From ea5e0379947aec3df30b9cfa1849d4441ae2a11c Mon Sep 17 00:00:00 2001 From: Emmanuel Levijarvi Date: Sat, 29 Aug 2026 08:17:09 -0700 Subject: [PATCH 1/2] Make RELEASE.md describe the release we actually run Two places where the documented process contradicted the tooling, both of which have already cost us something. The version bump section told you to hand-write the `Version X.Y.Z (YYYY-MM-DD)` heading into CHANGELOG.rst and commit it, then run `make version-bump`. But bump_version.py inserts that heading itself and commits it. Following the document produces a duplicate heading and a redundant commit - which happened on 9.3.1 - and a duplicate heading also breaks extract_changelog.py, so the GitHub release notes would come out wrong. The section now says to write entries under `Unreleased` only, and lists what the script does, including the clean-tree and up-to-date checks that decide whether it runs at all. Sections 6-8 documented `make publish` / `twine upload` with TWINE_USERNAME and TWINE_PASSWORD. That path is obsolete: release.yml publishes through PyPI trusted publishing when a v* tag is pushed. A maintainer following the document after pushing a tag would attempt to upload a version PyPI had already accepted. Those sections are replaced by a description of what the workflow does, an explicit "do not publish by hand", and a note that no credentials need setting up. The Makefile targets stay as an emergency fallback and are labelled as such - in the docs and in `make release`'s own output, which was telling you to run `make publish` when it finished. Also adds the recovery for a release that fails after the tag is pushed, since the tag is the trigger and a red pre-release check leaves one behind. That is what 9.3.1 needed, and it was not written down. The CI section no longer suggests setting up CI that has existed for a while; it lists the jobs, including why `Test without CLI extras` is there. Claude-Session: https://claude.ai/code/session_01XVj9BYvuLj7Th3iFUVoeCn --- Makefile | 7 +- RELEASE.md | 256 +++++++++++++++++++++++++++++------------------------ 2 files changed, 143 insertions(+), 120 deletions(-) diff --git a/Makefile b/Makefile index 8c70a215..1546eb0e 100644 --- a/Makefile +++ b/Makefile @@ -53,10 +53,11 @@ build: clean ## Build distribution packages check-release: lint format-check test validate-version ## Run all checks before release (lint, format check, tests, version validation) @echo "✓ All checks passed! Ready for release." -release: check-release build ## Prepare and build a release (run checks, then build) +release: check-release build ## Run the release checks and build locally (does not publish) @echo "✓ Release build complete!" - @echo "To publish to TestPyPI: make publish-test" - @echo "To publish to PyPI: make publish" + @echo "This built locally and published nothing." + @echo "To release: make version-bump BUMP=patch, then push main and the tag." + @echo "Pushing the tag is what publishes to PyPI. See RELEASE.md." publish-test: ## Publish to TestPyPI python -m twine check dist/* diff --git a/RELEASE.md b/RELEASE.md index ee56bc81..8a7c056f 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -14,7 +14,8 @@ make install-dev ## Quick Release -For a full automated release check and build: +Despite the name, this does not release anything - it runs the checks and +builds locally so you can see the release is in a fit state: ```bash make release @@ -24,8 +25,12 @@ This will: 1. Run linting checks 2. Verify code formatting 3. Run all tests -4. Clean build artifacts -5. Build distribution packages +4. Validate the version configuration +5. Clean build artifacts +6. Build distribution packages + +Releasing is `make version-bump` followed by pushing the tag, which hands off +to CI. See [The Release Runs Itself](#5-the-release-runs-itself). ## Step-by-Step Release Process @@ -105,52 +110,63 @@ That field stores the PyScaffold tool version (4.6), not the package version. #### Version Bump Process -1. Update `CHANGELOG.rst` with changes for this release: +The bump script owns the version heading, the changelog commit and the tag. +Do not write any of them by hand - `scripts/extract_changelog.py` parses the +heading it produces to build the GitHub release notes, and a hand-written +duplicate silently breaks that. -```bash -# Get current date -date +"%Y-%m-%d" +1. Make sure `CHANGELOG.rst` describes the release under `Unreleased`. -# Edit CHANGELOG.rst and add a new section: -# Version X.Y.Z (YYYY-MM-DD) -# ========================== -``` + Add entries only - no `Version X.Y.Z` heading, no date. Normally each PR + has already added its own entry, so this is a review rather than a write: -2. Commit the changelog: + ```rst + Unreleased + ========== -```bash -git add CHANGELOG.rst -git commit -m "Update changelog for vX.Y.Z" -``` + Fixed + ----- + - **Something was broken.** It is no longer broken. + ``` -3. Use the version bump script to create a git tag: + If you edit anything here, commit it before continuing: the script + refuses to run on a dirty working directory. -```bash -# For a patch release (X.Y.Z -> X.Y.Z+1) -make version-bump BUMP=patch +2. Run the bump script: -# For a minor release (X.Y.Z -> X.Y+1.0) -make version-bump BUMP=minor + ```bash + # For a patch release (X.Y.Z -> X.Y.Z+1) + make version-bump BUMP=patch -# For a major release (X.Y.Z -> X+1.0.0) -make version-bump BUMP=major + # For a minor release (X.Y.Z -> X.Y+1.0) + make version-bump BUMP=minor -# Or specify an explicit version -make version-bump BUMP=3.1.5 -``` + # For a major release (X.Y.Z -> X+1.0.0) + make version-bump BUMP=major -The script will: -- Get the current version from git tags -- Calculate the new version -- Validate the version progression (prevents large jumps) -- Create a git tag (e.g., `v3.1.5`) -- Display next steps + # Or specify an explicit version + make version-bump BUMP=3.1.5 + ``` -4. Push the tag to trigger the release: + The script will: + - Refuse to run unless the working directory is clean and in sync with + the remote + - Get the current version from git tags + - Calculate the new version and validate the progression (prevents large + jumps) + - Insert `Version X.Y.Z (YYYY-MM-DD)` under `Unreleased` in `CHANGELOG.rst` + - Commit that as `Update changelog for vX.Y.Z` + - Create the annotated tag `vX.Y.Z` -```bash -git push origin vX.Y.Z -``` +3. Push the commit, then the tag: + + ```bash + git push origin main + git push origin vX.Y.Z + ``` + + Push the commit first. The tag triggers the release, and the release notes + come from the changelog at that commit. #### Manual Version Tagging (Not Recommended) @@ -161,64 +177,86 @@ git tag -a vX.Y.Z -m "Release version X.Y.Z" git push origin vX.Y.Z ``` -**Warning**: Manual tagging bypasses validation checks. Use the version bump script instead. - -### 5. Build Distribution +**Warning**: Manual tagging bypasses the clean-tree, up-to-date and version +progression checks, and does not touch the changelog - so the GitHub release +notes will be empty unless you added the heading yourself. Use the version +bump script instead. -Clean and build distribution packages: +### 5. The Release Runs Itself -```bash -make build -``` +Pushing a `v*` tag starts `.github/workflows/release.yml`, which is the only +thing that publishes. In order, it: -This creates: -- `dist/nwp500_python-X.Y.Z.tar.gz` (source distribution) -- `dist/nwp500_python-X.Y.Z-py3-none-any.whl` (wheel) +1. **Pre-release Checks** - `ruff check`, `ruff format --check`, and `pytest` + against an install of `.[testing]` +2. **Build and Publish to PyPI** - `python -m build`, `twine check`, then + `pypa/gh-action-pypi-publish` via PyPI trusted publishing (no API tokens + are stored anywhere) +3. **Create GitHub Release** - release notes extracted from the `Version + X.Y.Z` section of `CHANGELOG.rst` -### 6. Test the Build +Each job depends on the one before it, so a failing check publishes nothing. -Test the distribution on TestPyPI first: +Watch it: ```bash -make publish-test +gh run watch $(gh run list --workflow=release.yml --limit 1 --json databaseId --jq '.[0].databaseId') ``` -Or manually: +Then confirm the result: ```bash -python -m twine upload --repository testpypi dist/* +gh release view vX.Y.Z +pip index versions nwp500-python ``` -Test installation from TestPyPI: +#### Do Not Publish By Hand -```bash -pip install --index-url https://test.pypi.org/simple/ nwp500-python -``` +`make publish` and `make publish-test` still exist as an emergency fallback, +but a normal release must not use them. The workflow has already uploaded the +artifacts by the time you could, and PyPI refuses to accept a version twice - +so running them after a successful release fails, and running them before it +makes the workflow fail instead. Publishing is CI's job. -### 7. Publish to PyPI +#### If the Release Fails After You Pushed the Tag -Once verified on TestPyPI, publish to production PyPI: +The tag is the trigger, so a failing pre-release check leaves a tag on the +remote pointing at a commit that was never released. Nothing was published - +the publish job is gated behind the checks - so the version number is still +free. Fix the problem, then move the tag: ```bash -make publish +# after committing the fix and pushing main +git tag -d vX.Y.Z +git push origin :refs/tags/vX.Y.Z +git tag -a vX.Y.Z -m "Release version X.Y.Z" +git push origin vX.Y.Z ``` -Or manually: +Confirm first that nothing was published - if the publish job did succeed, +that version is gone for good and you need a new patch version instead: ```bash -python -m twine upload dist/* +gh run view --json jobs --jq '.jobs[] | "\(.name): \(.conclusion)"' +pip index versions nwp500-python ``` -### 8. Tag the Release +### 6. Building Locally (Optional) -**Note**: If you used the version bump script, the tag is already created. -Just push it: +Building locally is a sanity check, not part of publishing: ```bash -git push origin vX.Y.Z +make build ``` -If you created a tag manually, push it now. +This creates: +- `dist/nwp500_python-X.Y.Z.tar.gz` (source distribution) +- `dist/nwp500_python-X.Y.Z-py3-none-any.whl` (wheel) + +The version comes from `git describe`, so building on an untagged commit +produces a development version like `9.3.1.dev3+g1a2b3c4` - that is expected, +and it is a quick way to confirm the tag you just created is what +`setuptools_scm` sees. ## Using Tox @@ -301,72 +339,56 @@ If build fails: Before releasing, ensure: +- [ ] Everything intended for the release is merged to `main` +- [ ] `CHANGELOG.rst` describes it all under `Unreleased`, with no + hand-written `Version X.Y.Z` heading - [ ] All code is formatted: `make format` - [ ] Linting passes: `make lint` - [ ] All tests pass: `make test` - [ ] Version configuration is valid: `make validate-version` -- [ ] Changelog is updated -- [ ] Version is bumped appropriately using `make version-bump` - [ ] Documentation is up to date - [ ] Examples work correctly +- [ ] Working directory is clean and `main` is in sync with the remote + (the bump script enforces both) - [ ] Build succeeds: `make build` -- [ ] TestPyPI upload works: `make publish-test` -## Environment Variables for Publishing +Then bump, push, and let the workflow publish - see +[The Release Runs Itself](#5-the-release-runs-itself). -Set these environment variables for Twine: +## Publishing Credentials -```bash -export TWINE_USERNAME=__token__ -export TWINE_PASSWORD=pypi-your-api-token-here -``` - -Or use a `.pypirc` file: - -```ini -[distutils] -index-servers = - pypi - testpypi +There are none to set up. The release workflow publishes through +[PyPI trusted publishing](https://docs.pypi.org/trusted-publishers/): PyPI is +configured to trust `release.yml` in this repository, and the job mints a +short-lived token through the `pypi` GitHub environment using its +`id-token: write` permission. No API token is stored in the repository, in +GitHub secrets, or on your machine. -[pypi] -username = __token__ -password = pypi-your-api-token - -[testpypi] -repository = https://test.pypi.org/legacy/ -username = __token__ -password = pypi-your-test-api-token -``` +`TWINE_USERNAME`/`TWINE_PASSWORD` and `~/.pypirc` are only relevant if you +ever need the emergency `make publish` fallback, which a normal release must +not use. ## Continuous Integration -Consider setting up CI/CD to automatically: +Two workflows already cover this: -1. Run linting on pull requests -2. Run tests on multiple Python versions -3. Check code formatting -4. Build and verify distributions -5. Publish releases automatically on git tags +**`.github/workflows/ci.yml`** - on pushes and pull requests: -Example GitHub Actions workflow could run: +| Job | What it does | +|-----|--------------| +| Lint and Format Check | `tox -e lint` (`ruff check` and `ruff format --check`) | +| Security Check | `ruff check --select S src/` | +| Test on Python 3.14 | `tox -e default` - pytest and pyright, with the `cli` and `testing` extras | +| Test without CLI extras | pytest against a `.[testing]`-only install, matching what `release.yml` does | +| Build Distribution | `python -m build` plus `twine check` | -```yaml -- name: Install dependencies - run: pip install -e ".[dev]" +**`.github/workflows/release.yml`** - on a `v*` tag: pre-release checks, then +build and publish to PyPI, then create the GitHub release. -- name: Lint with ruff - run: make lint - -- name: Check formatting - run: make format-check - -- name: Run tests - run: make test - -- name: Build - run: make build -``` +`Test without CLI extras` exists because `tox` installs the `cli` extra while +`release.yml` does not. Without it, a test module that imports `click` outside +its `ImportError` guard passes every pull request and fails only once the +release tag has been pushed. ## Quick Commands Reference @@ -380,10 +402,10 @@ Example GitHub Actions workflow could run: | `make test` | Run tests | | `make check-release` | Run all pre-release checks | | `make release` | Full release build process | -| `make build` | Build distribution packages | -| `make publish-test` | Upload to TestPyPI | -| `make publish` | Upload to PyPI | +| `make build` | Build distribution packages locally (CI builds what is published) | | `make clean` | Remove build artifacts | +| `make publish-test` | Emergency fallback only - upload to TestPyPI by hand | +| `make publish` | Emergency fallback only - upload to PyPI by hand | ## More Information From f7aacf52c4c77825be16a1146d3b8722f154bc13 Mon Sep 17 00:00:00 2001 From: Emmanuel Levijarvi Date: Tue, 1 Sep 2026 06:28:41 -0700 Subject: [PATCH 2/2] Correct the bump-script details RELEASE.md got wrong The script refuses to run when the branch is behind upstream; being ahead or having no upstream is only a warning, so "in sync with the remote" overstated it. A manual tag does not produce empty release notes either - extract_changelog.py falls back to "Release X.Y.Z". The script's own closing output still pointed at make publish-test / make publish, contradicting the document it is meant to match; it now points at the tag flow. Claude-Session: https://claude.ai/code/session_01YDSWT2RqH9T5r7Q4ZUsaEK --- RELEASE.md | 19 +++++++++++-------- scripts/bump_version.py | 9 ++++----- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index 8a7c056f..6664e1f2 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -149,8 +149,9 @@ duplicate silently breaks that. ``` The script will: - - Refuse to run unless the working directory is clean and in sync with - the remote + - Refuse to run unless the working directory is clean + - Refuse to run if the branch is behind its upstream (it warns, but + continues, if the branch is ahead or has no upstream) - Get the current version from git tags - Calculate the new version and validate the progression (prevents large jumps) @@ -177,10 +178,12 @@ git tag -a vX.Y.Z -m "Release version X.Y.Z" git push origin vX.Y.Z ``` -**Warning**: Manual tagging bypasses the clean-tree, up-to-date and version -progression checks, and does not touch the changelog - so the GitHub release -notes will be empty unless you added the heading yourself. Use the version -bump script instead. +**Warning**: Manual tagging bypasses the clean-tree, not-behind-upstream and +version progression checks, and does not touch the changelog - so unless you +added the `Version X.Y.Z (YYYY-MM-DD)` heading yourself, +`scripts/extract_changelog.py` finds no matching section and the GitHub release +notes fall back to the bare line `Release X.Y.Z`. Use the version bump script +instead. ### 5. The Release Runs Itself @@ -188,7 +191,7 @@ Pushing a `v*` tag starts `.github/workflows/release.yml`, which is the only thing that publishes. In order, it: 1. **Pre-release Checks** - `ruff check`, `ruff format --check`, and `pytest` - against an install of `.[testing]` + against an install of `-e ".[testing]"` 2. **Build and Publish to PyPI** - `python -m build`, `twine check`, then `pypa/gh-action-pypi-publish` via PyPI trusted publishing (no API tokens are stored anywhere) @@ -379,7 +382,7 @@ Two workflows already cover this: | Lint and Format Check | `tox -e lint` (`ruff check` and `ruff format --check`) | | Security Check | `ruff check --select S src/` | | Test on Python 3.14 | `tox -e default` - pytest and pyright, with the `cli` and `testing` extras | -| Test without CLI extras | pytest against a `.[testing]`-only install, matching what `release.yml` does | +| Test without CLI extras | pytest against an `-e ".[testing]"` install, matching what `release.yml` does | | Build Distribution | `python -m build` plus `twine check` | **`.github/workflows/release.yml`** - on a `v*` tag: pre-release checks, then diff --git a/scripts/bump_version.py b/scripts/bump_version.py index ec0996b8..42f4671b 100755 --- a/scripts/bump_version.py +++ b/scripts/bump_version.py @@ -375,11 +375,10 @@ def main() -> None: print("\n[OK] Version bump complete!") print("\nNext steps:") - print(f" 1. Push the tag: git push origin v{new_version}") - print(" (also push the changelog commit: git push origin HEAD)") - print(" 2. Build release: make build") - print(" 3. Test on TestPyPI: make publish-test") - print(" 4. Publish to PyPI: make publish") + print(" 1. Push the changelog commit: git push origin HEAD") + print(f" 2. Push the tag: git push origin v{new_version}") + print("\nThe tag triggers .github/workflows/release.yml, which runs the") + print("pre-release checks and publishes to PyPI. Do not publish by hand.") if __name__ == "__main__":