diff --git a/Makefile b/Makefile index 8c70a21..1546eb0 100644 --- a/Makefile +++ b/Makefile @@ -53,10 +53,11 @@ build: clean ## Build distribution packages check-release: lint format-check test validate-version ## Run all checks before release (lint, format check, tests, version validation) @echo "✓ All checks passed! Ready for release." -release: check-release build ## Prepare and build a release (run checks, then build) +release: check-release build ## Run the release checks and build locally (does not publish) @echo "✓ Release build complete!" - @echo "To publish to TestPyPI: make publish-test" - @echo "To publish to PyPI: make publish" + @echo "This built locally and published nothing." + @echo "To release: make version-bump BUMP=patch, then push main and the tag." + @echo "Pushing the tag is what publishes to PyPI. See RELEASE.md." publish-test: ## Publish to TestPyPI python -m twine check dist/* diff --git a/RELEASE.md b/RELEASE.md index ee56bc8..6664e1f 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -14,7 +14,8 @@ make install-dev ## Quick Release -For a full automated release check and build: +Despite the name, this does not release anything - it runs the checks and +builds locally so you can see the release is in a fit state: ```bash make release @@ -24,8 +25,12 @@ This will: 1. Run linting checks 2. Verify code formatting 3. Run all tests -4. Clean build artifacts -5. Build distribution packages +4. Validate the version configuration +5. Clean build artifacts +6. Build distribution packages + +Releasing is `make version-bump` followed by pushing the tag, which hands off +to CI. See [The Release Runs Itself](#5-the-release-runs-itself). ## Step-by-Step Release Process @@ -105,52 +110,64 @@ That field stores the PyScaffold tool version (4.6), not the package version. #### Version Bump Process -1. Update `CHANGELOG.rst` with changes for this release: +The bump script owns the version heading, the changelog commit and the tag. +Do not write any of them by hand - `scripts/extract_changelog.py` parses the +heading it produces to build the GitHub release notes, and a hand-written +duplicate silently breaks that. -```bash -# Get current date -date +"%Y-%m-%d" +1. Make sure `CHANGELOG.rst` describes the release under `Unreleased`. -# Edit CHANGELOG.rst and add a new section: -# Version X.Y.Z (YYYY-MM-DD) -# ========================== -``` + Add entries only - no `Version X.Y.Z` heading, no date. Normally each PR + has already added its own entry, so this is a review rather than a write: -2. Commit the changelog: + ```rst + Unreleased + ========== -```bash -git add CHANGELOG.rst -git commit -m "Update changelog for vX.Y.Z" -``` + Fixed + ----- + - **Something was broken.** It is no longer broken. + ``` -3. Use the version bump script to create a git tag: + If you edit anything here, commit it before continuing: the script + refuses to run on a dirty working directory. -```bash -# For a patch release (X.Y.Z -> X.Y.Z+1) -make version-bump BUMP=patch +2. Run the bump script: -# For a minor release (X.Y.Z -> X.Y+1.0) -make version-bump BUMP=minor + ```bash + # For a patch release (X.Y.Z -> X.Y.Z+1) + make version-bump BUMP=patch -# For a major release (X.Y.Z -> X+1.0.0) -make version-bump BUMP=major + # For a minor release (X.Y.Z -> X.Y+1.0) + make version-bump BUMP=minor -# Or specify an explicit version -make version-bump BUMP=3.1.5 -``` + # For a major release (X.Y.Z -> X+1.0.0) + make version-bump BUMP=major -The script will: -- Get the current version from git tags -- Calculate the new version -- Validate the version progression (prevents large jumps) -- Create a git tag (e.g., `v3.1.5`) -- Display next steps + # Or specify an explicit version + make version-bump BUMP=3.1.5 + ``` -4. Push the tag to trigger the release: + The script will: + - Refuse to run unless the working directory is clean + - Refuse to run if the branch is behind its upstream (it warns, but + continues, if the branch is ahead or has no upstream) + - Get the current version from git tags + - Calculate the new version and validate the progression (prevents large + jumps) + - Insert `Version X.Y.Z (YYYY-MM-DD)` under `Unreleased` in `CHANGELOG.rst` + - Commit that as `Update changelog for vX.Y.Z` + - Create the annotated tag `vX.Y.Z` -```bash -git push origin vX.Y.Z -``` +3. Push the commit, then the tag: + + ```bash + git push origin main + git push origin vX.Y.Z + ``` + + Push the commit first. The tag triggers the release, and the release notes + come from the changelog at that commit. #### Manual Version Tagging (Not Recommended) @@ -161,64 +178,88 @@ git tag -a vX.Y.Z -m "Release version X.Y.Z" git push origin vX.Y.Z ``` -**Warning**: Manual tagging bypasses validation checks. Use the version bump script instead. - -### 5. Build Distribution +**Warning**: Manual tagging bypasses the clean-tree, not-behind-upstream and +version progression checks, and does not touch the changelog - so unless you +added the `Version X.Y.Z (YYYY-MM-DD)` heading yourself, +`scripts/extract_changelog.py` finds no matching section and the GitHub release +notes fall back to the bare line `Release X.Y.Z`. Use the version bump script +instead. -Clean and build distribution packages: +### 5. The Release Runs Itself -```bash -make build -``` +Pushing a `v*` tag starts `.github/workflows/release.yml`, which is the only +thing that publishes. In order, it: -This creates: -- `dist/nwp500_python-X.Y.Z.tar.gz` (source distribution) -- `dist/nwp500_python-X.Y.Z-py3-none-any.whl` (wheel) +1. **Pre-release Checks** - `ruff check`, `ruff format --check`, and `pytest` + against an install of `-e ".[testing]"` +2. **Build and Publish to PyPI** - `python -m build`, `twine check`, then + `pypa/gh-action-pypi-publish` via PyPI trusted publishing (no API tokens + are stored anywhere) +3. **Create GitHub Release** - release notes extracted from the `Version + X.Y.Z` section of `CHANGELOG.rst` -### 6. Test the Build +Each job depends on the one before it, so a failing check publishes nothing. -Test the distribution on TestPyPI first: +Watch it: ```bash -make publish-test +gh run watch $(gh run list --workflow=release.yml --limit 1 --json databaseId --jq '.[0].databaseId') ``` -Or manually: +Then confirm the result: ```bash -python -m twine upload --repository testpypi dist/* +gh release view vX.Y.Z +pip index versions nwp500-python ``` -Test installation from TestPyPI: +#### Do Not Publish By Hand -```bash -pip install --index-url https://test.pypi.org/simple/ nwp500-python -``` +`make publish` and `make publish-test` still exist as an emergency fallback, +but a normal release must not use them. The workflow has already uploaded the +artifacts by the time you could, and PyPI refuses to accept a version twice - +so running them after a successful release fails, and running them before it +makes the workflow fail instead. Publishing is CI's job. -### 7. Publish to PyPI +#### If the Release Fails After You Pushed the Tag -Once verified on TestPyPI, publish to production PyPI: +The tag is the trigger, so a failing pre-release check leaves a tag on the +remote pointing at a commit that was never released. Nothing was published - +the publish job is gated behind the checks - so the version number is still +free. Fix the problem, then move the tag: ```bash -make publish +# after committing the fix and pushing main +git tag -d vX.Y.Z +git push origin :refs/tags/vX.Y.Z +git tag -a vX.Y.Z -m "Release version X.Y.Z" +git push origin vX.Y.Z ``` -Or manually: +Confirm first that nothing was published - if the publish job did succeed, +that version is gone for good and you need a new patch version instead: ```bash -python -m twine upload dist/* +gh run view --json jobs --jq '.jobs[] | "\(.name): \(.conclusion)"' +pip index versions nwp500-python ``` -### 8. Tag the Release +### 6. Building Locally (Optional) -**Note**: If you used the version bump script, the tag is already created. -Just push it: +Building locally is a sanity check, not part of publishing: ```bash -git push origin vX.Y.Z +make build ``` -If you created a tag manually, push it now. +This creates: +- `dist/nwp500_python-X.Y.Z.tar.gz` (source distribution) +- `dist/nwp500_python-X.Y.Z-py3-none-any.whl` (wheel) + +The version comes from `git describe`, so building on an untagged commit +produces a development version like `9.3.1.dev3+g1a2b3c4` - that is expected, +and it is a quick way to confirm the tag you just created is what +`setuptools_scm` sees. ## Using Tox @@ -301,72 +342,56 @@ If build fails: Before releasing, ensure: +- [ ] Everything intended for the release is merged to `main` +- [ ] `CHANGELOG.rst` describes it all under `Unreleased`, with no + hand-written `Version X.Y.Z` heading - [ ] All code is formatted: `make format` - [ ] Linting passes: `make lint` - [ ] All tests pass: `make test` - [ ] Version configuration is valid: `make validate-version` -- [ ] Changelog is updated -- [ ] Version is bumped appropriately using `make version-bump` - [ ] Documentation is up to date - [ ] Examples work correctly +- [ ] Working directory is clean and `main` is in sync with the remote + (the bump script enforces both) - [ ] Build succeeds: `make build` -- [ ] TestPyPI upload works: `make publish-test` -## Environment Variables for Publishing +Then bump, push, and let the workflow publish - see +[The Release Runs Itself](#5-the-release-runs-itself). -Set these environment variables for Twine: +## Publishing Credentials -```bash -export TWINE_USERNAME=__token__ -export TWINE_PASSWORD=pypi-your-api-token-here -``` - -Or use a `.pypirc` file: - -```ini -[distutils] -index-servers = - pypi - testpypi +There are none to set up. The release workflow publishes through +[PyPI trusted publishing](https://docs.pypi.org/trusted-publishers/): PyPI is +configured to trust `release.yml` in this repository, and the job mints a +short-lived token through the `pypi` GitHub environment using its +`id-token: write` permission. No API token is stored in the repository, in +GitHub secrets, or on your machine. -[pypi] -username = __token__ -password = pypi-your-api-token - -[testpypi] -repository = https://test.pypi.org/legacy/ -username = __token__ -password = pypi-your-test-api-token -``` +`TWINE_USERNAME`/`TWINE_PASSWORD` and `~/.pypirc` are only relevant if you +ever need the emergency `make publish` fallback, which a normal release must +not use. ## Continuous Integration -Consider setting up CI/CD to automatically: +Two workflows already cover this: -1. Run linting on pull requests -2. Run tests on multiple Python versions -3. Check code formatting -4. Build and verify distributions -5. Publish releases automatically on git tags +**`.github/workflows/ci.yml`** - on pushes and pull requests: -Example GitHub Actions workflow could run: +| Job | What it does | +|-----|--------------| +| Lint and Format Check | `tox -e lint` (`ruff check` and `ruff format --check`) | +| Security Check | `ruff check --select S src/` | +| Test on Python 3.14 | `tox -e default` - pytest and pyright, with the `cli` and `testing` extras | +| Test without CLI extras | pytest against an `-e ".[testing]"` install, matching what `release.yml` does | +| Build Distribution | `python -m build` plus `twine check` | -```yaml -- name: Install dependencies - run: pip install -e ".[dev]" +**`.github/workflows/release.yml`** - on a `v*` tag: pre-release checks, then +build and publish to PyPI, then create the GitHub release. -- name: Lint with ruff - run: make lint - -- name: Check formatting - run: make format-check - -- name: Run tests - run: make test - -- name: Build - run: make build -``` +`Test without CLI extras` exists because `tox` installs the `cli` extra while +`release.yml` does not. Without it, a test module that imports `click` outside +its `ImportError` guard passes every pull request and fails only once the +release tag has been pushed. ## Quick Commands Reference @@ -380,10 +405,10 @@ Example GitHub Actions workflow could run: | `make test` | Run tests | | `make check-release` | Run all pre-release checks | | `make release` | Full release build process | -| `make build` | Build distribution packages | -| `make publish-test` | Upload to TestPyPI | -| `make publish` | Upload to PyPI | +| `make build` | Build distribution packages locally (CI builds what is published) | | `make clean` | Remove build artifacts | +| `make publish-test` | Emergency fallback only - upload to TestPyPI by hand | +| `make publish` | Emergency fallback only - upload to PyPI by hand | ## More Information diff --git a/scripts/bump_version.py b/scripts/bump_version.py index ec0996b..42f4671 100755 --- a/scripts/bump_version.py +++ b/scripts/bump_version.py @@ -375,11 +375,10 @@ def main() -> None: print("\n[OK] Version bump complete!") print("\nNext steps:") - print(f" 1. Push the tag: git push origin v{new_version}") - print(" (also push the changelog commit: git push origin HEAD)") - print(" 2. Build release: make build") - print(" 3. Test on TestPyPI: make publish-test") - print(" 4. Publish to PyPI: make publish") + print(" 1. Push the changelog commit: git push origin HEAD") + print(f" 2. Push the tag: git push origin v{new_version}") + print("\nThe tag triggers .github/workflows/release.yml, which runs the") + print("pre-release checks and publishes to PyPI. Do not publish by hand.") if __name__ == "__main__":