Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,11 @@ build: clean ## Build distribution packages
check-release: lint format-check test validate-version ## Run all checks before release (lint, format check, tests, version validation)
@echo "✓ All checks passed! Ready for release."

release: check-release build ## Prepare and build a release (run checks, then build)
release: check-release build ## Run the release checks and build locally (does not publish)
@echo "✓ Release build complete!"
@echo "To publish to TestPyPI: make publish-test"
@echo "To publish to PyPI: make publish"
@echo "This built locally and published nothing."
@echo "To release: make version-bump BUMP=patch, then push main and the tag."
@echo "Pushing the tag is what publishes to PyPI. See RELEASE.md."

publish-test: ## Publish to TestPyPI
python -m twine check dist/*
Expand Down
259 changes: 142 additions & 117 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ make install-dev

## Quick Release

For a full automated release check and build:
Despite the name, this does not release anything - it runs the checks and
builds locally so you can see the release is in a fit state:

```bash
make release
Expand All @@ -24,8 +25,12 @@ This will:
1. Run linting checks
2. Verify code formatting
3. Run all tests
4. Clean build artifacts
5. Build distribution packages
4. Validate the version configuration
5. Clean build artifacts
6. Build distribution packages

Releasing is `make version-bump` followed by pushing the tag, which hands off
to CI. See [The Release Runs Itself](#5-the-release-runs-itself).

## Step-by-Step Release Process

Expand Down Expand Up @@ -105,52 +110,64 @@ That field stores the PyScaffold tool version (4.6), not the package version.

#### Version Bump Process

1. Update `CHANGELOG.rst` with changes for this release:
The bump script owns the version heading, the changelog commit and the tag.
Do not write any of them by hand - `scripts/extract_changelog.py` parses the
heading it produces to build the GitHub release notes, and a hand-written
duplicate silently breaks that.

```bash
# Get current date
date +"%Y-%m-%d"
1. Make sure `CHANGELOG.rst` describes the release under `Unreleased`.

# Edit CHANGELOG.rst and add a new section:
# Version X.Y.Z (YYYY-MM-DD)
# ==========================
```
Add entries only - no `Version X.Y.Z` heading, no date. Normally each PR
has already added its own entry, so this is a review rather than a write:

2. Commit the changelog:
```rst
Unreleased
==========

```bash
git add CHANGELOG.rst
git commit -m "Update changelog for vX.Y.Z"
```
Fixed
-----
- **Something was broken.** It is no longer broken.
```

3. Use the version bump script to create a git tag:
If you edit anything here, commit it before continuing: the script
refuses to run on a dirty working directory.

```bash
# For a patch release (X.Y.Z -> X.Y.Z+1)
make version-bump BUMP=patch
2. Run the bump script:

# For a minor release (X.Y.Z -> X.Y+1.0)
make version-bump BUMP=minor
```bash
# For a patch release (X.Y.Z -> X.Y.Z+1)
make version-bump BUMP=patch

# For a major release (X.Y.Z -> X+1.0.0)
make version-bump BUMP=major
# For a minor release (X.Y.Z -> X.Y+1.0)
make version-bump BUMP=minor

# Or specify an explicit version
make version-bump BUMP=3.1.5
```
# For a major release (X.Y.Z -> X+1.0.0)
make version-bump BUMP=major

The script will:
- Get the current version from git tags
- Calculate the new version
- Validate the version progression (prevents large jumps)
- Create a git tag (e.g., `v3.1.5`)
- Display next steps
# Or specify an explicit version
make version-bump BUMP=3.1.5
```

4. Push the tag to trigger the release:
The script will:
- Refuse to run unless the working directory is clean
- Refuse to run if the branch is behind its upstream (it warns, but
continues, if the branch is ahead or has no upstream)
- Get the current version from git tags
- Calculate the new version and validate the progression (prevents large
jumps)
- Insert `Version X.Y.Z (YYYY-MM-DD)` under `Unreleased` in `CHANGELOG.rst`
- Commit that as `Update changelog for vX.Y.Z`
- Create the annotated tag `vX.Y.Z`

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f7aacf5, at the source: bump_version.py no longer prints the make publish-test / make publish next steps. It now says to push the commit and the tag, and that the tag triggers release.yml — so there is nothing left for the document to warn about.

```bash
git push origin vX.Y.Z
```
3. Push the commit, then the tag:

```bash
git push origin main
git push origin vX.Y.Z
```

Push the commit first. The tag triggers the release, and the release notes
come from the changelog at that commit.

#### Manual Version Tagging (Not Recommended)

Expand All @@ -161,64 +178,88 @@ git tag -a vX.Y.Z -m "Release version X.Y.Z"
git push origin vX.Y.Z
```

**Warning**: Manual tagging bypasses validation checks. Use the version bump script instead.

### 5. Build Distribution
**Warning**: Manual tagging bypasses the clean-tree, not-behind-upstream and
version progression checks, and does not touch the changelog - so unless you
added the `Version X.Y.Z (YYYY-MM-DD)` heading yourself,
`scripts/extract_changelog.py` finds no matching section and the GitHub release
notes fall back to the bare line `Release X.Y.Z`. Use the version bump script
instead.

Clean and build distribution packages:
### 5. The Release Runs Itself

```bash
make build
```
Pushing a `v*` tag starts `.github/workflows/release.yml`, which is the only
thing that publishes. In order, it:

This creates:
- `dist/nwp500_python-X.Y.Z.tar.gz` (source distribution)
- `dist/nwp500_python-X.Y.Z-py3-none-any.whl` (wheel)
1. **Pre-release Checks** - `ruff check`, `ruff format --check`, and `pytest`
against an install of `-e ".[testing]"`
2. **Build and Publish to PyPI** - `python -m build`, `twine check`, then
`pypa/gh-action-pypi-publish` via PyPI trusted publishing (no API tokens
are stored anywhere)
3. **Create GitHub Release** - release notes extracted from the `Version
X.Y.Z` section of `CHANGELOG.rst`

### 6. Test the Build
Each job depends on the one before it, so a failing check publishes nothing.

Test the distribution on TestPyPI first:
Watch it:

```bash
make publish-test
gh run watch $(gh run list --workflow=release.yml --limit 1 --json databaseId --jq '.[0].databaseId')
```

Or manually:
Then confirm the result:

```bash
python -m twine upload --repository testpypi dist/*
gh release view vX.Y.Z
pip index versions nwp500-python
```

Test installation from TestPyPI:
#### Do Not Publish By Hand

```bash
pip install --index-url https://test.pypi.org/simple/ nwp500-python
```
`make publish` and `make publish-test` still exist as an emergency fallback,
but a normal release must not use them. The workflow has already uploaded the
artifacts by the time you could, and PyPI refuses to accept a version twice -
so running them after a successful release fails, and running them before it
makes the workflow fail instead. Publishing is CI's job.

### 7. Publish to PyPI
#### If the Release Fails After You Pushed the Tag

Once verified on TestPyPI, publish to production PyPI:
The tag is the trigger, so a failing pre-release check leaves a tag on the
remote pointing at a commit that was never released. Nothing was published -
the publish job is gated behind the checks - so the version number is still
free. Fix the problem, then move the tag:

```bash
make publish
# after committing the fix and pushing main
git tag -d vX.Y.Z
git push origin :refs/tags/vX.Y.Z
git tag -a vX.Y.Z -m "Release version X.Y.Z"
git push origin vX.Y.Z
```

Or manually:
Confirm first that nothing was published - if the publish job did succeed,
that version is gone for good and you need a new patch version instead:

```bash
python -m twine upload dist/*
gh run view <run-id> --json jobs --jq '.jobs[] | "\(.name): \(.conclusion)"'
pip index versions nwp500-python
```

### 8. Tag the Release
### 6. Building Locally (Optional)

**Note**: If you used the version bump script, the tag is already created.
Just push it:
Building locally is a sanity check, not part of publishing:

```bash
git push origin vX.Y.Z
make build
```

If you created a tag manually, push it now.
This creates:
- `dist/nwp500_python-X.Y.Z.tar.gz` (source distribution)
- `dist/nwp500_python-X.Y.Z-py3-none-any.whl` (wheel)

The version comes from `git describe`, so building on an untagged commit
produces a development version like `9.3.1.dev3+g1a2b3c4` - that is expected,
and it is a quick way to confirm the tag you just created is what
`setuptools_scm` sees.

## Using Tox

Expand Down Expand Up @@ -301,72 +342,56 @@ If build fails:

Before releasing, ensure:

- [ ] Everything intended for the release is merged to `main`
- [ ] `CHANGELOG.rst` describes it all under `Unreleased`, with no
hand-written `Version X.Y.Z` heading
- [ ] All code is formatted: `make format`
- [ ] Linting passes: `make lint`
- [ ] All tests pass: `make test`
- [ ] Version configuration is valid: `make validate-version`
- [ ] Changelog is updated
- [ ] Version is bumped appropriately using `make version-bump`
- [ ] Documentation is up to date
- [ ] Examples work correctly
- [ ] Working directory is clean and `main` is in sync with the remote
(the bump script enforces both)
- [ ] Build succeeds: `make build`
- [ ] TestPyPI upload works: `make publish-test`

## Environment Variables for Publishing
Then bump, push, and let the workflow publish - see
[The Release Runs Itself](#5-the-release-runs-itself).

Set these environment variables for Twine:
## Publishing Credentials

```bash
export TWINE_USERNAME=__token__
export TWINE_PASSWORD=pypi-your-api-token-here
```

Or use a `.pypirc` file:

```ini
[distutils]
index-servers =
pypi
testpypi
There are none to set up. The release workflow publishes through
[PyPI trusted publishing](https://docs.pypi.org/trusted-publishers/): PyPI is
configured to trust `release.yml` in this repository, and the job mints a
short-lived token through the `pypi` GitHub environment using its
`id-token: write` permission. No API token is stored in the repository, in
GitHub secrets, or on your machine.

[pypi]
username = __token__
password = pypi-your-api-token

[testpypi]
repository = https://test.pypi.org/legacy/
username = __token__
password = pypi-your-test-api-token
```
`TWINE_USERNAME`/`TWINE_PASSWORD` and `~/.pypirc` are only relevant if you
ever need the emergency `make publish` fallback, which a normal release must
not use.

## Continuous Integration

Consider setting up CI/CD to automatically:
Two workflows already cover this:

1. Run linting on pull requests
2. Run tests on multiple Python versions
3. Check code formatting
4. Build and verify distributions
5. Publish releases automatically on git tags
**`.github/workflows/ci.yml`** - on pushes and pull requests:

Example GitHub Actions workflow could run:
| Job | What it does |
|-----|--------------|
| Lint and Format Check | `tox -e lint` (`ruff check` and `ruff format --check`) |
| Security Check | `ruff check --select S src/` |
| Test on Python 3.14 | `tox -e default` - pytest and pyright, with the `cli` and `testing` extras |
| Test without CLI extras | pytest against an `-e ".[testing]"` install, matching what `release.yml` does |
| Build Distribution | `python -m build` plus `twine check` |

```yaml
- name: Install dependencies
run: pip install -e ".[dev]"
**`.github/workflows/release.yml`** - on a `v*` tag: pre-release checks, then
build and publish to PyPI, then create the GitHub release.

- name: Lint with ruff
run: make lint

- name: Check formatting
run: make format-check

- name: Run tests
run: make test

- name: Build
run: make build
```
`Test without CLI extras` exists because `tox` installs the `cli` extra while
`release.yml` does not. Without it, a test module that imports `click` outside
its `ImportError` guard passes every pull request and fails only once the
release tag has been pushed.

## Quick Commands Reference

Expand All @@ -380,10 +405,10 @@ Example GitHub Actions workflow could run:
| `make test` | Run tests |
| `make check-release` | Run all pre-release checks |
| `make release` | Full release build process |
| `make build` | Build distribution packages |
| `make publish-test` | Upload to TestPyPI |
| `make publish` | Upload to PyPI |
| `make build` | Build distribution packages locally (CI builds what is published) |
| `make clean` | Remove build artifacts |
| `make publish-test` | Emergency fallback only - upload to TestPyPI by hand |
| `make publish` | Emergency fallback only - upload to PyPI by hand |

## More Information

Expand Down
9 changes: 4 additions & 5 deletions scripts/bump_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -375,11 +375,10 @@ def main() -> None:

print("\n[OK] Version bump complete!")
print("\nNext steps:")
print(f" 1. Push the tag: git push origin v{new_version}")
print(" (also push the changelog commit: git push origin HEAD)")
print(" 2. Build release: make build")
print(" 3. Test on TestPyPI: make publish-test")
print(" 4. Publish to PyPI: make publish")
print(" 1. Push the changelog commit: git push origin HEAD")
print(f" 2. Push the tag: git push origin v{new_version}")
print("\nThe tag triggers .github/workflows/release.yml, which runs the")
print("pre-release checks and publishes to PyPI. Do not publish by hand.")


if __name__ == "__main__":
Expand Down
Loading