From eebe7ba4b48b1a4df48d1fbc0cbb4bf967b1f749 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 15:41:57 +0200 Subject: [PATCH 1/6] Define reproducible signed APT delivery and verification boundaries Track the issue 35 development slice, artifact trust model, publication contract, guest lifecycle evidence, and explicit human production activation boundary. Constraint: Keep Ubuntu and Debian package identity separate and preserve native release bytes Directive: Do not advertise repository installation before public verification and activation Scope-risk: narrow Tested: Plan mapped to all eight development requirements in issue 35 --- .../260905-kyo-signed-apt/260905-kyo-PLAN.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 .planning/quick/260905-kyo-signed-apt/260905-kyo-PLAN.md diff --git a/.planning/quick/260905-kyo-signed-apt/260905-kyo-PLAN.md b/.planning/quick/260905-kyo-signed-apt/260905-kyo-PLAN.md new file mode 100644 index 0000000..cf63523 --- /dev/null +++ b/.planning/quick/260905-kyo-signed-apt/260905-kyo-PLAN.md @@ -0,0 +1,50 @@ +--- +status: implementing +issue: 35 +--- + +# Signed APT repository development + +Goal: complete the development checklist in #35, verify it, and open one dedicated PR containing `resolves #35`. +The overall goal continues with #36 and then #37 after this PR is opened. Production accounts, keys, credentials, +and the first public activation remain the separately listed human operational work. + +## Contract and decisions + +- Target Ubuntu 24.04 and Debian 13, amd64, using the existing tested native package payload and recipes. +- Reuse existing Python/Bash/Git/OpenSSH tooling and distro APT/GnuPG utilities; no new application dependencies. +- Project-owned HTTPS publication uses a POSIX server over SSH. This provides a verifiable same-filesystem atomic + InRelease replacement; Bunny's documented PUT interface does not establish the required publication guarantee. +- Suites are ubuntu-24.04 and debian-13, component main. Preserve immutable pool and by-hash URLs indefinitely in + the first implementation. InRelease is the per-suite commit point; no cross-suite instantaneous transaction claim. +- Keep the server HTTP document root separate from private publication state and retained snapshots. Serialize + writes, compare the expected current revision, reject immutable collisions, and recover interrupted promotion. +- OpenPGP repository signatures are independent of the existing OpenSSL release checksum signatures. Verify both. +- Metadata is valid for 30 days; provide protected scheduled refresh of the same package set and explicit rollback + that produces fresh signed metadata. Rollback requires an explicit package downgrade on already upgraded clients. +- Homepage repository commands activate only through validated channel configuration after human public proof; + until then retain the functional existing installation options. Implement and test the activated UI in fixtures. + +## Tasks and ownership + +1. Generator and trust verification (`apt_protocol`): scripts/apt-repository.py and tests; Packages/Release/InRelease, + exact release-package validation, immutable inventory, prior-version retention, version ordering, fresh rollback, + tamper/path/key/architecture failure tests using real signing and APT tools. +2. Publication (`apt_publisher`): scripts/publish-package-repository.py and tests; local and SSH transports, locking, + compare-and-swap, immutable snapshots, atomic per-suite promotion/recovery, dry-run and cache configuration. +3. Lifecycle (`apt_guest_surface`): explicit APT mode in the existing VM runner, guest lifecycle script and independent + evidence verifier; fresh matching guests install/reinstall/upgrade/rollback/remove through HTTPS APT and perform + real GUI/provider/linkage checks. Preserve historical native proof; label synthetic package revisions as fixtures. +4. Integration (root): scoped bootstrap, release/refresh/rollback workflows, CI inputs, configuration contract, + gated homepage/install documentation, regression coverage, review, final validation and PR delivery. + +## Required evidence + +- Every development checkbox in #35 maps to implemented files and an executed check in the summary. +- Repository signatures and actual APT reject wrong keys, altered metadata/packages and incomplete publication. +- Retry, concurrent publication, downgrade and rollback rules are exercised, including actual SSH transport. +- Clean Ubuntu and Debian KVM guests consume the served HTTPS repository, with version/origin/signature and real + installed GUI/provider checks recorded for all applicable lifecycle transitions. +- Workflow syntax/contract checks, docs/build checks, focused tests and the full local validation pass. +- PR targets the current default branch, has a reviewable diff and `resolves #35`, and explicitly lists the human + production setup/activation still required. Do not claim public production installation was verified without it. From 51f8a411186211a3ba874b314ead15f5f08dcaa1 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 15:41:57 +0200 Subject: [PATCH 2/6] Enable native APT installation through a verified signed repository Generate and verify signed Ubuntu and Debian suites from existing release packages. Retain immutable package and by-hash objects, serialize publication over authenticated SSH, and atomically replace each suite's InRelease file. Add expiry refresh, recoverable promotion and explicit rollback, while keeping the public channel pending until operator verification activates its setup. Provide a scoped bootstrap, protected release integration, matching-guest lifecycle harness, and homepage/documentation support for the activated channel. Constraint: Initial native repository targets are Ubuntu 24.04 and Debian 13 amd64 Constraint: OpenPGP repository trust is separate from the OpenSSL release checksum key Rejected: Reuse Bunny storage PUT as atomic publication | documented overwrite guarantees are insufficient Confidence: high Scope-risk: moderate Directive: Preserve old immutable URLs and require expected-revision checks during publication Directive: Fixture credentials and synthetic package revisions are not production release proof Tested: Repository tests on Ubuntu 24.04 and Debian 13; real APT trust and tamper checks Tested: Full APT suite including real SSH, interruption recovery, bootstrap, HTTPS and proof validation Tested: Pending and activated-fixture browser flows, docs build and static site verification Tested: Workflow contracts, actionlint, Node/Python/Bash checks and ShellCheck Not-tested: Fresh KVM lifecycle execution and full workspace checks run after this reproducible commit Not-tested: Production hosting, credentials and public channel activation remain operator tasks --- .github/workflows/ci.yml | 2 + .github/workflows/deploy-docs.yml | 1 + .github/workflows/publish-apt.yml | 86 +++ .github/workflows/release.yml | 16 + .github/workflows/web.yml | 2 + .github/workflows/workflow-checks.yml | 2 + .gitignore | 1 + apps/docs/docs/.vitepress/config.ts | 2 + apps/docs/docs/developer/apt-repository.md | 253 ++++++++ apps/docs/docs/developer/release.md | 14 + apps/docs/docs/guide/apt-repository.md | 122 ++++ apps/docs/docs/guide/install.md | 26 +- apps/docs/docs/guide/support-matrix.md | 8 +- apps/docs/docs/release-notes/unreleased.md | 16 + apps/site/src/lib/aptChannel.mjs | 63 ++ apps/site/src/lib/aptChannel.test.mjs | 80 +++ apps/site/src/pages/index.astro | 18 +- package.json | 3 +- packaging/repositories/Dockerfile.apt-tools | 10 + packaging/repositories/apt-channel.json | 9 + packaging/repositories/nginx-apt.conf | 31 + packaging/vm/guest-apt-repository-smoke.sh | 253 ++++++++ scripts/apt-repository.py | 553 +++++++++++++++++ scripts/native-package-vm.sh | 50 +- scripts/publish-apt-workflow.sh | 103 ++++ scripts/publish-package-repository.py | 621 ++++++++++++++++++++ scripts/setup-apt-repository.sh | 167 ++++++ scripts/test-apt-bootstrap.py | 196 ++++++ scripts/test-apt-public.py | 152 +++++ scripts/test-apt-repository-vm-proof.mjs | 78 +++ scripts/test-apt-repository.py | 333 +++++++++++ scripts/test-apt-workflow-preflight.py | 61 ++ scripts/test-apt-workflow.rb | 47 ++ scripts/test-ci-workflow-paths.rb | 3 +- scripts/test-publish-package-repository.py | 539 +++++++++++++++++ scripts/verify-apt-public.py | 106 ++++ scripts/verify-apt-repository-vm-proof.mjs | 261 ++++++++ scripts/verify-apt-repository.sh | 16 + scripts/verify-docs.sh | 8 + scripts/verify-github-workflows.sh | 2 + scripts/verify-scripts.sh | 2 + scripts/with-apt-tools.sh | 34 ++ 42 files changed, 4323 insertions(+), 27 deletions(-) create mode 100644 .github/workflows/publish-apt.yml create mode 100644 apps/docs/docs/developer/apt-repository.md create mode 100644 apps/docs/docs/guide/apt-repository.md create mode 100644 apps/site/src/lib/aptChannel.mjs create mode 100644 apps/site/src/lib/aptChannel.test.mjs create mode 100644 packaging/repositories/Dockerfile.apt-tools create mode 100644 packaging/repositories/apt-channel.json create mode 100644 packaging/repositories/nginx-apt.conf create mode 100755 packaging/vm/guest-apt-repository-smoke.sh create mode 100644 scripts/apt-repository.py create mode 100755 scripts/publish-apt-workflow.sh create mode 100644 scripts/publish-package-repository.py create mode 100755 scripts/setup-apt-repository.sh create mode 100755 scripts/test-apt-bootstrap.py create mode 100755 scripts/test-apt-public.py create mode 100644 scripts/test-apt-repository-vm-proof.mjs create mode 100644 scripts/test-apt-repository.py create mode 100644 scripts/test-apt-workflow-preflight.py create mode 100755 scripts/test-apt-workflow.rb create mode 100644 scripts/test-publish-package-repository.py create mode 100755 scripts/verify-apt-public.py create mode 100644 scripts/verify-apt-repository-vm-proof.mjs create mode 100755 scripts/verify-apt-repository.sh create mode 100755 scripts/with-apt-tools.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ed5efb2..b81bbf8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,6 +19,7 @@ on: - "tsconfig.base.json" - ".npmrc" - "!**/*.md" + - "!packaging/repositories/*-channel.json" - "!scripts/*docs*" - "!scripts/build-static-site.mjs" - "!scripts/verify-static-site.mjs" @@ -46,6 +47,7 @@ on: - "tsconfig.base.json" - ".npmrc" - "!**/*.md" + - "!packaging/repositories/*-channel.json" - "!scripts/*docs*" - "!scripts/build-static-site.mjs" - "!scripts/verify-static-site.mjs" diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml index d80adff..c0e942f 100644 --- a/.github/workflows/deploy-docs.yml +++ b/.github/workflows/deploy-docs.yml @@ -11,6 +11,7 @@ on: paths: - ".github/workflows/deploy-docs.yml" - "apps/site/**" + - "packaging/repositories/apt-channel.json" - "apps/docs/**" - "assets/product-screenshot.png" - "package.json" diff --git a/.github/workflows/publish-apt.yml b/.github/workflows/publish-apt.yml new file mode 100644 index 0000000..d495abb --- /dev/null +++ b/.github/workflows/publish-apt.yml @@ -0,0 +1,86 @@ +name: Publish APT Repository + +on: + workflow_call: + inputs: + tag: + type: string + required: true + operation: + type: string + default: publish + workflow_dispatch: + inputs: + operation: + description: Publish a stable release, refresh expiry, or roll back to a retained revision + type: choice + options: [publish, refresh, rollback] + default: publish + tag: + description: Existing stable release tag for publish + type: string + revision: + description: Retained repository revision SHA-256 for rollback + type: string + schedule: + - cron: "37 5 * * 1" + +permissions: + contents: read + +concurrency: + group: apt-repository-production + cancel-in-progress: false + +jobs: + publish: + if: >- + ${{ + vars.APT_REPOSITORY_ENABLED == 'true' && + (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) || + (github.workflow == 'Release' && startsWith(github.ref, 'refs/tags/v'))) + }} + runs-on: ubuntu-24.04 + timeout-minutes: 30 + environment: packages-production + steps: + - name: Checkout publisher + uses: actions/checkout@v7.0.0 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Set up Node for release verification + uses: actions/setup-node@v6.4.0 + with: + node-version: 22.23.1 + + - name: Install repository tools + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends apt-utils dpkg-dev gnupg gpgv openssh-client python3 + + - name: Build, publish, and verify repository + env: + GH_TOKEN: ${{ github.token }} + OPERATION: ${{ inputs.operation || 'refresh' }} + RELEASE_TAG: ${{ inputs.tag }} + ROLLBACK_REVISION: ${{ inputs.revision }} + APT_REPOSITORY_URL: ${{ vars.APT_REPOSITORY_URL }} + APT_REPOSITORY_HOST: ${{ vars.APT_REPOSITORY_HOST }} + APT_REPOSITORY_ROOT: ${{ vars.APT_REPOSITORY_ROOT }} + APT_SSH_PORT: ${{ vars.APT_SSH_PORT || '22' }} + APT_SIGNING_FINGERPRINT: ${{ vars.APT_SIGNING_FINGERPRINT }} + APT_SSH_PRIVATE_KEY: ${{ secrets.APT_SSH_PRIVATE_KEY }} + APT_SSH_KNOWN_HOSTS: ${{ secrets.APT_SSH_KNOWN_HOSTS }} + APT_SIGNING_KEY: ${{ secrets.APT_SIGNING_KEY }} + APT_SIGNING_PASSPHRASE: ${{ secrets.APT_SIGNING_PASSPHRASE }} + run: bash scripts/publish-apt-workflow.sh + + - name: Upload public verification and activation record + uses: actions/upload-artifact@v7.0.1 + with: + name: loopwire-apt-publication-${{ github.run_id }} + path: dist/apt-publication + if-no-files-found: error + retention-days: 90 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d2aea31..fd12162 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -207,6 +207,8 @@ jobs: needs: build-linux runs-on: ubuntu-22.04 timeout-minutes: 45 + outputs: + tag: ${{ steps.verified-tag.outputs.tag }} steps: - name: Checkout uses: actions/checkout@v7.0.0 @@ -497,3 +499,17 @@ jobs: ${{ env.LOOPWIRE_RELEASE_EVIDENCE_ARCHIVE }} if-no-files-found: error retention-days: 90 + + - name: Export verified release tag + id: verified-tag + run: printf 'tag=%s\n' "$LOOPWIRE_RELEASE_TAG" >>"$GITHUB_OUTPUT" + + publish-apt: + name: Publish signed APT channel + needs: publish-release + if: ${{ vars.APT_REPOSITORY_ENABLED == 'true' && !contains(needs.publish-release.outputs.tag, '-') }} + uses: ./.github/workflows/publish-apt.yml + with: + tag: ${{ needs.publish-release.outputs.tag }} + operation: publish + secrets: inherit diff --git a/.github/workflows/web.yml b/.github/workflows/web.yml index 69684d9..8e80ef8 100644 --- a/.github/workflows/web.yml +++ b/.github/workflows/web.yml @@ -5,6 +5,7 @@ on: paths: - ".github/workflows/web.yml" - "apps/site/**" + - "packaging/repositories/apt-channel.json" - "apps/docs/**" - "assets/product-screenshot.png" - "README.md" @@ -30,6 +31,7 @@ on: paths: - ".github/workflows/web.yml" - "apps/site/**" + - "packaging/repositories/apt-channel.json" - "apps/docs/**" - "assets/product-screenshot.png" - "README.md" diff --git a/.github/workflows/workflow-checks.yml b/.github/workflows/workflow-checks.yml index f013a00..a6b5667 100644 --- a/.github/workflows/workflow-checks.yml +++ b/.github/workflows/workflow-checks.yml @@ -7,6 +7,7 @@ on: - "scripts/ci-impact.rb" - "scripts/test-ci-impact.rb" - "scripts/test-ci-workflow-paths.rb" + - "scripts/test-apt-workflow.rb" - "scripts/*native-package-proof-snapshot.mjs" - "scripts/verify-github-workflows.sh" - "scripts/verify-requirements.sh" @@ -21,6 +22,7 @@ on: - "scripts/ci-impact.rb" - "scripts/test-ci-impact.rb" - "scripts/test-ci-workflow-paths.rb" + - "scripts/test-apt-workflow.rb" - "scripts/*native-package-proof-snapshot.mjs" - "scripts/verify-github-workflows.sh" - "scripts/verify-requirements.sh" diff --git a/.gitignore b/.gitignore index 7f86e37..741ba06 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ node_modules/ dist/ dist-ssr/ coverage/ +__pycache__/ .vitepress/cache/ .vitepress/dist/ .astro/ diff --git a/apps/docs/docs/.vitepress/config.ts b/apps/docs/docs/.vitepress/config.ts index 4048e27..afd70bc 100644 --- a/apps/docs/docs/.vitepress/config.ts +++ b/apps/docs/docs/.vitepress/config.ts @@ -43,6 +43,7 @@ export default defineConfig({ text: "Guide", items: [ { text: "Install", link: "/guide/install" }, + { text: "APT Repository", link: "/guide/apt-repository" }, { text: "Basic Usage", link: "/guide/basic-usage" }, { text: "Configurations", link: "/guide/configurations" }, { text: "Audio Backends", link: "/guide/backends" }, @@ -60,6 +61,7 @@ export default defineConfig({ { text: "GitHub Actions Setup", link: "/developer/github-actions-setup" }, { text: "VM Matrix", link: "/developer/vm-matrix" }, { text: "Release", link: "/developer/release" }, + { text: "APT Repository Operations", link: "/developer/apt-repository" }, { text: "Release Notes", link: "/developer/release-notes" } ] }, diff --git a/apps/docs/docs/developer/apt-repository.md b/apps/docs/docs/developer/apt-repository.md new file mode 100644 index 0000000..ea83833 --- /dev/null +++ b/apps/docs/docs/developer/apt-repository.md @@ -0,0 +1,253 @@ +# Signed APT repository operations + +This runbook is for maintainers preparing, publishing, or recovering Loopwire's APT channel. Development of the +channel is separate from public activation. The checked-in channel record starts `pending`: provisioning the server, +signing identity, protected GitHub environment, and first public publication remain human operations. Keep the +existing direct-download instructions working until those operations and public verification are complete. + +## Scope and trust boundary + +The channel serves `main`/`amd64` in two independent suites: `ubuntu-24.04` and `debian-13`. It packages the existing +native release payload; no UI or audio-backend behavior belongs in this layer. Stable `X.Y.Z` versions, optionally +with `+build` metadata, are accepted. Prereleases and cross-distro substitutions are rejected. + +Two signatures have different jobs: + +1. The existing project **OpenSSL release key** authenticates `SHA256SUMS.sig` over `SHA256SUMS`. The generator checks + the exact Ubuntu and Debian artifacts against that manifest and their internal package metadata before indexing. +2. A separate **OpenPGP APT key** signs `InRelease` and `Release.gpg`. APT authenticates metadata and follows its SHA-256 + hashes through `Packages` to each deb. Clients trust that key only for the Loopwire source through `Signed-By`. + +The web root contains public packages, indexes, signed release metadata, and public keys. Private signing material, +publication state, retained snapshots, locks, and transaction staging must never be served over HTTP. + +## Human provisioning + +Provision a project-owned HTTPS host with a valid public certificate and an SSH account restricted to the repository +storage root. The host needs Python 3 and POSIX filesystem semantics; GnuPG runs on the publishing runner. Put staging +and the public root on the same filesystem so rename is atomic. Serve **`ROOT/public` only**, disable directory +listing, and deny sibling `ROOT/snapshots` and publication state. Back up private snapshots and state independently. + +The SSH approach is deliberate: the existing Bunny PUT upload surface does not establish an atomic replacement +contract for repository metadata. The ordinary website deployment remains separate. + +Configure HTTP caching so clients cannot receive a new index behind stale release metadata: + +- `InRelease`, `Release`, `Release.gpg`, and ordinary `Packages`/compressed indexes: `Cache-Control: no-store` or + equivalent mandatory revalidation. +- Content-addressed `by-hash` indexes and immutable package pool paths: long cache lifetime with `immutable`. +- Public keys and the current manifest: revalidate. Do not cache failures for paths that will soon be published. + +Create a dedicated OpenPGP signing identity on a trusted machine. Record its complete 40-character fingerprint, +expiration, custodian, backup, and revocation-certificate location. Keep the offline recovery copy and revocation +certificate separate from CI secrets. Do not reuse the OpenSSL release key. Export an ASCII-armored public key for +verification and a private export solely for the protected publishing environment. + +Configure the GitHub environment **`packages-production`**, restrict its permitted branches/tags to reviewed release +inputs, and apply its human approval policy. The workflow validates required configuration before remote writes. + +| Kind | Name | Purpose | +| --- | --- | --- | +| Repository variable | `APT_REPOSITORY_ENABLED` | Set to `true` to call APT publication after a successful tagged GitHub Release publication. Leave disabled until provisioned. | +| Variable | `APT_REPOSITORY_URL` | Canonical public HTTPS URL, without credentials, query, or fragment. | +| Variable | `APT_REPOSITORY_HOST` | SSH `USER@HOST` for the publication account. | +| Variable | `APT_REPOSITORY_ROOT` | Absolute private storage root; HTTP serves its `public` child. | +| Variable | `APT_SIGNING_FINGERPRINT` | Full uppercase OpenPGP fingerprint. | +| Optional variable | `APT_SSH_PORT` | SSH port when not 22. | +| Secret | `APT_SSH_PRIVATE_KEY` | Restricted SSH identity for publication. | +| Secret | `APT_SSH_KNOWN_HOSTS` | Host-key pins obtained through a trusted channel. | +| Secret | `APT_SIGNING_KEY` | Dedicated ASCII-armored OpenPGP private signing key. | +| Optional secret | `APT_SIGNING_PASSPHRASE` | Passphrase for that private export. | + +The enabled variable gates manual and scheduled runs as well as release-triggered publication. Leave it disabled +until all production inputs are ready; enabling it permits those workflows to write the repository. It does not +activate the public website's install commands. + +Do not generate trusted SSH pins from an unauthenticated scan in the publishing job. Configure monitoring for failed +publication/refresh runs, certificate expiry, signing-key expiry, and metadata approaching its 30-day expiry. + +## Build and verify a candidate + +Use a reviewed checkout and a directory containing published stable GitHub Release artifacts, `SHA256SUMS`, and +`SHA256SUMS.sig`. Set `APT_FPR` to the full signing fingerprint and `APT_GNUPG_HOME` to a protected GnuPG home containing +that key. The following paths are operator-chosen local staging directories; never expose the GnuPG home to HTTP. +Local generation needs Python 3, `dpkg-deb`, `dpkg`, GnuPG (`gpg` and `gpgv`), and OpenSSL. The protected Ubuntu runner +provides these tools; other development hosts can use the pinned APT tools container described below. + +```bash +python3 scripts/apt-repository.py build \ + --release-dir dist/release --version 0.1.0 --output dist/apt-candidate \ + --signing-key "$APT_FPR" --gnupg-home "$APT_GNUPG_HOME" +python3 scripts/apt-repository.py verify \ + --repository dist/apt-candidate --public-key apt-public.asc --fingerprint "$APT_FPR" +``` + +The default release verifier uses `packaging/release-signing-public.pem`; `--release-public-key FILE` selects an +explicit alternative for isolated fixtures. Production must retain the project trust anchor. Use `--previous DIR` +when advancing an existing repository so old pool objects and by-hash indexes remain available. Supply +`--passphrase-file FILE` when the signing key needs one; keep it private and delete temporary key material after use. + +`--date EPOCH` and `--valid-for-days 30` control signed timestamps. Fixed dates are for reproducible fixtures; normal +publication uses fresh dates. The verifier accepts `--now EPOCH` for expiry tests. Never publish already expired +metadata or turn off client expiry checks. + +## Publish, refresh, and recover + +Prefer the protected **Publish APT Repository** workflow. It supports manual `publish`, `refresh`, and `rollback`, +and weekly refresh. Tagged release publication calls it only after the GitHub Release has been published and only +when `APT_REPOSITORY_ENABLED=true`. It downloads and checks published release inputs instead of trusting an arbitrary +local build directory. An APT failure does not undo the existing GitHub Release; repair it and retry the APT job. +For a manual run, use the default branch: choose `publish` with an existing stable `tag`, `refresh` with no release +input, or `rollback` with the retained 64-character `revision`. The weekly run selects refresh automatically. + +For a reviewed local rehearsal, the same publisher can operate without SSH. For production, supply all SSH identity +and host-key arguments. In these examples `APT_ROOT`, `APT_HOST`, and `APT_REVISION` name the provisioned root, host, and +the current manifest revision. `APT_REVISION` is the empty string for an initial publication only. + +```bash +python3 scripts/publish-package-repository.py fetch \ + --root "$APT_ROOT" --output dist/apt-previous \ + --public-key apt-public.asc --fingerprint "$APT_FPR" \ + --ssh "$APT_HOST" --identity-file apt-ssh-key --known-hosts apt-known-hosts +python3 scripts/publish-package-repository.py publish \ + --repository dist/apt-candidate --root "$APT_ROOT" \ + --public-key apt-public.asc --fingerprint "$APT_FPR" --expected-revision "$APT_REVISION" \ + --ssh "$APT_HOST" --identity-file apt-ssh-key --known-hosts apt-known-hosts --dry-run +``` + +Read the dry-run result, then repeat publication without `--dry-run`. Add `--ssh-port PORT` if needed. Initial +publication skips `fetch`; subsequent publication builds with the fetched repository as `--previous`. The expected +revision provides compare-and-swap protection: a conflicting publisher must refetch and rebuild, not force a stale +candidate over the current channel. A lock serializes publication on the server. +Take `APT_REVISION` from the verified `fetch` JSON result. Fetching an empty root exits with code 3; a pending +transaction blocks fetch until recovery. Private state files are diagnostics, not a replacement for verified fetch. +Fetch verifies snapshots at their original signed creation time so expired but authentic snapshots remain usable for +refresh and rollback. Successful fetch alone does not prove current client usability; publication and public +verification also require metadata that is valid now. + +Immutable package and by-hash objects are installed first; existing paths with different content are rejected. +Each suite's `InRelease` rename is its commit point. Clients must see either the previous complete suite or the next +complete suite. Ubuntu and Debian may transition at different instants; there is no cross-suite atomicity claim. +Interrupted promotion retains recovery state. Recover using the pending candidate's key before retrying a failed +publication, then inspect the current revision and verify served metadata: + +```bash +python3 scripts/publish-package-repository.py recover \ + --root "$APT_ROOT" --public-key apt-public.asc --fingerprint "$APT_FPR" \ + --ssh "$APT_HOST" --identity-file apt-ssh-key --known-hosts apt-known-hosts +``` + +Recovery verifies the pending snapshot at the current time before resuming its exact journal. If a transaction has +remained pending beyond metadata expiry, inspect it with `recover --allow-expired --dry-run`. The explicit +`--allow-expired` recovery option validates its signature and hash chain at its original signed creation time, +finishes only that journal, and returns `requiresRefresh: true`. Use it only for an operator-reviewed expired +transaction, then immediately fetch, re-sign, and publish fresh metadata. Clients still reject expired metadata; +the option does not disable APT expiry checks. The workflow does not apply this override automatically. Do not change +the clock or edit public files and transaction state by hand to bypass a conflict. + +Refresh republishes the same package set with a new signed date and 30-day validity. It does not rebuild the +application or manufacture a new application release. Run it manually after missed schedules and verify both suites. +GitHub schedules can be delayed or disabled, so the weekly job is not the expiry monitor. + +## Retention and rollback + +Version 1 retains immutable package pool paths, by-hash indexes, and publication snapshots indefinitely. This keeps +old signed metadata and rollback references resolvable. There is no automatic garbage collection. Monitor storage +growth; a future deletion policy needs an explicit design covering metadata validity, cache lifetime, active clients, +and recovery retention before any objects are removed. + +Select a known-good snapshot revision and fetch it using `fetch --revision SHA`. Rollback signs that snapshot's +package set with **fresh** metadata; copying old expired `InRelease` files is not a valid rollback: + +```bash +python3 scripts/apt-repository.py rollback \ + --repository dist/apt-known-good --output dist/apt-rollback \ + --signing-key "$APT_FPR" --gnupg-home "$APT_GNUPG_HOME" +python3 scripts/apt-repository.py verify \ + --repository dist/apt-rollback --public-key apt-public.asc --fingerprint "$APT_FPR" +``` + +Publish the rollback candidate against the **current** revision, then run public verification. Previously installed +newer packages remain installed: clients must explicitly choose the distro-specific version with +`sudo apt-get install --allow-downgrades loopwire=VERSION`. Communicate that command and the reason for rollback; +the [user guide](../guide/apt-repository.md#earlier-versions) explains the client steps. + +## Signing-key rotation and revocation + +Treat a routine key change as a coordinated release operation. Generate the successor identity offline, publish its +full fingerprint through trusted project channels, and retain both keys' public material and historical snapshots. +Clients need the updated scoped keyring **before** they accept metadata signed only by the successor. The bootstrap +helper can replace its managed source with the newly verified key; ordinary package upgrades do not silently change +the trust anchor. Test the transition on both clean and existing-client guests, including rollback, before production. + +The conservative version 1 rotation path uses a **new repository root and HTTPS prefix**. Build a fresh candidate +from authenticated release files under the successor key without `--previous`, publish and verify that prefix, then +update the protected environment and client setup to the new URL/fingerprint. Leave the old prefix and key available +for the announced migration window if the old key remains trustworthy. Existing clients rerun the helper to replace +their source. A fetched old snapshot still requires its old key; `build --previous` and `rollback` accept snapshots +signed by their current signer, so they cannot silently re-sign old-key history as a new trust identity. + +The fingerprint-named public key file is immutable too: changing its expiry or subkeys changes its bytes and cannot +overwrite that URL in place. Use the reviewed successor-key procedure. Publishing itself can accept a new signer, +but that does not establish client trust or migrate old snapshots. The current single-key bootstrap provides no +unattended cross-signing or automatic rotation. Keep the public record `pending` during any unverified transition, +then review a new public verification record before reactivating instructions. + +If the key is compromised, disable publication and refresh immediately, remove its private export from CI, publish +the revocation certificate and an incident notice through trusted channels, and take the affected channel out of +service. A revocation certificate alone does not update existing local keyrings. Direct clients to remove the old +managed source/keyring, inspect the announced replacement fingerprint, and bootstrap the replacement explicitly. +Verify package provenance independently before republishing with a new key; re-signing compromised content does not +repair it. Preserve private evidence and recover only from known-good snapshots or authenticated release inputs. + +## Public verification and final activation + +Local signing tests and isolated VM fixtures establish development behavior. They do not prove that users can reach +the production repository. After initial publication, verify the actual HTTPS-served bytes against the candidate: + +```bash +python3 scripts/verify-apt-public.py \ + --repository dist/apt-candidate --public-key apt-public.asc --fingerprint "$APT_FPR" \ + --base-url "$APT_URL" --proof-url "$APT_PROOF_URL" --output dist/apt-channel.json +``` + +`APT_URL` is the canonical production URL and `APT_PROOF_URL` is that successful GitHub Actions run's URL. The checker +validates the local signed chain, fetches every manifest file over HTTPS without redirects, compares exact hashes and +sizes, and emits a verified record only on success. `--ca-file FILE` is for isolated HTTPS test CAs; it is not public +production proof. Do not create a production activation record from a fixture server or a synthetic package upgrade. + +The workflow uploads `loopwire-apt-publication-RUN_ID` with `apt-channel.json`, `publication.json`, and +`repository-manifest.json`. **Final activation is a human operation:** inspect the successful run, review the URL, +signing fingerprint, revision, timestamp, and package versions, complete initial public clean-client installation +checks, then copy its `apt-channel.json` into `packaging/repositories/apt-channel.json` in a reviewed commit. Build and +deploy the website from that commit. Do not hand-edit `status` alone or place operator credentials in this file. + +The schema is version 1. A pending record has null URL, fingerprint, revision, verification timestamp, and proof URL. +A verified record needs an HTTPS base URL, 40 uppercase hex fingerprint characters, a 64 lowercase hex revision, +an ISO timestamp, and the project GitHub Actions run URL. The homepage and user setup page validate every field; +missing or invalid data retains manual installation commands. On activation, Ubuntu and Debian tabs show +`sudo apt install loopwire` and link separately to one-time setup. Automatic, other distributions, and direct +download instructions remain available. + +## Development evidence + +Run generator, bootstrap, publisher, public-checker, and channel-gating tests, plus workflow, docs, and site checks. +For example, `bash scripts/with-apt-tools.sh --container python3 scripts/test-apt-repository.py` runs signing and real +APT checks inside the pinned Debian 13 tools image. The wrapper mounts the repository read-only and keeps test +temporary writes inside the container, without requiring a developer's host distro to install APT. It does not +replace the separate clean-guest lifecycle runs or their real GUI/provider evidence. +The package lifecycle harness has dedicated modes: + +```bash +pnpm vm:native-packages -- run-apt --target ubuntu-24.04 --version 0.1.0 --release-dir dist/release +pnpm vm:native-packages -- run-apt --target debian-13 --version 0.1.0 --release-dir dist/release +pnpm vm:native-packages -- verify-apt --target ubuntu-24.04 +pnpm vm:native-packages -- verify-apt --target debian-13 +``` + +These boot fresh checksum-pinned distro guests, use HTTPS APT with scoped trust, and exercise package installation, +reinstall, upgrade, downgrade/rollback, removal, and rejection of untrusted or broken repository state. Record exact +versions, source URLs, key fingerprint, candidate selection, signature results, GUI launch, and provider-command +checks. A synthetic `+aptfixture1` upgrade reuses the authenticated `0.1.0` payload to test lifecycle behavior; label +it as fixture evidence, never as a newly released application or public production proof. A package lifecycle pass +does not promote desktop-session or live audio-backend support claims. diff --git a/apps/docs/docs/developer/release.md b/apps/docs/docs/developer/release.md index 50e5f18..dbbba88 100644 --- a/apps/docs/docs/developer/release.md +++ b/apps/docs/docs/developer/release.md @@ -3,6 +3,20 @@ Loopwire releases are artifact-first. Every install channel must consume the same tarballs, `SHA256SUMS`, and `SHA256SUMS.sig`. +## Signed APT channel + +Ubuntu 24.04 and Debian 13 package publication has a separate [APT operations runbook](./apt-repository.md). It covers +the independent OpenPGP trust anchor, required protected environment and SSH/HTTPS hosting, release publication, +weekly metadata refresh, retention, rollback, key changes, and client removal. The optional **Publish APT Repository** +workflow runs after GitHub Release publication when `APT_REPOSITORY_ENABLED=true`; an APT failure leaves the existing +GitHub Release intact for repair and retry. + +Development and public activation are separate gates. Provisioning production infrastructure and signing keys, +running initial public verification, and reviewing the emitted channel record remain human operations. Until the +verified record is committed and the site deployed, Ubuntu and Debian homepage tabs retain signed direct-download +commands. Fixture lifecycle tests do not count as production proof. Follow the runbook's final activation procedure +before announcing repository availability. + ## Local Artifact Smoke ```bash diff --git a/apps/docs/docs/guide/apt-repository.md b/apps/docs/docs/guide/apt-repository.md new file mode 100644 index 0000000..a386c53 --- /dev/null +++ b/apps/docs/docs/guide/apt-repository.md @@ -0,0 +1,122 @@ + + +# APT repository + +Loopwire's APT channel targets **Ubuntu 24.04 and Debian 13 on x86_64 (`amd64`)**. It is a project repository that +requires one-time setup; Loopwire is not included in either distribution's default repositories. Other releases, +derivatives, and ARM64 should use the matching options in the [installation guide](./install.md). + +
+

Public channel pending

+

The repository tooling is available in the source tree. A public repository URL and signing key have not been activated. + Use the Ubuntu or Debian signed direct + downloads, or the automatic installer. The setup command will appear here after public verification.

+
+ +
+

Verified public channel

+

Repository: {{ channel.baseUrl }}
+ OpenPGP fingerprint: {{ channel.signingFingerprint }}
+ Public verification record, recorded {{ channel.verifiedAt }}.

+
+ +## One-time setup + +The following procedure applies once this page displays a verified public channel. You need Bash, curl, GnuPG, +Python 3, and `dpkg`, plus sudo access to configure APT. Download and inspect the small setup helper first: + +```bash +curl -fsSLo setup-apt-repository.sh \ + https://raw.githubusercontent.com/sandwichfarm/loopwire/master/scripts/setup-apt-repository.sh +less setup-apt-repository.sh +``` + +
+

Run the helper with the published URL and full signing fingerprint:

+ 
{{ setupCommand }}
+
+ +The helper detects the exact distribution version and architecture, downloads the repository's OpenPGP key over +HTTPS, and checks its full fingerprint before making changes. It writes only the Loopwire source and scoped +keyring. An existing unrelated source with the same filename is an error; other APT sources are preserved. +Add `--dry-run` and omit `sudo` to preview the selected suite and paths without downloads or changes. + +After successful setup, refresh package metadata and install: + +```bash +sudo apt update && +sudo apt install loopwire +``` + +The setup helper does not run either command for you. Confirm `apt update` succeeds for the Loopwire source before +installing. `apt-cache policy loopwire` shows the candidate version and repository URL; the URL should match the +verified channel above. The package includes the desktop application and background/provider commands, without +enabling startup services or applying audio routes during installation. + +## Updates and reinstall + +APT can update Loopwire alongside your other packages. To update only Loopwire: + +```bash +sudo apt update && +sudo apt install --only-upgrade loopwire +``` + +To repair package-owned files at the installed version, first find the exact version with +`dpkg-query -W -f='${Version}\n' loopwire`, then run `sudo apt install --reinstall loopwire=VERSION` with that value. +Saved routing configurations are outside package ownership and are preserved. + +## Earlier versions + +Use `apt-cache policy loopwire` to inspect available versions. Repository rollback changes the recommended package +set, but APT will not automatically downgrade a newer installed version. If a maintainer recommends rollback, +replace `VERSION` with the exact distro-specific version and opt into it: + +```bash +sudo apt update && +sudo apt-get install --allow-downgrades loopwire=VERSION +``` + +Keep the distro suffix: an Ubuntu package is not interchangeable with the Debian package. Older downloads are +retained for recovery; the active package index determines which versions APT can select. Ask for recovery guidance +if the required version is absent, rather than bypassing package authentication. + +## Remove Loopwire or the repository + +Uninstall the application with `sudo apt remove loopwire`. APT removes package-owned files and leaves your saved +configuration intact. Remove any startup integration you explicitly enabled using the +[start-on-boot guide](./start-on-boot.md). + +To stop receiving Loopwire repository updates, use the same inspected helper: + +```bash +sudo bash setup-apt-repository.sh --remove && +sudo apt update +``` + +This removes the managed `loopwire.sources` file and its referenced Loopwire keyring. It does not uninstall Loopwire +or affect other repositories. Manually provisioned source files require manual removal of the matching source and +keyring. Do not remove shared distro keyrings. + +## Trust and troubleshooting + +APT checks signed repository metadata, which binds package indexes and their package checksums. The repository +OpenPGP key is separate from the OpenSSL key used to verify direct GitHub Release downloads. The setup uses a +per-source `Signed-By` keyring; it does not grant Loopwire's key authority over other repositories. + +- **Wrong or changed fingerprint:** stop and compare this page's fingerprint with the project's announced key + change. Rerun the inspected setup helper with the new full fingerprint only after that change is confirmed. +- **Expired metadata:** check the system clock, retry `sudo apt update`, and report the error if it persists. + Repository metadata expires after 30 days and should be refreshed by the project before then. +- **Invalid signature, checksum mismatch, or missing file:** stop the install, retry metadata refresh, and report + the failing URL and APT error. Do not disable authentication, TLS validation, or expiry checks. +- **Unsupported distribution or architecture:** use the [installation guide](./install.md). Changing a suite name + to force a different distro package does not make that package compatible. + +Diagnostics need the distro version, architecture, `apt-cache policy loopwire`, and the APT error text. Redact local +usernames and private URLs; never include audio recordings or private keys. diff --git a/apps/docs/docs/guide/install.md b/apps/docs/docs/guide/install.md index b58f93a..2c39325 100644 --- a/apps/docs/docs/guide/install.md +++ b/apps/docs/docs/guide/install.md @@ -19,8 +19,8 @@ installer changes an existing installation. | Platform | Default installation path | | --- | --- | -| Ubuntu 24.04, x86_64 | Signed release deb through APT | -| Debian 13, x86_64 | Signed release deb through APT | +| Ubuntu 24.04, x86_64 | Signed direct-download deb installed by APT | +| Debian 13, x86_64 | Signed direct-download deb installed by APT | | Fedora 44, x86_64 | Signed release RPM through DNF | | openSUSE Tumbleweed, x86_64 | Signed release RPM through Zypper | | Arch Linux, x86_64 or ARM64 | `loopwire-bin` through an existing yay or paru; portable fallback without a helper | @@ -81,6 +81,17 @@ Automatic performs the download and verification steps for you. For manual insta below together in an empty directory. Commands are connected with `&&` so failed downloads or checks stop the install. These are direct `v0.1.0` downloads, not distro repositories. Use Automatic to select the latest available release. +The [APT repository guide](./apt-repository.md) shows channel availability, the verified URL and key when activated, +and one-time setup for Ubuntu 24.04 and Debian 13 on x86_64. After that setup and a successful `sudo apt update`, the +repository install command is: + +```bash +sudo apt install loopwire +``` + +Loopwire is not in the default Ubuntu or Debian repositories. Until the guide displays a verified public channel, +use Automatic or the signed manual downloads below. The automatic installer does not configure the APT repository. + The RPM files have no embedded RPM signature. The commands authenticate the download using the signed SHA-256 manifest first, then permit this local RPM for that install; repository dependency checks remain enabled. @@ -98,8 +109,7 @@ sha256sum --check --ignore-missing SHA256SUMS && sudo apt install ./loopwire_0.1.0-1ubuntu24.04_amd64.deb ``` -[Repository work to shorten this setup](https://github.com/sandwichfarm/loopwire/issues/35) tracks signed metadata, -release publication, clean-guest install/upgrade verification, and updated instructions. +[APT repository setup and availability](./apt-repository.md) includes upgrade, rollback, removal, and key guidance. ### Debian 13 @@ -115,8 +125,7 @@ sha256sum --check --ignore-missing SHA256SUMS && sudo apt install ./loopwire_0.1.0-1debian13_amd64.deb ``` -[Repository work to shorten this setup](https://github.com/sandwichfarm/loopwire/issues/35) tracks signed metadata, -release publication, clean-guest install/upgrade verification, and updated instructions. +[APT repository setup and availability](./apt-repository.md) includes upgrade, rollback, removal, and key guidance. ### Fedora 44 @@ -263,8 +272,9 @@ Run the metadata smoke: pnpm verify:packaging ``` -The AppImages and native deb/RPM files are published as direct downloads on the `v0.1.0` GitHub Release. They are not -yet served through an APT, DNF/COPR, or OBS repository. Their matching-guest proof command boots official, +The AppImages and native deb/RPM files are published as direct downloads on the `v0.1.0` GitHub Release. The APT +repository has a separate [public activation gate](./apt-repository.md); DNF/COPR and OBS repositories remain planned. +Their matching-guest proof command boots official, checksum-pinned cloud images under KVM and stores local evidence without changing host audio: ```bash diff --git a/apps/docs/docs/guide/support-matrix.md b/apps/docs/docs/guide/support-matrix.md index 254a9ac..28d489d 100644 --- a/apps/docs/docs/guide/support-matrix.md +++ b/apps/docs/docs/guide/support-matrix.md @@ -70,7 +70,8 @@ the Tauri shell command bridge. | Source checkout | `pnpm check` | Supported for contributors. | | Signed curl installer | Local verification plus live `/install.sh` byte comparison | Published for 0.1.0 at `loopwire.app`. | | AppImage | Published-artifact and Tauri bundle smoke | Published for 0.1.0 on GitHub Releases. | -| Ubuntu 24.04 / Debian 13 deb | Verified in matching KVM guests at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as direct downloads; no APT repository. | +| Ubuntu 24.04 / Debian 13 deb | Verified in matching KVM guests at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as direct downloads. | +| Ubuntu 24.04 / Debian 13 signed APT repository, amd64 | Separate signed-metadata, publication, bootstrap, and clean-guest lifecycle checks | Public activation is gated; see [current channel availability](./apt-repository.md). | | Fedora 44 / openSUSE Tumbleweed RPM | Verified in matching KVM guests at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as direct downloads; no COPR/OBS repository. | | AUR `loopwire` | Tagged source build through `pnpm verify:aur:source` | Published for 0.1.0. | | AUR `loopwire-bin` | Signed release-artifact build through `pnpm verify:aur` | Published for 0.1.0. | @@ -81,6 +82,11 @@ The flake package output is `packages..loopwire-bin`; `flake.nix` now pi for `x86_64-linux` and `aarch64-linux`. Fresh local non-skipped Nix build evidence in this repository currently covers `x86_64-linux` only; `aarch64-linux` still needs native proof. +APT lifecycle evidence is separate from the direct-download snapshot. Isolated HTTPS guests and synthetic +`+aptfixture1` package upgrades demonstrate development behavior with the existing release payload; they do not +establish a new public release or production channel. Only reviewed production HTTPS verification activates APT +instructions. Repository support covers the named distro versions on amd64, not derivatives or other architectures. + Native package verification is narrower than audio-backend support. The committed snapshot proves that each official, checksum-pinned guest built and installed its target package, ran the packaged background/provider/backend commands, resolved GUI libraries, created a Loopwire X11 window under Xvfb, and removed all package-owned files. It does not diff --git a/apps/docs/docs/release-notes/unreleased.md b/apps/docs/docs/release-notes/unreleased.md index 8b32292..dcc19d4 100644 --- a/apps/docs/docs/release-notes/unreleased.md +++ b/apps/docs/docs/release-notes/unreleased.md @@ -2,6 +2,22 @@ These notes describe source-tree progress. They are not a public release announcement. +## Signed APT repository development + +- Added signed APT metadata generation and verification for Ubuntu 24.04 and Debian 13 on amd64, consuming the + existing authenticated native release artifacts with a separate OpenPGP repository key. +- Added guarded SSH publication, retained snapshots and immutable package/index paths, fresh-metadata rollback, + protected release publication, and weekly metadata refresh. +- Added scoped repository bootstrap/removal and dedicated clean-guest APT lifecycle verification. Synthetic + `+aptfixture1` upgrades test the existing `0.1.0` payload; they are development evidence, not a new application release. +- Ubuntu and Debian homepage tabs switch to the short APT command only after a complete public verification record + is reviewed and committed. Production hosting, key/environment provisioning, and first public activation remain + separate human operations. Existing automatic and signed direct-download installation paths remain available. +- Added [user setup and recovery guidance](../guide/apt-repository.md) and the + [maintainer publication runbook](../developer/apt-repository.md). + +## Other distribution updates + - Root-level Bunny deployments now pass the post-upload live-site check when `BUNNY_REMOTE_PREFIX` is intentionally empty, while an omitted command argument remains an error. - The signed installer is now live at `https://loopwire.app/install.sh`; AppImage, deb, and RPM artifacts are available diff --git a/apps/site/src/lib/aptChannel.mjs b/apps/site/src/lib/aptChannel.mjs new file mode 100644 index 0000000..2788726 --- /dev/null +++ b/apps/site/src/lib/aptChannel.mjs @@ -0,0 +1,63 @@ +/** + * A public channel is advertised only after its complete HTTPS verification record + * has been reviewed and committed. Invalid or incomplete records keep manual installs. + * @param {unknown} value + */ +export function verifiedAptChannel(value) { + if (!value || typeof value !== "object") return null; + const channel = /** @type {Record} */ (value); + if (channel.schemaVersion !== 1 || channel.status !== "verified" || + typeof channel.baseUrl !== "string" || !validBaseUrl(channel.baseUrl) || + typeof channel.signingFingerprint !== "string" || !/^[A-F0-9]{40}$/.test(channel.signingFingerprint) || + typeof channel.revision !== "string" || !/^[a-f0-9]{64}$/.test(channel.revision) || + typeof channel.verifiedAt !== "string" || !validTimestamp(channel.verifiedAt) || + typeof channel.proofUrl !== "string" || + !/^https:\/\/github\.com\/sandwichfarm\/loopwire\/actions\/runs\/[1-9][0-9]*$/.test(channel.proofUrl)) { + return null; + } + return { + baseUrl: channel.baseUrl.replace(/\/+$/, ""), + signingFingerprint: channel.signingFingerprint, + revision: channel.revision, + verifiedAt: channel.verifiedAt, + proofUrl: channel.proofUrl + }; +} + +/** @param {string} value */ +function validBaseUrl(value) { + try { + const url = new URL(value); + return value.startsWith("https://") && !/[^\x21-\x7e]|[\\'"`$<>?#]/.test(value) && + url.protocol === "https:" && Boolean(url.hostname) && !url.username && !url.password && + (!url.port || Number(url.port) >= 1) && !url.search && !url.hash; + } catch { + return false; + } +} + +/** @param {string} value */ +function validTimestamp(value) { + if (!/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,6})?(?:Z|[+-]\d{2}:\d{2})$/.test(value) || + !Number.isFinite(Date.parse(value))) return false; + const date = value.slice(0, 10); + return new Date(`${date}T00:00:00Z`).toISOString().slice(0, 10) === date; +} + +/** + * @template {{command: string, note: string, detail: string, href: string, link: string}} T + * @param {unknown} channel + * @param {T} manual + * @returns {T} + */ +export function aptInstallOption(channel, manual) { + if (!verifiedAptChannel(channel)) return manual; + return { + ...manual, + command: "sudo apt install loopwire", + note: "After one-time setup, install and update Loopwire through its signed APT repository.", + detail: "Ubuntu 24.04 and Debian 13 on x86_64 are supported. Other versions and ARM64 use the portable path.", + href: "/docs/guide/apt-repository.html#one-time-setup", + link: "Set up the APT repository" + }; +} diff --git a/apps/site/src/lib/aptChannel.test.mjs b/apps/site/src/lib/aptChannel.test.mjs new file mode 100644 index 0000000..e0829c3 --- /dev/null +++ b/apps/site/src/lib/aptChannel.test.mjs @@ -0,0 +1,80 @@ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import test from "node:test"; +import { aptInstallOption, verifiedAptChannel } from "./aptChannel.mjs"; + +const verified = { + schemaVersion: 1, + status: "verified", + baseUrl: "https://packages.example.test/loopwire/", + signingFingerprint: "ABCDEF0123456789ABCDEF0123456789ABCDEF01", + revision: "a".repeat(64), + verifiedAt: "2026-09-05T10:20:30+00:00", + proofUrl: "https://github.com/sandwichfarm/loopwire/actions/runs/123456" +}; +const manual = { + id: "ubuntu", + command: "sudo apt install ./loopwire_0.1.0-1ubuntu24.04_amd64.deb", + note: "Verify the signed download first.", + detail: "Other versions use portable installation.", + href: "/docs/guide/apt-repository.html", + link: "APT repository setup and availability" +}; + +test("pending and incomplete channel records preserve the functional manual option", () => { + for (const value of [null, undefined, {}, [], { ...verified, status: "pending" }]) { + assert.equal(verifiedAptChannel(value), null); + assert.equal(aptInstallOption(value, manual), manual); + } + for (const key of Object.keys(verified)) { + const value = { ...verified }; + delete value[key]; + assert.equal(verifiedAptChannel(value), null, `missing ${key}`); + assert.equal(aptInstallOption(value, manual), manual); + } +}); + +test("verified channel exposes an install command and separate one-time setup link", () => { + assert.equal(verifiedAptChannel(verified).baseUrl, "https://packages.example.test/loopwire"); + for (const id of ["ubuntu", "debian"]) { + const option = aptInstallOption(verified, { ...manual, id }); + assert.equal(option.id, id); + assert.equal(option.command, "sudo apt install loopwire"); + assert.equal(option.href, "/docs/guide/apt-repository.html#one-time-setup"); + assert.doesNotMatch(`${option.note} ${option.detail}`, /workflow|revision|activation|operator/); + } + assert.match(manual.command, /\.deb$/); +}); + +test("malformed proof records never activate the channel", () => { + const invalid = { + schemaVersion: [0, "1"], status: [true, "ready"], + baseUrl: ["http://packages.example.test", "https://user:password@packages.example.test", "https://packages.example.test?a=1", + "https://packages.example.test#fragment", " https://packages.example.test", "https://packages.example.test/\n", "not a URL", + "https://packages.example.test:0", "https://packages.example.test:65536", + "https://packages.example.test?", "https://packages.example.test#", + "https://packages.example.test/$(id)", "https://packages.example.test/`id`", "https://packages.example.test/\\wrong"], + signingFingerprint: ["a".repeat(40), "A".repeat(39), "G".repeat(40)], + revision: ["A".repeat(64), "a".repeat(63)], + verifiedAt: ["yesterday", "2026-09-05", "2026-02-30T00:00:00Z", "2026-09-05T25:00:00Z"], + proofUrl: ["https://github.com/other/repo/actions/runs/123", "https://github.com/sandwichfarm/loopwire/pull/35", + "https://github.com/sandwichfarm/loopwire/actions/runs/123?fixture=1", "https://github.com/sandwichfarm/loopwire/actions/runs/0"] + }; + for (const [key, values] of Object.entries(invalid)) { + for (const value of values) { + const record = { ...verified, [key]: value }; + assert.equal(verifiedAptChannel(record), null, `${key}: ${value}`); + assert.equal(aptInstallOption(record, manual), manual); + } + } +}); + +test("checked-in channel either remains pending or has a complete verification record", () => { + const channel = JSON.parse(readFileSync(new URL("../../../../packaging/repositories/apt-channel.json", import.meta.url), "utf8")); + if (channel.status === "pending") { + assert.deepEqual(channel, { schemaVersion: 1, status: "pending", baseUrl: null, + signingFingerprint: null, revision: null, verifiedAt: null, proofUrl: null }); + } else { + assert.ok(verifiedAptChannel(channel)); + } +}); diff --git a/apps/site/src/pages/index.astro b/apps/site/src/pages/index.astro index 0dcbc56..ebaee03 100644 --- a/apps/site/src/pages/index.astro +++ b/apps/site/src/pages/index.astro @@ -1,6 +1,8 @@ --- import SiteLayout from "../layouts/SiteLayout.astro"; import screenshot from "../../../../assets/product-screenshot.png"; +import aptChannel from "../../../../packaging/repositories/apt-channel.json"; +import { aptInstallOption } from "../lib/aptChannel.mjs"; const title = "Loopwire | Linux virtual audio routing"; const description = @@ -43,22 +45,22 @@ const installOptions = [ detail: "No AUR helper? Use Automatic for a portable install. The loopwire, loopwire-bin, and loopwire-git packages conflict; choose one.", href: "https://aur.archlinux.org/packages/loopwire-bin", link: "View AUR package" }, - { + aptInstallOption(aptChannel, { id: "ubuntu", label: "Ubuntu", heading: "Ubuntu 24.04 · x86_64", command: nativeInstall("loopwire_0.1.0-1ubuntu24.04_amd64.deb", "sudo apt install ./loopwire_0.1.0-1ubuntu24.04_amd64.deb"), note: "Manual signed v0.1.0 package. Run these steps together in an empty folder; a failed check stops the " + "install. Automatic handles these steps for you.", - detail: "Other Ubuntu versions and ARM64 use the portable path. A native APT repository is planned to shorten this setup.", - href: "https://github.com/sandwichfarm/loopwire/issues/35", link: "Track simpler APT installs" - }, - { + detail: "Other Ubuntu versions and ARM64 use the portable path.", + href: "/docs/guide/apt-repository.html", link: "APT repository setup and availability" + }), + aptInstallOption(aptChannel, { id: "debian", label: "Debian", heading: "Debian 13 · x86_64", command: nativeInstall("loopwire_0.1.0-1debian13_amd64.deb", "sudo apt install ./loopwire_0.1.0-1debian13_amd64.deb"), note: "Manual signed v0.1.0 package. Run these steps together in an empty folder; a failed check stops the " + "install. Automatic handles these steps for you.", - detail: "Other Debian versions and ARM64 use the portable path. A native APT repository is planned to shorten this setup.", - href: "https://github.com/sandwichfarm/loopwire/issues/35", link: "Track simpler APT installs" - }, + detail: "Other Debian versions and ARM64 use the portable path.", + href: "/docs/guide/apt-repository.html", link: "APT repository setup and availability" + }), { id: "fedora", label: "Fedora", heading: "Fedora 44 · x86_64", command: nativeInstall("loopwire-0.1.0-1.fc44.x86_64.rpm", "sudo dnf --setopt=localpkg_gpgcheck=0 install ./loopwire-0.1.0-1.fc44.x86_64.rpm"), diff --git a/package.json b/package.json index b0ae2dc..ae0294b 100644 --- a/package.json +++ b/package.json @@ -15,7 +15,7 @@ "build:site": "pnpm --filter @loopwire/site build", "build:web": "pnpm build:site && pnpm build:docs && node scripts/build-static-site.mjs", "check": "pnpm check:verify && pnpm lint && pnpm typecheck && pnpm test && pnpm build && pnpm verify:site", - "check:verify": "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri", + "check:verify": "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri && pnpm verify:apt", "collect:evidence": "node scripts/collect-release-evidence.mjs", "collect:support": "node scripts/collect-support-bundle.mjs", "release:handoff": "bash scripts/plan-final-release-handoff.sh", @@ -66,6 +66,7 @@ "verify:docs-deployment": "node scripts/verify-docs-deployment-manifest.mjs", "verify:docs-live": "bash scripts/verify-docs-live.sh", "verify:packaging": "bash scripts/verify-packaging.sh", + "verify:apt": "bash scripts/verify-apt-repository.sh", "verify:native-packaging": "bash scripts/verify-native-packaging.sh", "build:portable-linux": "bash scripts/build-portable-linux-binary.sh", "package:deb": "bash scripts/build-deb-package.sh", diff --git a/packaging/repositories/Dockerfile.apt-tools b/packaging/repositories/Dockerfile.apt-tools new file mode 100644 index 0000000..0935599 --- /dev/null +++ b/packaging/repositories/Dockerfile.apt-tools @@ -0,0 +1,10 @@ +FROM debian:13@sha256:f324c7ff54321e8d9c588493a20244965938ce0aa50bbd1022d38010e9ffc4b1 + +RUN apt-get update \ + && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + apt ca-certificates curl dpkg-dev git gnupg gpgv nodejs openssh-client openssh-server openssl python3 \ + && rm -rf /var/lib/apt/lists/* \ + && mkdir -p /run/sshd + +ENV PYTHONDONTWRITEBYTECODE=1 +WORKDIR /workspace diff --git a/packaging/repositories/apt-channel.json b/packaging/repositories/apt-channel.json new file mode 100644 index 0000000..0535f2b --- /dev/null +++ b/packaging/repositories/apt-channel.json @@ -0,0 +1,9 @@ +{ + "schemaVersion": 1, + "status": "pending", + "baseUrl": null, + "signingFingerprint": null, + "revision": null, + "verifiedAt": null, + "proofUrl": null +} diff --git a/packaging/repositories/nginx-apt.conf b/packaging/repositories/nginx-apt.conf new file mode 100644 index 0000000..2ba3c12 --- /dev/null +++ b/packaging/repositories/nginx-apt.conf @@ -0,0 +1,31 @@ +# Include these locations in a TLS-enabled server, with an operator-provisioned +# certificate and server_name. The SSH publisher writes to /srv/loopwire-apt; +# only its public child may be served. snapshots/ and state/ stay private. +# Give the HTTP account read/traverse access to public, never origin write access. +root /srv/loopwire-apt/public; +autoindex off; +disable_symlinks on; + +# Interrupted same-filesystem writes may leave hidden temporary files. +location ~ (^|/)\. { + deny all; +} + +location ~ "^/(pool/.+\.deb|dists/[^/]+/main/binary-amd64/by-hash/SHA256/[0-9a-f]{64}|keys/[A-F0-9]+\.asc)$" { + try_files $uri =404; + error_page 404 = @apt_missing; + add_header Cache-Control "public, max-age=31536000, immutable"; +} + +location @apt_missing { + add_header Cache-Control "no-store, no-cache, must-revalidate" always; + return 404; +} + +# Never cache metadata or missing-file responses at an origin/CDN. InRelease is +# atomic per suite; clients use Acquire-By-Hash for indexes referenced by it. +location / { + try_files $uri =404; + add_header Cache-Control "no-store, no-cache, must-revalidate" always; + etag off; +} diff --git a/packaging/vm/guest-apt-repository-smoke.sh b/packaging/vm/guest-apt-repository-smoke.sh new file mode 100755 index 0000000..8077feb --- /dev/null +++ b/packaging/vm/guest-apt-repository-smoke.sh @@ -0,0 +1,253 @@ +#!/usr/bin/env bash +# The unprivileged guest user owns proof logs; sudo applies only to the package commands. +# shellcheck disable=SC2024 +set -euo pipefail + +target="${1:?target is required}" +package_target="${2:?package target is required}" +format="${3:?format is required}" +version="${4:?version is required}" +git_head="${5:?git head is required}" +kit_dir="${6:-$PWD}" +case "$target" in ubuntu-24.04 | debian-13) ;; *) echo "unsupported APT guest target" >&2; exit 2 ;; esac +[ "$package_target" = "$target" ] && [ "$format" = deb ] +[[ "$version" =~ ^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(\+[0-9A-Za-z]+(\.[0-9A-Za-z]+)*)?$ ]] +[[ "$git_head" =~ ^[0-9a-f]{40}$ ]] +cd "$kit_dir" +proof_dir="$kit_dir/proof" +fixture_dir="$kit_dir/apt-fixture" +base_url="https://127.0.0.1:8443" +upgrade_version="${version}+aptfixture1" +[[ "$version" != *+* ]] || upgrade_version="${version}.aptfixture1" +case "$target" in + ubuntu-24.04) suffix=1ubuntu24.04 ;; + debian-13) suffix=1debian13 ;; +esac +baseline_package_version="${version}-${suffix}" +upgrade_package_version="${upgrade_version}-${suffix}" +mkdir -p "$proof_dir/packages" "$proof_dir/repositories" "$fixture_dir" +exec > >(tee "$proof_dir/commands.log") 2>&1 +set -x + +cat /etc/os-release >"$proof_dir/os-release" +uname -a >"$proof_dir/uname.txt" +systemd-detect-virt --vm >"$proof_dir/virtualization.txt" +grep -Eq '^(kvm|qemu)$' "$proof_dir/virtualization.txt" +if dpkg-query -W -f='${Status}' loopwire 2>/dev/null | grep -qx 'install ok installed'; then + echo 'clean guest already has Loopwire installed' >&2 + exit 1 +fi +printf 'absent\n' >"$proof_dir/initial-package-status.txt" +( + cd "$kit_dir/release" + sha256sum --check --strict SHA256SUMS >/dev/null + sha256sum loopwire-linux-x86_64.tar.gz +) >"$proof_dir/release-payload.sha256" +tar -xOf "$kit_dir/release/loopwire-linux-x86_64.tar.gz" RELEASE >"$proof_dir/payload-release.txt" + +sudo apt-get update +sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + apt-utils ca-certificates curl dpkg-dev gnupg nodejs openssl python3 xdotool xz-utils xvfb + +# Signing material is generated inside this disposable guest and never copied into evidence. +gnupg_home="$fixture_dir/gnupg" +mkdir -m 0700 "$gnupg_home" +gpg --homedir "$gnupg_home" --batch --pinentry-mode loopback --passphrase '' \ + --quick-generate-key 'Loopwire disposable APT guest fixture' ed25519 sign 0 +fingerprint="$(gpg --homedir "$gnupg_home" --batch --with-colons --list-keys | awk -F: '$1 == "fpr" { print $10; exit }')" +[[ "$fingerprint" =~ ^[0-9A-F]{40}$ ]] +gpg --homedir "$gnupg_home" --batch --armor --export "$fingerprint" >"$proof_dir/repository-key.asc" +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out "$fixture_dir/release-key.pem" +openssl pkey -in "$fixture_dir/release-key.pem" -pubout -out "$fixture_dir/release-public.pem" +cp "$fixture_dir/release-public.pem" "$proof_dir/release-public.pem" + +build_fixture_release() { + local fixture_version="$1" destination="$2" package_distro + mkdir -p "$destination" + for package_distro in ubuntu-24.04 debian-13; do + SOURCE_DATE_EPOCH=0 bash scripts/build-deb-package.sh --target "$package_distro" \ + --version "$fixture_version" --arch x86_64 --release-dir "$kit_dir/release" --output-dir "$destination" + done + (cd "$destination" && sha256sum ./*.deb | sed 's| ./| |' >SHA256SUMS) + openssl dgst -sha256 -sign "$fixture_dir/release-key.pem" -out "$destination/SHA256SUMS.sig" "$destination/SHA256SUMS" + cp "$destination/loopwire_${fixture_version}-${suffix}_amd64.deb" "$proof_dir/packages/" +} + +build_fixture_release "$version" "$fixture_dir/baseline-release" +build_fixture_release "$upgrade_version" "$fixture_dir/upgrade-release" +python3 scripts/apt-repository.py build --release-dir "$fixture_dir/baseline-release" --version "$version" \ + --output "$fixture_dir/initial" --signing-key "$fingerprint" --gnupg-home "$gnupg_home" \ + --release-public-key "$fixture_dir/release-public.pem" +python3 scripts/apt-repository.py build --release-dir "$fixture_dir/upgrade-release" --version "$upgrade_version" \ + --output "$fixture_dir/upgraded" --previous "$fixture_dir/initial" --signing-key "$fingerprint" \ + --gnupg-home "$gnupg_home" --release-public-key "$fixture_dir/release-public.pem" +python3 scripts/apt-repository.py rollback --repository "$fixture_dir/initial" --output "$fixture_dir/rolled-back" \ + --signing-key "$fingerprint" --gnupg-home "$gnupg_home" + +for repository_stage in initial upgraded rolled-back; do + python3 scripts/apt-repository.py verify --repository "$fixture_dir/$repository_stage" \ + --public-key "$proof_dir/repository-key.asc" --fingerprint "$fingerprint" \ + >"$proof_dir/repositories/${repository_stage}-verification.json" + cp -a "$fixture_dir/$repository_stage" "$proof_dir/repositories/$repository_stage" +done + +# A guest-only CA exercises real TLS verification; no production trust is imported. +openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj '/CN=Loopwire disposable guest CA' \ + -keyout "$fixture_dir/ca-key.pem" -out "$fixture_dir/ca.crt" \ + -addext 'basicConstraints=critical,CA:TRUE' -addext 'keyUsage=critical,keyCertSign,cRLSign' +openssl req -newkey rsa:2048 -nodes -subj '/CN=127.0.0.1' \ + -keyout "$fixture_dir/tls-key.pem" -out "$fixture_dir/tls.csr" +printf 'subjectAltName=IP:127.0.0.1\nbasicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth\n' >"$fixture_dir/tls.ext" +openssl x509 -req -in "$fixture_dir/tls.csr" -CA "$fixture_dir/ca.crt" -CAkey "$fixture_dir/ca-key.pem" \ + -CAcreateserial -days 1 -extfile "$fixture_dir/tls.ext" -out "$fixture_dir/tls.crt" +cp "$fixture_dir/ca.crt" "$proof_dir/tls-ca.crt" +cp "$fixture_dir/tls.crt" "$proof_dir/tls-server.crt" +sudo install -m 0644 "$fixture_dir/ca.crt" /usr/local/share/ca-certificates/loopwire-guest-fixture.crt +sudo update-ca-certificates +mkdir -p "$fixture_dir/www" +ln -s "$fixture_dir/initial" "$fixture_dir/www/repository" +cat >"$fixture_dir/https-server.py" <<'PY' +import functools +import http.server +import ssl +import sys +class Handler(http.server.SimpleHTTPRequestHandler): + def do_GET(self): + # Fixture revisions may share a filesystem timestamp to the second. + # Always deliver their signed bytes when APT refreshes the source. + if "If-Modified-Since" in self.headers: + del self.headers["If-Modified-Since"] + super().do_GET() +server = http.server.ThreadingHTTPServer(("127.0.0.1", 8443), functools.partial(Handler, directory=sys.argv[1])) +context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) +context.load_cert_chain(sys.argv[2], sys.argv[3]) +server.socket = context.wrap_socket(server.socket, server_side=True) +server.serve_forever() +PY +python3 "$fixture_dir/https-server.py" "$fixture_dir/www/repository" "$fixture_dir/tls.crt" "$fixture_dir/tls-key.pem" \ + >"$proof_dir/https-server.log" 2>&1 & +server_pid=$! +cleanup() { kill "$server_pid" 2>/dev/null || true; } +trap cleanup EXIT +for _attempt in $(seq 1 20); do + if curl --fail --silent --show-error "$base_url/keys/$fingerprint.asc" >"$proof_dir/https-key.asc"; then break; fi + sleep 1 +done +cmp "$proof_dir/repository-key.asc" "$proof_dir/https-key.asc" +python3 scripts/verify-apt-public.py --repository "$fixture_dir/initial" --public-key "$proof_dir/repository-key.asc" \ + --fingerprint "$fingerprint" --base-url "$base_url" --ca-file "$fixture_dir/ca.crt" \ + | tee "$proof_dir/repositories/initial-public-verification.json" +sudo bash scripts/setup-apt-repository.sh --base-url "$base_url" --fingerprint "$fingerprint" \ + >"$proof_dir/bootstrap.log" 2>&1 +cat /etc/apt/sources.list.d/loopwire.sources >"$proof_dir/loopwire.sources" +sudo apt-get update >"$proof_dir/bootstrap-update.log" 2>&1 + +smoke_installed() { + local stage="$1" expected_version="$2" stage_dir="$proof_dir/$1" + mkdir -p "$stage_dir" + dpkg-query -W -f='${Package}\t${Version}\t${Architecture}\t${Status}\n' loopwire >"$stage_dir/package-metadata.tsv" + [ "$(dpkg-query -W -f='${Version}' loopwire)" = "$expected_version" ] + dpkg -L loopwire | sort >"$stage_dir/package-files.txt" + while IFS= read -r installed_file; do + if [ -f "$installed_file" ]; then sha256sum "$installed_file"; fi + done <"$stage_dir/package-files.txt" >"$stage_dir/installed-files.sha256" + apt-cache policy loopwire >"$stage_dir/apt-policy.txt" + grep -Fq "$base_url" "$stage_dir/apt-policy.txt" + loopwire --background --help >"$stage_dir/background-help.txt" + loopwire-dsp-provider --help >"$stage_dir/dsp-provider-help.txt" + loopwire-jack-ports --help >"$stage_dir/jack-provider-help.txt" + loopwire-detect-audio --pretty >"$stage_dir/detect-audio.json" + ldd /usr/lib/loopwire/loopwire-gui >"$stage_dir/gui-ldd.txt" + if grep -Fq 'not found' "$stage_dir/gui-ldd.txt"; then + echo 'Installed GUI has unresolved shared libraries' >&2 + return 1 + fi + local gui_status=0 + # Runtime process/window variables must expand in the child shell. + # shellcheck disable=SC2016 + timeout 35s bash -c ' + stage_dir="$1" + app_pid="" + Xvfb :99 -screen 0 1280x720x24 -nolisten tcp >"$stage_dir/xvfb.log" 2>&1 & + xvfb_pid=$! + cleanup_gui() { [ -z "$app_pid" ] || kill "$app_pid" 2>/dev/null || true; kill "$xvfb_pid" 2>/dev/null || true; wait || true; } + trap cleanup_gui EXIT + sleep 1 + DISPLAY=:99 GDK_BACKEND=x11 WEBKIT_DISABLE_DMABUF_RENDERER=1 \ + /usr/lib/loopwire/loopwire-gui >"$stage_dir/gui-launch.log" 2>&1 & + app_pid=$! + for attempt in $(seq 1 20); do + kill -0 "$app_pid" 2>/dev/null || exit 1 + if DISPLAY=:99 xdotool search --name "^(Loopwire|loopwire-gui)$" >"$stage_dir/gui-window-ids.txt" 2>/dev/null; then + while read -r window_id; do DISPLAY=:99 xdotool getwindowname "$window_id"; done \ + <"$stage_dir/gui-window-ids.txt" >"$stage_dir/gui-window-names.txt" + exit 0 + fi + sleep 1 + done + exit 124 + ' bash "$stage_dir" || gui_status=$? + printf '%s\n' "$gui_status" >"$stage_dir/gui-launch-status.txt" + [ "$gui_status" -eq 0 ] + [ -s "$stage_dir/gui-window-ids.txt" ] + if grep -Eiq 'error while loading shared libraries|panic|protocol error|missing acquire timeline' "$stage_dir/gui-launch.log"; then + echo 'Installed GUI reported a startup failure' >&2 + return 1 + fi + printf '%s\t%s\tinstalled\n' "$stage" "$expected_version" >>"$proof_dir/lifecycle.tsv" +} + +sudo DEBIAN_FRONTEND=noninteractive apt-get install -y "loopwire=$baseline_package_version" >"$proof_dir/install.log" 2>&1 +smoke_installed install "$baseline_package_version" +sudo DEBIAN_FRONTEND=noninteractive apt-get install --reinstall -y "loopwire=$baseline_package_version" >"$proof_dir/reinstall.log" 2>&1 +smoke_installed reinstall "$baseline_package_version" +ln -sfn "$fixture_dir/upgraded" "$fixture_dir/www/repository" +python3 scripts/verify-apt-public.py --repository "$fixture_dir/upgraded" --public-key "$proof_dir/repository-key.asc" \ + --fingerprint "$fingerprint" --base-url "$base_url" --ca-file "$fixture_dir/ca.crt" \ + | tee "$proof_dir/repositories/upgraded-public-verification.json" +sudo apt-get update >"$proof_dir/upgrade-update.log" 2>&1 +sudo DEBIAN_FRONTEND=noninteractive apt-get install --only-upgrade -y loopwire >"$proof_dir/upgrade.log" 2>&1 +smoke_installed upgrade "$upgrade_package_version" +ln -sfn "$fixture_dir/rolled-back" "$fixture_dir/www/repository" +python3 scripts/verify-apt-public.py --repository "$fixture_dir/rolled-back" --public-key "$proof_dir/repository-key.asc" \ + --fingerprint "$fingerprint" --base-url "$base_url" --ca-file "$fixture_dir/ca.crt" \ + | tee "$proof_dir/repositories/rolled-back-public-verification.json" +sudo apt-get update >"$proof_dir/rollback-update.log" 2>&1 +sudo DEBIAN_FRONTEND=noninteractive apt-get install --allow-downgrades -y "loopwire=$baseline_package_version" >"$proof_dir/rollback.log" 2>&1 +smoke_installed rollback "$baseline_package_version" +sudo DEBIAN_FRONTEND=noninteractive apt-get remove -y loopwire >"$proof_dir/remove.log" 2>&1 +if dpkg-query -W loopwire >/dev/null 2>&1; then + echo 'Loopwire remains registered after removal' >&2 + exit 1 +fi +for removed_file in /usr/bin/loopwire /usr/bin/loopwire-dsp-provider /usr/bin/loopwire-jack-ports \ + /usr/bin/loopwire-detect-audio /usr/lib/loopwire /usr/share/applications/loopwire.desktop \ + /usr/share/icons/hicolor/scalable/apps/loopwire.svg; do + test ! -e "$removed_file" + printf '%s\tabsent\n' "$removed_file" >>"$proof_dir/removed-files.tsv" +done +printf 'remove\t%s\tabsent\n' "$baseline_package_version" >>"$proof_dir/lifecycle.tsv" +sudo bash scripts/setup-apt-repository.sh --remove >"$proof_dir/source-removal.log" 2>&1 +test ! -e /etc/apt/sources.list.d/loopwire.sources +test ! -e "/etc/apt/keyrings/loopwire-$fingerprint.asc" +sudo apt-get update >"$proof_dir/source-removal-update.log" 2>&1 +apt-cache policy loopwire >"$proof_dir/source-removal-policy.txt" +if grep -Fq "$base_url" "$proof_dir/source-removal-policy.txt"; then + echo 'APT still lists the removed repository' >&2 + exit 1 +fi +cat >"$proof_dir/summary.tsv" <= minimum, f"invalid {label}") + return value + + +def object_pairs(pairs): + result = {} + for key, value in pairs: + require(key not in result, f"duplicate JSON field: {key}") + result[key] = value + return result + + +def read_json(path): + return json.loads(path.read_text(encoding="utf-8"), object_pairs_hook=object_pairs) + + +def safe_path(value): + require(isinstance(value, str) and value and not any(ord(char) < 33 or ord(char) > 126 for char in value), + "inventory paths must contain printable ASCII without whitespace") + path = PurePosixPath(value) + require(not path.is_absolute() and str(path) == value and ".." not in path.parts and "\\" not in value, + f"unsafe inventory path: {value}") + return value + + +def classify_path(value): + safe_path(value) + if re.fullmatch(rf"keys/{FINGERPRINT}\.asc", value): + return "immutable" + for suite, revision in SUITES.items(): + if re.fullmatch(rf"pool/{re.escape(suite)}/main/l/loopwire/loopwire_{VERSION}-1{revision}_amd64\.deb", value): + return "immutable" + prefix = f"dists/{suite}/" + if value in (prefix + "InRelease", prefix + "Release", + prefix + "main/binary-amd64/Packages", prefix + "main/binary-amd64/Packages.gz"): + return "metadata" + if re.fullmatch(re.escape(prefix) + rf"main/binary-amd64/by-hash/SHA256/{HASH}", value): + return "immutable" + raise RepositoryError(f"path is outside the APT repository contract: {value}") + + +def regular_file(path): + info = path.lstat() + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, + f"only regular files without symlinks or hardlinks are allowed: {path}") + return info + + +def tree_files(root): + require(root.is_dir() and not root.is_symlink(), "repository must be a real directory") + files = set() + for directory, directories, names in os.walk(root, followlinks=False): + for name in directories: + path = Path(directory) / name + require(not path.is_symlink(), f"symlink directory is forbidden: {path}") + for name in names: + path = Path(directory) / name + regular_file(path) + files.add(path.relative_to(root).as_posix()) + return files + + +def parse_control(data): + text = data.decode("utf-8") if isinstance(data, bytes) else data + require("\r" not in text and "\x00" not in text, "invalid control-file characters") + records = [] + current = {} + field = None + for line in text.splitlines(): + if not line: + if current: + records.append(current) + current = {} + field = None + elif line[0] in " \t": + require(field is not None, "control-file continuation without a field") + current[field] += "\n" + line + else: + require(":" in line, "malformed control-file field") + key, value = line.split(":", 1) + require(re.fullmatch(r"[A-Za-z][A-Za-z0-9-]*", key) is not None, "invalid control-file field name") + field = key.lower() + require(field not in current, f"duplicate control-file field: {key}") + current[field] = value.lstrip(" ") + if current: + records.append(current) + return records + + +def single_control(data): + records = parse_control(data) + require(len(records) == 1, "expected one control-file record") + return records[0] + + +def package_info(root, path, suite): + classify_path(path) + require(path.startswith(f"pool/{suite}/"), "package belongs to the wrong suite") + file = root / path + regular_file(file) + raw = run("dpkg-deb", "--field", file) + control = single_control(raw) + require(control.get("package") == "loopwire", "repository only accepts Package: loopwire") + require(control.get("architecture") == "amd64", "repository only accepts Architecture: amd64") + version = control.get("version", "") + require(re.fullmatch(rf"{VERSION}-1{SUITES[suite]}", version) is not None, + f"package version does not match suite {suite}: {version}") + require(Path(path).name == f"loopwire_{version}_amd64.deb", "package filename does not match control identity") + require(not ({"filename", "size", "md5sum", "sha1", "sha256", "sha512"} & set(control)), + "package control must not supply repository-owned hash/path fields") + return {"name": "loopwire", "version": version, "architecture": "amd64", "path": path, + "sha256": sha256(file), "size": file.stat().st_size}, raw.rstrip(b"\n") + + +def key_fingerprint(key, home): + data = run("gpg", "--batch", "--homedir", home, "--with-colons", "--import-options", "show-only", + "--import", key).decode() + primary = [] + want_fingerprint = False + for line in data.splitlines(): + fields = line.split(":") + if fields[0] == "pub": + want_fingerprint = True + elif fields[0] == "fpr" and want_fingerprint: + primary.append(fingerprint(fields[9])) + want_fingerprint = False + elif fields[0] == "sub": + want_fingerprint = False + require(len(primary) == 1, "public key must contain exactly one primary OpenPGP key") + return primary[0] + + +def signed_release(root, suite, ring, home, expected_fingerprint): + decoded = Path(home) / f"{suite}.Release" + status = run("gpgv", "--homedir", home, "--keyring", ring, "--status-fd", "1", + "--output", decoded, root / f"dists/{suite}/InRelease").decode() + valid = [line.split() for line in status.splitlines() if line.startswith("[GNUPG:] VALIDSIG ")] + require(len(valid) == 1 and valid[0][-1] == expected_fingerprint, + f"{suite}: signature does not match the pinned primary fingerprint") + require(not any(f"[GNUPG:] {flag}" in status for flag in + ("EXPKEYSIG", "EXPSIG", "REVKEYSIG", "KEYREVOKED", "KEYEXPIRED", "SIGEXPIRED")), + f"{suite}: expired or revoked signing identity") + data = decoded.read_bytes() + require(data == (root / f"dists/{suite}/Release").read_bytes(), + f"{suite}: Release differs from signed InRelease payload") + return single_control(data) + + +def manifest_revision(manifest): + unsigned = {key: value for key, value in manifest.items() if key != "revision"} + return hashlib.sha256(canonical(unsigned)).hexdigest() + + +def load_inventory(root): + actual = tree_files(root) + require(MANIFEST in actual, "missing repository-manifest.json") + manifest = read_json(root / MANIFEST) + exact_keys(manifest, {"schemaVersion", "revision", "createdAt", "validUntil", "signingFingerprint", "suites", "files"}, + "repository manifest") + require(type(manifest["schemaVersion"]) is int and manifest["schemaVersion"] == 1, "unsupported manifest schema") + fingerprint(manifest["signingFingerprint"]) + integer(manifest["createdAt"], "createdAt") + integer(manifest["validUntil"], "validUntil") + require(manifest["validUntil"] > manifest["createdAt"], "metadata expiry must follow creation") + require(manifest["revision"] == manifest_revision(manifest), "manifest revision digest mismatch") + require(isinstance(manifest["files"], list), "files must be an array") + inventory = {} + for entry in manifest["files"]: + exact_keys(entry, {"path", "sha256", "size", "kind"}, "inventory entry") + path = safe_path(entry["path"]) + require(path not in inventory, f"duplicate inventory path: {path}") + require(entry["kind"] == classify_path(path), f"incorrect file classification: {path}") + integer(entry["size"], "file size") + require(isinstance(entry["sha256"], str) and re.fullmatch(HASH, entry["sha256"]) is not None, + f"invalid SHA256: {path}") + require(path in actual, f"missing inventory file: {path}") + require((root / path).stat().st_size == entry["size"] and sha256(root / path) == entry["sha256"], + f"inventory checksum mismatch: {path}") + if "/by-hash/" in path: + require(Path(path).name == entry["sha256"], f"by-hash filename/content mismatch: {path}") + inventory[path] = entry + require(actual == set(inventory) | {MANIFEST}, "repository has unlisted files") + return manifest, inventory + + +def verify_repository(root, public_key, expected_fingerprint, now=None): + """Verify inventory, pinned signatures, index hashes, and exact package identities.""" + manifest, inventory = load_inventory(root) + expected = fingerprint(expected_fingerprint) if expected_fingerprint else manifest["signingFingerprint"] + require(expected == manifest["signingFingerprint"], "manifest fingerprint differs from operator pin") + now = int(time.time()) if now is None else integer(now, "verification time") + require(manifest["createdAt"] <= now + 10, "repository metadata is from the future") + require(manifest["validUntil"] > now, "repository metadata has expired; refresh and re-sign it") + require(isinstance(manifest["suites"], list) and len(manifest["suites"]) == len(SUITES), + "repository must contain exactly both supported suites") + with tempfile.TemporaryDirectory(prefix="loopwire-apt-verify-") as temporary: + home = Path(temporary) + require(key_fingerprint(public_key, home) == expected, "trusted public key differs from operator pin") + ring = home / "trusted.gpg" + run("gpg", "--batch", "--homedir", home, "--dearmor", "--output", ring, public_key) + for path in inventory: + if path.startswith("keys/"): + require(key_fingerprint(root / path, home) == Path(path).stem, "exported key fingerprint/path mismatch") + key_path = f"keys/{expected}.asc" + require(key_path in inventory, "missing fingerprint-addressed bootstrap key") + # The candidate's bootstrap asset must actually verify the same metadata, + # not merely carry another packet set with the same primary fingerprint. + exported_ring = home / "exported.gpg" + run("gpg", "--batch", "--homedir", home, "--dearmor", "--output", exported_ring, root / key_path) + seen_suites = set() + for suite in manifest["suites"]: + exact_keys(suite, {"name", "architecture", "component", "packages"}, "suite") + name = suite["name"] + require(name in SUITES and name not in seen_suites, "invalid or duplicate suite") + seen_suites.add(name) + require(suite["architecture"] == "amd64" and suite["component"] == "main", "unsupported suite layout") + release = signed_release(root, name, ring, home, expected) + (home / f"{name}.Release").unlink() + signed_release(root, name, exported_ring, home, expected) + require(set(release) == {"origin", "label", "suite", "codename", "architectures", "components", + "date", "valid-until", "acquire-by-hash", "sha256"}, + "signed Release fields are outside the supported repository contract") + require(release.get("origin") == "Loopwire" and release.get("label") == "Loopwire", + "incorrect repository identity") + require(release.get("suite") == name and release.get("codename") == name, "signed suite mismatch") + require(release.get("architectures") == "amd64" and release.get("components") == "main", + "signed architecture/component mismatch") + require(release.get("acquire-by-hash") == "yes", "signed metadata must enable by-hash") + for field, expected_time in (("date", manifest["createdAt"]), ("valid-until", manifest["validUntil"])): + date = parsedate_to_datetime(release.get(field, "")) + require(date.tzinfo is not None and int(date.timestamp()) == expected_time, + f"signed {field} differs from inventory") + hashes = {} + for row in release.get("sha256", "").splitlines(): + if not row.strip(): + continue + parts = row.split() + require(len(parts) == 3 and re.fullmatch(HASH, parts[0]) and parts[1].isdigit(), + "invalid signed SHA256 row") + checksum, size, path = parts + require(path not in hashes, "duplicate signed index path") + hashes[path] = (checksum, int(size)) + require(set(hashes) == {"main/binary-amd64/Packages", "main/binary-amd64/Packages.gz"}, + "signed metadata must cover exactly the supported package indexes") + for path, (checksum, size) in hashes.items(): + canonical_path = f"dists/{name}/{path}" + by_hash = f"dists/{name}/main/binary-amd64/by-hash/SHA256/{checksum}" + for candidate in (canonical_path, by_hash): + require(candidate in inventory and inventory[candidate]["sha256"] == checksum + and inventory[candidate]["size"] == size, f"signed index checksum mismatch: {candidate}") + index = (root / f"dists/{name}/main/binary-amd64/Packages").read_bytes() + require(gzip.decompress((root / f"dists/{name}/main/binary-amd64/Packages.gz").read_bytes()) == index, + "compressed index does not match Packages") + records = parse_control(index) + require(isinstance(suite["packages"], list) and suite["packages"], "suite has no packages") + require(len(records) == len(suite["packages"]), "package inventory/index count mismatch") + packages = {} + versions = set() + for entry in suite["packages"]: + exact_keys(entry, PACKAGE_FIELDS, "package") + path = safe_path(entry["path"]) + require(path not in packages and path in inventory, "duplicate or missing package inventory path") + info, _raw = package_info(root, path, name) + require(entry == info, "package identity/hash does not match inventory") + require(info["version"] not in versions, "duplicate package version") + versions.add(info["version"]) + packages[path] = entry + seen = set() + for record in records: + path = record.get("filename") + require(path in packages and path not in seen, "index contains missing or duplicate package") + seen.add(path) + package = packages[path] + require(record.get("package") == package["name"] and record.get("version") == package["version"] + and record.get("architecture") == package["architecture"], "signed package identity mismatch") + require(record.get("sha256") == package["sha256"] and record.get("size") == str(package["size"]), + "signed package checksum mismatch") + require(seen_suites == set(SUITES), "missing suite") + # Old immutable packages are outside current indexes after rollback, but + # still need to satisfy the allowed native-package identity contract. + for path in inventory: + if path.startswith("pool/"): + package_info(root, path, path.split("/")[1]) + return manifest + + +def export_signer(args, directory): + expected = fingerprint(args.signing_key) + gpg = ["gpg", "--batch", "--no-tty"] + if args.gnupg_home: + gpg.extend(["--homedir", str(args.gnupg_home)]) + if args.passphrase_file: + regular_file(args.passphrase_file) + gpg.extend(["--pinentry-mode", "loopback", "--passphrase-file", str(args.passphrase_file)]) + key = directory / "signer.asc" + key.write_bytes(run(*gpg, "--export-options", "export-minimal", "--armor", "--export", expected)) + require(key.stat().st_size > 0, "signing public key is missing") + require(key_fingerprint(key, directory) == expected, "signing key must identify one primary key") + return gpg, key, expected + + +def previous_repository(path, key, expected): + # Refresh/rollback must work after expiry. Authenticate the old snapshot at + # its signed Date; current validity is checked again on the newly signed output. + manifest, _inventory = load_inventory(path) + return verify_repository(path, key, expected, manifest["createdAt"]) + + +def copy_immutable(source, target, manifest): + for entry in manifest["files"]: + if entry["kind"] == "immutable": + destination = target / entry["path"] + destination.parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(source / entry["path"], destination) + require(sha256(destination) == entry["sha256"] and destination.stat().st_size == entry["size"], + "previous immutable file changed while copying the snapshot") + + +def source_packages(args, output): + require(re.fullmatch(VERSION, args.version) is not None, + "APT publication requires X.Y.Z with optional +build metadata; prereleases need a separate version policy") + source = args.release_dir + checksums = source / "SHA256SUMS" + signature = source / "SHA256SUMS.sig" + regular_file(checksums) + regular_file(signature) + run("openssl", "dgst", "-sha256", "-verify", args.release_public_key, + "-signature", signature, checksums) + signed = {} + for line in checksums.read_text(encoding="utf-8").splitlines(): + match = re.fullmatch(rf"({HASH}) [ *]([^/\\\s]+)", line) + require(match is not None, "invalid signed release checksum line") + checksum, name = match.groups() + require(name not in signed and name not in (".", ".."), "duplicate or invalid release checksum asset") + signed[name] = checksum + expected_assets = {f"loopwire_{args.version}-1{revision}_amd64.deb" for revision in SUITES.values()} + actual_debs = {path.name for path in source.glob("*.deb")} + require(actual_debs == expected_assets, "release must contain exactly the two expected native amd64 .deb files") + result = {} + for suite, revision in SUITES.items(): + name = f"loopwire_{args.version}-1{revision}_amd64.deb" + original = source / name + regular_file(original) + require(name in signed and sha256(original) == signed[name], f"release package checksum mismatch: {name}") + path = f"pool/{suite}/main/l/loopwire/{name}" + destination = output / path + destination.parent.mkdir(parents=True, exist_ok=True) + if destination.exists(): + require(sha256(destination) == signed[name], f"same package version has different bytes: {name}") + else: + shutil.copyfile(original, destination) + require(sha256(destination) == signed[name], "release package changed while staging its signed bytes") + result[suite], _raw = package_info(output, path, suite) + return result + + +def write_candidate(args, rollback=False): + output = args.output + require(not output.exists() and not output.is_symlink(), "output must not already exist; reuse the completed candidate for publication retries") + date = int(time.time()) if args.date is None else integer(args.date, "date") + require(1 <= args.valid_for_days <= 90, "valid-for-days must be between 1 and 90") + valid_until = date + args.valid_for_days * 86400 + output.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix=f".{output.name}-", dir=output.parent) as temporary: + staging_root = Path(temporary) + working = staging_root / "repository" + working.mkdir() + gpg, key, expected = export_signer(args, staging_root) + previous_path = args.repository if rollback else args.previous + previous = previous_repository(previous_path, key, expected) if previous_path else None + if previous: + require(date >= previous["createdAt"], "new metadata Date must not precede the previous revision") + copy_immutable(previous_path, working, previous) + suites = previous["suites"] if previous else [ + {"name": name, "architecture": "amd64", "component": "main", "packages": []} for name in SUITES] + if not rollback: + added = source_packages(args, working) + for suite in suites: + package = added[suite["name"]] + for old in suite["packages"]: + comparison = subprocess.run(["dpkg", "--compare-versions", package["version"], "ge", old["version"]], + stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) + require(comparison.returncode == 0, + "new package version is lower than a published version; use explicit rollback") + if not any(old["version"] == package["version"] for old in suite["packages"]): + suite["packages"].append(package) + suite["packages"].sort(key=lambda entry: entry["path"]) + exported = working / f"keys/{expected}.asc" + exported.parent.mkdir(exist_ok=True) + if exported.exists(): + require(exported.read_bytes() == key.read_bytes(), + "fingerprint-addressed key bytes changed; rotate with a new identity and bootstrap trust first") + else: + shutil.copyfile(key, exported) + for suite in suites: + name = suite["name"] + suite_dir = working / f"dists/{name}" + binary = suite_dir / "main/binary-amd64" + by_hash = binary / "by-hash/SHA256" + by_hash.mkdir(parents=True, exist_ok=True) + paragraphs = [] + for entry in suite["packages"]: + info, raw = package_info(working, entry["path"], name) + require(info == entry, "retained package differs from validated inventory") + paragraphs.append(raw + (f"\nFilename: {entry['path']}\nSize: {entry['size']}\n" + f"SHA256: {entry['sha256']}\n\n").encode()) + index = b"".join(paragraphs) + (binary / "Packages").write_bytes(index) + (binary / "Packages.gz").write_bytes(gzip.compress(index, compresslevel=9, mtime=0)) + hash_rows = [] + for index_name in ("Packages", "Packages.gz"): + file = binary / index_name + checksum = sha256(file) + immutable = by_hash / checksum + if immutable.exists(): + require(sha256(immutable) == checksum, "immutable by-hash collision") + else: + shutil.copyfile(file, immutable) + hash_rows.append(f" {checksum} {file.stat().st_size} main/binary-amd64/{index_name}\n") + release = ("Origin: Loopwire\nLabel: Loopwire\n" + f"Suite: {name}\nCodename: {name}\nArchitectures: amd64\nComponents: main\n" + f"Date: {format_datetime(datetime.fromtimestamp(date, timezone.utc), usegmt=True)}\n" + f"Valid-Until: {format_datetime(datetime.fromtimestamp(valid_until, timezone.utc), usegmt=True)}\n" + "Acquire-By-Hash: yes\nSHA256:\n" + "".join(hash_rows)) + (suite_dir / "Release").write_text(release, encoding="utf-8") + run(*gpg, "--yes", "--faked-system-time", f"{date}!", "--digest-algo", "SHA256", + "--local-user", expected, "--armor", "--clearsign", "--output", suite_dir / "InRelease", + suite_dir / "Release") + files = [{"path": path, "sha256": sha256(working / path), "size": (working / path).stat().st_size, + "kind": classify_path(path)} for path in sorted(tree_files(working))] + manifest = {"schemaVersion": 1, "createdAt": date, "validUntil": valid_until, + "signingFingerprint": expected, "suites": suites, "files": files} + manifest["revision"] = manifest_revision(manifest) + (working / MANIFEST).write_text(json.dumps(manifest, indent=2, sort_keys=True) + "\n", encoding="utf-8") + verify_repository(working, key, expected, date) + working.rename(output) + return manifest + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + commands = parser.add_subparsers(dest="command", required=True) + build = commands.add_parser("build", help="generate a candidate from signed native release assets") + build.add_argument("--release-dir", type=Path, required=True) + build.add_argument("--version", required=True) + build.add_argument("--release-public-key", type=Path, default=ROOT / "packaging/release-signing-public.pem", + help="trusted release checksum PEM (override for fixture keys)") + build.add_argument("--previous", type=Path, help="verified previous snapshot whose immutable history is retained") + rollback = commands.add_parser("rollback", help="freshly sign a previous snapshot's package set (also refreshes expiry)") + rollback.add_argument("--repository", type=Path, required=True) + for command in (build, rollback): + command.add_argument("--output", type=Path, required=True) + command.add_argument("--signing-key", required=True, help="uppercase primary OpenPGP fingerprint") + command.add_argument("--gnupg-home", type=Path, help="isolated GnuPG home containing the signing identity") + command.add_argument("--passphrase-file", type=Path, + help="optional protected file containing the signing-key passphrase; never pass it as a value") + command.add_argument("--date", type=int, help="metadata/signature creation time as Unix epoch seconds (default: now)") + command.add_argument("--valid-for-days", type=int, default=30) + verify = commands.add_parser("verify", help="verify the inventory and complete pinned APT trust chain") + verify.add_argument("--repository", type=Path, required=True) + verify.add_argument("--public-key", type=Path, required=True, help="independently trusted ASCII-armored public key") + verify.add_argument("--fingerprint", help="expected uppercase primary fingerprint (otherwise derived from trusted public key)") + verify.add_argument("--now", type=int, help="explicit verification time for fixture or historical-snapshot validation") + args = parser.parse_args() + if args.command == "verify": + manifest = verify_repository(args.repository, args.public_key, args.fingerprint, args.now) + else: + manifest = write_candidate(args, rollback=args.command == "rollback") + print(json.dumps({key: manifest[key] for key in + ("revision", "signingFingerprint", "createdAt", "validUntil", "suites")}, sort_keys=True)) + + +if __name__ == "__main__": + try: + main() + except (RepositoryError, OSError, ValueError, KeyError, TypeError, EOFError, OverflowError) as error: + print(f"apt-repository: {error}", file=sys.stderr) + sys.exit(1) diff --git a/scripts/native-package-vm.sh b/scripts/native-package-vm.sh index 0378cda..3f41104 100755 --- a/scripts/native-package-vm.sh +++ b/scripts/native-package-vm.sh @@ -4,6 +4,8 @@ set -euo pipefail root="$(git rev-parse --show-toplevel 2>/dev/null || true)" manifest="${LOOPWIRE_NATIVE_VM_TARGETS:-packaging/vm/native-package-targets.tsv}" vm_root="${LOOPWIRE_NATIVE_VM_ROOT:-.vm/native-packages}" +image_root="$vm_root" +proof_kind="native" qemu_image="${LOOPWIRE_QEMU_IMAGE:-loopwire-native-package-qemu:ubuntu-24.04}" ssh_user="loopwire" active_vm_container="" @@ -21,15 +23,19 @@ Usage: native-package-vm.sh run-all --version VERSION --release-dir DIR native-package-vm.sh verify --target TARGET [--git-head COMMIT] native-package-vm.sh verify-all [--git-head COMMIT] + native-package-vm.sh run-apt --target ubuntu-24.04|debian-13 --version VERSION --release-dir DIR + native-package-vm.sh verify-apt --target ubuntu-24.04|debian-13 [--git-head COMMIT] Environment: LOOPWIRE_NATIVE_VM_ROOT Cache/run/evidence root (default: .vm/native-packages) + LOOPWIRE_APT_VM_ROOT APT run/evidence root (default: .vm/apt-repository) LOOPWIRE_NATIVE_VM_TARGETS Target manifest override LOOPWIRE_QEMU_IMAGE Docker QEMU tool image tag The host needs Docker, OpenSSH, /dev/kvm access, and enough disk for the official cloud images. Containers only provide QEMU tools; every proof is collected from -a separately booted guest kernel and checked by verify-native-package-vm-proof.mjs. +a separately booted guest kernel. APT proofs use verify-apt-repository-vm-proof.mjs; +the original native-package proofs use verify-native-package-vm-proof.mjs. USAGE } @@ -113,7 +119,7 @@ image_path_for() { local id="$1" url="$2" local suffix="${url##*.}" case "$suffix" in img | qcow2) ;; *) suffix="qcow2" ;; esac - printf '%s/images/%s.%s\n' "$vm_root" "$id" "$suffix" + printf '%s/images/%s.%s\n' "$image_root" "$id" "$suffix" } download_target() { @@ -262,6 +268,10 @@ run_target() { evidence_parent="$(realpath -m "$vm_root/evidence/$id")" evidence_dir="$evidence_parent/$git_head" container="loopwire-native-$id" + if [ "$proof_kind" = "apt" ]; then + container="loopwire-apt-$id" + port=$((port + 10)) + fi key="$(ensure_ssh_key)" public_key="$(cat "${key}.pub")" known_hosts="$target_dir/known_hosts" @@ -306,8 +316,17 @@ run_target() { mapfile -d '' -t ssh_args < <(ssh_args_for "$key" "$port" "$known_hosts") ssh "${ssh_args[@]}" "$ssh_user@127.0.0.1" 'rm -rf /home/loopwire/loopwire-native-kit' copy_to_guest "$key" "$port" "$known_hosts" "$target_dir/kit" '/home/loopwire/loopwire-native-kit' + local guest_script="packaging/vm/guest-native-package-smoke.sh" + local verifier="scripts/verify-native-package-vm-proof.mjs" + if [ "$proof_kind" = "apt" ]; then + guest_script="packaging/vm/guest-apt-repository-smoke.sh" + verifier="scripts/verify-apt-repository-vm-proof.mjs" + fi + local guest_status=0 + # Script paths are fixed and all client-expanded arguments are validated above. + # shellcheck disable=SC2029 ssh "${ssh_args[@]}" "$ssh_user@127.0.0.1" \ - "cd /home/loopwire/loopwire-native-kit && bash packaging/vm/guest-native-package-smoke.sh '$id' '$package_target' '$format' '$version' '$git_head' \"\$PWD\"" + "cd /home/loopwire/loopwire-native-kit && bash '$guest_script' '$id' '$package_target' '$format' '$version' '$git_head' \"\$PWD\"" || guest_status=$? mkdir -p "$evidence_dir" ssh "${ssh_args[@]}" "$ssh_user@127.0.0.1" \ 'tar -C /home/loopwire/loopwire-native-kit/proof -cf - .' | tar -xf - -C "$evidence_dir" @@ -322,13 +341,16 @@ run_target() { checksum "$checksum" \ actual_checksum "$(checksum_file "$algorithm" "$image_path")" \ firmware "$firmware" >"$evidence_dir/image.tsv" - node scripts/verify-native-package-vm-proof.mjs \ + [ "$guest_status" -eq 0 ] || fail "$id guest smoke failed ($guest_status); evidence: $evidence_dir" + node "$verifier" \ --target "$id" --evidence-dir "$evidence_dir" --git-head "$git_head" cleanup_active_vm active_vm_container="" active_vm_console="" trap - EXIT INT TERM - echo "Verified native package in matching KVM guest: $id" + local proof_label="native package" + [ "$proof_kind" != "apt" ] || proof_label="APT repository lifecycle" + echo "Verified $proof_label in matching KVM guest: $id" echo "Evidence: $evidence_dir" } @@ -336,7 +358,9 @@ verify_target() { local selected="$1" git_head="$2" validate_target_value "$selected" [ -n "$git_head" ] || git_head="$(git rev-parse HEAD)" - node scripts/verify-native-package-vm-proof.mjs \ + local verifier="scripts/verify-native-package-vm-proof.mjs" + [ "$proof_kind" != "apt" ] || verifier="scripts/verify-apt-repository-vm-proof.mjs" + node "$verifier" \ --target "$selected" --evidence-dir "$vm_root/evidence/$selected/$git_head" --git-head "$git_head" } @@ -366,6 +390,16 @@ while [ "$#" -gt 0 ]; do esac done +case "$command" in + run-apt | verify-apt) + case "$selected" in ubuntu-24.04 | debian-13) ;; *) fail "$command requires an Ubuntu 24.04 or Debian 13 target" ;; esac + proof_kind="apt" + vm_root="${LOOPWIRE_APT_VM_ROOT:-.vm/apt-repository}" + [ "$(realpath -m "$vm_root")" != "$(realpath -m "$image_root")" ] || + fail "APT VM state must use a different root from native-package state" + ;; +esac + case "$command" in list) printf '%-22s %-24s %-5s %-5s\n' TARGET DISTRO TYPE PORT @@ -382,7 +416,7 @@ case "$command" in require_host while read -r id; do download_target "$id"; done < <(target_ids) ;; - run) + run | run-apt) [ -n "$selected" ] || fail "run requires --target" [ -n "$version" ] || fail "run requires --version" [ -n "$release_dir" ] || fail "run requires --release-dir" @@ -393,7 +427,7 @@ case "$command" in [ -n "$release_dir" ] || fail "run-all requires --release-dir" while read -r id; do run_target "$id" "$version" "$release_dir"; done < <(target_ids) ;; - verify) + verify | verify-apt) [ -n "$selected" ] || fail "verify requires --target" verify_target "$selected" "$git_head" ;; diff --git a/scripts/publish-apt-workflow.sh b/scripts/publish-apt-workflow.sh new file mode 100755 index 0000000..3787cc8 --- /dev/null +++ b/scripts/publish-apt-workflow.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# Entrypoint for protected GitHub release publication and metadata maintenance. +set -euo pipefail + +fail() { printf 'publish-apt-workflow: %s\n' "$*" >&2; exit 1; } +for name in APT_REPOSITORY_URL APT_REPOSITORY_HOST APT_REPOSITORY_ROOT APT_SIGNING_FINGERPRINT \ + APT_SSH_PRIVATE_KEY APT_SSH_KNOWN_HOSTS APT_SIGNING_KEY RUNNER_TEMP GITHUB_REPOSITORY GITHUB_SERVER_URL GITHUB_RUN_ID; do + [ -n "${!name:-}" ] || fail "missing configuration: $name" +done +operation="${OPERATION:-refresh}" +case "$operation" in + publish) [[ "${RELEASE_TAG:-}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || fail "publish requires a stable vX.Y.Z release tag" ;; + refresh) ;; + rollback) [[ "${ROLLBACK_REVISION:-}" =~ ^[a-f0-9]{64}$ ]] || fail "rollback requires a retained revision SHA-256" ;; + *) fail "unknown operation" ;; +esac +[[ "$APT_SIGNING_FINGERPRINT" =~ ^[A-F0-9]{40}$ ]] || fail "signing fingerprint must be complete uppercase hexadecimal" +[[ "${APT_SSH_PORT:-22}" =~ ^[0-9]+$ ]] || fail "SSH port must be numeric" +python3 - "$APT_REPOSITORY_URL" <<'PY' +import runpy, sys +validate = runpy.run_path('scripts/verify-apt-public.py')['validate_base_url'] +try: + validate(sys.argv[1]) +except ValueError as error: + sys.exit(f'publish-apt-workflow: {error}') +PY + +work="$(mktemp -d "$RUNNER_TEMP/loopwire-apt.XXXXXX")" +cleanup() { + gpgconf --homedir "$work/gnupg" --kill all >/dev/null 2>&1 || true + rm -rf -- "$work" +} +trap cleanup EXIT +umask 077 +mkdir "$work/gnupg" +printf '%s\n' "$APT_SSH_PRIVATE_KEY" >"$work/ssh-key" +printf '%s\n' "$APT_SSH_KNOWN_HOSTS" >"$work/known-hosts" +printf '%s\n' "$APT_SIGNING_KEY" >"$work/signing-key.asc" +gpg --no-options --batch --homedir "$work/gnupg" --import "$work/signing-key.asc" +gpg --no-options --batch --homedir "$work/gnupg" --armor --export "$APT_SIGNING_FINGERPRINT" >"$work/public-key.asc" +[ -s "$work/public-key.asc" ] || fail "configured key does not match the expected fingerprint" +sign_args=(--signing-key "$APT_SIGNING_FINGERPRINT" --gnupg-home "$work/gnupg" --valid-for-days 30) +if [ -n "${APT_SIGNING_PASSPHRASE:-}" ]; then + printf '%s' "$APT_SIGNING_PASSPHRASE" >"$work/passphrase" + sign_args+=(--passphrase-file "$work/passphrase") +fi +unset APT_SSH_PRIVATE_KEY APT_SSH_KNOWN_HOSTS APT_SIGNING_KEY APT_SIGNING_PASSPHRASE +transport=(--root "$APT_REPOSITORY_ROOT" --ssh "$APT_REPOSITORY_HOST" --ssh-port "${APT_SSH_PORT:-22}" + --identity-file "$work/ssh-key" --known-hosts "$work/known-hosts" + --public-key "$work/public-key.asc" --fingerprint "$APT_SIGNING_FINGERPRINT") + +set +e +python3 scripts/publish-package-repository.py fetch "${transport[@]}" --output "$work/current" +fetch_status=$? +set -e +previous_args=() +if [ "$fetch_status" -eq 0 ]; then + expected="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["revision"])' \ + "$work/current/repository-manifest.json")" + previous_args=(--previous "$work/current") +elif [ "$fetch_status" -eq 3 ] && [ "$operation" = publish ]; then + expected=empty +else + fail "could not load the existing repository (status $fetch_status); no publication attempted" +fi + +case "$operation" in + publish) + gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" --json isDraft,isPrerelease,tagName >"$work/release.json" + python3 - "$work/release.json" "$RELEASE_TAG" <<'PY' +import json, sys +release = json.load(open(sys.argv[1])) +if release['isDraft'] or release['isPrerelease'] or release['tagName'] != sys.argv[2]: + sys.exit('APT publication requires the requested published stable release') +PY + mkdir "$work/release" + gh release download "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" --dir "$work/release" + release_commit="$(git rev-parse --verify "refs/tags/${RELEASE_TAG}^{commit}")" + bash scripts/verify-release-signature.sh --release-dir "$work/release" --public-key packaging/release-signing-public.pem + node scripts/release-asset-manifest.mjs verify --release-dir "$work/release" --tag "$RELEASE_TAG" \ + --git-head "$release_commit" --require-checksum --require-evidence + python3 scripts/apt-repository.py build --release-dir "$work/release" --version "${RELEASE_TAG#v}" \ + --output "$work/candidate" "${sign_args[@]}" "${previous_args[@]}" + ;; + refresh) + python3 scripts/apt-repository.py rollback --repository "$work/current" --output "$work/candidate" "${sign_args[@]}" + ;; + rollback) + python3 scripts/publish-package-repository.py fetch "${transport[@]}" --revision "$ROLLBACK_REVISION" \ + --output "$work/rollback" + python3 scripts/apt-repository.py rollback --repository "$work/rollback" --output "$work/candidate" "${sign_args[@]}" + ;; +esac + +mkdir -p dist/apt-publication +python3 scripts/publish-package-repository.py publish "${transport[@]}" --repository "$work/candidate" \ + --expected-revision "$expected" >dist/apt-publication/publication.json +python3 scripts/verify-apt-public.py --repository "$work/candidate" --public-key "$work/public-key.asc" \ + --fingerprint "$APT_SIGNING_FINGERPRINT" --base-url "$APT_REPOSITORY_URL" \ + --proof-url "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \ + --output dist/apt-publication/apt-channel.json +cp "$work/candidate/repository-manifest.json" dist/apt-publication/repository-manifest.json +printf 'APT repository published and verified; activation record is in the workflow artifact.\n' diff --git a/scripts/publish-package-repository.py b/scripts/publish-package-repository.py new file mode 100644 index 0000000..f72bf04 --- /dev/null +++ b/scripts/publish-package-repository.py @@ -0,0 +1,621 @@ +#!/usr/bin/env python3 +"""Publish verified APT trees to a POSIX origin; no third-party Python modules. + +ROOT/public is the HTTP document root. ROOT/snapshots and ROOT/state are private. +Snapshots and pool/by-hash URLs are retained indefinitely. Each suite's InRelease +is an independent atomic commit point, not a transaction across suites. The +durable pending journal must be completed before any competing publication. + +The SSH transport runs this same source with Python on the origin. All arguments +are encoded data, host keys must already be trusted, and no credentials are sent +in arguments or output. Signature verification happens on the client; the +authenticated transport and origin independently check the full file inventory. +The origin needs Python 3, SSH, and a dedicated account with exclusive ownership +of ROOT on a filesystem implementing flock, fsync, and same-directory rename. +""" + +import argparse +import base64 +import contextlib +import fcntl +import hashlib +import json +import os +from pathlib import Path, PurePosixPath +import re +import shlex +import shutil +import stat +import subprocess +import sys +import tarfile +import tempfile +import time + + +SUITES = ("debian-13", "ubuntu-24.04") +MANIFEST = "repository-manifest.json" +REVISION = re.compile(r"[0-9a-f]{64}\Z") +FINGERPRINT = re.compile(r"(?:[A-F0-9]{40}|[A-F0-9]{64})\Z") + + +class PublicationError(Exception): + """An actionable publication failure, with no private transport details.""" + + +class EmptyRepository(PublicationError): + """No committed snapshot exists (distinct exit status 3).""" + + +def require(condition, message): + if not condition: + raise PublicationError(message) + + +def canonical(value): + return json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8") + + +def digest(path): + with path.open("rb") as source: + return hashlib.file_digest(source, "sha256").hexdigest() if hasattr(hashlib, "file_digest") else _digest_stream(source) + + +def _digest_stream(source): + result = hashlib.sha256() + for chunk in iter(lambda: source.read(1024 * 1024), b""): + result.update(chunk) + return result.hexdigest() + + +def safe_path(value): + require(isinstance(value, str) and value and "\\" not in value, + "inventory contains an invalid path") + path = PurePosixPath(value) + require(not path.is_absolute() and path.as_posix() == value + and all(part not in (".", "..") for part in path.parts), + "inventory path must be normalized and relative") + return path + + +def classify(path): + """Derive ownership from the protocol, never from an arbitrary manifest kind.""" + safe_path(path) + if re.fullmatch(r"pool/(ubuntu-24\.04|debian-13)/main/l/loopwire/[A-Za-z0-9][A-Za-z0-9.+_~-]*\.deb", path): + return "immutable" + if re.fullmatch(r"keys/(?:[A-F0-9]{40}|[A-F0-9]{64})\.asc", path): + return "immutable" + if re.fullmatch(r"dists/(ubuntu-24\.04|debian-13)/main/binary-amd64/by-hash/SHA256/[0-9a-f]{64}", path): + return "immutable" + if re.fullmatch(r"dists/(ubuntu-24\.04|debian-13)/(InRelease|Release|main/binary-amd64/Packages(?:\.gz)?)", path): + return "metadata" + raise PublicationError("inventory contains a path outside the APT protocol") + + +def plain_path(path, directory=False, missing=False): + """Reject symlinks in every ancestor, including deployment/output roots.""" + path = Path(path).absolute() + require(".." not in path.parts, "paths must not contain parent traversal") + for item in reversed((path, *path.parents)): + try: + mode = item.lstat().st_mode + except FileNotFoundError: + if missing: + continue + raise PublicationError("required path does not exist") from None + require(not stat.S_ISLNK(mode), "symlinks are forbidden in repository paths") + if item != path or directory: + require(stat.S_ISDIR(mode), "repository ancestor is not a directory") + return path + + +def tree_files(root): + plain_path(root, directory=True) + result = set() + for directory, dirs, files in os.walk(root, followlinks=False): + for name in dirs + files: + item = Path(directory) / name + info = item.lstat() + require(not stat.S_ISLNK(info.st_mode), "candidate contains a symlink") + if name in dirs: + require(stat.S_ISDIR(info.st_mode), "candidate contains a non-directory") + else: + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, + "candidate contains a nonregular file or hardlink") + result.add(item.relative_to(root).as_posix()) + return result + + +def inventory(root, fingerprint): + root = plain_path(root, directory=True) + actual = tree_files(root) + require(MANIFEST in actual, "candidate is missing its manifest") + manifest = read_json(root / MANIFEST) + require(isinstance(manifest, dict) and manifest.get("schemaVersion") == 1, + "unsupported repository manifest") + revision = manifest.get("revision") + require(isinstance(revision, str) and REVISION.fullmatch(revision), "invalid candidate revision") + unsigned = {key: value for key, value in manifest.items() if key != "revision"} + require(hashlib.sha256(canonical(unsigned)).hexdigest() == revision, + "candidate revision does not match its manifest") + require(manifest.get("signingFingerprint") == fingerprint, "candidate signing fingerprint differs") + require(isinstance(manifest.get("files"), list), "candidate inventory must be a list") + expected = {MANIFEST} + for entry in manifest["files"]: + require(isinstance(entry, dict), "invalid candidate file entry") + path = entry.get("path") + kind = classify(path) + require(path not in expected and entry.get("kind") == kind, + "duplicate file or incorrect inventory kind") + require(type(entry.get("size")) is int and entry["size"] >= 0, + "invalid inventory file size") + require(isinstance(entry.get("sha256"), str) and REVISION.fullmatch(entry["sha256"]), + "invalid inventory digest") + target = root / path + require(path in actual and target.stat().st_size == entry["size"] + and digest(target) == entry["sha256"], "candidate file checksum or size differs") + expected.add(path) + require(actual == expected, "candidate contains unlisted files") + require({suite.get("name") for suite in manifest.get("suites", [])} == set(SUITES), + "candidate must contain both supported suites") + for suite in SUITES: + for suffix in ("Release", "InRelease", "main/binary-amd64/Packages", "main/binary-amd64/Packages.gz"): + require(f"dists/{suite}/{suffix}" in expected, "candidate is missing required suite metadata") + require(f"keys/{fingerprint}.asc" in expected, "candidate is missing its public key") + return manifest + + +def verify_signed(root, key, fingerprint, historical=False): + manifest = inventory(root, fingerprint) + verifier = Path(__file__).resolve().with_name("apt-repository.py") + require(verifier.is_file(), "apt-repository.py verifier is missing") + command = [sys.executable, str(verifier), "verify", "--repository", str(root), + "--public-key", str(key), "--fingerprint", fingerprint] + if historical: + require(type(manifest.get("createdAt")) is int, "invalid repository creation time") + command += ["--now", str(manifest["createdAt"])] + result = subprocess.run(command, capture_output=True, text=True, check=False) + require(result.returncode == 0, "signed repository verification failed; run apt-repository.py verify for diagnostics") + return manifest + + +def fsync_directory(path): + descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW) + try: + os.fsync(descriptor) + finally: + os.close(descriptor) + + +def make_directory(path, mode=0o755): + path = plain_path(path, directory=True, missing=True) + if path.exists(): + return + make_directory(path.parent, mode=mode) + path.mkdir(mode=mode) + descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW) + try: + # mkdir applies the SSH/workflow umask; set our intended mode only on + # newly created directories, preserving operator-provisioned ancestors. + os.fchmod(descriptor, mode) + os.fsync(descriptor) + finally: + os.close(descriptor) + fsync_directory(path.parent) + + +def atomic_write(target, source=None, data=None, mode=0o644, directory_mode=0o755): + plain_path(target, missing=True) + make_directory(target.parent, mode=directory_mode) + descriptor, temporary = tempfile.mkstemp(prefix=".upload-", dir=target.parent) + try: + with os.fdopen(descriptor, "wb") as output: + if source is not None: + with source.open("rb") as incoming: + shutil.copyfileobj(incoming, output, 1024 * 1024) + else: + output.write(data) + output.flush() + os.fchmod(output.fileno(), mode) + os.fsync(output.fileno()) + os.replace(temporary, target) + fsync_directory(target.parent) + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +def read_json(path): + plain_path(path) + try: + with path.open(encoding="utf-8") as source: + return json.load(source) + except (ValueError, UnicodeError) as error: + raise PublicationError("invalid repository JSON") from error + + +def root_path(value): + path = Path(value) + require(path.is_absolute() and path != Path("/"), "--root must be an absolute, non-root directory") + return plain_path(path, directory=True, missing=True) + + +@contextlib.contextmanager +def locked(root, create=False): + if create: + make_directory(root) + if not root.exists(): + raise EmptyRepository("repository has no committed snapshot") + lock = root / ".publish.lock" + plain_path(lock, missing=True) + if not create and not lock.exists(): + require(not (root / "state").exists() and not (root / "public").exists(), + "repository state exists without its publication lock") + raise EmptyRepository("repository has no committed snapshot") + descriptor = os.open(lock, os.O_NOFOLLOW | (os.O_RDWR | os.O_CREAT if create else os.O_RDONLY), 0o600) + try: + info = os.fstat(descriptor) + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, "invalid publication lock file") + try: + fcntl.flock(descriptor, (fcntl.LOCK_EX if create else fcntl.LOCK_SH) | fcntl.LOCK_NB) + except BlockingIOError: + raise PublicationError("repository is locked by another operation; retry later") from None + yield + finally: + os.close(descriptor) + + +def state(root, name): + path = root / "state" / f"{name}.json" + plain_path(path, missing=True) + if not path.exists(): + return None + record = read_json(path) + require(isinstance(record, dict) and isinstance(record.get("revision"), str) + and REVISION.fullmatch(record["revision"]), "invalid publication state") + return record + + +def _checkpoint(label): + """No-op hook for process-interruption tests; never controlled by environment.""" + + +def check_public(root, manifest, immutable_only=False): + for entry in manifest["files"]: + if immutable_only and entry["kind"] != "immutable": + continue + target = root / "public" / entry["path"] + plain_path(target, missing=True) + if target.exists(): + info = target.stat() + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, + "public target is not a standalone regular file") + require(info.st_size == entry["size"] and digest(target) == entry["sha256"], + "immutable URL collision" if immutable_only else "committed public repository has drifted") + elif not immutable_only: + raise PublicationError("committed public repository is missing files") + + +def save_snapshot(root, repository, manifest): + snapshots = root / "snapshots" + make_directory(snapshots, mode=0o700) + snapshot = snapshots / manifest["revision"] + plain_path(snapshot, directory=True, missing=True) + if snapshot.exists(): + require(inventory(snapshot, manifest["signingFingerprint"]) == manifest, + "retained snapshot differs from candidate") + return snapshot + temporary = Path(tempfile.mkdtemp(prefix=".staging-", dir=snapshots)) + try: + for entry in manifest["files"]: + atomic_write(temporary / entry["path"], source=repository / entry["path"], + mode=0o600, directory_mode=0o700) + atomic_write(temporary / MANIFEST, source=repository / MANIFEST, mode=0o600, directory_mode=0o700) + inventory(temporary, manifest["signingFingerprint"]) + fsync_directory(temporary) + os.replace(temporary, snapshot) + fsync_directory(snapshots) + finally: + if temporary.exists(): + shutil.rmtree(temporary) + return snapshot + + +def promote(root, snapshot, manifest): + """Resumable order: all immutable objects, metadata, per-suite InRelease.""" + check_public(root, manifest, immutable_only=True) + for entry in manifest["files"]: + if entry["kind"] == "immutable": + target = root / "public" / entry["path"] + if not target.exists(): + atomic_write(target, source=snapshot / entry["path"]) + _checkpoint("immutable") + for suite in SUITES: + prefix = f"dists/{suite}/" + for entry in manifest["files"]: + if entry["kind"] == "metadata" and entry["path"].startswith(prefix) and not entry["path"].endswith("/InRelease"): + atomic_write(root / "public" / entry["path"], source=snapshot / entry["path"]) + _checkpoint("metadata:" + suite) + relative = f"dists/{suite}/InRelease" + atomic_write(root / "public" / relative, source=snapshot / relative) + _checkpoint("committed:" + suite) + atomic_write(root / "public" / MANIFEST, source=snapshot / MANIFEST) + check_public(root, manifest) + atomic_write(root / "state" / "current.json", data=canonical({"revision": manifest["revision"]}) + b"\n", mode=0o600) + _checkpoint("current") + (root / "state" / "pending.json").unlink() + fsync_directory(root / "state") + return {"status": "published", "revision": manifest["revision"], "suites": list(SUITES)} + + +def publish_at(root, repository, fingerprint, expected): + manifest = inventory(repository, fingerprint) + with locked(root, create=True): + current = state(root, "current") + pending = state(root, "pending") + revision = current["revision"] if current else "empty" + if pending: + require(pending["revision"] == manifest["revision"], + "interrupted publication pending; recover it before publishing another revision") + require(expected == pending.get("previousRevision"), "expected revision differs from interrupted publication") + require(revision in (pending["previousRevision"], pending["revision"]), "current revision conflicts with pending journal") + snapshot = root / "snapshots" / pending["revision"] + require(inventory(snapshot, fingerprint) == manifest, "pending snapshot differs from candidate") + return promote(root, snapshot, manifest) + if revision == manifest["revision"]: + check_public(root, manifest) + return {"status": "unchanged", "revision": revision, "suites": list(SUITES)} + require(expected == revision, "expected revision differs from current publication (compare-and-swap failed)") + if current is None: + public = root / "public" + plain_path(public, directory=True, missing=True) + require(not public.exists() or not any(public.iterdir()), "refusing to adopt an unmanaged public repository") + check_public(root, manifest, immutable_only=True) + snapshot = save_snapshot(root, repository, manifest) + make_directory(root / "state", mode=0o700) + atomic_write(root / "state" / "pending.json", data=canonical({ + "revision": manifest["revision"], "previousRevision": revision, + }) + b"\n", mode=0o600) + _checkpoint("journal") + return promote(root, snapshot, manifest) + + +def recover_at(root, fingerprint, revision): + with locked(root, create=True): + pending = state(root, "pending") + require(pending is not None and pending["revision"] == revision, + "pending publication changed; fetch and verify it again before recovery") + current = state(root, "current") + require((current["revision"] if current else "empty") in (pending.get("previousRevision"), revision), + "current revision conflicts with pending journal") + snapshot = root / "snapshots" / revision + return promote(root, snapshot, inventory(snapshot, fingerprint)) + + +@contextlib.contextmanager +def selected_snapshot(root, fingerprint, revision=None, pending_only=False): + with locked(root): + pending = state(root, "pending") + if pending_only: + if not pending: + raise EmptyRepository("repository has no pending publication") + revision = pending["revision"] + else: + require(pending is None, "interrupted publication pending; recover before fetching snapshots") + if revision is None: + current = state(root, "current") + if not current: + raise EmptyRepository("repository has no committed snapshot") + revision = current["revision"] + snapshot = root / "snapshots" / revision + manifest = inventory(snapshot, fingerprint) + require(manifest["revision"] == revision, "snapshot does not match selected revision") + yield snapshot, manifest + + +def write_archive(repository, output): + with tarfile.open(fileobj=output, mode="w|") as archive: + for relative in sorted(tree_files(repository)): + archive.add(repository / relative, arcname=relative, recursive=False) + + +def read_archive(source, output): + seen = set() + with tarfile.open(fileobj=source, mode="r|*") as archive: + for member in archive: + safe_path(member.name) + require(member.name == MANIFEST or classify(member.name), "unexpected archive path") + require(member.isfile() and not member.issym() and not member.islnk() + and member.name not in seen, "archive contains a link, special file, or duplicate") + seen.add(member.name) + target = output / member.name + make_directory(target.parent) + with archive.extractfile(member) as incoming, target.open("xb") as destination: + shutil.copyfileobj(incoming, destination, 1024 * 1024) + + +def ssh_command(args, request): + require(re.fullmatch(r"[A-Za-z0-9_][A-Za-z0-9_.-]*@[A-Za-z0-9][A-Za-z0-9_.-]*", args.ssh), + "--ssh must be USER@HOST using an SSH alias, hostname, or IPv4 address") + command = ["ssh", "-T", "-o", "BatchMode=yes", "-o", "StrictHostKeyChecking=yes", + "-o", "PasswordAuthentication=no", "-o", "KbdInteractiveAuthentication=no", + "-o", "ForwardAgent=no", "-o", "ClearAllForwardings=yes", "-o", "ConnectTimeout=15"] + if args.known_hosts: + key_file = plain_path(args.known_hosts) + require(key_file.is_file(), "--known-hosts must name a regular file") + command += ["-o", f"UserKnownHostsFile={key_file}"] + if args.ssh_port: + command += ["-p", str(args.ssh_port)] + if args.identity_file: + identity = plain_path(args.identity_file) + require(identity.is_file(), "--identity-file must name a regular file") + command += ["-i", str(identity), "-o", "IdentitiesOnly=yes"] + encoded = base64.urlsafe_b64encode(canonical(request)).decode("ascii") + source = Path(__file__).read_text(encoding="utf-8") + remote = "python3 -c " + shlex.quote(source) + " _serve " + shlex.quote(encoded) + return command + ["--", args.ssh, remote] + + +def remote_call(args, request, repository=None, output=None): + command = ssh_command(args, request) + with tempfile.TemporaryFile() as incoming, tempfile.TemporaryFile() as outgoing: + if repository: + write_archive(repository, incoming) + incoming.seek(0) + result = subprocess.run(command, stdin=incoming, stdout=outgoing, stderr=subprocess.PIPE, check=False) + if result.returncode == 3: + raise EmptyRepository("remote repository has no selected snapshot") + if result.returncode == 1: + # Forward only the server's deliberately sanitized structured error. + # SSH diagnostics can contain hostnames/paths and stay private. + try: + failure = json.loads(result.stderr) + if failure.get("status") == "error" and isinstance(failure.get("message"), str): + raise PublicationError(failure["message"]) + except (ValueError, AttributeError): + pass + require(result.returncode == 0, + "SSH repository operation failed; check trusted host keys, connectivity, remote Python, and publisher state") + outgoing.seek(0) + if output: + read_archive(outgoing, output) + return None + try: + return json.load(outgoing) + except (ValueError, UnicodeError) as error: + raise PublicationError("SSH repository response is not valid JSON") from error + + +def serve(request): + root = root_path(request["root"]) + fingerprint = request["fingerprint"] + require(FINGERPRINT.fullmatch(fingerprint), "invalid signing fingerprint") + action = request["action"] + if action == "publish": + require(request["expected"] == "empty" or REVISION.fullmatch(request["expected"]), "invalid expected revision") + with tempfile.TemporaryDirectory(prefix="loopwire-apt-upload-") as directory: + repository = Path(directory) + read_archive(sys.stdin.buffer, repository) + return publish_at(root, repository, fingerprint, request["expected"]) + if action == "recover": + require(REVISION.fullmatch(request["revision"]), "invalid recovery revision") + return recover_at(root, fingerprint, request["revision"]) + require(action in ("fetch", "fetch-pending"), "unknown remote operation") + revision = request.get("revision") + require(revision is None or REVISION.fullmatch(revision), "invalid fetch revision") + with selected_snapshot(root, fingerprint, revision, action == "fetch-pending") as (snapshot, _): + write_archive(snapshot, sys.stdout.buffer) + return None + + +def fetch_into(args, output, pending_only=False): + if args.ssh: + remote_call(args, {"action": "fetch-pending" if pending_only else "fetch", "root": args.root, + "fingerprint": args.fingerprint, "revision": getattr(args, "revision", None)}, output=output) + else: + with selected_snapshot(root_path(args.root), args.fingerprint, + getattr(args, "revision", None), pending_only) as (snapshot, _): + shutil.copytree(snapshot, output, dirs_exist_ok=True) + return verify_signed(output, args.public_key, args.fingerprint, + historical=not pending_only or getattr(args, "allow_expired", False)) + + +def parser(): + result = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + actions = result.add_subparsers(dest="action", required=True) + for name in ("publish", "fetch", "recover"): + action = actions.add_parser(name) + action.add_argument("--root", required=True, help="absolute origin root; HTTP serves ROOT/public") + action.add_argument("--public-key", required=True, type=Path) + action.add_argument("--fingerprint", required=True) + action.add_argument("--ssh", metavar="USER@HOST", help="omit for a local POSIX origin") + action.add_argument("--ssh-port", type=int) + action.add_argument("--identity-file", type=Path, help="existing SSH private key; alternatively use the caller's agent") + action.add_argument("--known-hosts", type=Path, help="pre-provisioned known_hosts; strict checking is mandatory") + if name == "publish": + action.add_argument("--repository", type=Path, required=True) + action.add_argument("--expected-revision", required=True, help="observed revision, or literal empty for first publication") + action.add_argument("--dry-run", action="store_true", help="verify locally; perform no origin access or upload") + elif name == "fetch": + action.add_argument("--output", type=Path, required=True, help="new destination directory (must not exist)") + action.add_argument("--revision", help="retained snapshot revision; defaults to committed current") + else: + action.add_argument("--dry-run", action="store_true", help="fetch and verify pending snapshot without promotion") + action.add_argument("--allow-expired", action="store_true", + help="finish an expired signed journal, then immediately fetch, re-sign, and publish fresh metadata") + return result + + +def run(args): + root_path(args.root) if not args.ssh else require(Path(args.root).is_absolute() and args.root != "/" + and ".." not in Path(args.root).parts, + "--root must be an absolute normalized non-root path") + require(FINGERPRINT.fullmatch(args.fingerprint), "--fingerprint must be a full uppercase fingerprint") + require(args.ssh_port is None or 1 <= args.ssh_port <= 65535, "invalid SSH port") + require(args.ssh or (args.ssh_port is None and args.known_hosts is None and args.identity_file is None), + "SSH options require --ssh") + args.public_key = plain_path(args.public_key) + require(args.public_key.is_file(), "--public-key must name a file") + if args.action == "publish": + require(args.expected_revision == "empty" or REVISION.fullmatch(args.expected_revision), "invalid expected revision") + manifest = verify_signed(args.repository, args.public_key, args.fingerprint) + if args.dry_run: + return {"status": "validated", "revision": manifest["revision"], "originChecked": False} + with tempfile.TemporaryDirectory(prefix="loopwire-apt-candidate-") as directory: + candidate = Path(directory) / "repository" + shutil.copytree(args.repository, candidate, symlinks=True) + require(verify_signed(candidate, args.public_key, args.fingerprint) == manifest, + "candidate changed during verification") + if args.ssh: + return remote_call(args, {"action": "publish", "root": args.root, "fingerprint": args.fingerprint, + "expected": args.expected_revision}, repository=candidate) + return publish_at(root_path(args.root), candidate, args.fingerprint, args.expected_revision) + if args.action == "fetch": + require(args.revision is None or REVISION.fullmatch(args.revision), "invalid selected revision") + output = plain_path(args.output, directory=True, missing=True) + require(not output.exists(), "--output must not exist") + require(output.parent.is_dir(), "--output parent must already exist") + with tempfile.TemporaryDirectory(prefix=".loopwire-apt-fetch-", dir=output.parent) as directory: + fetched = Path(directory) / "repository" + fetched.mkdir() + manifest = fetch_into(args, fetched) + os.rename(fetched, output) + fsync_directory(output.parent) + return {"status": "fetched", "revision": manifest["revision"]} + with tempfile.TemporaryDirectory(prefix="loopwire-apt-recovery-") as directory: + manifest = fetch_into(args, Path(directory), pending_only=True) + needs_refresh = manifest["validUntil"] <= int(time.time()) + if args.dry_run: + return {"status": "recovery-validated", "revision": manifest["revision"], "requiresRefresh": needs_refresh} + if args.ssh: + result = remote_call(args, {"action": "recover", "root": args.root, "fingerprint": args.fingerprint, + "revision": manifest["revision"]}) + else: + result = recover_at(root_path(args.root), args.fingerprint, manifest["revision"]) + result["requiresRefresh"] = needs_refresh + if needs_refresh: + result["nextAction"] = "Immediately fetch, re-sign, and publish fresh metadata; APT rejects the expired snapshot." + return result + + +def main(): + try: + if len(sys.argv) == 3 and sys.argv[1] == "_serve": + result = serve(json.loads(base64.urlsafe_b64decode(sys.argv[2]))) + else: + result = run(parser().parse_args()) + if result is not None: + print(json.dumps(result, sort_keys=True)) + return 0 + except EmptyRepository: + print(json.dumps({"status": "empty", "revision": None})) + return 3 + except (PublicationError, OSError, ValueError, KeyError, TypeError, tarfile.TarError) as error: + # OS/transport exceptions can include machine-local paths; do not print them. + message = str(error) if isinstance(error, PublicationError) else "repository operation failed; check filesystem and inputs" + print(json.dumps({"status": "error", "message": message}), file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/setup-apt-repository.sh b/scripts/setup-apt-repository.sh new file mode 100755 index 0000000..6b1a9bb --- /dev/null +++ b/scripts/setup-apt-repository.sh @@ -0,0 +1,167 @@ +#!/usr/bin/env bash +set -euo pipefail + +base_url="" +fingerprint="" +install_root="/" +remove="false" +dry_run="false" + +fail() { printf 'setup-apt-repository: %s\n' "$*" >&2; exit 1; } +usage() { + cat <<'USAGE' +Configure Loopwire's signed APT repository on Ubuntu 24.04 or Debian 13 (amd64). + +Usage: + sudo bash setup-apt-repository.sh --base-url HTTPS_URL --fingerprint OPENPGP_FINGERPRINT + sudo bash setup-apt-repository.sh --remove + bash setup-apt-repository.sh --base-url HTTPS_URL --fingerprint FINGERPRINT --dry-run + +Options: + --root DIR Configure an offline filesystem tree instead of / (including its etc/os-release). + +Obtain the URL and fingerprint from the verified Loopwire channel documentation. +Requires curl, GnuPG, Python 3, and dpkg. Existing unrelated APT sources are preserved. +This only writes the source/keyring configuration. Run apt update and apt install yourself afterward. +USAGE +} +while [ "$#" -gt 0 ]; do + case "$1" in + --base-url) base_url="${2:?missing --base-url value}"; shift 2 ;; + --fingerprint) fingerprint="${2:?missing --fingerprint value}"; shift 2 ;; + --root) install_root="${2:?missing --root value}"; shift 2 ;; + --remove) remove="true"; shift ;; + --dry-run) dry_run="true"; shift ;; + -h|--help) usage; exit 0 ;; + *) fail "unknown option: $1" ;; + esac +done + +install_root="$(realpath -e "$install_root")" +[ -d "$install_root" ] || fail "root must be an existing directory" +source_file="${install_root%/}/etc/apt/sources.list.d/loopwire.sources" +key_directory="${install_root%/}/etc/apt/keyrings" +python3 - "$install_root" "$source_file" "$key_directory" <<'PY' +import sys +from pathlib import Path +root = Path(sys.argv[1]) +for name in sys.argv[2:]: + path = Path(name) + for part in (path, *path.parents): + if part == root: + break + if part.is_symlink(): + sys.exit('setup-apt-repository: refusing symbolic links inside the target APT configuration tree') + if not part.is_relative_to(root): + sys.exit('setup-apt-repository: APT configuration path leaves the target root') +PY +owner_marker="# Managed by Loopwire APT repository setup" +[ ! -L "$source_file" ] || fail "refusing a symbolic-link source file" +if [ -e "$source_file" ] && ! head -n 1 "$source_file" | grep -Fxq "$owner_marker"; then + fail "loopwire.sources already exists and is not managed by this helper" +fi +if [ "$install_root" = / ] && [ "$dry_run" != true ] && [ "$EUID" -ne 0 ]; then + fail "run with sudo to change /etc/apt, or use --dry-run" +fi + +if [ "$remove" = true ]; then + if [ "$dry_run" = true ]; then + printf 'Would remove the managed Loopwire APT source and its keyring.\n' + exit 0 + fi + if [ -f "$source_file" ]; then + key_name="$(sed -n 's|^Signed-By: /etc/apt/keyrings/\(loopwire-[A-F0-9]*\.asc\)$|\1|p' "$source_file")" + [[ "$key_name" =~ ^loopwire-[A-F0-9]{40}\.asc$ ]] || fail "managed source has an unexpected keyring path" + rm -- "$source_file" + rm -f -- "$key_directory/$key_name" + fi + printf 'Loopwire APT source removed. Installed packages and other sources are unchanged.\n' + exit 0 +fi + +for command in curl gpg python3 dpkg; do + command -v "$command" >/dev/null 2>&1 || fail "$command is required" +done +fingerprint="${fingerprint^^}" +[[ "$fingerprint" =~ ^[A-F0-9]{40}$ ]] || fail "provide a complete 40-character OpenPGP fingerprint" +base_url="$(python3 - "$base_url" <<'PY' +import sys +from urllib.parse import urlsplit +value = sys.argv[1] +try: + url = urlsplit(value) + valid = (url.scheme == 'https' and url.hostname and not url.username and not url.password + and not any(char in value for char in "\\'\"`$<>?#") + and all(32 < ord(char) < 127 for char in value)) + if not valid: + raise ValueError('invalid URL') + if url.port is not None and not 1 <= url.port <= 65535: + raise ValueError('invalid port') +except ValueError: + sys.exit('setup-apt-repository: base URL must be HTTPS without credentials, whitespace, query, or fragment') +print(value.rstrip('/')) +PY +)" + +# Read os-release as data; do not execute a file supplied through --root. +suite="$(python3 - "${install_root%/}/etc/os-release" <<'PY' +import shlex +import sys +from pathlib import Path +values = {} +for line in Path(sys.argv[1]).read_text().splitlines(): + if '=' in line and not line.lstrip().startswith('#'): + key, value = line.split('=', 1) + fields = shlex.split(value) + if len(fields) == 1: + values[key] = fields[0] +suite = {('ubuntu', '24.04'): 'ubuntu-24.04', ('debian', '13'): 'debian-13'}.get( + (values.get('ID'), values.get('VERSION_ID'))) +if not suite: + sys.exit('setup-apt-repository: supported systems are Ubuntu 24.04 and Debian 13') +print(suite) +PY +)" +[ "$(dpkg --print-architecture)" = amd64 ] || fail "this channel currently supports amd64 only" +key_name="loopwire-${fingerprint}.asc" +[ ! -L "$key_directory/$key_name" ] || fail "refusing a symbolic-link keyring" +if [ "$dry_run" = true ]; then + printf 'Would verify %s/keys/%s.asc and configure suite %s with a scoped Signed-By keyring.\n' \ + "$base_url" "$fingerprint" "$suite" + exit 0 +fi + +temporary="$(mktemp -d)" +key_temporary="" +source_temporary="" +cleanup() { + rm -rf -- "$temporary" + [ -z "$key_temporary" ] || rm -f -- "$key_temporary" + [ -z "$source_temporary" ] || rm -f -- "$source_temporary" +} +trap cleanup EXIT +mkdir -m 0700 "$temporary/gnupg" +curl --disable --fail --silent --show-error --proto '=https' --tlsv1.2 \ + --connect-timeout 10 --max-time 60 --output "$temporary/key.asc" "$base_url/keys/$fingerprint.asc" +actual="$(gpg --no-options --batch --homedir "$temporary/gnupg" --with-colons --show-keys "$temporary/key.asc" | + awk -F: '$1 == "pub" { count++ } $1 == "fpr" && !seen { print $10; seen=1 } END { if (count != 1) exit 1 }')" +[ "$actual" = "$fingerprint" ] || fail "downloaded key does not match the expected fingerprint" +cat >"$temporary/loopwire.sources" <", "rsa2048", "sign", "1d") + keys = run("gpg", "--homedir", str(cls.home), "--with-colons", "--list-keys").stdout + cls.fingerprint = next(line.split(":")[9] for line in keys.splitlines() if line.startswith("fpr:")) + cls.public = run("gpg", "--homedir", str(cls.home), "--armor", "--export", cls.fingerprint).stdout + cls.web = cls.work / "web" + (cls.web / "keys").mkdir(parents=True) + (cls.web / "keys" / f"{cls.fingerprint}.asc").write_text(cls.public) + (cls.web / "keys" / f"{'A' * 40}.asc").write_text(cls.public) + cert, key = cls.work / "cert.pem", cls.work / "key.pem" + run("openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-days", "1", + "-subj", "/CN=127.0.0.1", "-addext", "subjectAltName=IP:127.0.0.1", + "-keyout", str(key), "-out", str(cert)) + cls.requests = [] + + class Handler(http.server.SimpleHTTPRequestHandler): + def log_message(self, *_args): + pass + + def do_GET(self): + cls.requests.append(self.path) + super().do_GET() + + cls.server = http.server.ThreadingHTTPServer( + ("127.0.0.1", 0), functools.partial(Handler, directory=str(cls.web))) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain(cert, key) + cls.server.socket = context.wrap_socket(cls.server.socket, server_side=True) + cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True) + cls.thread.start() + cls.url = f"https://127.0.0.1:{cls.server.server_port}" + cls.binary = cls.work / "bin" + cls.binary.mkdir() + dpkg = cls.binary / "dpkg" + dpkg.write_text('#!/bin/sh\nprintf "%s\\n" "${TEST_ARCH:-amd64}"\n') + dpkg.chmod(0o755) + cls.environment = {**os.environ, "PATH": f"{cls.binary}:{os.environ['PATH']}", "CURL_CA_BUNDLE": str(cert)} + + @classmethod + def tearDownClass(cls): + cls.server.shutdown() + cls.server.server_close() + subprocess.run(["gpgconf", "--homedir", str(cls.home), "--kill", "all"], check=False, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + cls.temporary.cleanup() + + def setUp(self): + self.root = self.work / "root" + shutil.rmtree(self.root, ignore_errors=True) + (self.root / "etc").mkdir(parents=True) + self.os_release("ubuntu", "24.04") + self.source = self.root / "etc/apt/sources.list.d/loopwire.sources" + self.key = self.root / f"etc/apt/keyrings/loopwire-{self.fingerprint}.asc" + self.requests.clear() + + def os_release(self, identity, version): + (self.root / "etc/os-release").write_text(f'ID={identity}\nVERSION_ID="{version}"\n') + + def invoke(self, *args, fingerprint=None, url=None, env=None): + return subprocess.run([ + "bash", str(SCRIPT), "--root", str(self.root), "--base-url", url or self.url, + "--fingerprint", fingerprint or self.fingerprint, *args, + ], env={**self.environment, **(env or {})}, capture_output=True, text=True) + + def test_supported_suites_and_idempotence(self): + for identity, version, suite in [("ubuntu", "24.04", "ubuntu-24.04"), ("debian", "13", "debian-13")]: + with self.subTest(suite=suite): + self.os_release(identity, version) + result = self.invoke() + self.assertEqual(result.returncode, 0, result.stderr) + self.assertIn(f"Suites: {suite}\n", self.source.read_text()) + self.assertIn(f"Signed-By: /etc/apt/keyrings/loopwire-{self.fingerprint}.asc", self.source.read_text()) + self.assertEqual(self.key.read_text(), self.public) + self.assertEqual(self.key.stat().st_mode & 0o777, 0o644) + source_bytes = self.source.read_bytes() + again = self.invoke() + self.assertEqual(again.returncode, 0, again.stderr) + self.assertEqual(self.source.read_bytes(), source_bytes) + + def test_dry_run_has_no_network_or_writes(self): + result = self.invoke("--dry-run") + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(self.requests, []) + self.assertFalse(self.source.exists()) + self.assertFalse(self.key.exists()) + + def test_wrong_fingerprint_preserves_existing_configuration(self): + self.assertEqual(self.invoke().returncode, 0) + before = self.source.read_bytes() + result = self.invoke(fingerprint="A" * 40) + self.assertNotEqual(result.returncode, 0) + self.assertIn("does not match", result.stderr) + self.assertEqual(self.source.read_bytes(), before) + self.assertEqual(self.key.read_text(), self.public) + + def test_tls_authentication_required(self): + result = self.invoke(env={"CURL_CA_BUNDLE": str(self.work / "absent-ca.pem")}) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(self.source.exists()) + + def test_bad_urls_rejected_without_network(self): + for url in ["http://example.invalid", "https://user:pass@example.invalid", "https://example.invalid/#fragment", + "https://example.invalid/?query=x", "https://example.invalid/\ninjected", "https://example.invalid:99999"]: + with self.subTest(url=url): + self.assertNotEqual(self.invoke(url=url).returncode, 0) + self.assertEqual(self.requests, []) + self.assertFalse(self.source.exists()) + + def test_unsupported_distribution_and_architecture(self): + self.os_release("ubuntu", "22.04") + self.assertNotEqual(self.invoke().returncode, 0) + self.os_release("ubuntu", "24.04") + self.assertNotEqual(self.invoke(env={"TEST_ARCH": "arm64"}).returncode, 0) + self.assertEqual(self.requests, []) + + def test_os_release_is_never_executed(self): + sentinel = self.work / "must-not-exist" + self.os_release(f'"$(touch {sentinel})"', "24.04") + self.assertNotEqual(self.invoke().returncode, 0) + self.assertFalse(sentinel.exists()) + + def test_unmanaged_source_preserved(self): + self.source.parent.mkdir(parents=True) + self.source.write_text("# Maintainer-owned source\n") + self.assertNotEqual(self.invoke().returncode, 0) + self.assertNotEqual(self.invoke("--remove").returncode, 0) + self.assertEqual(self.source.read_text(), "# Maintainer-owned source\n") + + def test_source_symlink_rejected(self): + self.source.parent.mkdir(parents=True) + destination = self.root / "unrelated" + destination.write_text("keep") + self.source.symlink_to(destination) + self.assertNotEqual(self.invoke().returncode, 0) + self.assertEqual(destination.read_text(), "keep") + + def test_symlinked_parent_cannot_escape_offline_root(self): + outside = self.work / "outside" + outside.mkdir(exist_ok=True) + for relative in ["etc/apt", "etc/apt/sources.list.d", "etc/apt/keyrings"]: + with self.subTest(relative=relative): + shutil.rmtree(self.root / "etc/apt", ignore_errors=True) + parent = self.root / relative + parent.parent.mkdir(parents=True, exist_ok=True) + parent.symlink_to(outside, target_is_directory=True) + try: + result = self.invoke() + self.assertNotEqual(result.returncode, 0) + self.assertEqual(list(outside.iterdir()), []) + finally: + parent.unlink(missing_ok=True) + shutil.rmtree(outside) + outside.mkdir() + + def test_remove_is_idempotent_and_preserves_other_sources(self): + self.assertEqual(self.invoke().returncode, 0) + other = self.source.with_name("unrelated.sources") + other.write_text("keep") + for _ in range(2): + result = self.invoke("--remove") + self.assertEqual(result.returncode, 0, result.stderr) + self.assertFalse(self.source.exists()) + self.assertFalse(self.key.exists()) + self.assertEqual(other.read_text(), "keep") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test-apt-public.py b/scripts/test-apt-public.py new file mode 100755 index 0000000..afb7530 --- /dev/null +++ b/scripts/test-apt-public.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +"""Test public byte verification and activation output independently of the signed-chain verifier.""" +import contextlib +import functools +import hashlib +import http.server +import importlib.util +import io +import json +from pathlib import Path +import ssl +import subprocess +import sys +import tempfile +import threading +import unittest +from unittest.mock import patch + +SCRIPT = Path(__file__).with_name("verify-apt-public.py") +spec = importlib.util.spec_from_file_location("apt_public", SCRIPT) +module = importlib.util.module_from_spec(spec) +spec.loader.exec_module(module) + + +class PublicTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.temporary = tempfile.TemporaryDirectory(prefix="loopwire-apt-public-") + cls.root = Path(cls.temporary.name) + cls.web = cls.root / "web" + cls.web.mkdir() + cls.cert, key = cls.root / "cert.pem", cls.root / "key.pem" + subprocess.run([ + "openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-days", "1", + "-subj", "/CN=127.0.0.1", "-addext", "subjectAltName=IP:127.0.0.1", + "-keyout", str(key), "-out", str(cls.cert), + ], check=True, capture_output=True) + cls.requests = [] + + class Handler(http.server.SimpleHTTPRequestHandler): + def log_message(self, *_args): + pass + + def do_GET(self): + cls.requests.append(self.path) + if self.path.startswith("/redirect/"): + self.send_response(302) + self.send_header("Location", "/must-not-follow") + self.end_headers() + else: + super().do_GET() + + cls.server = http.server.ThreadingHTTPServer( + ("127.0.0.1", 0), functools.partial(Handler, directory=str(cls.web))) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain(cls.cert, key) + cls.server.socket = context.wrap_socket(cls.server.socket, server_side=True) + cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True) + cls.thread.start() + cls.url = f"https://127.0.0.1:{cls.server.server_port}" + + @classmethod + def tearDownClass(cls): + cls.server.shutdown() + cls.server.server_close() + cls.temporary.cleanup() + + def setUp(self): + self.requests.clear() + (self.web / "payload").write_bytes(b"expected package bytes") + (self.root / "repository-manifest.json").write_text(json.dumps({ + "revision": "a" * 64, + "files": [{"path": "payload", "size": 22, "sha256": hashlib.sha256(b"expected package bytes").hexdigest()}], + })) + self.output = self.root / "activation.json" + self.output.unlink(missing_ok=True) + + def invoke(self, *extra, verifier_error=None): + args = [str(SCRIPT), "--repository", str(self.root), "--public-key", str(self.root / "trusted.asc"), + "--fingerprint", "A" * 40, "--base-url", self.url] + if "--output" not in extra: + args += ["--ca-file", str(self.cert)] + args += extra + trust = ssl.create_default_context(cafile=str(self.cert)) + with patch.object(sys, "argv", args), patch.object(module.subprocess, "run") as verify, \ + patch.object(module.ssl, "create_default_context", return_value=trust), \ + contextlib.redirect_stdout(io.StringIO()) as output: + if verifier_error: + verify.side_effect = verifier_error + module.main() + verify.assert_called_once() + self.assertTrue(verify.call_args.kwargs["check"]) + self.assertIn("--fingerprint", verify.call_args.args[0]) + self.assertIn(str(self.root / "trusted.asc"), verify.call_args.args[0]) + return json.loads(output.getvalue()) + + def test_verified_bytes_produce_activation_record(self): + result = self.invoke("--output", str(self.output), "--proof-url", "https://github.com/sandwichfarm/loopwire/actions/runs/123") + self.assertEqual(result["files"], 1) + record = json.loads(self.output.read_text()) + self.assertEqual(record["status"], "verified") + self.assertEqual(record["baseUrl"], self.url) + self.assertEqual(record["revision"], "a" * 64) + self.assertEqual(record["signingFingerprint"], "A" * 40) + self.assertIn("verifiedAt", record) + + def test_remote_tamper_never_overwrites_activation(self): + self.output.write_text("previous activation") + (self.web / "payload").write_bytes(b"changed package bytes!") + with self.assertRaises(ValueError): + self.invoke("--output", str(self.output), "--proof-url", "https://github.com/sandwichfarm/loopwire/actions/runs/123") + self.assertEqual(self.output.read_text(), "previous activation") + + def test_missing_file_fails(self): + (self.web / "payload").unlink() + with self.assertRaisesRegex(ValueError, "HTTP 404"): + self.invoke() + self.assertFalse(self.output.exists()) + + def test_redirect_is_rejected(self): + with self.assertRaisesRegex(ValueError, "does not follow redirects"): + self.invoke("--base-url", self.url + "/redirect") + self.assertEqual(self.requests, ["/redirect/payload"]) + + def test_invalid_local_signature_prevents_network(self): + with self.assertRaises(subprocess.CalledProcessError): + self.invoke(verifier_error=subprocess.CalledProcessError(1, "signed verifier")) + self.assertEqual(self.requests, []) + + def test_activation_requires_workflow_evidence_url(self): + for url in ["", "https://example.invalid/run/123", "https://github.com/test/repo/actions/runs/not-a-number"]: + with self.subTest(url=url), self.assertRaisesRegex(ValueError, "GitHub Actions"): + self.invoke("--output", str(self.output), "--proof-url", url) + self.assertEqual(self.requests, []) + + def test_custom_ca_fixture_cannot_produce_public_activation(self): + with self.assertRaisesRegex(ValueError, "custom-CA fixture"): + self.invoke("--output", str(self.output), "--ca-file", str(self.cert), "--proof-url", + "https://github.com/sandwichfarm/loopwire/actions/runs/123") + self.assertFalse(self.output.exists()) + self.assertEqual(self.requests, []) + + def test_https_and_independent_fingerprint_required(self): + for args in [("--base-url", "http://example.invalid"), ("--base-url", "https://user:pass@example.invalid"), + ("--fingerprint", "A" * 8)]: + with self.subTest(args=args), self.assertRaises(ValueError): + self.invoke(*args) + self.assertEqual(self.requests, []) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test-apt-repository-vm-proof.mjs b/scripts/test-apt-repository-vm-proof.mjs new file mode 100644 index 0000000..7ceab49 --- /dev/null +++ b/scripts/test-apt-repository-vm-proof.mjs @@ -0,0 +1,78 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { parseInstalledHashes, verifyInstalledStage, verifyLifecycle } from "./verify-apt-repository-vm-proof.mjs"; + +const directory = await mkdtemp(path.join(tmpdir(), "loopwire-apt-proof-test-")); +const baseline = "0.1.0-1ubuntu24.04"; +const upgrade = "0.1.0+aptfixture1-1ubuntu24.04"; +const baseUrl = "https://127.0.0.1:8443"; +const expectedHashes = Object.fromEntries([ + "/usr/bin/loopwire", "/usr/bin/loopwire-dsp-provider", "/usr/bin/loopwire-jack-ports", + "/usr/bin/loopwire-detect-audio", "/usr/lib/loopwire/loopwire-gui", + "/usr/share/applications/loopwire.desktop", "/usr/share/icons/hicolor/scalable/apps/loopwire.svg", +].map((name) => [name, "a".repeat(64)])); +const transitions = [ + `install\t${baseline}\tinstalled`, `reinstall\t${baseline}\tinstalled`, `upgrade\t${upgrade}\tinstalled`, + `rollback\t${baseline}\tinstalled`, `remove\t${baseline}\tabsent`, +].join("\n"); +let passed = 0; +async function test(name, action) { + await action(); + passed += 1; + console.log(`PASS ${name}`); +} +async function stageFixture() { + await mkdir(path.join(directory, "install"), { recursive: true }); + const files = { + "package-metadata.tsv": `loopwire\t${baseline}\tamd64\tinstall ok installed\n`, + "package-files.txt": `${Object.keys(expectedHashes).join("\n")}\n`, + "installed-files.sha256": `${Object.entries(expectedHashes).map(([name, hash]) => `${hash} ${name}`).join("\n")}\n`, + "apt-policy.txt": `loopwire:\n Installed: ${baseline}\n Candidate: ${baseline}\n 500 ${baseUrl} ubuntu-24.04/main amd64 Packages\n`, + "background-help.txt": "Usage: Loopwire background restore\n", + "dsp-provider-help.txt": "Usage: Loopwire DSP provider\n", + "jack-provider-help.txt": "Usage: Loopwire JACK provider\n", + "detect-audio.json": "{\"backends\":[]}\n", + "gui-ldd.txt": "libgtk-3.so => /lib/libgtk-3.so\nlibwebkit2gtk-4.1.so => /lib/libwebkit2gtk-4.1.so\n", + "gui-launch-status.txt": "0\n", "gui-window-ids.txt": "1234\n", "gui-window-names.txt": "Loopwire\n", + "gui-launch.log": "", "xvfb.log": "", + }; + for (const [name, content] of Object.entries(files)) await writeFile(path.join(directory, "install", name), content); +} +async function verifyStage() { + await verifyInstalledStage(directory, "install", baseline, baseUrl, expectedHashes); +} +async function change(name, transform) { + const file = path.join(directory, "install", name); + await writeFile(file, transform(await readFile(file, "utf8"))); +} +try { + await test("complete ordered lifecycle accepted", () => verifyLifecycle(transitions, baseline, upgrade)); + await test("missing reinstall rejected", () => assert.throws(() => verifyLifecycle(transitions.split("\n").filter((line) => !line.startsWith("reinstall")).join("\n"), baseline, upgrade), /lifecycle transitions/)); + await test("rollback remaining at new version rejected", () => assert.throws(() => verifyLifecycle(transitions.replace(`rollback\t${baseline}`, `rollback\t${upgrade}`), baseline, upgrade), /lifecycle transitions/)); + await test("fabricated summary pass is insufficient", () => assert.throws(() => verifyLifecycle("pass", baseline, upgrade), /lifecycle transitions/)); + await test("duplicate file hash rejected", () => assert.throws(() => parseInstalledHashes(`${"a".repeat(64)} /usr/bin/loopwire\n${"b".repeat(64)} /usr/bin/loopwire\n`), /duplicate/)); + await test("parent traversal hash rejected", () => assert.throws(() => parseInstalledHashes(`${"a".repeat(64)} /usr/../etc/passwd\n`), /invalid/)); + await stageFixture(); + await test("complete stage fixture accepted", verifyStage); + for (const [name, file, mutation, pattern] of [ + ["changed installed bytes rejected", "installed-files.sha256", (value) => value.replace("a".repeat(64), "b".repeat(64)), /signed package payload/], + ["wrong installed version rejected", "package-metadata.tsv", (value) => value.replace(baseline, upgrade), /package metadata/], + ["local file installation without origin rejected", "apt-policy.txt", (value) => value.replace(baseUrl, "file:\/tmp/local"), /repository origin/], + ["unresolved GUI dependency rejected", "gui-ldd.txt", (value) => `${value}libmissing.so => not found\n`, /linkage/], + ["unrelated X11 window rejected", "gui-window-names.txt", () => "xterm\n", /application window/], + ["failed GUI process rejected", "gui-launch-status.txt", () => "124\n", /GUI launch/], + ["GUI panic rejected", "gui-launch.log", () => "thread main panicked\n", /fatal GUI log/], + ["empty provider output rejected", "dsp-provider-help.txt", () => "", /empty evidence/], + ["missing installed helper rejected", "package-files.txt", (value) => value.replace("/usr/bin/loopwire-dsp-provider\n", ""), /missing/], + ]) { + await stageFixture(); + await change(file, mutation); + await test(name, () => assert.rejects(verifyStage, pattern)); + } + console.log(`APT VM proof verifier tests passed: ${passed}`); +} finally { + await rm(directory, { recursive: true, force: true }); +} diff --git a/scripts/test-apt-repository.py b/scripts/test-apt-repository.py new file mode 100644 index 0000000..a264d9e --- /dev/null +++ b/scripts/test-apt-repository.py @@ -0,0 +1,333 @@ +#!/usr/bin/env python3 +"""Credential-free APT repository regression tests using real dpkg, GPG and APT. + +Run on Ubuntu 24.04 or Debian 13 with python3, dpkg-dev, apt-utils, gnupg, +and openssl installed. All keys, package fixtures, servers and APT state are +temporary; the host's sources, trusted keyrings and package database are unused. +""" + +import functools +import hashlib +import http.server +import json +import os +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import threading +import time +import unittest + + +SCRIPT = Path(__file__).with_name("apt-repository.py") +SUITES = {"ubuntu-24.04": "ubuntu24.04", "debian-13": "debian13"} + + +def run(*args, ok=True, **kwargs): + result = subprocess.run([str(arg) for arg in args], capture_output=True, text=True, **kwargs) + if ok and result.returncode: + raise AssertionError(f"{args!r}\n{result.stdout}\n{result.stderr}") + return result + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +class QuietHandler(http.server.SimpleHTTPRequestHandler): + def log_message(self, *_args): + pass + + +class RepositoryTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + for tool in ("dpkg-deb", "dpkg", "gpg", "gpgv", "apt-get", "openssl"): + if not shutil.which(tool): + raise RuntimeError(f"{tool} required; run these tests in an Ubuntu/Debian container") + cls.temporary = tempfile.TemporaryDirectory(prefix="loopwire-apt-tests-") + cls.root = Path(cls.temporary.name) + cls.root.chmod(0o755) + cls.gnupg = cls.root / "gnupg" + cls.gnupg.mkdir(mode=0o700) + cls.fingerprints = [] + for identity in ("Loopwire Test", "Wrong Test"): + run("gpg", "--homedir", cls.gnupg, "--batch", "--pinentry-mode", "loopback", + "--passphrase", "", "--quick-generate-key", identity, "rsa2048", "sign", "1d") + listing = run("gpg", "--homedir", cls.gnupg, "--with-colons", "--list-keys", identity).stdout + cls.fingerprints.append(next(line.split(":")[9] for line in listing.splitlines() + if line.startswith("fpr:"))) + cls.fingerprint, cls.wrong_fingerprint = cls.fingerprints + cls.key = cls.root / "archive.asc" + cls.key.write_text(run("gpg", "--homedir", cls.gnupg, "--armor", "--export", + cls.fingerprint).stdout) + cls.release_private = cls.root / "release-private.pem" + cls.release_public = cls.root / "release-public.pem" + run("openssl", "genpkey", "-algorithm", "RSA", "-pkeyopt", "rsa_keygen_bits:2048", + "-out", cls.release_private) + run("openssl", "pkey", "-in", cls.release_private, "-pubout", "-out", cls.release_public) + cls.release1 = cls.make_release("1.0.0") + cls.release2 = cls.make_release("1.1.0") + cls.date = int(time.time()) + cls.base = cls.root / "base" + cls.build(cls.release1, "1.0.0", cls.base) + + @classmethod + def tearDownClass(cls): + run("gpgconf", "--homedir", cls.gnupg, "--kill", "gpg-agent", ok=False) + cls.temporary.cleanup() + + @classmethod + def make_release(cls, version, architecture="amd64", package_name="loopwire", suffix=""): + release = cls.root / f"release-{version}-{architecture}-{package_name}{suffix}" + release.mkdir() + for suite, revision in SUITES.items(): + package_root = cls.root / f"pkg-{version}-{suite}-{architecture}-{package_name}{suffix}" + (package_root / "DEBIAN").mkdir(parents=True) + (package_root / "usr/share/loopwire").mkdir(parents=True) + (package_root / "usr/share/loopwire/fixture").write_text(version + suffix) + (package_root / "DEBIAN/control").write_text( + f"Package: {package_name}\nVersion: {version}-1{revision}\n" + f"Architecture: {architecture}\nMaintainer: Test \n" + "Description: Loopwire repository fixture\n A multiline description.\n") + output = release / f"loopwire_{version}-1{revision}_amd64.deb" + run("dpkg-deb", "--root-owner-group", "--build", package_root, output) + cls.sign_release(release) + return release + + @classmethod + def sign_release(cls, release): + (release / "SHA256SUMS").write_text("".join( + f"{digest(path)} {path.name}\n" for path in sorted(release.glob("*.deb")))) + run("openssl", "dgst", "-sha256", "-sign", cls.release_private, + "-out", release / "SHA256SUMS.sig", release / "SHA256SUMS") + + @classmethod + def build(cls, release, version, output, *extra, ok=True): + return run(sys.executable, SCRIPT, "build", "--release-dir", release, "--version", version, + "--output", output, "--signing-key", cls.fingerprint, "--gnupg-home", cls.gnupg, + "--release-public-key", cls.release_public, "--date", cls.date, *extra, ok=ok) + + def setUp(self): + self.case_dir = self.root / self.id().split(".")[-1] + self.case_dir.mkdir(mode=0o755) + self.repo = self.case_dir / "repository" + shutil.copytree(self.base, self.repo) + + def verify(self, *extra, ok=True): + return run(sys.executable, SCRIPT, "verify", "--repository", self.repo, + "--public-key", self.key, "--fingerprint", self.fingerprint, *extra, ok=ok) + + def rewrite_manifest(self): + manifest_path = self.repo / "repository-manifest.json" + manifest = json.loads(manifest_path.read_text()) + for entry in manifest["files"]: + path = self.repo / entry["path"] + if path.is_file(): + entry.update(sha256=digest(path), size=path.stat().st_size) + manifest.pop("revision", None) + encoded = json.dumps(manifest, sort_keys=True, separators=(",", ":")).encode() + manifest["revision"] = hashlib.sha256(encoded).hexdigest() + manifest_path.write_text(json.dumps(manifest)) + + def apt(self, suite="ubuntu-24.04", operation=("update",), key=None): + handler = functools.partial(QuietHandler, directory=str(self.repo)) + server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + state = self.case_dir / "apt" + state.mkdir(exist_ok=True) + state.chmod(0o755) + for directory in ("lists", "cache", "downloads"): + (state / directory).mkdir(exist_ok=True) + (state / directory).chmod(0o777) + source = state / "loopwire.sources" + source.write_text( + f"Types: deb\nURIs: http://127.0.0.1:{server.server_port}\nSuites: {suite}\n" + f"Components: main\nArchitectures: amd64\nSigned-By: {key or self.key}\nBy-Hash: force\n") + args = ["apt-get", "-o", f"Dir::Etc::sourcelist={source}", "-o", "Dir::Etc::sourceparts=-", + "-o", f"Dir::State::lists={state / 'lists'}", "-o", f"Dir::Cache={state / 'cache'}", + "-o", "Dir::State::status=/dev/null", "-o", "APT::Architecture=amd64", + "-o", "APT::Architectures::=amd64", "-o", "Acquire::Languages=none", + "-o", "APT::Update::Error-Mode=any"] + try: + update = run(*args, "update", ok=False, cwd=state / "downloads") + if operation == ("update",) or update.returncode: + return update + return run(*args, *operation, ok=False, cwd=state / "downloads") + finally: + server.shutdown() + server.server_close() + thread.join() + + def test_signed_chain_and_real_apt_download_both_suites(self): + summary = json.loads(self.verify().stdout) + self.assertEqual(summary["signingFingerprint"], self.fingerprint) + for suite, revision in SUITES.items(): + result = self.apt(suite, ("download", f"loopwire=1.0.0-1{revision}")) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + downloaded = self.case_dir / "apt/downloads" / f"loopwire_1.0.0-1{revision}_amd64.deb" + self.assertEqual(digest(downloaded), digest(self.release1 / downloaded.name)) + + def test_retention_version_order_and_fresh_rollback(self): + upgraded = self.case_dir / "upgraded" + self.build(self.release2, "1.1.0", upgraded, "--previous", self.base) + old = json.loads((self.base / "repository-manifest.json").read_text()) + new = json.loads((upgraded / "repository-manifest.json").read_text()) + for entry in old["files"]: + if entry["kind"] == "immutable": + self.assertEqual(digest(upgraded / entry["path"]), entry["sha256"]) + for suite in new["suites"]: + self.assertEqual(len(suite["packages"]), 2) + failed = self.build(self.release1, "1.0.0", self.case_dir / "downgrade", + "--previous", upgraded, ok=False) + self.assertNotEqual(failed.returncode, 0) + rollback = self.case_dir / "rollback" + run(sys.executable, SCRIPT, "rollback", "--repository", self.base, "--output", rollback, + "--signing-key", self.fingerprint, "--gnupg-home", self.gnupg, + "--date", self.date + 60) + rolled = json.loads((rollback / "repository-manifest.json").read_text()) + self.assertEqual(rolled["suites"], old["suites"]) + self.assertGreater(rolled["createdAt"], old["createdAt"]) + self.assertNotEqual(rolled["revision"], old["revision"]) + run(sys.executable, SCRIPT, "verify", "--repository", rollback, "--public-key", self.key, + "--fingerprint", self.fingerprint, "--now", self.date + 60) + + def test_release_input_signature_and_duplicate_checksum_rejected(self): + release = self.case_dir / "release" + shutil.copytree(self.release1, release) + (release / "SHA256SUMS.sig").write_bytes(b"invalid") + self.assertNotEqual(self.build(release, "1.0.0", self.case_dir / "bad", ok=False).returncode, 0) + self.sign_release(release) + checksums = release / "SHA256SUMS" + checksums.write_text(checksums.read_text() * 2) + run("openssl", "dgst", "-sha256", "-sign", self.release_private, + "-out", release / "SHA256SUMS.sig", checksums) + self.assertNotEqual(self.build(release, "1.0.0", self.case_dir / "duplicate", ok=False).returncode, 0) + self.sign_release(release) + shutil.copyfile(next(release.glob("*.deb")), release / "duplicate.deb") + self.sign_release(release) + self.assertNotEqual(self.build(release, "1.0.0", self.case_dir / "extra-deb", ok=False).returncode, 0) + + def test_reproducible_signed_candidate_and_build_metadata_upgrade(self): + second = self.case_dir / "second" + self.build(self.release1, "1.0.0", second) + self.assertEqual((second / "repository-manifest.json").read_bytes(), + (self.base / "repository-manifest.json").read_bytes()) + release = self.make_release("1.0.0+aptfixture1") + upgraded = self.case_dir / "upgraded" + self.build(release, "1.0.0+aptfixture1", upgraded, "--previous", self.base) + self.repo = upgraded + result = self.apt(operation=("download", "loopwire")) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertTrue((self.case_dir / "apt/downloads/loopwire_1.0.0+aptfixture1-1ubuntu24.04_amd64.deb").is_file()) + + def test_encrypted_signing_key_uses_passphrase_file(self): + # GnuPG's Unix socket pathname must fit the platform's short limit. + home = self.root / "encrypted-gnupg" + home.mkdir(mode=0o700) + passphrase = self.case_dir / "passphrase" + passphrase.write_text("fixture passphrase with spaces\n") + passphrase.chmod(0o600) + try: + run("gpg", "--homedir", home, "--batch", "--pinentry-mode", "loopback", "--passphrase-file", + passphrase, "--quick-generate-key", "Encrypted Fixture", "rsa2048", "sign", "1d") + listing = run("gpg", "--homedir", home, "--with-colons", "--list-keys").stdout + identity = next(line.split(":")[9] for line in listing.splitlines() if line.startswith("fpr:")) + run("gpgconf", "--homedir", home, "--kill", "gpg-agent") + output = self.case_dir / "encrypted" + self.build(self.release1, "1.0.0", output, "--gnupg-home", home, "--signing-key", identity, + "--passphrase-file", passphrase, "--date", int(time.time())) + run(sys.executable, SCRIPT, "verify", "--repository", output, "--fingerprint", identity, + "--public-key", output / f"keys/{identity}.asc") + finally: + run("gpgconf", "--homedir", home, "--kill", "gpg-agent", ok=False) + + def test_package_architecture_name_and_version_rejected(self): + for architecture, name in (("arm64", "loopwire"), ("amd64", "other")): + release = self.make_release("1.2.0", architecture, name) + self.assertNotEqual(self.build(release, "1.2.0", self.case_dir / name / architecture, + ok=False).returncode, 0) + self.assertNotEqual(self.build(self.release1, "1.0.0-rc.1", self.case_dir / "prerelease", + ok=False).returncode, 0) + release = self.case_dir / "mismatched-suite" + shutil.copytree(self.release1, release) + ubuntu = release / "loopwire_1.0.0-1ubuntu24.04_amd64.deb" + debian = release / "loopwire_1.0.0-1debian13_amd64.deb" + shutil.copyfile(ubuntu, debian) + self.sign_release(release) + self.assertNotEqual(self.build(release, "1.0.0", self.case_dir / "bad-suite", ok=False).returncode, 0) + + def test_wrong_signer_rejected(self): + self.assertNotEqual(self.verify("--fingerprint", self.wrong_fingerprint, ok=False).returncode, 0) + wrong_key = self.case_dir / "wrong.asc" + wrong_key.write_text(run("gpg", "--homedir", self.gnupg, "--armor", "--export", + self.wrong_fingerprint).stdout) + self.assertNotEqual(self.apt(key=wrong_key).returncode, 0) + + def test_signed_metadata_tampering_and_unsigned_repository_rejected(self): + inrelease = self.repo / "dists/ubuntu-24.04/InRelease" + inrelease.write_text(inrelease.read_text().replace("Origin: Loopwire", "Origin: Forged!!")) + self.rewrite_manifest() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + self.assertNotEqual(self.apt().returncode, 0) + inrelease.unlink() + self.assertNotEqual(self.apt().returncode, 0) + + def test_modified_package_rejected_by_verifier_and_apt(self): + package = next(self.repo.glob("pool/ubuntu-24.04/**/*.deb")) + package.write_bytes(package.read_bytes() + b"tampered") + self.rewrite_manifest() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + result = self.apt(operation=("download", "loopwire=1.0.0-1ubuntu24.04")) + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + + def test_tampered_index_and_previous_snapshot_rejected(self): + index = self.repo / "dists/ubuntu-24.04/main/binary-amd64/Packages" + index.write_text(index.read_text().replace("Version: 1.0.0", "Version: 9.0.0")) + self.rewrite_manifest() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + self.assertNotEqual(self.build(self.release2, "1.1.0", self.case_dir / "from-invalid", + "--previous", self.repo, ok=False).returncode, 0) + + def test_expired_and_future_metadata_rejected(self): + self.assertNotEqual(self.verify("--now", self.date + 31 * 86400, ok=False).returncode, 0) + self.assertNotEqual(self.verify("--now", self.date - 60, ok=False).returncode, 0) + + def test_path_manifest_classification_and_extras_rejected(self): + path = self.repo / "repository-manifest.json" + original = path.read_text() + for replacement in ("../../outside", "/absolute", "dists/../secret", "pool//double"): + manifest = json.loads(original) + manifest["files"][0]["path"] = replacement + path.write_text(json.dumps(manifest)) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + path.write_text(original) + manifest = json.loads(original) + manifest["files"][0]["kind"] = "metadata" if manifest["files"][0]["kind"] == "immutable" else "immutable" + path.write_text(json.dumps(manifest)) + self.rewrite_manifest() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + path.write_text(original) + (self.repo / "unadvertised").write_text("extra") + self.assertNotEqual(self.verify(ok=False).returncode, 0) + + def test_symlink_hardlink_and_same_version_repack_rejected(self): + package = next(self.repo.glob("pool/**/*.deb")) + original = package.read_bytes() + package.unlink() + package.symlink_to(self.release1 / package.name) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + package.unlink() + package.write_bytes(original) + os.link(package, self.case_dir / "hardlink") + self.assertNotEqual(self.verify(ok=False).returncode, 0) + release = self.make_release("1.0.0", suffix="repack") + self.assertNotEqual(self.build(release, "1.0.0", self.case_dir / "repack", + "--previous", self.base, ok=False).returncode, 0) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/scripts/test-apt-workflow-preflight.py b/scripts/test-apt-workflow-preflight.py new file mode 100644 index 0000000..f3aedc4 --- /dev/null +++ b/scripts/test-apt-workflow-preflight.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python3 +import os +from pathlib import Path +import subprocess +import tempfile +import unittest + +SCRIPT = Path(__file__).with_name("publish-apt-workflow.sh").resolve() + + +class PreflightTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory(prefix="loopwire-apt-preflight-") + self.root = Path(self.temp.name) + self.addCleanup(self.temp.cleanup) + binary = self.root / "bin" + binary.mkdir() + gpg = binary / "gpg" + gpg.write_text('#!/bin/sh\nprintf called > "$APT_TEST_MARKER"\nexit 1\n') + gpg.chmod(0o755) + self.marker = self.root / "used-key" + self.env = { + **os.environ, "PATH": f"{binary}:{os.environ['PATH']}", "APT_TEST_MARKER": str(self.marker), + "APT_REPOSITORY_URL": "https://packages.example.invalid/apt", + "APT_REPOSITORY_HOST": "publisher@example.invalid", "APT_REPOSITORY_ROOT": "/srv/loopwire", + "APT_SIGNING_FINGERPRINT": "A" * 40, "APT_SSH_PRIVATE_KEY": "private-ssh-fixture", + "APT_SSH_KNOWN_HOSTS": "known-hosts-fixture", "APT_SIGNING_KEY": "private-gpg-fixture", + "RUNNER_TEMP": str(self.root), "GITHUB_REPOSITORY": "sandwichfarm/loopwire", + "GITHUB_SERVER_URL": "https://github.com", "GITHUB_RUN_ID": "123", + "OPERATION": "publish", "RELEASE_TAG": "v1.2.3", + } + + def rejected(self, values, message): + result = subprocess.run(["bash", str(SCRIPT)], env={**self.env, **values}, capture_output=True, text=True) + self.assertNotEqual(result.returncode, 0) + self.assertIn(message, result.stderr) + self.assertFalse(self.marker.exists(), "invalid input reached signing-key operations") + self.assertNotIn("private-ssh-fixture", result.stdout + result.stderr) + self.assertNotIn("private-gpg-fixture", result.stdout + result.stderr) + + def test_https_url_rejected_before_keys_or_origin_access(self): + for url in ["http://example.invalid", "https://user:pass@example.invalid", "https://example.invalid/?", + "https://example.invalid/#", "https://example.invalid/\ninjected"]: + with self.subTest(url=url): + self.rejected({"APT_REPOSITORY_URL": url}, "base URL") + + def test_missing_configuration(self): + self.rejected({"APT_SIGNING_KEY": ""}, "missing configuration: APT_SIGNING_KEY") + + def test_tag_is_validated(self): + self.rejected({"RELEASE_TAG": "v1.2.3; echo unexpected"}, "stable vX.Y.Z") + + def test_rollback_revision_is_validated(self): + self.rejected({"OPERATION": "rollback", "ROLLBACK_REVISION": "HEAD"}, "revision SHA-256") + + def test_fingerprint_is_validated(self): + self.rejected({"APT_SIGNING_FINGERPRINT": "short"}, "fingerprint") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test-apt-workflow.rb b/scripts/test-apt-workflow.rb new file mode 100755 index 0000000..abb4ae3 --- /dev/null +++ b/scripts/test-apt-workflow.rb @@ -0,0 +1,47 @@ +#!/usr/bin/env ruby +require 'yaml' + +root = File.expand_path('..', __dir__) +load_workflow = ->(name) { YAML.safe_load_file(File.join(root, '.github/workflows', name)) } +check = ->(condition, message) { raise message unless condition } +apt = load_workflow.call('publish-apt.yml') +release = load_workflow.call('release.yml') +events = apt['on'] || apt[true] +check.call(!events.key?('push') && !events.key?('pull_request'), 'APT publication must not run on arbitrary pushes or PRs') +check.call(events.key?('workflow_call') && events.key?('workflow_dispatch'), 'release and operator entrypoints required') +check.call(events.fetch('schedule').any? { |schedule| schedule['cron'] == '37 5 * * 1' }, 'weekly expiry refresh required') +check.call(events.dig('workflow_dispatch', 'inputs', 'operation', 'options') == %w[publish refresh rollback], 'operator actions drifted') +check.call(apt.dig('permissions', 'contents') == 'read', 'APT publisher needs only read access to GitHub') +check.call(apt.dig('concurrency', 'cancel-in-progress') == false, 'do not interrupt an active metadata promotion') +job = apt.dig('jobs', 'publish') +check.call(job['environment'] == 'packages-production', 'production secrets must remain environment-scoped') +check.call(job['if'].include?("vars.APT_REPOSITORY_ENABLED == 'true'"), 'production must be explicitly enabled') +check.call(job['if'].include?('github.event.repository.default_branch'), 'operator publication must restrict its code ref') +check.call(job['if'] == job['if'].strip, 'job condition must not have trailing literal whitespace') +publisher = job.fetch('steps').find { |step| step['run'] == 'bash scripts/publish-apt-workflow.sh' } +check.call(publisher, 'workflow must use the reviewed publication entrypoint') +check.call(publisher.dig('env', 'OPERATION') == "${{ inputs.operation || 'refresh' }}", 'scheduled runs must refresh') +check.call(publisher.dig('env', 'RELEASE_TAG') == '${{ inputs.tag }}', 'tag input must be passed as data') +%w[APT_SIGNING_KEY APT_SSH_PRIVATE_KEY APT_SSH_KNOWN_HOSTS APT_SIGNING_PASSPHRASE].each do |name| + check.call(publisher.dig('env', name) == "${{ secrets.#{name} }}", "#{name} must come from secrets") +end +caller = release.dig('jobs', 'publish-apt') +check.call(caller['needs'] == 'publish-release', 'APT must wait for all existing release gates') +check.call(caller['uses'] == './.github/workflows/publish-apt.yml', 'release should reuse the publication workflow') +check.call(caller.dig('with', 'tag') == '${{ needs.publish-release.outputs.tag }}', 'use the verified release tag') +publish_release = release.dig('jobs', 'publish-release') +check.call(publish_release.dig('outputs', 'tag') == '${{ steps.verified-tag.outputs.tag }}', 'release output must be verified') +check.call(publish_release.fetch('steps').last['id'] == 'verified-tag', 'tag export must follow all release evidence gates') + +script = File.read(File.join(root, 'scripts/publish-apt-workflow.sh')) +publish_index = script.index('scripts/publish-package-repository.py publish') +%w[verify-release-signature.sh release-asset-manifest.mjs].each do |gate| + check.call(script.index(gate) && script.index(gate) < publish_index, "#{gate} must precede publication") +end +check.call(script.include?('--require-checksum --require-evidence'), 'manual publication must validate release evidence inventory') +check.call(script.include?('fetch_status" -eq 3') && script.include?('operation" = publish'), 'only initial publish accepts empty origin') +check.call(script.index('python3 scripts/verify-apt-public.py') > publish_index, 'activation record requires verification of served bytes') +check.call(script.include?('--expected-revision "$expected"'), 'publication must use compare-and-swap') +check.call(script.include?('trap cleanup EXIT'), 'private signing/SSH files must be removed') +check.call(script.include?('unset APT_SSH_PRIVATE_KEY'), 'do not pass raw credentials to publisher child processes') +puts 'APT workflow contract passed: protected release ordering, explicit activation, expiry refresh, secret transport and public proof.' diff --git a/scripts/test-ci-workflow-paths.rb b/scripts/test-ci-workflow-paths.rb index 7b73470..c96512c 100644 --- a/scripts/test-ci-workflow-paths.rb +++ b/scripts/test-ci-workflow-paths.rb @@ -28,6 +28,7 @@ def selected_paths(path, event, parsed) cases = { 'apps/site/src/pages/index.astro' => [%w[web], %w[deploy web]], 'apps/site/package.json' => [%w[web], %w[deploy web]], + 'packaging/repositories/apt-channel.json' => [%w[web], %w[deploy web]], 'apps/docs/docs/guide/install.md' => [%w[web], %w[deploy web]], 'README.md' => [%w[web], %w[web]], 'packaging/README.md' => [%w[web], %w[web]], @@ -106,7 +107,7 @@ def selected_paths(path, event, parsed) check(parsed['deploy'].dig('jobs', 'deploy-bunny', 'environment') == 'docs-production', 'preserve deployment environment') condition = parsed['deploy'].dig('jobs', 'deploy-bunny', 'if') check(condition == condition.strip, 'deployment condition must not become an always-true string with trailing whitespace') -%w[release final-release-proof publish-aur continuous-tests].each do |name| +%w[release final-release-proof publish-aur publish-apt continuous-tests].each do |name| workflow = YAML.safe_load_file(File.join(ROOT, '.github/workflows', "#{name}.yml")) events = workflow['on'] || workflow[true] check(!events.key?('pull_request'), "#{name}: deliberate operator workflows must not gain PR triggers") diff --git a/scripts/test-publish-package-repository.py b/scripts/test-publish-package-repository.py new file mode 100644 index 0000000..df0731b --- /dev/null +++ b/scripts/test-publish-package-repository.py @@ -0,0 +1,539 @@ +#!/usr/bin/env python3 +"""Publisher regression tests; real signed fixtures require Debian/Ubuntu tools. + +Run: python3 scripts/test-publish-package-repository.py +Add --with-ssh inside a disposable root Docker container with openssh-server to +exercise the real transport. It starts sshd only in that container, uses temporary +host/client keys and known_hosts, and removes them and the server on completion. +No production credentials, services, or audio configuration are used. +""" + +import argparse +import base64 +import hashlib +import importlib.util +import io +import json +import os +from pathlib import Path +import shutil +import socket +import stat +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +from unittest import mock + + +SCRIPT = Path(__file__).with_name("publish-package-repository.py") + + +def load(name, path): + specification = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(specification) + specification.loader.exec_module(module) + return module + + +publisher = load("publisher", SCRIPT) +FPR = "A" * 40 +WITH_SSH = False + + +def fixture(root, version="1", revision_date=1, previous=None): + root.mkdir() + files = {} + if previous: + prior = json.loads((previous / publisher.MANIFEST).read_text()) + for entry in prior["files"]: + if entry["kind"] == "immutable": + files[entry["path"]] = (previous / entry["path"]).read_bytes() + files[f"keys/{FPR}.asc"] = b"synthetic key for filesystem-only tests" + suites = [] + for suite in publisher.SUITES: + package = f"pool/{suite}/main/l/loopwire/loopwire_{version}_amd64.deb" + files[package] = b"package " + version.encode() + suites.append({"name": suite, "architecture": "amd64", "component": "main", "packages": []}) + for suffix in ("Packages", "Packages.gz"): + data = f"index {suite} {version} {suffix}".encode() + prefix = f"dists/{suite}/main/binary-amd64" + files[f"{prefix}/{suffix}"] = data + files[f"{prefix}/by-hash/SHA256/{hashlib.sha256(data).hexdigest()}"] = data + for suffix in ("Release", "InRelease"): + files[f"dists/{suite}/{suffix}"] = f"{suite} {version} {revision_date} {suffix}".encode() + entries = [] + for path, data in sorted(files.items()): + target = root / path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(data) + entries.append({"path": path, "kind": publisher.classify(path), "size": len(data), + "sha256": hashlib.sha256(data).hexdigest()}) + manifest = {"schemaVersion": 1, "createdAt": revision_date, "validUntil": revision_date + 2592000, + "signingFingerprint": FPR, "suites": suites, "files": entries} + write_manifest(root, manifest) + return json.loads((root / publisher.MANIFEST).read_text()) + + +def write_manifest(root, manifest): + manifest.pop("revision", None) + manifest["revision"] = hashlib.sha256(publisher.canonical(manifest)).hexdigest() + (root / publisher.MANIFEST).write_text(json.dumps(manifest)) + + +class PublicationTests(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory(prefix="loopwire-publisher-tests-") + self.directory = Path(self.temporary.name) + self.root = self.directory / "origin" + self.first = self.directory / "first" + self.second = self.directory / "second" + self.one = fixture(self.first) + self.two = fixture(self.second, "2", 2, self.first) + + def tearDown(self): + self.temporary.cleanup() + + def publish_first(self): + return publisher.publish_at(self.root, self.first, FPR, "empty") + + def assert_current(self, revision): + self.assertEqual(publisher.state(self.root, "current"), {"revision": revision}) + + def test_publish_idempotence_and_revision_compare_and_swap(self): + self.assertEqual(self.publish_first()["status"], "published") + self.assert_current(self.one["revision"]) + self.assertEqual(self.publish_first()["status"], "unchanged") + with self.assertRaisesRegex(publisher.PublicationError, "compare-and-swap"): + publisher.publish_at(self.root, self.second, FPR, "empty") + self.assert_current(self.one["revision"]) + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assert_current(self.two["revision"]) + self.assertTrue((self.root / "snapshots" / self.one["revision"]).is_dir()) + + def test_restrictive_umask_preserves_public_and_private_permissions(self): + previous_umask = os.umask(0o077) + try: + self.publish_first() + + def permissions(path): + return stat.S_IMODE(path.stat().st_mode) + + self.assertEqual(permissions(self.root), 0o755) + public = self.root / "public" + for path in (public, *public.rglob("*")): + self.assertEqual(permissions(path), 0o755 if path.is_dir() else 0o644, str(path)) + for private in (self.root / "snapshots", self.root / "state"): + for path in (private, *private.rglob("*")): + self.assertEqual(permissions(path), 0o700 if path.is_dir() else 0o600, str(path)) + self.assertEqual(permissions(self.root / ".publish.lock"), 0o600) + + def interrupt(label): + if label == "journal": + raise InterruptedError("inspect durable pending journal permissions") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assertEqual(permissions(self.root / "state/pending.json"), 0o600) + finally: + os.umask(previous_umask) + + def test_existing_operator_directory_permissions_are_preserved(self): + self.root.mkdir(mode=0o750) + (self.root / "public").mkdir(mode=0o750) + self.root.chmod(0o750) + (self.root / "public").chmod(0o750) + self.publish_first() + self.assertEqual(stat.S_IMODE(self.root.stat().st_mode), 0o750) + self.assertEqual(stat.S_IMODE((self.root / "public").stat().st_mode), 0o750) + + def test_old_pool_and_by_hash_survive_without_previous_in_candidate(self): + self.publish_first() + independent = self.directory / "independent" + fixture(independent, "3", 3) + publisher.publish_at(self.root, independent, FPR, self.one["revision"]) + for entry in self.one["files"]: + if entry["kind"] == "immutable": + self.assertEqual(publisher.digest(self.root / "public" / entry["path"]), entry["sha256"]) + + def test_immutable_collision_does_not_change_metadata_or_snapshot(self): + self.publish_first() + entry = next(item for item in self.two["files"] if item["path"].endswith("loopwire_1_amd64.deb")) + target = self.second / entry["path"] + target.write_bytes(b"replaced published package") + entry.update(size=target.stat().st_size, sha256=publisher.digest(target)) + write_manifest(self.second, self.two) + with self.assertRaisesRegex(publisher.PublicationError, "immutable URL collision"): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assert_current(self.one["revision"]) + self.assertIsNone(publisher.state(self.root, "pending")) + self.assertFalse((self.root / "snapshots" / self.two["revision"]).exists()) + + def test_order_uploads_every_immutable_before_suite_commit(self): + self.publish_first() + events = [] + + def check(label): + events.append(label) + if label == "immutable": + for entry in self.two["files"]: + if entry["kind"] == "immutable": + self.assertEqual(publisher.digest(self.root / "public" / entry["path"]), entry["sha256"]) + for suite in publisher.SUITES: + path = f"dists/{suite}/InRelease" + self.assertEqual((self.root / "public" / path).read_bytes(), (self.first / path).read_bytes()) + if label == "committed:debian-13": + path = "dists/ubuntu-24.04/InRelease" + self.assertEqual((self.root / "public" / path).read_bytes(), (self.first / path).read_bytes()) + + with mock.patch.object(publisher, "_checkpoint", side_effect=check): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assertLess(events.index("immutable"), events.index("committed:debian-13")) + self.assertLess(events.index("committed:debian-13"), events.index("committed:ubuntu-24.04")) + + def test_killed_process_leaves_recoverable_journal_and_blocks_fetch(self): + self.publish_first() + code = ( + "import importlib.util,os,sys; from pathlib import Path; " + "s=importlib.util.spec_from_file_location('publisher',sys.argv[1]); " + "p=importlib.util.module_from_spec(s); s.loader.exec_module(p); " + "p._checkpoint=lambda label: os._exit(97) if label=='committed:debian-13' else None; " + "p.publish_at(Path(sys.argv[2]),Path(sys.argv[3]),sys.argv[4],sys.argv[5])" + ) + process = subprocess.run([sys.executable, "-c", code, str(SCRIPT), str(self.root), + str(self.second), FPR, self.one["revision"]], check=False) + self.assertEqual(process.returncode, 97) + self.assert_current(self.one["revision"]) + self.assertEqual(publisher.state(self.root, "pending")["revision"], self.two["revision"]) + with self.assertRaisesRegex(publisher.PublicationError, "recover"): + with publisher.selected_snapshot(self.root, FPR): + pass + with self.assertRaisesRegex(publisher.PublicationError, "recover"): + publisher.publish_at(self.root, self.first, FPR, self.one["revision"]) + publisher.recover_at(self.root, FPR, self.two["revision"]) + self.assert_current(self.two["revision"]) + self.assertIsNone(publisher.state(self.root, "pending")) + + def test_every_promotion_checkpoint_is_resumable_with_same_candidate(self): + checkpoints = ("journal", "immutable", "metadata:debian-13", "committed:debian-13", + "metadata:ubuntu-24.04", "committed:ubuntu-24.04", "current") + for index, checkpoint in enumerate(checkpoints): + with self.subTest(checkpoint=checkpoint): + root = self.directory / f"origin-{index}" + + def interrupt(label): + if label == checkpoint: + raise InterruptedError("simulated process interruption") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(root, self.first, FPR, "empty") + self.assertIsNotNone(publisher.state(root, "pending")) + publisher.publish_at(root, self.first, FPR, "empty") + self.assertEqual(publisher.state(root, "current")["revision"], self.one["revision"]) + self.assertIsNone(publisher.state(root, "pending")) + + def test_exclusive_lock_blocks_publish_and_fetch(self): + self.publish_first() + with publisher.locked(self.root, create=True): + with self.assertRaisesRegex(publisher.PublicationError, "locked"): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + with self.assertRaisesRegex(publisher.PublicationError, "locked"): + with publisher.selected_snapshot(self.root, FPR): + pass + + def test_empty_fetch_has_no_filesystem_side_effects(self): + with self.assertRaises(publisher.EmptyRepository): + with publisher.selected_snapshot(self.root, FPR): + pass + self.assertFalse(self.root.exists()) + + def test_fetch_selects_retained_snapshot(self): + self.publish_first() + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + with publisher.selected_snapshot(self.root, FPR, self.one["revision"]) as (snapshot, manifest): + self.assertEqual(snapshot.name, self.one["revision"]) + self.assertEqual(manifest, self.one) + + def test_malicious_path_kind_and_revision_fail_before_root_writes(self): + cases = ("../../escape", "/etc/passwd", "dists/../escape", "state/current.json", "pool//x") + for index, bad in enumerate(cases): + with self.subTest(path=bad): + candidate = self.directory / f"bad-{index}" + manifest = fixture(candidate) + manifest["files"][0]["path"] = bad + write_manifest(candidate, manifest) + with self.assertRaises(publisher.PublicationError): + publisher.publish_at(self.root, candidate, FPR, "empty") + self.assertFalse(self.root.exists()) + self.one["files"][0]["kind"] = "immutable" + write_manifest(self.first, self.one) + with self.assertRaisesRegex(publisher.PublicationError, "kind"): + publisher.publish_at(self.root, self.first, FPR, "empty") + + def test_candidate_symlinks_and_hardlinks_are_rejected(self): + target = self.first / "unlisted" + target.symlink_to(self.second / publisher.MANIFEST) + with self.assertRaisesRegex(publisher.PublicationError, "symlink"): + self.publish_first() + target.unlink() + os.link(self.first / publisher.MANIFEST, target) + with self.assertRaisesRegex(publisher.PublicationError, "hardlink"): + self.publish_first() + self.assertFalse(self.root.exists()) + + def test_origin_symlink_and_unmanaged_content_are_rejected(self): + elsewhere = self.directory / "elsewhere" + elsewhere.mkdir() + self.root.symlink_to(elsewhere, target_is_directory=True) + with self.assertRaisesRegex(publisher.PublicationError, "symlink"): + self.publish_first() + self.assertEqual(list(elsewhere.iterdir()), []) + self.root.unlink() + (self.root / "public").mkdir(parents=True) + (self.root / "public/existing").write_text("unmanaged") + with self.assertRaisesRegex(publisher.PublicationError, "unmanaged"): + self.publish_first() + + def test_public_symlink_and_drift_rejected(self): + self.publish_first() + target = self.root / "public" / self.one["files"][0]["path"] + target.unlink() + target.symlink_to(self.first / self.one["files"][0]["path"]) + with self.assertRaisesRegex(publisher.PublicationError, "symlink"): + self.publish_first() + target.unlink() + target.write_bytes(b"drift") + with self.assertRaisesRegex(publisher.PublicationError, "drifted"): + self.publish_first() + + def test_archive_rejects_traversal_links_and_duplicate_entries(self): + for kind in ("traversal", "symlink", "hardlink", "duplicate"): + with self.subTest(kind=kind): + archive = io.BytesIO() + with tarfile.open(fileobj=archive, mode="w") as output: + member = tarfile.TarInfo("../escape" if kind == "traversal" else publisher.MANIFEST) + member.size = 2 + if kind in ("symlink", "hardlink"): + member.type = tarfile.SYMTYPE if kind == "symlink" else tarfile.LNKTYPE + member.linkname = "/etc/passwd" + output.addfile(member, io.BytesIO(b"{}")) + if kind == "duplicate": + output.addfile(member, io.BytesIO(b"{}")) + archive.seek(0) + destination = self.directory / kind + destination.mkdir() + with self.assertRaises(publisher.PublicationError): + publisher.read_archive(archive, destination) + + def test_remote_arguments_are_data_and_host_trust_is_mandatory(self): + args = argparse.Namespace(ssh="publisher@example.invalid", ssh_port=2222, known_hosts=None, identity_file=None) + request = {"root": "/tmp/path with 'quotes';$(touch /tmp/unsafe)", "action": "fetch"} + command = publisher.ssh_command(args, request) + self.assertIn("StrictHostKeyChecking=yes", command) + self.assertIn("BatchMode=yes", command) + self.assertIn("ForwardAgent=no", command) + import shlex + remote = shlex.split(command[-1]) + self.assertEqual(remote[:2], ["python3", "-c"]) + self.assertEqual(json.loads(base64.urlsafe_b64decode(remote[-1])), request) + args.ssh = "publisher@host;touch /tmp/unsafe" + with self.assertRaises(publisher.PublicationError): + publisher.ssh_command(args, request) + + +class SignedPublicationTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + # Share the generator suite's real dpkg/OpenSSL/GPG fixture builders. + cls.fixtures = load("apt_repository_test_fixtures", SCRIPT.with_name("test-apt-repository.py")).RepositoryTests + cls.fixtures.setUpClass() + cls.root = cls.fixtures.root + cls.upgraded = cls.root / "publisher-upgraded" + cls.fixtures.build(cls.fixtures.release2, "1.1.0", cls.upgraded, "--previous", cls.fixtures.base) + cls.one = json.loads((cls.fixtures.base / publisher.MANIFEST).read_text()) + cls.two = json.loads((cls.upgraded / publisher.MANIFEST).read_text()) + + @classmethod + def tearDownClass(cls): + cls.fixtures.tearDownClass() + + def setUp(self): + self.case_dir = self.root / self.id().split(".")[-1] + self.case_dir.mkdir() + self.origin = self.case_dir / "origin" + + def command(self, action, *extra, ok=True): + command = [sys.executable, str(SCRIPT), action, "--root", str(self.origin), + "--public-key", str(self.fixtures.key), "--fingerprint", self.fixtures.fingerprint, + *map(str, extra)] + result = subprocess.run(command, capture_output=True, text=True, check=False) + if ok: + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + return result + + def publish(self, *extra, **kwargs): + return self.command("publish", "--repository", self.fixtures.base, "--expected-revision", "empty", *extra, **kwargs) + + def test_signed_publish_fetch_refresh_and_retained_rollback_input(self): + self.publish() + self.command("publish", "--repository", self.upgraded, "--expected-revision", self.one["revision"]) + fetched = self.case_dir / "fetched" + result = self.command("fetch", "--output", fetched) + self.assertEqual(json.loads(result.stdout)["revision"], self.two["revision"]) + retained = self.case_dir / "retained" + result = self.command("fetch", "--revision", self.one["revision"], "--output", retained) + self.assertEqual(json.loads(result.stdout)["revision"], self.one["revision"]) + self.assertEqual((retained / publisher.MANIFEST).read_bytes(), (self.fixtures.base / publisher.MANIFEST).read_bytes()) + + def test_invalid_signature_dry_run_and_empty_fetch_do_not_touch_origin(self): + self.publish("--dry-run", "--ssh", "unused@example.invalid") + self.assertFalse(self.origin.exists()) + result = self.command("fetch", "--output", self.case_dir / "empty", ok=False) + self.assertEqual(result.returncode, 3) + self.assertEqual(json.loads(result.stdout), {"status": "empty", "revision": None}) + self.assertFalse(self.origin.exists()) + altered = self.case_dir / "altered" + shutil.copytree(self.fixtures.base, altered) + manifest = json.loads((altered / publisher.MANIFEST).read_text()) + entry = next(item for item in manifest["files"] if item["path"].endswith("/InRelease")) + (altered / entry["path"]).write_text("invalid signature") + entry.update(size=len("invalid signature"), sha256=publisher.digest(altered / entry["path"])) + write_manifest(altered, manifest) + result = self.command("publish", "--repository", altered, "--expected-revision", "empty", ok=False) + self.assertNotEqual(result.returncode, 0) + self.assertIn("signed repository verification failed", result.stderr) + self.assertFalse(self.origin.exists()) + + def test_recovery_verifies_pending_signed_snapshot(self): + self.publish() + + def interrupt(label): + if label == "committed:debian-13": + raise InterruptedError("simulated interruption") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(self.origin, self.upgraded, self.fixtures.fingerprint, self.one["revision"]) + self.command("recover", "--dry-run") + self.assertIsNotNone(publisher.state(self.origin, "pending")) + self.command("recover") + self.assertIsNone(publisher.state(self.origin, "pending")) + self.assertEqual(publisher.state(self.origin, "current")["revision"], self.two["revision"]) + + def test_expired_journal_requires_explicit_recovery_then_fresh_signing(self): + # GnuPG agent sockets must fit the platform's short Unix socket path limit. + gnupg = self.root / "historical-gnupg" + gnupg.mkdir(mode=0o700) + date = int(time.time()) - 3 * 86400 + + def run(*command): + result = subprocess.run(list(map(str, command)), capture_output=True, text=True, check=False) + self.assertEqual(result.returncode, 0, result.stderr) + return result.stdout + + try: + run("gpg", "--homedir", gnupg, "--batch", "--pinentry-mode", "loopback", "--passphrase", "", + "--faked-system-time", f"{date - 60}!", "--quick-generate-key", "Historical fixture", "rsa2048", "sign", "1y") + listing = run("gpg", "--homedir", gnupg, "--with-colons", "--list-keys") + fingerprint = next(line.split(":")[9] for line in listing.splitlines() if line.startswith("fpr:")) + key = self.case_dir / "historical.asc" + key.write_text(run("gpg", "--homedir", gnupg, "--armor", "--export", fingerprint)) + candidate = self.case_dir / "expired" + run(sys.executable, SCRIPT.with_name("apt-repository.py"), "build", "--release-dir", self.fixtures.release1, + "--version", "1.0.0", "--output", candidate, "--signing-key", fingerprint, + "--gnupg-home", gnupg, "--date", date, "--valid-for-days", "1", + "--release-public-key", self.fixtures.release_public) + + def interrupt(label): + if label == "journal": + raise InterruptedError("interrupted before promotion three days ago") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(self.origin, candidate, fingerprint, "empty") + options = ["--public-key", key, "--fingerprint", fingerprint] + rejected = self.command("recover", *options, ok=False) + self.assertNotEqual(rejected.returncode, 0) + dry_run = self.command("recover", "--allow-expired", "--dry-run", *options) + self.assertTrue(json.loads(dry_run.stdout)["requiresRefresh"]) + self.assertIsNotNone(publisher.state(self.origin, "pending")) + completed = self.command("recover", "--allow-expired", *options) + self.assertTrue(json.loads(completed.stdout)["requiresRefresh"]) + self.assertIn("APT rejects", json.loads(completed.stdout)["nextAction"]) + self.command("fetch", "--output", self.case_dir / "expired-fetched", *options) + rejected = self.command("publish", "--repository", candidate, "--expected-revision", "empty", *options, ok=False) + self.assertNotEqual(rejected.returncode, 0) + finally: + subprocess.run(["gpgconf", "--homedir", str(gnupg), "--kill", "gpg-agent"], check=False) + + def test_actual_ssh_publish_fetch_host_trust_and_no_remote_gpg(self): + if not WITH_SSH: + self.skipTest("use --with-ssh inside a disposable Docker container") + self.assertTrue(Path("/.dockerenv").exists() and os.geteuid() == 0, + "--with-ssh is restricted to root inside a disposable Docker container") + sshd = shutil.which("sshd") + self.assertIsNotNone(sshd, "openssh-server is required for --with-ssh") + identity = self.case_dir / "identity" + host_key = self.case_dir / "host-key" + for key in (identity, host_key): + subprocess.run(["ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-f", str(key)], check=True) + with socket.socket() as listener: + listener.bind(("127.0.0.1", 0)) + port = listener.getsockname()[1] + known = self.case_dir / "known_hosts" + known.write_text(f"[127.0.0.1]:{port} " + host_key.with_suffix(".pub").read_text()) + remote_bin = self.case_dir / "remote-bin" + remote_bin.mkdir() + (remote_bin / "python3").symlink_to(sys.executable) + configuration = self.case_dir / "sshd.conf" + configuration.write_text( + f"Port {port}\nListenAddress 127.0.0.1\nHostKey {host_key}\n" + f"PidFile {self.case_dir / 'sshd.pid'}\nAuthorizedKeysFile {identity}.pub\n" + "PermitRootLogin prohibit-password\nPasswordAuthentication no\nKbdInteractiveAuthentication no\n" + f"UsePAM no\nStrictModes no\nAllowUsers root\nLogLevel ERROR\nSetEnv PATH={remote_bin}\n") + Path("/run/sshd").mkdir(exist_ok=True) + with (self.case_dir / "sshd.log").open("wb") as log: + server = subprocess.Popen([sshd, "-D", "-e", "-f", str(configuration)], stdout=log, stderr=log) + try: + for _ in range(100): + self.assertIsNone(server.poll(), "temporary sshd exited") + try: + with socket.create_connection(("127.0.0.1", port), timeout=0.1): + break + except OSError: + time.sleep(0.05) + options = ["--ssh", "root@127.0.0.1", "--ssh-port", str(port), + "--identity-file", identity, "--known-hosts", known] + connection = argparse.Namespace(ssh="root@127.0.0.1", ssh_port=port, + identity_file=identity, known_hosts=known) + probe = publisher.ssh_command(connection, {}) + probe[-1] = "python3 -c 'import shutil; assert shutil.which(\"gpg\") is None'" + checked = subprocess.run(probe, capture_output=True, text=True, check=False) + self.assertEqual(checked.returncode, 0, checked.stderr) + # The server executes only the transported publisher source; no + # generator, GPG executable, or private signing key is uploaded. + self.publish(*options) + self.assertEqual(json.loads(self.publish(*options).stdout)["status"], "unchanged") + self.command("fetch", "--output", self.case_dir / "ssh-fetched", *options) + self.command("publish", "--repository", self.upgraded, "--expected-revision", self.one["revision"], *options) + result = self.command("publish", "--repository", self.fixtures.base, + "--expected-revision", "empty", *options, ok=False) + self.assertNotEqual(result.returncode, 0) + known.write_text("") + result = self.command("fetch", "--output", self.case_dir / "untrusted", *options, ok=False) + self.assertNotEqual(result.returncode, 0) + self.assertFalse((self.case_dir / "untrusted").exists()) + finally: + server.terminate() + server.wait(timeout=10) + + +if __name__ == "__main__": + if "--with-ssh" in sys.argv: + WITH_SSH = True + sys.argv.remove("--with-ssh") + unittest.main(verbosity=2) diff --git a/scripts/verify-apt-public.py b/scripts/verify-apt-public.py new file mode 100755 index 0000000..ab0e5ea --- /dev/null +++ b/scripts/verify-apt-public.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +"""Verify served repository bytes before producing a homepage activation record.""" +import argparse +from datetime import datetime, timezone +import hashlib +import json +from pathlib import Path +import re +import ssl +import subprocess +import sys +import tempfile +import urllib.error +import urllib.parse +import urllib.request + + +class NoRedirects(urllib.request.HTTPRedirectHandler): + def redirect_request(self, request, response, code, message, headers, new_url): + response.close() + raise ValueError("repository verification does not follow redirects; use the canonical HTTPS URL") + + +def validate_base_url(value): + base = urllib.parse.urlsplit(value) + if (base.scheme != "https" or not base.hostname or base.username or base.password + or any(char in value for char in "\\'\"`$<>?#") + or any(ord(char) <= 32 or ord(char) >= 127 for char in value)): + raise ValueError("base URL must be HTTPS without credentials, whitespace, query, fragment, or shell metacharacters") + if base.port is not None and not 1 <= base.port <= 65535: + raise ValueError("invalid HTTPS port in base URL") + return value.rstrip("/") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repository", required=True, type=Path) + parser.add_argument("--public-key", required=True, type=Path) + parser.add_argument("--fingerprint", required=True) + parser.add_argument("--base-url", required=True) + parser.add_argument("--ca-file", type=Path, help="custom CA for isolated test servers") + parser.add_argument("--proof-url", help="GitHub Actions run URL required for --output") + parser.add_argument("--output", type=Path, help="write verified public-channel configuration after all checks") + args = parser.parse_args() + base_url = validate_base_url(args.base_url) + if args.output and args.ca_file: + raise ValueError("custom-CA fixture checks cannot produce public activation records") + if args.output and (not args.proof_url or not re.fullmatch( + r"https://github\.com/sandwichfarm/loopwire/actions/runs/[1-9][0-9]*", args.proof_url)): + raise ValueError("activation output requires the verifying GitHub Actions run URL") + fingerprint = args.fingerprint.upper() + if not re.fullmatch(r"[A-F0-9]{40}", fingerprint): + raise ValueError("a complete OpenPGP fingerprint is required") + validator = Path(__file__).with_name("apt-repository.py") + subprocess.run([ + sys.executable, str(validator), "verify", "--repository", str(args.repository), + "--public-key", str(args.public_key), "--fingerprint", fingerprint, + ], check=True, stdout=subprocess.PIPE) + manifest = json.loads((args.repository / "repository-manifest.json").read_text()) + context = ssl.create_default_context(cafile=str(args.ca_file) if args.ca_file else None) + opener = urllib.request.build_opener(NoRedirects(), urllib.request.HTTPSHandler(context=context)) + for entry in manifest["files"]: + url = base_url + "/" + urllib.parse.quote(entry["path"], safe="/+") + request = urllib.request.Request(url, headers={"Cache-Control": "no-cache", "User-Agent": "Loopwire-APT-Proof/1"}) + digest, size = hashlib.sha256(), 0 + try: + response = opener.open(request, timeout=30) + except urllib.error.HTTPError as error: + status = error.code + error.close() + raise ValueError(f"repository returned HTTP {status} for {entry['path']}") from None + with response: + if response.status != 200: + raise ValueError(f"repository returned HTTP {response.status} for {entry['path']}") + while chunk := response.read(1024 * 1024): + size += len(chunk) + if size > entry["size"]: + raise ValueError(f"public file is larger than expected: {entry['path']}") + digest.update(chunk) + if size != entry["size"] or digest.hexdigest() != entry["sha256"]: + raise ValueError(f"public file differs from the verified candidate: {entry['path']}") + record = { + "schemaVersion": 1, + "status": "verified", + "baseUrl": base_url, + "signingFingerprint": fingerprint, + "revision": manifest["revision"], + "verifiedAt": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"), + "proofUrl": args.proof_url, + } + if args.output: + args.output.parent.mkdir(parents=True, exist_ok=True) + with tempfile.NamedTemporaryFile(mode="w", dir=args.output.parent, delete=False) as temporary: + json.dump(record, temporary, indent=2) + temporary.write("\n") + temporary_path = Path(temporary.name) + temporary_path.replace(args.output) + print(json.dumps({"status": "verified", "revision": manifest["revision"], "files": len(manifest["files"])})) + + +if __name__ == "__main__": + try: + main() + except (OSError, ValueError, KeyError, subprocess.CalledProcessError, urllib.error.URLError) as error: + print(f"verify-apt-public: {error}", file=sys.stderr) + sys.exit(1) diff --git a/scripts/verify-apt-repository-vm-proof.mjs b/scripts/verify-apt-repository-vm-proof.mjs new file mode 100644 index 0000000..c29e9fc --- /dev/null +++ b/scripts/verify-apt-repository-vm-proof.mjs @@ -0,0 +1,261 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { lstat, readFile, readdir } from "node:fs/promises"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const repositoryRoot = path.dirname(path.dirname(fileURLToPath(import.meta.url))); +const requiredPaths = [ + "/usr/bin/loopwire", "/usr/bin/loopwire-dsp-provider", "/usr/bin/loopwire-jack-ports", + "/usr/bin/loopwire-detect-audio", "/usr/lib/loopwire/loopwire-gui", + "/usr/share/applications/loopwire.desktop", "/usr/share/icons/hicolor/scalable/apps/loopwire.svg", +]; + +function requireThat(condition, message) { + if (!condition) throw new Error(message); +} +async function bytes(directory, name) { + const file = path.join(directory, name); + const stat = await lstat(file); + requireThat(stat.isFile() && !stat.isSymbolicLink(), `evidence must be a regular file: ${name}`); + return readFile(file); +} +async function text(directory, name, nonempty = true) { + const result = (await bytes(directory, name)).toString("utf8"); + requireThat(!nonempty || result.trim(), `empty evidence: ${name}`); + return result; +} +function tsvMap(value, label) { + const map = new Map(); + for (const line of value.trimEnd().split("\n")) { + const separator = line.indexOf("\t"); + requireThat(separator > 0, `${label} must contain key/value TSV`); + const key = line.slice(0, separator); + requireThat(!map.has(key), `${label} repeats ${key}`); + map.set(key, line.slice(separator + 1)); + } + return map; +} +function equal(actual, expected, label) { + requireThat(actual === expected, `${label}: expected ${JSON.stringify(expected)}, got ${JSON.stringify(actual)}`); +} +function command(program, args) { + const result = spawnSync(program, args, { encoding: "utf8", maxBuffer: 16 * 1024 * 1024 }); + requireThat(!result.error && result.status === 0, + `${program} verification failed: ${result.error?.message ?? result.stderr ?? result.stdout}`); + return result.stdout; +} +function sha256(buffer) { return createHash("sha256").update(buffer).digest("hex"); } + +export function parseInstalledHashes(value) { + const result = {}; + for (const line of value.trimEnd().split("\n")) { + const match = /^([a-f0-9]{64}) {2}(\/usr\/[^\r\n]+)$/.exec(line); + requireThat(match && !match[2].split("/").includes(".."), "invalid installed-file checksum record"); + requireThat(!Object.hasOwn(result, match[2]), `duplicate installed path: ${match[2]}`); + result[match[2]] = match[1]; + } + return result; +} + +export function debPayload(packageFile) { + // Read the signed .deb payload without extracting or executing package content. + return JSON.parse(command("python3", ["-c", ` +import hashlib, io, json, pathlib, sys, tarfile +raw = pathlib.Path(sys.argv[1]).read_bytes() +assert raw[:8] == b'!\\n', 'invalid deb archive' +position = 8 +payload = None +while position < len(raw): + header = raw[position:position + 60] + assert len(header) == 60 and header[58:60] == b'\\x60\\n', 'invalid ar header' + size = int(header[48:58].decode().strip()) + assert size >= 0 and position + 60 + size <= len(raw), 'truncated ar member' + name = header[:16].decode().strip().rstrip('/') + content = raw[position + 60:position + 60 + size] + if name.startswith('data.tar'): + assert payload is None, 'multiple package payloads' + payload = content + position += 60 + size + size % 2 +assert payload is not None, 'missing package payload' +files = {} +with tarfile.open(fileobj=io.BytesIO(payload), mode='r:*') as archive: + for member in archive: + if not member.isfile(): + continue + relative = pathlib.PurePosixPath(member.name) + assert not relative.is_absolute() and '..' not in relative.parts, 'unsafe package path' + name = '/' + str(relative) + assert name.startswith('/usr/') and name not in files, 'unexpected package path' + files[name] = hashlib.sha256(archive.extractfile(member).read()).hexdigest() +print(json.dumps(files)) +`, packageFile])); +} + +export function verifyLifecycle(value, baselineVersion, upgradeVersion) { + equal(value.trimEnd(), [ + `install\t${baselineVersion}\tinstalled`, `reinstall\t${baselineVersion}\tinstalled`, + `upgrade\t${upgradeVersion}\tinstalled`, `rollback\t${baselineVersion}\tinstalled`, + `remove\t${baselineVersion}\tabsent`, + ].join("\n"), "lifecycle transitions"); +} + +export async function verifyInstalledStage(directory, stage, version, baseUrl, payloadHashes) { + const prefix = `${stage}/`; + equal((await text(directory, `${prefix}package-metadata.tsv`)).trim(), + `loopwire\t${version}\tamd64\tinstall ok installed`, `${stage} package metadata`); + const files = (await text(directory, `${prefix}package-files.txt`)).trim().split("\n"); + for (const installed of requiredPaths) requireThat(files.includes(installed), `${stage} missing ${installed}`); + const hashes = parseInstalledHashes(await text(directory, `${prefix}installed-files.sha256`)); + assert.deepEqual(hashes, payloadHashes, `${stage} installed bytes must equal the signed package payload`); + const policy = await text(directory, `${prefix}apt-policy.txt`); + requireThat(policy.includes(baseUrl) && policy.includes(`Installed: ${version}`), `${stage} lacks installed version/repository origin`); + for (const help of ["background-help.txt", "dsp-provider-help.txt", "jack-provider-help.txt"]) { + await text(directory, `${prefix}${help}`); + } + const detection = JSON.parse(await text(directory, `${prefix}detect-audio.json`)); + requireThat(detection && typeof detection === "object", `${stage} detection must be JSON object/array`); + const linkage = await text(directory, `${prefix}gui-ldd.txt`); + requireThat(!linkage.includes("not found") && /libgtk-3/.test(linkage) && /libwebkit2gtk/.test(linkage), `${stage} GUI linkage missing or unresolved`); + equal((await text(directory, `${prefix}gui-launch-status.txt`)).trim(), "0", `${stage} GUI launch`); + requireThat(/^\d+(?:\n\d+)*\n?$/.test(await text(directory, `${prefix}gui-window-ids.txt`)), `${stage} lacks X11 window ids`); + const names = (await text(directory, `${prefix}gui-window-names.txt`)).trim().split("\n"); + requireThat(names.every((name) => /^(Loopwire|loopwire-gui)$/.test(name)), `${stage} lacks Loopwire application window`); + const launch = await text(directory, `${prefix}gui-launch.log`, false); + requireThat(!/error while loading shared libraries|panic|protocol error|missing acquire timeline/i.test(launch), `${stage} fatal GUI log`); + await text(directory, `${prefix}xvfb.log`, false); +} + +export async function verifyEvidence({ target, evidenceDir, gitHead }) { + requireThat(["ubuntu-24.04", "debian-13"].includes(target), "supported --target is required"); + requireThat(/^[a-f0-9]{40}$/.test(gitHead ?? ""), "--git-head must be a full lowercase commit hash"); + const summary = tsvMap(await text(evidenceDir, "summary.tsv"), "summary"); + equal(summary.get("schema"), "loopwire.apt-repository-vm-proof.v1", "schema"); + equal(summary.get("target"), target, "target"); + equal(summary.get("git_head"), gitHead, "summary commit"); + equal((await text(evidenceDir, "git-head.txt")).trim(), gitHead, "evidence commit"); + equal(summary.get("payload_kind"), "cached-release-lifecycle-fixture", "payload provenance kind"); + const version = summary.get("version"); + requireThat(/^(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)(?:\+[0-9A-Za-z]+(?:\.[0-9A-Za-z]+)*)?$/.test(version ?? ""), "invalid baseline version"); + const upgradedVersion = `${version}${version.includes("+") ? "." : "+"}aptfixture1`; + equal(summary.get("upgrade_version"), upgradedVersion, "synthetic fixture version"); + const suffix = target === "ubuntu-24.04" ? "1ubuntu24.04" : "1debian13"; + const baselineVersion = `${version}-${suffix}`; + const upgradeVersion = `${upgradedVersion}-${suffix}`; + equal(summary.get("baseline_package_version"), baselineVersion, "baseline package version"); + equal(summary.get("upgrade_package_version"), upgradeVersion, "upgrade package version"); + const fingerprint = summary.get("fingerprint"); + requireThat(/^[A-F0-9]{40}$/.test(fingerprint ?? ""), "invalid signing fingerprint"); + const baseUrl = summary.get("base_url"); + equal(baseUrl, "https://127.0.0.1:8443", "guest-only HTTPS origin"); + const epoch = summary.get("verification_epoch"); + requireThat(/^[0-9]{10}$/.test(epoch ?? "") && Number(epoch) <= Date.now() / 1000 + 300, "invalid proof timestamp"); + const os = new Map((await text(evidenceDir, "os-release")).split("\n").filter((line) => /^[A-Z_]+=/.test(line)).map((line) => { + const separator = line.indexOf("="); + return [line.slice(0, separator), line.slice(separator + 1).replace(/^"|"$/g, "")]; + })); + equal(os.get("ID"), target === "ubuntu-24.04" ? "ubuntu" : "debian", "guest OS"); + equal(os.get("VERSION_ID"), target === "ubuntu-24.04" ? "24.04" : "13", "guest OS version"); + requireThat(["kvm", "qemu"].includes((await text(evidenceDir, "virtualization.txt")).trim()), "not a VM proof"); + requireThat((await text(evidenceDir, "uname.txt")).includes("Linux"), "guest kernel evidence missing"); + await text(evidenceDir, "console.log"); + const targetRows = (await readFile(path.join(repositoryRoot, "packaging/vm/native-package-targets.tsv"), "utf8")) + .split("\n").map((line) => line.split("\t")).filter((row) => row[0] === target); + requireThat(targetRows.length === 1, "target missing or duplicate in image manifest"); + const row = targetRows[0]; + const image = tsvMap(await text(evidenceDir, "image.tsv"), "image"); + for (const [key, expected] of Object.entries({ schema: "loopwire.native-package-image.v1", target, + distro: row[1], url: row[4], checksum_algorithm: row[5], checksum: row[6], actual_checksum: row[6], firmware: row[8] })) { + equal(image.get(key), expected, `image ${key}`); + } + equal((await text(evidenceDir, "initial-package-status.txt")).trim(), "absent", "clean guest status"); + requireThat(/^[a-f0-9]{64} {2}loopwire-linux-x86_64.tar.gz\n?$/.test(await text(evidenceDir, "release-payload.sha256")), "missing original payload digest"); + await text(evidenceDir, "payload-release.txt"); + const source = await text(evidenceDir, "loopwire.sources"); + for (const line of [`URIs: ${baseUrl}`, `Suites: ${target}`, "Components: main", "Architectures: amd64", + `Signed-By: /etc/apt/keyrings/loopwire-${fingerprint}.asc`]) requireThat(source.split("\n").includes(line), `APT source lacks ${line}`); + requireThat(!/Trusted:|Allow-Insecure:/i.test(source), "APT proof bypasses authentication"); + equal(await text(evidenceDir, "https-key.asc"), await text(evidenceDir, "repository-key.asc"), "HTTPS public key"); + command("openssl", ["verify", "-attime", epoch, "-CAfile", path.join(evidenceDir, "tls-ca.crt"), + "-verify_ip", "127.0.0.1", path.join(evidenceDir, "tls-server.crt")]); + + const packageHashes = {}; + const payloadHashes = {}; + const packageNames = await readdir(path.join(evidenceDir, "packages")); + equal(packageNames.length, 2, "package evidence count"); + for (const packageVersion of [baselineVersion, upgradeVersion]) { + const name = `loopwire_${packageVersion}_amd64.deb`; + requireThat(packageNames.includes(name), `missing package ${name}`); + packageHashes[packageVersion] = sha256(await bytes(evidenceDir, `packages/${name}`)); + payloadHashes[packageVersion] = debPayload(path.join(evidenceDir, "packages", name)); + } + for (const stage of ["initial", "upgraded", "rolled-back"]) { + const directory = path.join(evidenceDir, "repositories", stage); + const result = command("bash", [path.join(repositoryRoot, "scripts/with-apt-tools.sh"), "--read-only-path", evidenceDir, + "python3", path.join(repositoryRoot, "scripts/apt-repository.py"), "verify", "--repository", directory, + "--public-key", path.join(evidenceDir, "repository-key.asc"), "--fingerprint", fingerprint, "--now", epoch]); + JSON.parse(result); + JSON.parse(await text(evidenceDir, `repositories/${stage}-verification.json`)); + const snapshot = JSON.parse(await text(directory, "repository-manifest.json")); + const served = JSON.parse(await text(evidenceDir, `repositories/${stage}-public-verification.json`)); + equal(served.status, "verified", `${stage} HTTPS verification`); + equal(served.revision, snapshot.revision, `${stage} HTTPS revision`); + equal(served.files, snapshot.files.length, `${stage} HTTPS file count`); + const packages = (await text(directory, `dists/${target}/main/binary-amd64/Packages`)).trim().split(/\n\n+/).map((block) => { + return new Map(block.split("\n").filter((line) => /^[^ :]+:/.test(line)).map((line) => { + const separator = line.indexOf(":"); + return [line.slice(0, separator), line.slice(separator + 1).trim()]; + })); + }); + for (const expectedVersion of stage === "upgraded" ? [baselineVersion, upgradeVersion] : [baselineVersion]) { + const matches = packages.filter((item) => item.get("Package") === "loopwire" && item.get("Version") === expectedVersion); + requireThat(matches.length === 1, `${stage} must contain exactly one ${expectedVersion} package`); + equal(matches[0].get("SHA256"), packageHashes[expectedVersion], `${stage} signed package digest`); + equal(matches[0].get("Architecture"), "amd64", `${stage} signed architecture`); + } + if (stage !== "upgraded") requireThat(!packages.some((item) => item.get("Version") === upgradeVersion), `${stage} unexpectedly advertises upgrade`); + } + verifyLifecycle(await text(evidenceDir, "lifecycle.tsv"), baselineVersion, upgradeVersion); + for (const [stage, expectedVersion] of Object.entries({ install: baselineVersion, reinstall: baselineVersion, upgrade: upgradeVersion, rollback: baselineVersion })) { + await verifyInstalledStage(evidenceDir, stage, expectedVersion, baseUrl, payloadHashes[expectedVersion]); + const log = await text(evidenceDir, `${stage}.log`); + requireThat(log.includes("loopwire") && log.includes(expectedVersion), `${stage} lacks APT operation/version log`); + } + const commands = await text(evidenceDir, "commands.log"); + for (const needle of ["apt-get install -y loopwire=", "apt-get install --reinstall", "apt-get install --only-upgrade", + "apt-get install --allow-downgrades", "apt-get remove -y loopwire", "smoke_installed", "xdotool"]) { + requireThat(commands.includes(needle), `missing executed command: ${needle}`); + } + for (const file of ["bootstrap.log", "bootstrap-update.log", "upgrade-update.log", "rollback-update.log", "remove.log", "source-removal.log", "source-removal-update.log"]) { + await text(evidenceDir, file); + } + const requests = await text(evidenceDir, "https-server.log"); + requireThat(requests.includes(`/keys/${fingerprint}.asc`) && requests.includes(`/dists/${target}/InRelease`) && requests.includes(".deb"), "missing real HTTPS key/index/package request evidence"); + const removal = tsvMap(await text(evidenceDir, "removed-files.tsv"), "removed paths"); + for (const removed of [...requiredPaths.filter((value) => value !== "/usr/lib/loopwire/loopwire-gui"), "/usr/lib/loopwire"]) { + equal(removal.get(removed), "absent", `removed ${removed}`); + } + requireThat(!(await text(evidenceDir, "source-removal-policy.txt", false)).includes(baseUrl), "removed APT source remains active"); + return { target, gitHead, baselineVersion, upgradeVersion, fingerprint, packageHashes }; +} + +if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { + const args = {}; + for (let index = 2; index < process.argv.length; index += 2) { + const option = process.argv[index]; + requireThat(["--target", "--evidence-dir", "--git-head"].includes(option) && process.argv[index + 1], `invalid option: ${option}`); + requireThat(!Object.hasOwn(args, option), `duplicate option: ${option}`); + args[option] = process.argv[index + 1]; + } + requireThat(args["--evidence-dir"], "--evidence-dir is required"); + const result = await verifyEvidence({ target: args["--target"], evidenceDir: path.resolve(args["--evidence-dir"]), gitHead: args["--git-head"] }); + console.log(`APT repository VM proof verified: ${result.target}`); + console.log(JSON.stringify(result)); + } catch (error) { + console.error(`verify-apt-repository-vm-proof: ${error.message}`); + process.exitCode = 1; + } +} diff --git a/scripts/verify-apt-repository.sh b/scripts/verify-apt-repository.sh new file mode 100755 index 0000000..5a070a1 --- /dev/null +++ b/scripts/verify-apt-repository.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +set -euo pipefail +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +if [ "${1:-}" != --inside ]; then + exec bash "$root/scripts/with-apt-tools.sh" --container bash "$root/scripts/verify-apt-repository.sh" --inside +fi +cd "$root" +export PYTHONDONTWRITEBYTECODE=1 +python3 scripts/test-apt-repository.py +python3 scripts/test-publish-package-repository.py --with-ssh +python3 scripts/test-apt-bootstrap.py +python3 scripts/test-apt-public.py +python3 scripts/test-apt-workflow-preflight.py +node scripts/test-apt-repository-vm-proof.mjs +node --test apps/site/src/lib/aptChannel.test.mjs +echo 'APT repository development verification passed.' diff --git a/scripts/verify-docs.sh b/scripts/verify-docs.sh index 76c7a5e..713dae8 100644 --- a/scripts/verify-docs.sh +++ b/scripts/verify-docs.sh @@ -6,6 +6,7 @@ root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" required_files=( "apps/docs/docs/index.md" "apps/docs/docs/guide/install.md" + "apps/docs/docs/guide/apt-repository.md" "apps/docs/docs/guide/basic-usage.md" "apps/docs/docs/guide/start-on-boot.md" "apps/docs/docs/guide/backends.md" @@ -17,6 +18,7 @@ required_files=( "apps/docs/docs/developer/screenshots.md" "apps/docs/docs/developer/vm-matrix.md" "apps/docs/docs/developer/release.md" + "apps/docs/docs/developer/apt-repository.md" "apps/docs/docs/developer/release-notes.md" "apps/docs/docs/release-notes/0.1.0.md" "apps/docs/docs/release-notes/unreleased.md" @@ -63,6 +65,12 @@ node "$root/scripts/verify-support-matrix.mjs" assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/support-matrix" assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/troubleshooting" assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/basic-usage" +assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/apt-repository" +assert_contains "apps/docs/docs/.vitepress/config.ts" "/developer/apt-repository" +assert_contains "apps/docs/docs/guide/apt-repository.md" "Signed-By" +assert_contains "apps/docs/docs/guide/apt-repository.md" "--allow-downgrades" +assert_contains "apps/docs/docs/developer/apt-repository.md" "APT_REPOSITORY_ENABLED" +assert_contains "apps/docs/docs/developer/apt-repository.md" "Final activation is a human operation" assert_contains "apps/docs/docs/.vitepress/config.ts" 'base: "/docs/"' assert_contains "apps/docs/docs/.vitepress/config.ts" "/developer/release-notes" assert_contains "apps/docs/docs/.vitepress/config.ts" "/release-notes/0.1.0" diff --git a/scripts/verify-github-workflows.sh b/scripts/verify-github-workflows.sh index b37e623..2c072d5 100755 --- a/scripts/verify-github-workflows.sh +++ b/scripts/verify-github-workflows.sh @@ -95,6 +95,7 @@ fi workflows=( ".github/workflows/ci.yml" + ".github/workflows/publish-apt.yml" ".github/workflows/web.yml" ".github/workflows/aur.yml" ".github/workflows/workflow-checks.yml" @@ -335,6 +336,7 @@ assert_contains ".github/workflows/vm-matrix.yml" "apps/docs/docs/guide/support- ruby "$root/scripts/test-ci-impact.rb" ruby "$root/scripts/test-ci-workflow-paths.rb" +ruby "$root/scripts/test-apt-workflow.rb" node "$root/scripts/test-native-package-proof-snapshot.mjs" echo "GitHub workflow contract verification passed." diff --git a/scripts/verify-scripts.sh b/scripts/verify-scripts.sh index 779e62a..bc29593 100755 --- a/scripts/verify-scripts.sh +++ b/scripts/verify-scripts.sh @@ -101,6 +101,8 @@ node --check scripts/verify-vm-evidence-archive-manifest.mjs node --check scripts/release-asset-manifest.mjs node --check scripts/verify-native-package-vm-proof.mjs node --check scripts/verify-native-package-proof-snapshot.mjs +node --check scripts/verify-apt-repository-vm-proof.mjs +node --check scripts/test-apt-repository-vm-proof.mjs bash scripts/build-portable-linux-binary.sh -- --help | grep -Fq -- "--output FILE" || { echo "verify-scripts: portable builder does not accept the package-script separator" >&2 exit 1 diff --git a/scripts/with-apt-tools.sh b/scripts/with-apt-tools.sh new file mode 100755 index 0000000..3bf2595 --- /dev/null +++ b/scripts/with-apt-tools.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +set -euo pipefail + +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +image="${LOOPWIRE_APT_TOOLS_IMAGE:-loopwire-apt-tools:debian-13}" +force_container=false +mounts=() +while [ "$#" -gt 0 ]; do + case "$1" in + --container) force_container=true; shift ;; + --read-only-path) + path="$(realpath -e "${2:?missing --read-only-path value}")" + mounts+=(--volume "$path:$path:ro") + shift 2 + ;; + *) break ;; + esac +done +[ "$#" -gt 0 ] || { echo 'Usage: with-apt-tools.sh [--container] [--read-only-path PATH] COMMAND [ARG ...]' >&2; exit 2; } +available=true +for command in python3 dpkg dpkg-deb gpg gpgv openssl; do + command -v "$command" >/dev/null 2>&1 || available=false +done +export PYTHONDONTWRITEBYTECODE=1 +if [ "$available" = true ] && [ "$force_container" = false ]; then + exec "$@" +fi +command -v docker >/dev/null 2>&1 || { echo 'APT tools or Docker are required; see the APT developer guide.' >&2; exit 1; } +if ! docker image inspect "$image" >/dev/null 2>&1; then + docker build --file "$root/packaging/repositories/Dockerfile.apt-tools" --tag "$image" "$root" >&2 +fi +# Keep absolute source/evidence paths valid for callers. Test writes belong in the disposable /tmp tree. +exec docker run --rm --network none --env PYTHONDONTWRITEBYTECODE=1 \ + --volume "$root:$root:ro" "${mounts[@]}" --workdir "$root" "$image" "$@" From 389db7c40045fa85060a350d089a33c1d2c8b856 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 15:51:15 +0200 Subject: [PATCH 3/6] Keep APT lifecycle proof consistent across target guests Debian installs GnuPG without the standalone gpgv verifier, while Ubuntu made it available transitively. Declare the verifier explicitly so both clean guest runs exercise the same signed repository checks. Keep the project validation contract synchronized with the new dedicated APT gate. Constraint: Guest proof must use clean distribution package state Confidence: high Scope-risk: narrow Tested: Requirements contract, Bash syntax, ShellCheck and whitespace checks Not-tested: Fresh Ubuntu and Debian KVM lifecycle reruns follow this commit --- packaging/vm/guest-apt-repository-smoke.sh | 2 +- scripts/verify-requirements.sh | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/packaging/vm/guest-apt-repository-smoke.sh b/packaging/vm/guest-apt-repository-smoke.sh index 8077feb..dbe955e 100755 --- a/packaging/vm/guest-apt-repository-smoke.sh +++ b/packaging/vm/guest-apt-repository-smoke.sh @@ -47,7 +47,7 @@ tar -xOf "$kit_dir/release/loopwire-linux-x86_64.tar.gz" RELEASE >"$proof_dir/pa sudo apt-get update sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - apt-utils ca-certificates curl dpkg-dev gnupg nodejs openssl python3 xdotool xz-utils xvfb + apt-utils ca-certificates curl dpkg-dev gnupg gpgv nodejs openssl python3 xdotool xz-utils xvfb # Signing material is generated inside this disposable guest and never copied into evidence. gnupg_home="$fixture_dir/gnupg" diff --git a/scripts/verify-requirements.sh b/scripts/verify-requirements.sh index 119a853..7dce409 100644 --- a/scripts/verify-requirements.sh +++ b/scripts/verify-requirements.sh @@ -124,7 +124,8 @@ done assert_script "package.json" "check" "pnpm check:verify && pnpm lint && pnpm typecheck && pnpm test && pnpm build && pnpm verify:site" assert_script "package.json" "check:verify" \ - "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri" + "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri && pnpm verify:apt" +assert_script "package.json" "verify:apt" "bash scripts/verify-apt-repository.sh" assert_script "package.json" "verify:requirements" "bash scripts/verify-requirements.sh" assert_script "package.json" "setup:github" "node scripts/setup-github-actions.mjs" assert_script "package.json" "test:setup-github" "node scripts/test-setup-github-actions.mjs" From 7849a1b6d6dcd543e2e7c4c9906ecb90135ffa73 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 15:58:35 +0200 Subject: [PATCH 4/6] Fail safely across public verification and guest TLS differences Compare the served repository manifest as well as every indexed file before emitting an activation record, and prevent custom test trust roots from producing production activation data. Make the disposable HTTPS server send a proper TLS shutdown so Debian's stricter APT client verifies the same fixture as Ubuntu. Correct the operator guide to match the publication protocol. Constraint: Fixture CA verification may prove behavior but cannot activate the public channel Confidence: high Scope-risk: narrow Tested: 9 public HTTPS verification cases and 16 APT VM proof-verifier cases Tested: Dedicated APT suite, workflow contracts, docs/site build, actionlint, Bash/Node/Python syntax and ShellCheck Not-tested: Fresh matching-guest lifecycle reruns follow this commit --- apps/docs/docs/developer/apt-repository.md | 9 +++++---- packaging/vm/guest-apt-repository-smoke.sh | 10 +++++++++- scripts/test-apt-public.py | 13 ++++++++++--- scripts/verify-apt-public.py | 9 +++++++-- 4 files changed, 31 insertions(+), 10 deletions(-) diff --git a/apps/docs/docs/developer/apt-repository.md b/apps/docs/docs/developer/apt-repository.md index ea83833..14ea678 100644 --- a/apps/docs/docs/developer/apt-repository.md +++ b/apps/docs/docs/developer/apt-repository.md @@ -15,7 +15,7 @@ Two signatures have different jobs: 1. The existing project **OpenSSL release key** authenticates `SHA256SUMS.sig` over `SHA256SUMS`. The generator checks the exact Ubuntu and Debian artifacts against that manifest and their internal package metadata before indexing. -2. A separate **OpenPGP APT key** signs `InRelease` and `Release.gpg`. APT authenticates metadata and follows its SHA-256 +2. A separate **OpenPGP APT key** signs the clear-signed `InRelease`. APT authenticates metadata and follows its SHA-256 hashes through `Packages` to each deb. Clients trust that key only for the Loopwire source through `Signed-By`. The web root contains public packages, indexes, signed release metadata, and public keys. Private signing material, @@ -33,7 +33,7 @@ contract for repository metadata. The ordinary website deployment remains separa Configure HTTP caching so clients cannot receive a new index behind stale release metadata: -- `InRelease`, `Release`, `Release.gpg`, and ordinary `Packages`/compressed indexes: `Cache-Control: no-store` or +- `InRelease`, `Release`, and ordinary `Packages`/compressed indexes: `Cache-Control: no-store` or equivalent mandatory revalidation. - Content-addressed `by-hash` indexes and immutable package pool paths: long cache lifetime with `immutable`. - Public keys and the current manifest: revalidate. Do not cache failures for paths that will soon be published. @@ -102,7 +102,7 @@ input, or `rollback` with the retained 64-character `revision`. The weekly run s For a reviewed local rehearsal, the same publisher can operate without SSH. For production, supply all SSH identity and host-key arguments. In these examples `APT_ROOT`, `APT_HOST`, and `APT_REVISION` name the provisioned root, host, and -the current manifest revision. `APT_REVISION` is the empty string for an initial publication only. +the current manifest revision. Set `APT_REVISION=empty` for an initial publication only. ```bash python3 scripts/publish-package-repository.py fetch \ @@ -246,7 +246,8 @@ pnpm vm:native-packages -- verify-apt --target debian-13 ``` These boot fresh checksum-pinned distro guests, use HTTPS APT with scoped trust, and exercise package installation, -reinstall, upgrade, downgrade/rollback, removal, and rejection of untrusted or broken repository state. Record exact +reinstall, upgrade, downgrade/rollback, and removal. The container regression suite separately proves rejection of +untrusted or broken repository state. Record exact versions, source URLs, key fingerprint, candidate selection, signature results, GUI launch, and provider-command checks. A synthetic `+aptfixture1` upgrade reuses the authenticated `0.1.0` payload to test lifecycle behavior; label it as fixture evidence, never as a newly released application or public production proof. A package lifecycle pass diff --git a/packaging/vm/guest-apt-repository-smoke.sh b/packaging/vm/guest-apt-repository-smoke.sh index dbe955e..b411250 100755 --- a/packaging/vm/guest-apt-repository-smoke.sh +++ b/packaging/vm/guest-apt-repository-smoke.sh @@ -118,7 +118,15 @@ class Handler(http.server.SimpleHTTPRequestHandler): if "If-Modified-Since" in self.headers: del self.headers["If-Modified-Since"] super().do_GET() -server = http.server.ThreadingHTTPServer(("127.0.0.1", 8443), functools.partial(Handler, directory=sys.argv[1])) +class Server(http.server.ThreadingHTTPServer): + def shutdown_request(self, request): + # Debian's APT rejects peers that close TLS without close_notify. + request.settimeout(5) + try: + request.unwrap() + except (OSError, ssl.SSLError): + request.close() +server = Server(("127.0.0.1", 8443), functools.partial(Handler, directory=sys.argv[1])) context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) context.load_cert_chain(sys.argv[2], sys.argv[3]) server.socket = context.wrap_socket(server.socket, server_side=True) diff --git a/scripts/test-apt-public.py b/scripts/test-apt-public.py index afb7530..0e0196c 100755 --- a/scripts/test-apt-public.py +++ b/scripts/test-apt-public.py @@ -68,10 +68,12 @@ def tearDownClass(cls): def setUp(self): self.requests.clear() (self.web / "payload").write_bytes(b"expected package bytes") - (self.root / "repository-manifest.json").write_text(json.dumps({ + manifest = json.dumps({ "revision": "a" * 64, "files": [{"path": "payload", "size": 22, "sha256": hashlib.sha256(b"expected package bytes").hexdigest()}], - })) + }) + (self.root / "repository-manifest.json").write_text(manifest) + (self.web / "repository-manifest.json").write_text(manifest) self.output = self.root / "activation.json" self.output.unlink(missing_ok=True) @@ -96,7 +98,7 @@ def invoke(self, *extra, verifier_error=None): def test_verified_bytes_produce_activation_record(self): result = self.invoke("--output", str(self.output), "--proof-url", "https://github.com/sandwichfarm/loopwire/actions/runs/123") - self.assertEqual(result["files"], 1) + self.assertEqual(result["files"], 2) record = json.loads(self.output.read_text()) self.assertEqual(record["status"], "verified") self.assertEqual(record["baseUrl"], self.url) @@ -111,6 +113,11 @@ def test_remote_tamper_never_overwrites_activation(self): self.invoke("--output", str(self.output), "--proof-url", "https://github.com/sandwichfarm/loopwire/actions/runs/123") self.assertEqual(self.output.read_text(), "previous activation") + def test_public_manifest_tamper_is_rejected(self): + (self.web / "repository-manifest.json").write_text("{}") + with self.assertRaisesRegex(ValueError, "repository-manifest.json"): + self.invoke() + def test_missing_file_fails(self): (self.web / "payload").unlink() with self.assertRaisesRegex(ValueError, "HTTP 404"): diff --git a/scripts/verify-apt-public.py b/scripts/verify-apt-public.py index ab0e5ea..3c97405 100755 --- a/scripts/verify-apt-public.py +++ b/scripts/verify-apt-public.py @@ -57,9 +57,14 @@ def main(): "--public-key", str(args.public_key), "--fingerprint", fingerprint, ], check=True, stdout=subprocess.PIPE) manifest = json.loads((args.repository / "repository-manifest.json").read_text()) + manifest_bytes = (args.repository / "repository-manifest.json").read_bytes() context = ssl.create_default_context(cafile=str(args.ca_file) if args.ca_file else None) opener = urllib.request.build_opener(NoRedirects(), urllib.request.HTTPSHandler(context=context)) - for entry in manifest["files"]: + public_entries = [*manifest["files"], { + "path": "repository-manifest.json", "size": len(manifest_bytes), + "sha256": hashlib.sha256(manifest_bytes).hexdigest(), + }] + for entry in public_entries: url = base_url + "/" + urllib.parse.quote(entry["path"], safe="/+") request = urllib.request.Request(url, headers={"Cache-Control": "no-cache", "User-Agent": "Loopwire-APT-Proof/1"}) digest, size = hashlib.sha256(), 0 @@ -95,7 +100,7 @@ def main(): temporary.write("\n") temporary_path = Path(temporary.name) temporary_path.replace(args.output) - print(json.dumps({"status": "verified", "revision": manifest["revision"], "files": len(manifest["files"])})) + print(json.dumps({"status": "verified", "revision": manifest["revision"], "files": len(public_entries)})) if __name__ == "__main__": From 639cbbb32c23ea593ed63a53b95a9565d86a7092 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:02:26 +0200 Subject: [PATCH 5/6] Make package evidence portable across supported distro toolchains Normalize compressed deb payloads through dpkg-deb before hashing their tar contents. This avoids relying on Python 3.14's Zstandard support when Ubuntu 24.04 and Debian 13 use older runtimes. Align HTTPS proof counts with the served manifest and correct the operator runbook. Constraint: Verification must work with the documented Ubuntu 24.04 and Debian 13 toolchains Confidence: high Scope-risk: narrow Directive: Keep real Zstandard package coverage in the pinned APT tools suite Tested: Real -Zzstd package regression and 17 APT proof-verifier cases Tested: Debian 13 clean-guest lifecycle passed at parent implementation commit 7849a1b Not-tested: Full suite and Ubuntu evidence revalidation follow this commit --- scripts/test-apt-repository-vm-proof.mjs | 21 ++++++++++++-- scripts/verify-apt-repository-vm-proof.mjs | 33 ++++++++-------------- 2 files changed, 31 insertions(+), 23 deletions(-) diff --git a/scripts/test-apt-repository-vm-proof.mjs b/scripts/test-apt-repository-vm-proof.mjs index 7ceab49..050c1ea 100644 --- a/scripts/test-apt-repository-vm-proof.mjs +++ b/scripts/test-apt-repository-vm-proof.mjs @@ -1,9 +1,10 @@ #!/usr/bin/env node import assert from "node:assert/strict"; -import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import { mkdtemp, mkdir, readFile, rm, writeFile, chmod } from "node:fs/promises"; import { tmpdir } from "node:os"; import path from "node:path"; -import { parseInstalledHashes, verifyInstalledStage, verifyLifecycle } from "./verify-apt-repository-vm-proof.mjs"; +import { spawnSync } from "node:child_process"; +import { debPayload, parseInstalledHashes, verifyInstalledStage, verifyLifecycle } from "./verify-apt-repository-vm-proof.mjs"; const directory = await mkdtemp(path.join(tmpdir(), "loopwire-apt-proof-test-")); const baseline = "0.1.0-1ubuntu24.04"; @@ -49,6 +50,22 @@ async function change(name, transform) { await writeFile(file, transform(await readFile(file, "utf8"))); } try { + await test("Zstandard deb payloads are read through dpkg-deb", async () => { + const packageRoot = path.join(directory, "zstd-package"); + await mkdir(path.join(packageRoot, "DEBIAN"), { recursive: true }); + await mkdir(path.join(packageRoot, "usr/bin"), { recursive: true }); + await writeFile(path.join(packageRoot, "DEBIAN/control"), + "Package: loopwire\nVersion: 0.1.0-1ubuntu24.04\nArchitecture: amd64\nMaintainer: Test \nDescription: fixture\n"); + await writeFile(path.join(packageRoot, "usr/bin/loopwire"), "#!/bin/sh\necho fixture\n"); + await chmod(path.join(packageRoot, "usr/bin/loopwire"), 0o755); + const packageFile = path.join(directory, "loopwire-zstd.deb"); + const built = spawnSync("dpkg-deb", ["-Zzstd", "--root-owner-group", "--build", packageRoot, packageFile], + { encoding: "utf8" }); + assert.equal(built.status, 0, built.stderr); + assert.deepEqual(debPayload(packageFile), { + "/usr/bin/loopwire": "6ecf06f6dbbab6a920b5b208bc7c4069ca266b150d6c00533a00b5975a8417ca" + }); + }); await test("complete ordered lifecycle accepted", () => verifyLifecycle(transitions, baseline, upgrade)); await test("missing reinstall rejected", () => assert.throws(() => verifyLifecycle(transitions.split("\n").filter((line) => !line.startsWith("reinstall")).join("\n"), baseline, upgrade), /lifecycle transitions/)); await test("rollback remaining at new version rejected", () => assert.throws(() => verifyLifecycle(transitions.replace(`rollback\t${baseline}`, `rollback\t${upgrade}`), baseline, upgrade), /lifecycle transitions/)); diff --git a/scripts/verify-apt-repository-vm-proof.mjs b/scripts/verify-apt-repository-vm-proof.mjs index c29e9fc..ae96aa1 100644 --- a/scripts/verify-apt-repository-vm-proof.mjs +++ b/scripts/verify-apt-repository-vm-proof.mjs @@ -61,27 +61,16 @@ export function parseInstalledHashes(value) { } export function debPayload(packageFile) { - // Read the signed .deb payload without extracting or executing package content. - return JSON.parse(command("python3", ["-c", ` -import hashlib, io, json, pathlib, sys, tarfile -raw = pathlib.Path(sys.argv[1]).read_bytes() -assert raw[:8] == b'!\\n', 'invalid deb archive' -position = 8 -payload = None -while position < len(raw): - header = raw[position:position + 60] - assert len(header) == 60 and header[58:60] == b'\\x60\\n', 'invalid ar header' - size = int(header[48:58].decode().strip()) - assert size >= 0 and position + 60 + size <= len(raw), 'truncated ar member' - name = header[:16].decode().strip().rstrip('/') - content = raw[position + 60:position + 60 + size] - if name.startswith('data.tar'): - assert payload is None, 'multiple package payloads' - payload = content - position += 60 + size + size % 2 -assert payload is not None, 'missing package payload' + // dpkg-deb handles the package's xz/zstd member. Python receives a plain tar + // stream so this works on Ubuntu 24.04's Python 3.12 as well as newer hosts. + const mount = path.dirname(packageFile); + return JSON.parse(command("bash", [path.join(repositoryRoot, "scripts/with-apt-tools.sh"), + "--read-only-path", mount, "python3", "-c", ` +import hashlib, json, pathlib, subprocess, sys, tarfile +package = pathlib.Path(sys.argv[1]) +process = subprocess.Popen(['dpkg-deb', '--fsys-tarfile', package], stdout=subprocess.PIPE, stderr=subprocess.PIPE) files = {} -with tarfile.open(fileobj=io.BytesIO(payload), mode='r:*') as archive: +with tarfile.open(fileobj=process.stdout, mode='r|') as archive: for member in archive: if not member.isfile(): continue @@ -90,6 +79,8 @@ with tarfile.open(fileobj=io.BytesIO(payload), mode='r:*') as archive: name = '/' + str(relative) assert name.startswith('/usr/') and name not in files, 'unexpected package path' files[name] = hashlib.sha256(archive.extractfile(member).read()).hexdigest() +stderr = process.stderr.read().decode('utf-8', errors='replace') +assert process.wait() == 0, stderr print(json.dumps(files)) `, packageFile])); } @@ -202,7 +193,7 @@ export async function verifyEvidence({ target, evidenceDir, gitHead }) { const served = JSON.parse(await text(evidenceDir, `repositories/${stage}-public-verification.json`)); equal(served.status, "verified", `${stage} HTTPS verification`); equal(served.revision, snapshot.revision, `${stage} HTTPS revision`); - equal(served.files, snapshot.files.length, `${stage} HTTPS file count`); + equal(served.files, snapshot.files.length + 1, `${stage} HTTPS file count including manifest`); const packages = (await text(directory, `dists/${target}/main/binary-amd64/Packages`)).trim().split(/\n\n+/).map((block) => { return new Map(block.split("\n").filter((line) => /^[^ :]+:/.test(line)).map((line) => { const separator = line.indexOf(":"); From a8ee704a2a71388a1c00290813a08af798792719 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:09:08 +0200 Subject: [PATCH 6/6] Preserve signed APT implementation evidence and operator boundary Map every issue 35 development task to its delivered files and fresh checks, including both clean-guest lifecycles. Keep production provisioning and public activation explicit as the remaining human-owned work. Constraint: Fixture trust and synthetic upgrades are development proof only Confidence: high Scope-risk: narrow Tested: pnpm check; Ubuntu and Debian KVM lifecycle evidence reverified Not-tested: Production publication and public client installation await operator provisioning --- .planning/STATE.md | 5 +- .../260905-kyo-SUMMARY.md | 83 +++++++++++++++++++ 2 files changed, 86 insertions(+), 2 deletions(-) create mode 100644 .planning/quick/260905-kyo-signed-apt/260905-kyo-SUMMARY.md diff --git a/.planning/STATE.md b/.planning/STATE.md index b822c41..5466819 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,7 +3,7 @@ gsd_state_version: 1.0 milestone: v0.5 milestone_name: GitHub Operator Setup status: Ready for Review -last_updated: "2026-09-05T11:55:11.036Z" +last_updated: "2026-09-05T14:08:23Z" last_activity: 2026-09-05 progress: total_phases: 1 @@ -27,7 +27,7 @@ See: .planning/PROJECT.md (updated 2026-07-03) Phase: 19 of 19 complete Plan: 19.1 — Hardened GitHub Actions Setup Status: Ready for review in PR #9 -Last activity: 2026-09-05 - completed quick task 260905-i4l: CI input scopes and full release-validation evidence +Last activity: 2026-09-05 - completed quick task 260905-kyo: signed APT repository development and clean-guest proof ## Blockers / Concerns @@ -93,6 +93,7 @@ Last activity: 2026-09-05 - completed quick task 260905-i4l: CI input scopes and | 260905-fld | Default platform installer and homepage tabs; native install/reinstall proof | 2026-09-05 | c415386 | [260905-fld-homepage-platform-installer](./quick/260905-fld-homepage-platform-installer/) | | 260905-hia | Minimal landing identity, GSAP reactions and screenshot proofs in PR #40 | 2026-09-05 | 6e84a9a | [260905-hia-landing-identity](./quick/260905-hia-landing-identity/) | | 260905-i4l | Scope CI to affected files while retaining release validation | 2026-09-05 | f9bc0d8 | [260905-i4l-scope-ci](./quick/260905-i4l-scope-ci/) | +| 260905-kyo | Signed APT repository development and clean-guest lifecycle proof | 2026-09-05 | 639cbbb | [260905-kyo-signed-apt](./quick/260905-kyo-signed-apt/) | ## Accumulated Context diff --git a/.planning/quick/260905-kyo-signed-apt/260905-kyo-SUMMARY.md b/.planning/quick/260905-kyo-signed-apt/260905-kyo-SUMMARY.md new file mode 100644 index 0000000..261415a --- /dev/null +++ b/.planning/quick/260905-kyo-signed-apt/260905-kyo-SUMMARY.md @@ -0,0 +1,83 @@ +--- +status: complete +issue: 35 +--- + +# Signed APT repository development + +Issue: https://github.com/sandwichfarm/loopwire/issues/35 + +## Result + +The development work for the Ubuntu 24.04 and Debian 13 amd64 APT channel is complete. The checked-in channel remains +`pending` until the separately listed human operations provision a production HTTPS/SSH origin, create the signing +identity, configure the protected environment, and perform the first public verification. Existing homepage install +commands remain usable; a complete reviewed activation record switches only the Ubuntu and Debian panels to +`sudo apt install loopwire` and exposes the repository-scoped bootstrap command. + +## Development checklist evidence + +1. **Layout/configuration:** `apt-repository.py` defines separate `ubuntu-24.04` and `debian-13` suites under + `main/binary-amd64`, suite-specific pool paths, retained SHA-256 by-hash indexes, stable dpkg version ordering, + 30-day signed metadata, indefinite v1 immutable retention, and a strict manifest. The operator runbook specifies + every variable, secret, path, permission, cache, monitoring, and key-rotation boundary. +2. **Generation:** build verifies the existing OpenSSL-signed release manifest and exact internal deb identity before + preserving those package bytes. It creates Packages/Packages.gz, Release, OpenPGP clear-signed InRelease, exported + fingerprint key, by-hash objects, and the independently validated inventory. Verify checks the entire trust chain. +3. **Publication/rollback:** the local/SSH publisher validates before writes, requires pinned host trust, locks the + POSIX origin, uses revision compare-and-swap, rejects immutable collisions, retains private snapshots, promotes + immutable objects before metadata, and atomically replaces each suite's InRelease. Durable journals resume every + interruption point; expired recovery is explicit and demands immediate refresh. Rollback re-signs selected + package sets with fresh dates. The Nginx example serves only `ROOT/public`, revalidates metadata and long-caches + immutable URLs. +4. **Protected automation:** Publish APT Repository supports release-triggered publish, operator publish/refresh/ + rollback, and weekly expiry refresh. `APT_REPOSITORY_ENABLED=true` and `packages-production` gate writes. Stable + release publication waits for the existing GitHub Release/evidence gates, re-downloads and verifies public release + assets, then verifies every HTTPS-served byte before producing a reviewable activation record. Preflight rejects + unsafe configuration before key or origin access. +5. **Bootstrap:** the repeat-safe helper supports Ubuntu 24.04 and Debian 13 amd64, downloads only HTTPS key material, + pins the full fingerprint, uses `/etc/apt/keyrings` and a deb822 source with `Signed-By`, preserves unrelated + sources, rejects symlink escapes, supports no-network dry-run and safe removal, and never uses `apt-key` or insecure + APT options. User docs cover install, updates, repair, explicit downgrade, source removal, trust and key changes. +6. **Regression tests:** the dedicated suite runs real GPG/OpenSSL/dpkg/APT checks plus an actual disposable SSH + server. It rejects unsigned or tampered metadata, modified packages, wrong signers, bad suite/package identity, + downgrades outside explicit rollback, unsafe files, stale CAS, concurrent access and origin drift. It exercises 22 + publisher cases, all resumable checkpoints, permissions under umask 077, expired-journal recovery, public HTTPS + tampering, bootstrap containment and a real Zstandard deb on the pinned Debian 13 toolchain. +7. **Matching guests:** clean checksum-pinned Ubuntu 24.04 and Debian 13 KVM guests installed from a guest-only HTTPS + repository using the real scoped bootstrap. Each performed install, reinstall, a synthetic `+aptfixture1` upgrade, + explicit downgrade/rollback, removal and source removal. The verifier binds repository origin, versions, signed + package hashes, every installed `/usr` file, providers/backend detector, GUI linkage and a real X11 application + window. The synthetic version reuses the authenticated v0.1.0 payload and is lifecycle evidence, not a release. +8. **Docs/UI:** homepage, install guide, support matrix, release guide, user APT guide, maintainer runbook, release + notes, and navigation are updated. Pending and verified-fixture browser tests prove the fallback and activated + states. Production activation remains the human step that supplies verified public values; Loopwire is never + described as part of a distribution's default repository. + +## Verification + +- `pnpm check` passed after the final review change: all project verification, types, 295 workspace tests, 22 Rust + tests, builds, static-site validation and the dedicated APT suite. +- `pnpm verify:apt` passed: 13 generator, 22 publisher (including actual SSH), 11 bootstrap, 9 public HTTPS, 5 workflow + preflight, 17 proof-verifier, and 4 homepage channel cases. +- Generator tests passed with real APT on both Debian 13 and Ubuntu 24.04; wrong-key, unsigned/tampered metadata and + modified-package downloads were rejected. +- Ubuntu and Debian lifecycle proof directories each contain 96 evidence files from commit `7849a1b`, revalidated + successfully with the final portable verifier at `639cbbb`. +- Pending production build browser tests passed all existing install/copy/keyboard/responsive/motion/fallback cases. + The verified fixture passed those same checks and additionally rendered both short APT commands, their separate + setup links, and the complete URL/fingerprint setup command. The checked-in record was restored to pending. +- Workflow contracts, actionlint, ShellCheck, Python/Node/Ruby/Bash syntax, docs/build checks and whitespace checks + passed. Final review's Zstandard portability finding was reproduced, fixed with `dpkg-deb --fsys-tarfile`, covered + by a real compressed package, and approved on re-review. + +## Human operations still open + +No production host/account, TLS certificate, SSH credential, OpenPGP identity, GitHub environment value, or public +repository was created or changed. No production publication was triggered. The five Human operational tasks in +issue #35 remain unchecked. The first public run must attach the public URL/fingerprint/revision and clean-client +evidence, then its reviewed `apt-channel.json` can activate the website through a separate commit. + +Power-loss behavior and network filesystems were not tested; the publication contract explicitly requires local +POSIX filesystem locking/fsync/atomic-rename semantics. Ubuntu/Debian guests used a disposable local CA and repository +key; fixture trust cannot generate a production activation record.