From 17e3820407f97c698d55585b0d36ef7ee5a56fe6 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:34:55 +0200 Subject: [PATCH 1/7] Define signed Fedora delivery on the reviewed publication foundation Record the project-owned provider decision, embedded RPM and repository signature boundaries, Fedora 44 target, rollback policy, public activation gate, and clean-guest evidence required by issue 36. Constraint: This is an intentional stack on PR 45 to reuse publication foundations Rejected: COPR | provider-built output requires separate provenance and promotion evidence Confidence: high Scope-risk: narrow Tested: Plan maps every development task and human activation boundary in issue 36 --- .../260905-mhp-PLAN.md | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 .planning/quick/260905-mhp-signed-fedora/260905-mhp-PLAN.md diff --git a/.planning/quick/260905-mhp-signed-fedora/260905-mhp-PLAN.md b/.planning/quick/260905-mhp-signed-fedora/260905-mhp-PLAN.md new file mode 100644 index 0000000..0487614 --- /dev/null +++ b/.planning/quick/260905-mhp-signed-fedora/260905-mhp-PLAN.md @@ -0,0 +1,57 @@ +--- +status: implementing +issue: 36 +depends_on: 45 +--- + +# Signed Fedora repository development + +Goal: complete every development task in #36 and open a dedicated PR containing `resolves #36`. This branch is an +intentional stack on #35/PR #45 because the Fedora channel reuses its reviewed SSH/POSIX publication, protected +environment, public activation, and guest-proof foundations. Production provider ownership, credentials, signing +identity and first public activation remain the separately listed human operational tasks. + +## Decisions + +- Choose a project-owned repository over COPR. It indexes the exact OpenSSL-authenticated release RPM, controls when + the RPM is signed and verified, supports reviewed revision/CAS/retention/recovery semantics, and can be tested + locally and over the same restricted SSH origin. COPR rebuilds from source and owns signing/publication timing, so + its output would need distinct provider build evidence and would not be the existing release artifact. +- Initial scope is Fedora 44 x86_64 only. #37 adds openSUSE independently after this PR is open. +- The repository-distributed copy of the GitHub Release RPM receives a separate OpenPGP RPM signature before its + final repository hash and lifecycle evidence are recorded. The source release hash and distributed hash stay + distinct and traceable. +- Require both `gpgcheck=1` for package signatures and `repo_gpgcheck=1` for the detached OpenPGP signature over + `repodata/repomd.xml`. No local-RPM signature exception applies to the repository path. +- Retain immutable signed RPMs and content-addressed repodata indefinitely in v1. Publication must serialize writers, + require an expected revision, reject immutable collisions, recover interruption, and publish metadata only after + every package/content object exists. The implementation must make clients either verify a complete revision or + fail safely during the promotion boundary; exact commit semantics are recorded after generator/publisher tests. +- Metadata expires after 30 days and gets protected weekly refresh. Rollback selects a retained package set, signs + fresh metadata, and documents the explicit DNF downgrade needed on already-upgraded clients. +- The checked-in Fedora channel remains pending. Existing signed direct-download and automatic-installer paths stay + functional until a human reviews the production public proof record and commits activation data. + +## Work lanes + +1. `fedora_repo_protocol`: exact release authentication, RPM/repository signing, createrepo metadata, manifest, + retention/rollback and real DNF/tamper tests in a pinned Fedora toolchain. +2. `fedora_publisher`: local/SSH POSIX publication, commit semantics, CAS/locks/recovery, immutable retention, actual + SSH and HTTP/cache tests. +3. `fedora_guest`: isolated clean Fedora 44 KVM lifecycle and strict raw evidence verifier, including package + signatures, installed payload hashes, providers and a real GUI window. +4. `fedora_docs`: provider comparison, pending/verified website gate and complete user/operator documentation. +5. Root integration: scoped setup/public verification, protected release/refresh/rollback workflow, configuration + contracts, CI gates, browser fixtures, final review, issue checklist and PR delivery. + +## Required verification + +- Real DNF with repository and package signature checks accepts correct content and rejects wrong/unsigned/tampered + RPMs and metadata. Version/architecture/target and downgrade rules are independently verified. +- Publication proves writer exclusion, CAS, ordering/commit behavior, idempotence, interruption recovery, permissions, + immutable retention, rollback, actual SSH transport, and public cache behavior without production credentials. +- A clean checksum-pinned Fedora 44 KVM guest performs repository install, reinstall, fixture upgrade, explicit + rollback/downgrade, removal and repository removal against the final committed development code. Raw proof binds + source/distributed hashes, signer, origin, versions, installed bytes, providers and GUI behavior. +- Pending and verified-fixture browser states, workflow/action syntax, documentation, focused tests and full project + gates pass. Public production remains disabled and no human task is reported complete without its real evidence. From 8e3f5ce96630cd87b44b03dbe895feabc6ca84ae Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:47:26 +0200 Subject: [PATCH 2/7] Enable Fedora installation through a verified signed repository Authenticate the published Fedora 44 release RPM, sign the staged repository copy and repomd metadata with a dedicated OpenPGP identity, and publish through the reviewed SSH/POSIX revision protocol. Add protected refresh and rollback, a scoped DNF bootstrap, public activation gating, and clean-guest proof. The checked-in channel remains pending until the human production tasks supply and verify its public URL, signing key, origin and workflow environment. Constraint: Fedora 44 x86_64 is the only repository target in this slice Constraint: DNF does not natively enforce the project-signed metadata deadline Rejected: COPR | provider-built packages require separate output and promotion proof Confidence: high Scope-risk: moderate Directive: Preserve global immutable RPM/repodata URLs and publish repomd.xml last Directive: Keep gpgcheck, repo_gpgcheck and sslverify enabled in client configuration Tested: 13 generator, 20 publisher, 7 bootstrap, 6 HTTPS, 3 preflight, 34 proof and 4 channel cases Tested: Real DNF package/metadata signatures, interruption recovery and actual SSH transport Tested: Pending/verified browser fixtures, docs/build, workflow contracts, actionlint and ShellCheck Not-tested: Fresh Fedora KVM lifecycle and full workspace gates follow this reproducible commit Not-tested: Production origin/signing/environment/public activation remain human tasks --- .github/workflows/deploy-docs.yml | 1 + .github/workflows/publish-fedora.yml | 83 ++ .github/workflows/release.yml | 10 + .github/workflows/web.yml | 2 + .github/workflows/workflow-checks.yml | 2 + apps/docs/docs/.vitepress/config.ts | 2 + apps/docs/docs/developer/fedora-repository.md | 277 ++++++ apps/docs/docs/developer/release.md | 18 + apps/docs/docs/guide/fedora-repository.md | 150 ++++ apps/docs/docs/guide/install.md | 27 +- apps/docs/docs/guide/support-matrix.md | 9 +- apps/docs/docs/release-notes/unreleased.md | 19 +- apps/site/src/lib/rpmChannel.mjs | 65 ++ apps/site/src/lib/rpmChannel.test.mjs | 81 ++ apps/site/src/pages/index.astro | 10 +- package.json | 3 +- packaging/README.md | 47 ++ packaging/repositories/Dockerfile.rpm-tools | 9 + packaging/repositories/fedora-channel.json | 10 + packaging/repositories/nginx-rpm.conf | 35 + packaging/vm/guest-fedora-repository-smoke.sh | 355 ++++++++ scripts/native-package-vm.sh | 39 +- scripts/publish-fedora-workflow.sh | 102 +++ scripts/publish-rpm-repository.py | 782 +++++++++++++++++ scripts/rpm-repository.py | 776 +++++++++++++++++ scripts/setup-fedora-repository.sh | 164 ++++ scripts/test-ci-workflow-paths.rb | 3 +- scripts/test-fedora-bootstrap.py | 154 ++++ scripts/test-fedora-repository-vm-proof.mjs | 175 ++++ scripts/test-fedora-workflow-preflight.py | 58 ++ scripts/test-fedora-workflow.rb | 41 + scripts/test-publish-rpm-repository.py | 794 ++++++++++++++++++ scripts/test-rpm-public.py | 125 +++ scripts/test-rpm-repository.py | 459 ++++++++++ scripts/verify-docs.sh | 8 + scripts/verify-fedora-repository-vm-proof.mjs | 456 ++++++++++ scripts/verify-github-workflows.sh | 2 + scripts/verify-requirements.sh | 3 +- scripts/verify-rpm-public.py | 108 +++ scripts/verify-rpm-repository.sh | 20 + scripts/verify-scripts.sh | 4 + scripts/with-rpm-tools.sh | 33 + 42 files changed, 5501 insertions(+), 20 deletions(-) create mode 100644 .github/workflows/publish-fedora.yml create mode 100644 apps/docs/docs/developer/fedora-repository.md create mode 100644 apps/docs/docs/guide/fedora-repository.md create mode 100644 apps/site/src/lib/rpmChannel.mjs create mode 100644 apps/site/src/lib/rpmChannel.test.mjs create mode 100644 packaging/repositories/Dockerfile.rpm-tools create mode 100644 packaging/repositories/fedora-channel.json create mode 100644 packaging/repositories/nginx-rpm.conf create mode 100755 packaging/vm/guest-fedora-repository-smoke.sh create mode 100755 scripts/publish-fedora-workflow.sh create mode 100644 scripts/publish-rpm-repository.py create mode 100644 scripts/rpm-repository.py create mode 100755 scripts/setup-fedora-repository.sh create mode 100755 scripts/test-fedora-bootstrap.py create mode 100755 scripts/test-fedora-repository-vm-proof.mjs create mode 100755 scripts/test-fedora-workflow-preflight.py create mode 100755 scripts/test-fedora-workflow.rb create mode 100644 scripts/test-publish-rpm-repository.py create mode 100755 scripts/test-rpm-public.py create mode 100644 scripts/test-rpm-repository.py create mode 100755 scripts/verify-fedora-repository-vm-proof.mjs create mode 100755 scripts/verify-rpm-public.py create mode 100755 scripts/verify-rpm-repository.sh create mode 100755 scripts/with-rpm-tools.sh diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml index c0e942f..c6e424f 100644 --- a/.github/workflows/deploy-docs.yml +++ b/.github/workflows/deploy-docs.yml @@ -12,6 +12,7 @@ on: - ".github/workflows/deploy-docs.yml" - "apps/site/**" - "packaging/repositories/apt-channel.json" + - "packaging/repositories/fedora-channel.json" - "apps/docs/**" - "assets/product-screenshot.png" - "package.json" diff --git a/.github/workflows/publish-fedora.yml b/.github/workflows/publish-fedora.yml new file mode 100644 index 0000000..2261e7f --- /dev/null +++ b/.github/workflows/publish-fedora.yml @@ -0,0 +1,83 @@ +name: Publish Fedora Repository + +on: + workflow_call: + inputs: + tag: + type: string + required: true + operation: + type: string + default: publish + workflow_dispatch: + inputs: + operation: + description: Publish a stable release, refresh expiry, or roll back to a retained revision + type: choice + options: [publish, refresh, rollback] + default: publish + tag: + description: Existing stable release tag for publish + type: string + revision: + description: Retained repository revision SHA-256 for rollback + type: string + schedule: + - cron: "53 5 * * 1" + +permissions: + contents: read + +concurrency: + group: fedora-repository-production + cancel-in-progress: false + +jobs: + publish: + if: >- + ${{ + vars.FEDORA_REPOSITORY_ENABLED == 'true' && + (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) || + (github.workflow == 'Release' && startsWith(github.ref, 'refs/tags/v'))) + }} + runs-on: ubuntu-24.04 + timeout-minutes: 35 + environment: packages-production + container: + image: fedora:44@sha256:be9d65e2344d805cc11114319c685ecaa96b6d9b4350a0a6460cdb931babbd19 + steps: + - name: Install repository and workflow tools + run: >- + dnf install -y + createrepo_c dnf git gh gnupg2 nodejs openssh-clients openssl python3 rpm-build rpm-sign + + - name: Checkout publisher + uses: actions/checkout@v7.0.0 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Build, publish, and verify repository + env: + GH_TOKEN: ${{ github.token }} + OPERATION: ${{ inputs.operation || 'refresh' }} + RELEASE_TAG: ${{ inputs.tag }} + ROLLBACK_REVISION: ${{ inputs.revision }} + FEDORA_REPOSITORY_URL: ${{ vars.FEDORA_REPOSITORY_URL }} + FEDORA_REPOSITORY_HOST: ${{ vars.FEDORA_REPOSITORY_HOST }} + FEDORA_REPOSITORY_ROOT: ${{ vars.FEDORA_REPOSITORY_ROOT }} + FEDORA_SSH_PORT: ${{ vars.FEDORA_SSH_PORT || '22' }} + FEDORA_SIGNING_FINGERPRINT: ${{ vars.FEDORA_SIGNING_FINGERPRINT }} + FEDORA_SSH_PRIVATE_KEY: ${{ secrets.FEDORA_SSH_PRIVATE_KEY }} + FEDORA_SSH_KNOWN_HOSTS: ${{ secrets.FEDORA_SSH_KNOWN_HOSTS }} + FEDORA_SIGNING_KEY: ${{ secrets.FEDORA_SIGNING_KEY }} + FEDORA_SIGNING_PASSPHRASE: ${{ secrets.FEDORA_SIGNING_PASSPHRASE }} + run: bash scripts/publish-fedora-workflow.sh + + - name: Upload public verification and activation record + uses: actions/upload-artifact@v7.0.1 + with: + name: loopwire-fedora-publication-${{ github.run_id }} + path: dist/fedora-publication + if-no-files-found: error + retention-days: 90 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index fd12162..b6d8443 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -513,3 +513,13 @@ jobs: tag: ${{ needs.publish-release.outputs.tag }} operation: publish secrets: inherit + + publish-fedora: + name: Publish signed Fedora channel + needs: publish-release + if: ${{ vars.FEDORA_REPOSITORY_ENABLED == 'true' && !contains(needs.publish-release.outputs.tag, '-') }} + uses: ./.github/workflows/publish-fedora.yml + with: + tag: ${{ needs.publish-release.outputs.tag }} + operation: publish + secrets: inherit diff --git a/.github/workflows/web.yml b/.github/workflows/web.yml index 8e80ef8..3937adf 100644 --- a/.github/workflows/web.yml +++ b/.github/workflows/web.yml @@ -6,6 +6,7 @@ on: - ".github/workflows/web.yml" - "apps/site/**" - "packaging/repositories/apt-channel.json" + - "packaging/repositories/fedora-channel.json" - "apps/docs/**" - "assets/product-screenshot.png" - "README.md" @@ -32,6 +33,7 @@ on: - ".github/workflows/web.yml" - "apps/site/**" - "packaging/repositories/apt-channel.json" + - "packaging/repositories/fedora-channel.json" - "apps/docs/**" - "assets/product-screenshot.png" - "README.md" diff --git a/.github/workflows/workflow-checks.yml b/.github/workflows/workflow-checks.yml index a6b5667..5d34f1b 100644 --- a/.github/workflows/workflow-checks.yml +++ b/.github/workflows/workflow-checks.yml @@ -8,6 +8,7 @@ on: - "scripts/test-ci-impact.rb" - "scripts/test-ci-workflow-paths.rb" - "scripts/test-apt-workflow.rb" + - "scripts/test-fedora-workflow.rb" - "scripts/*native-package-proof-snapshot.mjs" - "scripts/verify-github-workflows.sh" - "scripts/verify-requirements.sh" @@ -23,6 +24,7 @@ on: - "scripts/test-ci-impact.rb" - "scripts/test-ci-workflow-paths.rb" - "scripts/test-apt-workflow.rb" + - "scripts/test-fedora-workflow.rb" - "scripts/*native-package-proof-snapshot.mjs" - "scripts/verify-github-workflows.sh" - "scripts/verify-requirements.sh" diff --git a/apps/docs/docs/.vitepress/config.ts b/apps/docs/docs/.vitepress/config.ts index afd70bc..4ca04f9 100644 --- a/apps/docs/docs/.vitepress/config.ts +++ b/apps/docs/docs/.vitepress/config.ts @@ -44,6 +44,7 @@ export default defineConfig({ items: [ { text: "Install", link: "/guide/install" }, { text: "APT Repository", link: "/guide/apt-repository" }, + { text: "Fedora Repository", link: "/guide/fedora-repository" }, { text: "Basic Usage", link: "/guide/basic-usage" }, { text: "Configurations", link: "/guide/configurations" }, { text: "Audio Backends", link: "/guide/backends" }, @@ -62,6 +63,7 @@ export default defineConfig({ { text: "VM Matrix", link: "/developer/vm-matrix" }, { text: "Release", link: "/developer/release" }, { text: "APT Repository Operations", link: "/developer/apt-repository" }, + { text: "Fedora Repository Operations", link: "/developer/fedora-repository" }, { text: "Release Notes", link: "/developer/release-notes" } ] }, diff --git a/apps/docs/docs/developer/fedora-repository.md b/apps/docs/docs/developer/fedora-repository.md new file mode 100644 index 0000000..33b8589 --- /dev/null +++ b/apps/docs/docs/developer/fedora-repository.md @@ -0,0 +1,277 @@ +# Signed Fedora repository operations + +This runbook covers development, publication, and recovery for Loopwire's third-party Fedora channel. Tooling can be +tested without production access, but the checked-in channel record starts `pending`. Provisioning the origin and +signing identity, configuring the protected GitHub environment, completing the first public publication, and +reviewing its clean-client proof remain human operations. Keep the signed direct-download path available until those +steps are complete. + +## Scope and provider decision + +Version 1 serves one target: **Fedora 44, x86_64**, at a canonical HTTPS base such as +`https://HOST/fedora/44/x86_64`. It uses the existing Fedora package recipe and authenticated GitHub Release input; +no UI or audio-backend behavior belongs in this layer. + +The project chose a project-owned repository after comparing it with COPR: + +| Requirement | COPR | Project-owned repository | +| --- | --- | --- | +| Release artifact control | COPR builds provider output from submitted inputs; its RPM would need separate build-ID and exact-output proof. | The generator verifies the existing signed release manifest and exact Fedora RPM, then signs a staged copy without changing the GitHub Release. | +| Signing | Provider-managed package signing reduces private-key custody, but signed-metadata and stable-promotion behavior remain provider policy. | One dedicated project OpenPGP identity signs the distributed RPM and repository metadata under the protected environment. | +| Fedora target | COPR chroots can target supported Fedora releases, subject to service lifecycle. | The generator rejects every target except the tested Fedora 44 x86_64 contract. | +| Promotion and proof | Async build completion and repository visibility need provider-specific polling and build records. | A complete candidate is verified locally, published with compare-and-swap protection, and checked byte-for-byte over public HTTPS. | +| Retention and rollback | Build retention and project state depend on provider policy and configuration. | Immutable packages, keys, content metadata, and signed snapshots remain under project control; rollback republishes a retained package set with fresh metadata. | + +Project-owned hosting gives release artifact control, atomic promotion, explicit retention, and the same local/CI proof +surface as the existing release flow. It also keeps Fedora-specific proof independent from any future openSUSE channel. +The tradeoff is operational responsibility for the HTTPS origin, SSH access, OpenPGP recovery, caching, monitoring, +and storage growth. + +## Trust and repository layout + +The generator verifies the existing OpenSSL signature on `SHA256SUMS` and the input RPM checksum before staging it. +It then signs or re-signs only the staged RPM with the dedicated repository OpenPGP key. The GitHub Release artifact +is never rewritten. DNF validates two related signatures: + +1. `gpgcheck=1` validates the distributed RPM's embedded OpenPGP signature. +2. `repo_gpgcheck=1` validates `repodata/repomd.xml.asc`, which authenticates checksums for the retained metadata and + package index. + +The public target root contains: + +```text +fedora/44/x86_64/ +├── keys/FINGERPRINT.asc +├── packages/loopwire-VERSION-1.fc44.x86_64.rpm +├── repodata/repomd.xml +├── repodata/repomd.xml.asc +├── repodata/CHECKSUM-primary.xml.gz +└── repository-manifest.json +``` + +The manifest uses `loopwire.rpm-repository.v1` with schema version 1 and target +`{"distribution":"fedora","release":"44","architecture":"x86_64"}`. Its revision is the lowercase SHA-256 of the +canonical JSON excluding the `revision` field. It records the signing fingerprint, creation time, project verification +deadline, source/distributed RPM hashes, packages, and every public file. Do not hand-edit generated repository state. + +## Human provisioning + +Provision a project-owned HTTPS host with a valid public certificate and an SSH account restricted to the private +repository root. The host needs Python 3 and POSIX filesystem semantics. Put transaction staging and the public tree +on the same filesystem so `repomd.xml` replacement is atomic. Serve **`ROOT/public` only**; keep `ROOT/snapshots`, +`ROOT/state`, locks, incoming transactions, SSH identities, and private signing material outside the web root. Back up +private snapshots and state independently. + +The project-owned SSH/POSIX origin is required because the existing website upload surface does not provide the +compare-and-swap, retention, or atomic metadata-pointer contract. The ordinary website deployment remains separate. +Configure caching by object role: + +- `repodata/repomd.xml`, its detached signature, `repository-manifest.json`, and all HTTP 404 responses: `no-store` or + mandatory revalidation. +- RPMs, public key files, and checksum-named metadata objects: a one-year immutable policy. +- Never cache an absent mutable object that a publication or recovery operation will create. + +Create a dedicated OpenPGP identity offline. Record the full 40-character uppercase fingerprint, expiration, +custodian, encrypted backup, and revocation-certificate location. Keep recovery material outside CI. Do not reuse the +OpenSSL release key or export the private repository key to the origin. + +Configure the GitHub environment **`packages-production`**, restrict it to reviewed release/default-branch inputs, +and apply its human approval policy. `.github/workflows/publish-fedora.yml` validates these settings before remote +writes: + +| Kind | Name | Purpose | +| --- | --- | --- | +| Repository variable | `FEDORA_REPOSITORY_ENABLED` | Set to `true` only after provisioning to permit protected publication. | +| Variable | `FEDORA_REPOSITORY_URL` | Exact public HTTPS target URL ending in `/fedora/44/x86_64`, without credentials, query, or fragment. | +| Variable | `FEDORA_REPOSITORY_HOST` | Restricted SSH `USER@HOST` used by the publisher. | +| Variable | `FEDORA_REPOSITORY_ROOT` | Absolute private origin root; HTTP maps its `public` child as the document root. | +| Variable | `FEDORA_SIGNING_FINGERPRINT` | Complete 40-character uppercase OpenPGP fingerprint. | +| Optional variable | `FEDORA_SSH_PORT` | SSH port when it is not 22. | +| Secret | `FEDORA_SSH_PRIVATE_KEY` | Restricted SSH identity for publication. | +| Secret | `FEDORA_SSH_KNOWN_HOSTS` | Host-key pins obtained through a trusted channel. | +| Secret | `FEDORA_SIGNING_KEY` | Dedicated ASCII-armored OpenPGP private signing key. | +| Optional secret | `FEDORA_SIGNING_PASSPHRASE` | Passphrase for that private export. | + +Leave the enable variable false until every production value, secret, and protection rule is ready. Enabling +publication permits protected repository writes; it does not activate the website's short DNF command. Do not +generate SSH host pins from an unauthenticated scan inside the publishing job. Monitor failed publications, origin +drift, TLS expiry, signing-key expiry, the project metadata verification deadline, and storage growth. + +## Build and verify a candidate + +Use a reviewed checkout plus a directory containing the published Fedora RPM, `SHA256SUMS`, and `SHA256SUMS.sig`. +`RPM_FPR` is the complete OpenPGP fingerprint and `RPM_GNUPG_HOME` is a protected GnuPG home containing its private +key. Local generation needs Python 3, RPM tools including `rpmsign`, `createrepo_c`, GnuPG, and OpenSSL. + +```bash +python3 scripts/rpm-repository.py build \ + --release-dir dist/release --version 0.1.0 --output dist/rpm-repository \ + --signing-key "$RPM_FPR" --gnupg-home "$RPM_GNUPG_HOME" \ + --release-public-key packaging/release-signing-public.pem +python3 scripts/rpm-repository.py verify \ + --repository dist/rpm-repository \ + --public-key rpm-public.asc --fingerprint "$RPM_FPR" +``` + +The verifier rejects a missing or invalid RPM signature, wrong signer, changed RPM bytes, unsigned or changed +metadata, unexpected target/version/architecture, unlisted files, and a passed project verification deadline. +Production uses stable `X.Y.Z` versions and the project release trust anchor. `--release-public-key FILE` exists for +explicit fixture trust anchors; do not replace the production anchor with fixture material. + +Use `--previous DIR` to retain objects and older packages from the current repository. `--passphrase-file FILE` +supports a protected signing-key passphrase. `--date EPOCH` is for deterministic fixtures; production uses current +time. The custom signed `loopwire-valid-until` deadline is 30 days by default, and `--valid-for-days` accepts only 1 +through 90. Loopwire's verifier and publishing workflow enforce it; DNF does not understand this project tag or +provide APT-style signed `Valid-Until` enforcement. The public monitor must not rely on scheduled GitHub runs happening +exactly on time. DNF's client `metadata_expire=6h` setting only controls cache refresh frequency. + +An explicit `--date` fixes createrepo timestamps plus RPM and metadata signature creation times. Byte-identical output +also depends on the pinned Fedora tool versions, identical key material, and a deterministic OpenPGP algorithm. When +`--previous` already contains the same version with the same authenticated source-release hash, the generator reuses +that signed RPM instead of manufacturing different signed bytes for an unchanged release input. + +## Publish, fetch, and recover + +Use the protected **Publish Fedora Repository** workflow for production publication, refresh, or rollback. Tagged +release integration must run only after GitHub Release publication succeeds and `FEDORA_REPOSITORY_ENABLED=true`. +The workflow downloads and verifies published release inputs rather than trusting an arbitrary runner directory. A +Fedora publication failure leaves the GitHub Release and previous repository revision intact; repair the failure and +retry this channel. Its weekly schedule refreshes signed metadata without inventing a new application release; delayed +or disabled schedules still require external expiry monitoring. + +The publisher supports local rehearsal and SSH operation. For production, provide all SSH identity and host-key +arguments. Here `RPM_ROOT`, `RPM_HOST`, and `RPM_REVISION` refer to the provisioned private root, restricted SSH host, +and currently verified manifest revision. Use `empty` only for the first publication. + +```bash +python3 scripts/publish-rpm-repository.py fetch \ + --root "$RPM_ROOT" --output dist/rpm-previous \ + --public-key rpm-public.asc --fingerprint "$RPM_FPR" \ + --ssh "$RPM_HOST" --identity-file rpm-ssh-key --known-hosts rpm-known-hosts +python3 scripts/publish-rpm-repository.py publish \ + --repository dist/rpm-repository --root "$RPM_ROOT" \ + --public-key rpm-public.asc --fingerprint "$RPM_FPR" --expected-revision "$RPM_REVISION" \ + --ssh "$RPM_HOST" --identity-file rpm-ssh-key --known-hosts rpm-known-hosts --dry-run +``` + +Read the dry-run result, then repeat without `--dry-run`. Add `--ssh-port PORT` when required. Take the current +revision from verified `fetch` output. A conflicting writer must fetch and rebuild against the latest revision; do not +force stale state over it. Server locking serializes publication and the operation is repeat-safe for an already +committed revision. + +The publisher installs immutable RPM, key, and checksum-named metadata objects first. Existing immutable paths with +different bytes are rejected. It writes `repodata/repomd.xml.asc` before atomically replacing +`repodata/repomd.xml`, which is the metadata commit point. A client in the brief interval between those two writes may +see a signature/metadata mismatch and fail closed; it never accepts unauthenticated metadata. Retry after publication +completes. Fedora 44 DNF5 may report the bad signature, exclude the repository, and still return exit code 0 from +`dnf repoquery`; lifecycle and monitoring checks must reject the verification diagnostic and require the expected +Loopwire package result instead of trusting process status alone. A future protocol would need a versioned target URL +to eliminate even that fail-closed window. + +An interrupted promotion preserves a recovery journal. Inspect and resume the exact pending revision: + +```bash +python3 scripts/publish-rpm-repository.py recover \ + --root "$RPM_ROOT" --public-key rpm-public.asc --fingerprint "$RPM_FPR" \ + --ssh "$RPM_HOST" --identity-file rpm-ssh-key --known-hosts rpm-known-hosts --dry-run +``` + +Repeat without `--dry-run` after review. `--allow-expired` is limited to an operator-reviewed journal whose project +verification deadline passed: it can finish only that authenticated transition, then requires an immediate fetch, +fresh re-sign, and publication. It does not disable project verification for ordinary candidates. DNF signature checks +alone may accept replayed older correctly signed metadata, which is why immediate refresh plus HTTPS/origin monitoring +remain required. Never edit public metadata, snapshots, or state by hand to bypass a conflict. + +## Retention and rollback + +Version 1 retains RPMs, key files, checksum-named metadata, and signed snapshots indefinitely. There is no automatic +garbage collection. Monitor storage growth; any deletion policy needs a separate design covering cached clients, +manifest references, rollback availability, and incident evidence. + +Fetch a retained revision with `fetch --revision SHA`. Project policy requires rollback to generate fresh signed +metadata for that known-good package set. Copying an old `repomd.xml` whose project verification deadline passed would +be replay, even though DNF may still accept its valid signature: + +```bash +python3 scripts/rpm-repository.py rollback \ + --repository dist/rpm-known-good --output dist/rpm-rollback \ + --signing-key "$RPM_FPR" --gnupg-home "$RPM_GNUPG_HOME" +python3 scripts/rpm-repository.py verify \ + --repository dist/rpm-rollback --public-key rpm-public.asc --fingerprint "$RPM_FPR" +``` + +Publish the rollback candidate against the current revision and repeat public verification. Installed newer packages +do not downgrade automatically. Communicate either `sudo dnf downgrade loopwire` or the exact retained +`sudo dnf install loopwire-VERSION-RELEASE.x86_64` command; the +[user guide](../guide/fedora-repository.md#earlier-versions) +explains both choices. + +## Key rotation and revocation + +Treat routine key rotation as a coordinated release. Generate the successor offline, announce its complete +fingerprint through trusted channels, and test clean-client setup, existing-client migration, upgrade, reinstall, and +rollback. The conservative path uses a new HTTPS repository prefix signed only by the successor. Existing clients +rerun the setup helper with the reviewed new URL and fingerprint; package upgrades do not silently grant trust to a +new key. Keep the old prefix available during an announced migration window only while its key remains trustworthy. + +The fingerprint-named public key object is immutable. Do not overwrite it after an expiration/subkey change. A +fetched old snapshot still requires its original key, and re-signing unverified historical bytes does not establish +their provenance. + +For compromise, disable publication immediately, remove the private export from CI, publish the revocation and +incident notice through trusted channels, and mark `packaging/repositories/fedora-channel.json` pending. Existing +clients must remove the old repository/key file and bootstrap the reviewed replacement explicitly. Preserve evidence +and recover only from authenticated release inputs or known-good snapshots. + +## Public verification and final activation + +After publication, verify every production HTTPS byte against the signed local candidate: + +```bash +python3 scripts/verify-rpm-public.py \ + --repository dist/rpm-repository --public-key rpm-public.asc --fingerprint "$RPM_FPR" \ + --base-url "$FEDORA_URL" --proof-url "$FEDORA_PROOF_URL" --output dist/fedora-channel.json +``` + +The checker validates the local trust chain, fetches the production repository over HTTPS without redirects, compares +exact hashes and sizes, and emits a verified record only on success. A fixture CA or synthetic upgrade is development +evidence, never production proof. + +The workflow artifact `loopwire-fedora-publication-RUN_ID` contains `fedora-channel.json`, `publication.json`, and +`repository-manifest.json`. **Final activation is a human operation:** review the successful protected run, public +URL, target, signing fingerprint, revision, timestamp, and lifecycle evidence. Then copy the emitted channel record to +`packaging/repositories/fedora-channel.json` in a reviewed commit and deploy the website from that commit. Do not edit +`status` alone or place credentials in the channel file. + +A verified version 1 record requires `target: "fedora-44"`, an HTTPS base URL, a 40-character uppercase fingerprint, +a 64-character lowercase revision, an ISO timestamp, and a project GitHub Actions run URL. Missing or malformed data +keeps the homepage on the signed direct-download command. A valid record changes only the Fedora tab to +`sudo dnf install loopwire` and a separate one-time setup link. Automatic installation and other platform options +remain available. + +## Development and guest evidence + +Run the generator, publisher, bootstrap, public-checker, channel-gate, workflow, documentation, and site tests. The +pinned Fedora tools image gives non-Fedora hosts the RPM toolchain without changing host package state: + +```bash +bash scripts/with-rpm-tools.sh --container python3 scripts/test-rpm-repository.py +python3 scripts/test-fedora-bootstrap.py +python3 scripts/test-publish-rpm-repository.py +python3 scripts/test-rpm-public.py +node --test apps/site/src/lib/rpmChannel.test.mjs +``` + +Use the dedicated Fedora 44 guest lifecycle modes for stronger evidence: + +```bash +bash scripts/native-package-vm.sh run-fedora-repo \ + --target fedora-44 --version 0.1.0 --release-dir dist/release +bash scripts/native-package-vm.sh verify-fedora-repo --target fedora-44 +``` + +The clean, checksum-pinned guest must exercise repository setup, install, reinstall, synthetic upgrade, explicit +downgrade/rollback, removal, and repository removal. Record the target, repository origin, exact versions, package and +metadata signer, public URL, signature results, GUI/provider smokes, and command logs. Synthetic fixture releases +exercise lifecycle behavior with authenticated existing payloads; they do not create a public application release, +prove production reachability, or promote Fedora desktop/audio support. diff --git a/apps/docs/docs/developer/release.md b/apps/docs/docs/developer/release.md index dbbba88..eec7a4f 100644 --- a/apps/docs/docs/developer/release.md +++ b/apps/docs/docs/developer/release.md @@ -17,6 +17,24 @@ verified record is committed and the site deployed, Ubuntu and Debian homepage t commands. Fixture lifecycle tests do not count as production proof. Follow the runbook's final activation procedure before announcing repository availability. +## Signed Fedora channel + +Fedora 44 x86_64 publication has a separate +[Fedora repository operations runbook](./fedora-repository.md). The selected provider is a project-owned HTTPS +repository: this keeps control of the exact release artifact, RPM and metadata signing, atomic promotion, indefinite +retention, and rollback proof. COPR would produce provider-built RPMs that need a separate build-ID and +exact-provider-output proof path, while its retention and promotion behavior remain service-owned. + +The optional **Publish Fedora Repository** workflow runs after GitHub Release publication only when +`FEDORA_REPOSITORY_ENABLED=true` in the protected `packages-production` environment. A repository failure leaves the +published GitHub Release and previous Fedora revision intact for repair and retry. Production origin/signing +provisioning and first activation remain human tasks. + +The Fedora homepage tab remains on its signed direct-download RPM while +`packaging/repositories/fedora-channel.json` is pending. Only a complete public HTTPS verification record changes that +tab to `sudo dnf install loopwire` and links the separate one-time setup procedure. The automatic installer continues +to work through the direct-download path, and fixture lifecycle evidence cannot activate the production channel. + ## Local Artifact Smoke ```bash diff --git a/apps/docs/docs/guide/fedora-repository.md b/apps/docs/docs/guide/fedora-repository.md new file mode 100644 index 0000000..a93c0b9 --- /dev/null +++ b/apps/docs/docs/guide/fedora-repository.md @@ -0,0 +1,150 @@ + + +# Fedora repository + +Loopwire's third-party Fedora repository targets **Fedora 44 on x86_64**. It requires one-time setup because Loopwire +is not included in Fedora's default repositories. Other Fedora releases and architectures should use the matching +option in the [installation guide](./install.md). + +
+

Public channel pending

+

The repository implementation is available in the source tree, but its public URL and signing key have not been + activated. Use the signed Fedora direct download or the automatic installer. + The setup command will appear here only after the production repository passes public verification.

+
+ +
+

Verified public channel

+

Repository: {{ channel.baseUrl }}
+ OpenPGP fingerprint: {{ channel.signingFingerprint }}
+ Public verification record, recorded {{ channel.verifiedAt }}.

+
+ +## One-time setup + +The following procedure applies once this page displays a verified public channel. You need Bash, curl, GnuPG, RPM, +DNF, and sudo access. Download and inspect the small setup helper first: + +```bash +curl -fsSLo setup-fedora-repository.sh \ + https://raw.githubusercontent.com/sandwichfarm/loopwire/master/scripts/setup-fedora-repository.sh +less setup-fedora-repository.sh +``` + +
+

Run the helper with the published URL and complete signing fingerprint:

+ 
{{ setupCommand }}
+

It writes an equivalent repository definition:

+ 
{{ repoDefinition }}
+
+ +The helper accepts only Fedora 44 on x86_64. It downloads the OpenPGP public key over HTTPS, verifies the complete +fingerprint, and writes only `/etc/yum.repos.d/loopwire.repo` plus the fingerprint-named key under +`/etc/pki/rpm-gpg/`. Repeating setup with the same inputs is safe. An unrelated `loopwire.repo` or a symbolic link in +either managed path is an error; other DNF sources are preserved. Add `--dry-run` and omit `sudo` to preview the +target, URL, key fingerprint, and managed paths without downloads or changes. + +The generated `.repo` file keeps `gpgcheck=1`, `repo_gpgcheck=1`, and `sslverify=1`. DNF verifies the RPM signature and the +detached signature on repository metadata. Do not add `--nogpgcheck`, disable either setting, or copy the local-RPM +signature exception from the direct-download path into this repository path. + +After successful setup, refresh metadata and install: + +```bash +sudo dnf makecache --refresh && +sudo dnf install loopwire +``` + +The setup helper does not install Loopwire. Inspect `dnf repoquery --info --repo=loopwire loopwire` before installation +when you need to confirm the candidate version and repository. Fedora 44's DNF5 can exit successfully after reporting +a bad repository-metadata signature and excluding the source, so require an actual Loopwire package record and no GPG +verification error in the output. The package includes the desktop application and background/provider commands +without enabling startup services or applying audio routes during installation. + +## Update and reinstall + +DNF can update Loopwire with the rest of the system. To update only Loopwire: + +```bash +sudo dnf upgrade --refresh loopwire +``` + +To repair files owned by the installed package without changing versions: + +```bash +sudo dnf reinstall loopwire +``` + +Saved routing configurations are outside package ownership and are preserved. + +## Earlier versions + +List versions retained in the repository with `dnf --showduplicates list loopwire`. DNF does not automatically follow +a repository rollback to an older build. When maintainers recommend rollback, either select the next lower retained +version or name the exact Fedora package version: + +```bash +sudo dnf downgrade loopwire +# Or replace VERSION-RELEASE with an exact retained value, such as 0.1.0-1.fc44: +sudo dnf install loopwire-VERSION-RELEASE.x86_64 +``` + +Do not install an RPM built for another Fedora release or architecture. Ask for recovery guidance if the required +version is absent instead of disabling signature checks. + +## Remove Loopwire or the repository + +Uninstall the application with `sudo dnf remove loopwire`. DNF removes package-owned files and leaves saved Loopwire +configurations intact. Remove startup integration separately if you enabled it through the +[start-on-boot guide](./start-on-boot.md). + +To stop receiving repository updates, use the same inspected helper: + +```bash +sudo bash setup-fedora-repository.sh --remove && +sudo dnf clean metadata +``` + +This removes only the managed Loopwire `.repo` file and its scoped public-key file. It does not uninstall Loopwire, +change Fedora's repositories, or remove keys already accepted into the RPM database. Without the `.repo` file, that +key no longer authorizes a Loopwire source. Manually provisioned definitions and keys require matching manual cleanup. + +## Trust, key changes, and troubleshooting + +The repository OpenPGP key signs the Fedora RPM and repository metadata. It is separate from the OpenSSL release key +that authenticates checksums for direct GitHub Release downloads. The scoped `.repo` configuration does not grant the +Loopwire key authority over other repositories. + +`metadata_expire=6h` asks DNF to refetch metadata after six hours; it is a cache setting, not a signed expiry check. +DNF verifies that metadata was signed by the trusted key, but that signature alone cannot detect replay of older, +correctly signed metadata. Loopwire's publication verifier enforces a custom signed verification deadline and the +project monitors the HTTPS origin. Stop and report an unexpected version rollback rather than forcing installation. + +- **Wrong or changed fingerprint:** stop and compare this page's fingerprint with the project's announced key change. + Rerun the inspected helper only after the successor fingerprint is confirmed through a trusted project channel. +- **Revoked or compromised key:** disable or remove the repository immediately. Do not accept new metadata from that + key. Follow the incident notice to verify and bootstrap a replacement repository explicitly. +- **Bad RPM or metadata signature:** stop the install, run `sudo dnf clean metadata`, retry a refresh, and report the + exact DNF error. Do not disable `gpgcheck`, `repo_gpgcheck`, or TLS verification. +- **Stale or unavailable metadata:** run `sudo dnf clean metadata` and retry. `skip_if_unavailable=False` makes a + missing or invalid Loopwire repository visible instead of silently continuing with stale assumptions. +- **Unsupported Fedora release or architecture:** use the [installation guide](./install.md). Editing `$releasever` or + forcing another repository path does not make its package compatible. + +Useful diagnostics are `rpm -E %fedora`, `uname -m`, `dnf repolist --enabled`, +`dnf repoquery --info --repo=loopwire loopwire`, and the DNF error text. Redact usernames and private URLs. Never send +private keys, environment variables, or audio recordings. diff --git a/apps/docs/docs/guide/install.md b/apps/docs/docs/guide/install.md index 2c39325..15e5805 100644 --- a/apps/docs/docs/guide/install.md +++ b/apps/docs/docs/guide/install.md @@ -92,8 +92,20 @@ sudo apt install loopwire Loopwire is not in the default Ubuntu or Debian repositories. Until the guide displays a verified public channel, use Automatic or the signed manual downloads below. The automatic installer does not configure the APT repository. -The RPM files have no embedded RPM signature. The commands authenticate the download using the signed SHA-256 -manifest first, then permit this local RPM for that install; repository dependency checks remain enabled. +The [Fedora repository guide](./fedora-repository.md) provides the same gated availability record and one-time setup +for Fedora 44 on x86_64. Once that page displays a verified public channel, the normal repository install is: + +```bash +sudo dnf install loopwire +``` + +This is a third-party Loopwire repository, not a Fedora or COPR repository. Until its guide displays a verified +production URL and fingerprint, use Automatic or the signed Fedora direct download below. Automatic continues to use +the direct-download path and does not add the repository. + +The `v0.1.0` direct-download RPM files have no embedded RPM signature. The commands authenticate the download using +the signed SHA-256 manifest first, then permit this local RPM for that install; repository dependency checks remain +enabled. ### Ubuntu 24.04 @@ -141,8 +153,10 @@ sha256sum --check --ignore-missing SHA256SUMS && sudo dnf --setopt=localpkg_gpgcheck=0 install ./loopwire-0.1.0-1.fc44.x86_64.rpm ``` -[Repository work to shorten this setup](https://github.com/sandwichfarm/loopwire/issues/36) tracks signed metadata, -release publication, clean-guest install/upgrade verification, and updated instructions. +[Fedora repository setup and availability](./fedora-repository.md) includes the public activation gate, normal DNF +updates, reinstall, exact-version rollback, removal, and signing-key guidance. Its repository path keeps both package +and metadata signature checks enabled; the local package exception above applies only to this authenticated direct +download. ### openSUSE Tumbleweed @@ -272,8 +286,9 @@ Run the metadata smoke: pnpm verify:packaging ``` -The AppImages and native deb/RPM files are published as direct downloads on the `v0.1.0` GitHub Release. The APT -repository has a separate [public activation gate](./apt-repository.md); DNF/COPR and OBS repositories remain planned. +The AppImages and native deb/RPM files are published as direct downloads on the `v0.1.0` GitHub Release. The APT and +Fedora repositories have separate [APT](./apt-repository.md) and [Fedora](./fedora-repository.md) public activation +gates; the openSUSE repository remains planned. Their matching-guest proof command boots official, checksum-pinned cloud images under KVM and stores local evidence without changing host audio: diff --git a/apps/docs/docs/guide/support-matrix.md b/apps/docs/docs/guide/support-matrix.md index 28d489d..01c6397 100644 --- a/apps/docs/docs/guide/support-matrix.md +++ b/apps/docs/docs/guide/support-matrix.md @@ -72,7 +72,9 @@ the Tauri shell command bridge. | AppImage | Published-artifact and Tauri bundle smoke | Published for 0.1.0 on GitHub Releases. | | Ubuntu 24.04 / Debian 13 deb | Verified in matching KVM guests at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as direct downloads. | | Ubuntu 24.04 / Debian 13 signed APT repository, amd64 | Separate signed-metadata, publication, bootstrap, and clean-guest lifecycle checks | Public activation is gated; see [current channel availability](./apt-repository.md). | -| Fedora 44 / openSUSE Tumbleweed RPM | Verified in matching KVM guests at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as direct downloads; no COPR/OBS repository. | +| Fedora 44 RPM | Verified in a matching KVM guest at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as a direct download. | +| Fedora 44 signed project repository, x86_64 | Signed-RPM/metadata, publication, bootstrap, and clean-guest lifecycle checks | Public activation is gated; see [current channel availability](./fedora-repository.md). | +| openSUSE Tumbleweed RPM | Verified in a matching KVM guest at commit `70eee4e`; review snapshot in `vm/native-package-proof/` | Published as a direct download; no OBS repository. | | AUR `loopwire` | Tagged source build through `pnpm verify:aur:source` | Published for 0.1.0. | | AUR `loopwire-bin` | Signed release-artifact build through `pnpm verify:aur` | Published for 0.1.0. | | AUR `loopwire-git` | Rolling default-branch build through `pnpm verify:aur:git` | Published; development snapshots are not stable releases. | @@ -87,6 +89,11 @@ APT lifecycle evidence is separate from the direct-download snapshot. Isolated H establish a new public release or production channel. Only reviewed production HTTPS verification activates APT instructions. Repository support covers the named distro versions on amd64, not derivatives or other architectures. +Fedora repository lifecycle evidence is separate too. It proves signed RPM and repository-metadata handling, +publication recovery, and install/reinstall/upgrade/downgrade/removal behavior on a clean Fedora 44 x86_64 guest. +Synthetic versions and local HTTPS fixtures are development evidence. They do not prove the production URL is live or +promote Fedora desktop/audio support. Only a reviewed production verification record activates the short DNF command. + Native package verification is narrower than audio-backend support. The committed snapshot proves that each official, checksum-pinned guest built and installed its target package, ran the packaged background/provider/backend commands, resolved GUI libraries, created a Loopwire X11 window under Xvfb, and removed all package-owned files. It does not diff --git a/apps/docs/docs/release-notes/unreleased.md b/apps/docs/docs/release-notes/unreleased.md index dcc19d4..767f5e9 100644 --- a/apps/docs/docs/release-notes/unreleased.md +++ b/apps/docs/docs/release-notes/unreleased.md @@ -16,6 +16,20 @@ These notes describe source-tree progress. They are not a public release announc - Added [user setup and recovery guidance](../guide/apt-repository.md) and the [maintainer publication runbook](../developer/apt-repository.md). +## Signed Fedora repository development + +- Selected a project-owned Fedora repository over COPR so the project can control exact release artifacts, RPM and + metadata signing, atomic promotion, retained snapshots, and rollback verification. +- Added Fedora 44 x86_64 signed-RPM and repository-metadata generation, guarded SSH/POSIX publication, recovery, + public HTTPS verification, and clean-guest lifecycle proof surfaces. +- Added a repeat-safe setup/removal helper that verifies the complete OpenPGP fingerprint and keeps both `gpgcheck=1` + and `repo_gpgcheck=1`. The repository never uses the local direct-download RPM signature exception. +- The Fedora homepage tab switches to `sudo dnf install loopwire` only after a complete public verification record is + reviewed and committed. Production hosting, signing key/environment setup, and first activation remain human + operations; the automatic installer and signed direct-download path remain available. +- Added [Fedora user setup and rollback guidance](../guide/fedora-repository.md) and the + [maintainer operations runbook](../developer/fedora-repository.md). + ## Other distribution updates - Root-level Bunny deployments now pass the post-upload live-site check when `BUNNY_REMOTE_PREFIX` is intentionally @@ -50,8 +64,9 @@ These notes describe source-tree progress. They are not a public release announc release verification key, and handles repeat installs and upgrades. Portable installs stage and verify files before replacing existing files; native packages use the distro package manager. Automatic preserves earlier portable installations, and incomplete rollback retains recovery backups with explicit restoration paths. -- Multi-step manual package instructions link to repository work for [APT](https://github.com/sandwichfarm/loopwire/issues/35), - [Fedora](https://github.com/sandwichfarm/loopwire/issues/36), and [openSUSE](https://github.com/sandwichfarm/loopwire/issues/37). +- Multi-step manual package instructions link to gated repository setup for + [APT](../guide/apt-repository.md) and [Fedora](../guide/fedora-repository.md), while + [openSUSE repository work](https://github.com/sandwichfarm/loopwire/issues/37) remains tracked separately. ## Packaging diff --git a/apps/site/src/lib/rpmChannel.mjs b/apps/site/src/lib/rpmChannel.mjs new file mode 100644 index 0000000..1d8bb88 --- /dev/null +++ b/apps/site/src/lib/rpmChannel.mjs @@ -0,0 +1,65 @@ +/** + * Fedora repository instructions are advertised only after the complete public + * verification record for the supported target has been reviewed and committed. + * Invalid or incomplete records preserve the signed direct-download option. + * @param {unknown} value + */ +export function verifiedFedoraChannel(value) { + if (!value || typeof value !== "object") return null; + const channel = /** @type {Record} */ (value); + if (channel.schemaVersion !== 1 || channel.status !== "verified" || channel.target !== "fedora-44" || + typeof channel.baseUrl !== "string" || !validBaseUrl(channel.baseUrl) || + typeof channel.signingFingerprint !== "string" || !/^[A-F0-9]{40}$/.test(channel.signingFingerprint) || + typeof channel.revision !== "string" || !/^[a-f0-9]{64}$/.test(channel.revision) || + typeof channel.verifiedAt !== "string" || !validTimestamp(channel.verifiedAt) || + typeof channel.proofUrl !== "string" || + !/^https:\/\/github\.com\/sandwichfarm\/loopwire\/actions\/runs\/[1-9][0-9]*$/.test(channel.proofUrl)) { + return null; + } + return { + target: channel.target, + baseUrl: channel.baseUrl.replace(/\/+$/, ""), + signingFingerprint: channel.signingFingerprint, + revision: channel.revision, + verifiedAt: channel.verifiedAt, + proofUrl: channel.proofUrl + }; +} + +/** @param {string} value */ +function validBaseUrl(value) { + try { + const url = new URL(value); + return value.startsWith("https://") && !/[^\x21-\x7e]|[\\'"`$<>?#]/.test(value) && + url.protocol === "https:" && Boolean(url.hostname) && !url.username && !url.password && + (!url.port || Number(url.port) >= 1) && !url.search && !url.hash; + } catch { + return false; + } +} + +/** @param {string} value */ +function validTimestamp(value) { + if (!/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,6})?(?:Z|[+-]\d{2}:\d{2})$/.test(value) || + !Number.isFinite(Date.parse(value))) return false; + const date = value.slice(0, 10); + return new Date(`${date}T00:00:00Z`).toISOString().slice(0, 10) === date; +} + +/** + * @template {{command: string, note: string, detail: string, href: string, link: string}} T + * @param {unknown} channel + * @param {T} manual + * @returns {T} + */ +export function fedoraInstallOption(channel, manual) { + if (!verifiedFedoraChannel(channel)) return manual; + return { + ...manual, + command: "sudo dnf install loopwire", + note: "After one-time setup, install and update Loopwire through its signed Fedora repository.", + detail: "Fedora 44 on x86_64 is supported. Other releases and architectures use the portable path.", + href: "/docs/guide/fedora-repository.html#one-time-setup", + link: "Set up the Fedora repository" + }; +} diff --git a/apps/site/src/lib/rpmChannel.test.mjs b/apps/site/src/lib/rpmChannel.test.mjs new file mode 100644 index 0000000..7c4c50e --- /dev/null +++ b/apps/site/src/lib/rpmChannel.test.mjs @@ -0,0 +1,81 @@ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import test from "node:test"; +import { fedoraInstallOption, verifiedFedoraChannel } from "./rpmChannel.mjs"; + +const verified = { + schemaVersion: 1, + status: "verified", + target: "fedora-44", + baseUrl: "https://packages.example.test/fedora/44/x86_64/", + signingFingerprint: "ABCDEF0123456789ABCDEF0123456789ABCDEF01", + revision: "a".repeat(64), + verifiedAt: "2026-09-05T10:20:30+00:00", + proofUrl: "https://github.com/sandwichfarm/loopwire/actions/runs/123456" +}; +const manual = { + id: "fedora", + command: "sudo dnf --setopt=localpkg_gpgcheck=0 install ./loopwire-0.1.0-1.fc44.x86_64.rpm", + note: "Verify the signed download first.", + detail: "Other versions use portable installation.", + href: "/docs/guide/fedora-repository.html", + link: "Fedora repository setup and availability" +}; + +test("pending and incomplete channel records preserve the functional manual option", () => { + for (const value of [null, undefined, {}, [], { ...verified, status: "pending" }]) { + assert.equal(verifiedFedoraChannel(value), null); + assert.equal(fedoraInstallOption(value, manual), manual); + } + for (const key of Object.keys(verified)) { + const value = { ...verified }; + delete value[key]; + assert.equal(verifiedFedoraChannel(value), null, `missing ${key}`); + assert.equal(fedoraInstallOption(value, manual), manual); + } +}); + +test("verified Fedora 44 channel exposes installation and separate setup guidance", () => { + const channel = verifiedFedoraChannel(verified); + assert.equal(channel.target, "fedora-44"); + assert.equal(channel.baseUrl, "https://packages.example.test/fedora/44/x86_64"); + const option = fedoraInstallOption(verified, manual); + assert.equal(option.id, "fedora"); + assert.equal(option.command, "sudo dnf install loopwire"); + assert.equal(option.href, "/docs/guide/fedora-repository.html#one-time-setup"); + assert.doesNotMatch(`${option.note} ${option.detail}`, /workflow|revision|activation|operator/); + assert.match(manual.command, /localpkg_gpgcheck=0/); +}); + +test("malformed or wrong-target records never activate the channel", () => { + const invalid = { + schemaVersion: [0, "1"], status: [true, "ready"], target: ["fedora-43", "Fedora-44", null], + baseUrl: ["http://packages.example.test", "https://user:password@packages.example.test", "https://packages.example.test?a=1", + "https://packages.example.test#fragment", " https://packages.example.test", "https://packages.example.test/\n", "not a URL", + "https://packages.example.test:0", "https://packages.example.test:65536", + "https://packages.example.test?", "https://packages.example.test#", + "https://packages.example.test/$(id)", "https://packages.example.test/`id`", "https://packages.example.test/\\wrong"], + signingFingerprint: ["a".repeat(40), "A".repeat(39), "G".repeat(40)], + revision: ["A".repeat(64), "a".repeat(63)], + verifiedAt: ["yesterday", "2026-09-05", "2026-02-30T00:00:00Z", "2026-09-05T25:00:00Z"], + proofUrl: ["https://github.com/other/repo/actions/runs/123", "https://github.com/sandwichfarm/loopwire/pull/36", + "https://github.com/sandwichfarm/loopwire/actions/runs/123?fixture=1", "https://github.com/sandwichfarm/loopwire/actions/runs/0"] + }; + for (const [key, values] of Object.entries(invalid)) { + for (const value of values) { + const record = { ...verified, [key]: value }; + assert.equal(verifiedFedoraChannel(record), null, `${key}: ${value}`); + assert.equal(fedoraInstallOption(record, manual), manual); + } + } +}); + +test("checked-in Fedora channel remains pending or has a complete verification record", () => { + const channel = JSON.parse(readFileSync(new URL("../../../../packaging/repositories/fedora-channel.json", import.meta.url), "utf8")); + if (channel.status === "pending") { + assert.deepEqual(channel, { schemaVersion: 1, status: "pending", target: null, baseUrl: null, + signingFingerprint: null, revision: null, verifiedAt: null, proofUrl: null }); + } else { + assert.ok(verifiedFedoraChannel(channel)); + } +}); diff --git a/apps/site/src/pages/index.astro b/apps/site/src/pages/index.astro index ebaee03..38acfd9 100644 --- a/apps/site/src/pages/index.astro +++ b/apps/site/src/pages/index.astro @@ -2,7 +2,9 @@ import SiteLayout from "../layouts/SiteLayout.astro"; import screenshot from "../../../../assets/product-screenshot.png"; import aptChannel from "../../../../packaging/repositories/apt-channel.json"; +import fedoraChannel from "../../../../packaging/repositories/fedora-channel.json"; import { aptInstallOption } from "../lib/aptChannel.mjs"; +import { fedoraInstallOption } from "../lib/rpmChannel.mjs"; const title = "Loopwire | Linux virtual audio routing"; const description = @@ -61,15 +63,15 @@ const installOptions = [ detail: "Other Debian versions and ARM64 use the portable path.", href: "/docs/guide/apt-repository.html", link: "APT repository setup and availability" }), - { + fedoraInstallOption(fedoraChannel, { id: "fedora", label: "Fedora", heading: "Fedora 44 · x86_64", command: nativeInstall("loopwire-0.1.0-1.fc44.x86_64.rpm", "sudo dnf --setopt=localpkg_gpgcheck=0 install ./loopwire-0.1.0-1.fc44.x86_64.rpm"), note: "Manual signed v0.1.0 download. Run these steps together in an empty folder. The signed checksum " + "authenticates the RPM before DNF installs it.", detail: "The RPM has no embedded signature; the signature exception applies only to local packages. Automatic " + - "handles verification for you. A signed DNF repository is planned.", - href: "https://github.com/sandwichfarm/loopwire/issues/36", link: "Track simpler Fedora installs" - }, + "handles verification for you. The signed DNF repository is pending public verification.", + href: "/docs/guide/fedora-repository.html", link: "Fedora repository setup and availability" + }), { id: "opensuse", label: "openSUSE", heading: "openSUSE Tumbleweed · x86_64", command: nativeInstall("loopwire-0.1.0-1.x86_64.rpm", "sudo zypper install --allow-unsigned-rpm ./loopwire-0.1.0-1.x86_64.rpm"), diff --git a/package.json b/package.json index ae0294b..08efc2c 100644 --- a/package.json +++ b/package.json @@ -15,7 +15,7 @@ "build:site": "pnpm --filter @loopwire/site build", "build:web": "pnpm build:site && pnpm build:docs && node scripts/build-static-site.mjs", "check": "pnpm check:verify && pnpm lint && pnpm typecheck && pnpm test && pnpm build && pnpm verify:site", - "check:verify": "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri && pnpm verify:apt", + "check:verify": "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri && pnpm verify:apt && pnpm verify:rpm-repository", "collect:evidence": "node scripts/collect-release-evidence.mjs", "collect:support": "node scripts/collect-support-bundle.mjs", "release:handoff": "bash scripts/plan-final-release-handoff.sh", @@ -67,6 +67,7 @@ "verify:docs-live": "bash scripts/verify-docs-live.sh", "verify:packaging": "bash scripts/verify-packaging.sh", "verify:apt": "bash scripts/verify-apt-repository.sh", + "verify:rpm-repository": "bash scripts/verify-rpm-repository.sh", "verify:native-packaging": "bash scripts/verify-native-packaging.sh", "build:portable-linux": "bash scripts/build-portable-linux-binary.sh", "package:deb": "bash scripts/build-deb-package.sh", diff --git a/packaging/README.md b/packaging/README.md index ce615a2..ce0d11a 100644 --- a/packaging/README.md +++ b/packaging/README.md @@ -145,6 +145,51 @@ pnpm verify:nix-release -- \ Render-only mode proves manifest parsing and expression generation. It never proves the package builds. +## Signed Fedora repository + +The project-owned Fedora channel serves only Fedora 44 x86_64. It reuses the native Fedora recipe below, verifies the +exact RPM against the OpenSSL-signed GitHub Release checksum manifest, signs a staging copy with a dedicated OpenPGP +repository key, and generates signed DNF metadata. The original GitHub Release asset stays unchanged. This path was +selected over COPR to preserve exact release-artifact control, atomic promotion, indefinite retention, and local/CI +proof; maintainers accept responsibility for the SSH/POSIX origin and signing-key lifecycle. + +Build and verify a candidate with the pinned RPM tools image or the equivalent host tools: + +```bash +python3 scripts/rpm-repository.py build \ + --release-dir dist/release --version 0.1.0 --output dist/rpm-repository \ + --signing-key "$RPM_FPR" --gnupg-home "$RPM_GNUPG_HOME" \ + --release-public-key packaging/release-signing-public.pem +python3 scripts/rpm-repository.py verify \ + --repository dist/rpm-repository --public-key rpm-public.asc --fingerprint "$RPM_FPR" +``` + +The public target is `/fedora/44/x86_64/`. Package RPMs, fingerprint-named public keys, checksum-named metadata, and +private snapshots are retained. The SSH publisher installs immutable objects first, writes the new detached +`repomd.xml.asc`, and atomically replaces `repomd.xml` as the metadata commit point. Compare-and-swap revision checks, +a server lock, and a recovery journal prevent stale or incomplete promotion from being treated as success. + +Production publication uses `.github/workflows/publish-fedora.yml` and the protected `packages-production` +environment. `FEDORA_REPOSITORY_ENABLED` remains false until the origin, SSH host pins, dedicated signing identity, +approval policy, and public verification are ready. The checked-in `packaging/repositories/fedora-channel.json` +therefore remains pending and the homepage keeps the existing signed direct-download command. + +The client helper writes only `/etc/yum.repos.d/loopwire.repo` and a fingerprint-named public key under +`/etc/pki/rpm-gpg/`. It keeps `gpgcheck=1` and `repo_gpgcheck=1`; repository installation never uses the local-RPM +signature exception: + +```bash +sudo bash scripts/setup-fedora-repository.sh \ + --base-url 'https://HOST/fedora/44/x86_64' --fingerprint "$RPM_FPR" +sudo dnf makecache --refresh +sudo dnf install loopwire +``` + +These commands are illustrative until the channel record is verified. User instructions must take the URL and full +fingerprint from the [gated Fedora repository guide](../apps/docs/docs/guide/fedora-repository.md), not a source-tree +placeholder. See the [maintainer runbook](../apps/docs/docs/developer/fedora-repository.md) for the provider decision, +production layout, protected configuration, publication/recovery, rollback, caching, key rotation, and activation. + ## Native deb and RPM packages The native package recipes install the complete canonical payload: GUI, background restore, DSP and JACK provider @@ -217,6 +262,8 @@ pnpm verify:release pnpm verify:aur pnpm verify:nix-release -- --version 0.1.0 --release-dir dist/release --render-only pnpm verify:packaging +bash scripts/with-rpm-tools.sh --container python3 scripts/test-rpm-repository.py +node --test apps/site/src/lib/rpmChannel.test.mjs ``` `verify:install` creates a local fake release artifact, signs and verifies `SHA256SUMS`, installs it into a temp prefix, diff --git a/packaging/repositories/Dockerfile.rpm-tools b/packaging/repositories/Dockerfile.rpm-tools new file mode 100644 index 0000000..f015241 --- /dev/null +++ b/packaging/repositories/Dockerfile.rpm-tools @@ -0,0 +1,9 @@ +FROM fedora:44@sha256:be9d65e2344d805cc11114319c685ecaa96b6d9b4350a0a6460cdb931babbd19 + +RUN dnf -y install \ + cpio createrepo_c dnf dnf5-plugins gh git gnupg2 nginx nodejs openssh-clients openssh-server \ + openssl python3 rpm rpm-build rpm-sign \ + && dnf clean all + +ENV PYTHONDONTWRITEBYTECODE=1 +WORKDIR /workspace diff --git a/packaging/repositories/fedora-channel.json b/packaging/repositories/fedora-channel.json new file mode 100644 index 0000000..5aad5fd --- /dev/null +++ b/packaging/repositories/fedora-channel.json @@ -0,0 +1,10 @@ +{ + "schemaVersion": 1, + "status": "pending", + "target": null, + "baseUrl": null, + "signingFingerprint": null, + "revision": null, + "verifiedAt": null, + "proofUrl": null +} diff --git a/packaging/repositories/nginx-rpm.conf b/packaging/repositories/nginx-rpm.conf new file mode 100644 index 0000000..e12e8fe --- /dev/null +++ b/packaging/repositories/nginx-rpm.conf @@ -0,0 +1,35 @@ +# Include these locations in a TLS-enabled server. The SSH publisher writes to +# /srv/loopwire-rpm; only its public child is served. snapshots/ and state/ stay +# private. The DNF base URL is https://HOST/fedora/44/x86_64/. +root /srv/loopwire-rpm/public; +autoindex off; +disable_symlinks on; + +# Interrupted same-filesystem writes may leave hidden temporary files. +location ~ (^|/)\. { + deny all; +} + +# Package, fingerprinted key, and checksum-named repodata URLs never change. +# Old objects remain reachable so clients that cached an older repomd.xml can +# finish after a publication or rollback. +location ~ "^/fedora/44/x86_64/(packages/loopwire-[A-Za-z0-9][A-Za-z0-9.+_~]*-1\.fc44\.x86_64\.rpm|keys/([A-F0-9]{40}|[A-F0-9]{64})\.asc|repodata/[0-9a-f]{64}-(primary|filelists|other)\.xml\.gz)$" { + try_files $uri =404; + error_page 404 = @rpm_missing; + add_header Cache-Control "public, max-age=31536000, immutable"; +} + +location @rpm_missing { + add_header Cache-Control "no-store, no-cache, must-revalidate" always; + return 404; +} + +# repomd.xml is the commit point. The publisher atomically replaces its +# detached signature first and repomd.xml last. Never cache either file: a +# request sequence that crosses publication fails closed under repo_gpgcheck=1 +# and a retry obtains the matched pair. No CDN-wide purge is required. +location /fedora/44/x86_64/ { + try_files $uri =404; + add_header Cache-Control "no-store, no-cache, must-revalidate" always; + etag off; +} diff --git a/packaging/vm/guest-fedora-repository-smoke.sh b/packaging/vm/guest-fedora-repository-smoke.sh new file mode 100755 index 0000000..99999dc --- /dev/null +++ b/packaging/vm/guest-fedora-repository-smoke.sh @@ -0,0 +1,355 @@ +#!/usr/bin/env bash +# The unprivileged guest user owns proof logs; sudo applies only to package and trust-store commands. +# shellcheck disable=SC2024 +set -euo pipefail + +target="${1:?target is required}" +package_target="${2:?package target is required}" +format="${3:?format is required}" +version="${4:?version is required}" +git_head="${5:?git head is required}" +kit_dir="${6:-$PWD}" +[ "$target" = fedora-44 ] || { echo "unsupported Fedora repository guest target" >&2; exit 2; } +[ "$package_target" = fedora-44 ] && [ "$format" = rpm ] +[[ "$version" =~ ^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(\+[0-9A-Za-z]+(\.[0-9A-Za-z]+)*)?$ ]] +[[ "$git_head" =~ ^[0-9a-f]{40}$ ]] +cd "$kit_dir" + +proof_dir="$kit_dir/proof" +fixture_dir="$kit_dir/fedora-repository-fixture" +release_dir="$kit_dir/release" +public_release_proof="$proof_dir/public-release" +base_url="https://127.0.0.1:8444/fedora/44/x86_64" +upgrade_version="${version}+dnffixture1" +[[ "$version" != *+* ]] || upgrade_version="${version}.dnffixture1" +baseline_package_version="${version}-1.fc44" +upgrade_package_version="${upgrade_version}-1.fc44" +baseline_package="loopwire-${baseline_package_version}.x86_64.rpm" +upgrade_package="loopwire-${upgrade_package_version}.x86_64.rpm" +mkdir -p "$proof_dir/packages" "$proof_dir/repositories" "$public_release_proof" "$fixture_dir" +exec > >(tee "$proof_dir/commands.log") 2>&1 +set -x + +cat /etc/os-release >"$proof_dir/os-release" +uname -a >"$proof_dir/uname.txt" +systemd-detect-virt --vm >"$proof_dir/virtualization.txt" +grep -Eq '^(kvm|qemu)$' "$proof_dir/virtualization.txt" +if rpm -q loopwire >/dev/null 2>&1; then + echo 'clean guest already has Loopwire installed' >&2 + exit 1 +fi +printf 'absent\n' >"$proof_dir/initial-package-status.txt" + +# Authenticate the public GitHub Release manifest before using any release payload as repository input. +openssl dgst -sha256 -verify packaging/release-signing-public.pem \ + -signature "$release_dir/SHA256SUMS.sig" "$release_dir/SHA256SUMS" +python3 - "$release_dir" "$version" "$baseline_package" >"$proof_dir/public-release-validation.json" <<'PY' +import hashlib, json, pathlib, re, sys +root, version, rpm_name = pathlib.Path(sys.argv[1]), sys.argv[2], sys.argv[3] +selected = [rpm_name, "loopwire-linux-x86_64.tar.gz", "release-assets.json"] +entries = {} +for number, line in enumerate((root / "SHA256SUMS").read_text().splitlines(), 1): + match = re.fullmatch(r"([0-9a-f]{64}) ([^/\\\s]+)", line) + assert match and match.group(2) not in entries, f"invalid or duplicate SHA256SUMS entry at line {number}" + entries[match.group(2)] = match.group(1) +for name in selected: + assert list(entries).count(name) == 1, f"SHA256SUMS must contain exactly one {name} entry" + data = (root / name).read_bytes() + assert hashlib.sha256(data).hexdigest() == entries[name], f"signed checksum mismatch: {name}" +manifest = json.loads((root / "release-assets.json").read_text()) +assert set(manifest) == {"schema", "release", "artifacts"} and manifest["schema"] == "loopwire.release-assets.v1" +release = manifest["release"] +assert set(release) == {"tag", "version", "gitHead"} +assert release["tag"] == f"v{version}" and release["version"] == version +assert re.fullmatch(r"[0-9a-f]{40}", release["gitHead"]) +assert isinstance(manifest["artifacts"], list) +fedora = [item for item in manifest["artifacts"] if isinstance(item, dict) and item.get("target") == "fedora-44"] +assert len(fedora) == 1 and set(fedora[0]) == {"name", "kind", "target", "architecture", "bytes", "sha256"} +rpm = fedora[0] +assert (rpm["name"], rpm["kind"], rpm["target"], rpm["architecture"]) == (rpm_name, "native-rpm", "fedora-44", "x86_64") +assert rpm["bytes"] == (root / rpm_name).stat().st_size and rpm["sha256"] == entries[rpm_name] +portable = [item for item in manifest["artifacts"] if isinstance(item, dict) + and item.get("name") == "loopwire-linux-x86_64.tar.gz"] +assert len(portable) == 1 and portable[0].get("kind") == "portable-archive" +assert portable[0].get("target") == "linux-generic" and portable[0].get("architecture") == "x86_64" +assert portable[0].get("bytes") == (root / selected[1]).stat().st_size and portable[0].get("sha256") == entries[selected[1]] +print(json.dumps({"status": "verified", "releaseGitHead": release["gitHead"], + "rpmSha256": entries[rpm_name], "tarSha256": entries[selected[1]], + "manifestSha256": entries[selected[2]]}, sort_keys=True)) +PY +tar -xOf "$release_dir/loopwire-linux-x86_64.tar.gz" RELEASE >"$proof_dir/payload-release.txt" +python3 - "$proof_dir/payload-release.txt" "$version" <<'PY' +import pathlib, re, sys +values = {} +for line in pathlib.Path(sys.argv[1]).read_text().splitlines(): + assert "=" in line + key, value = line.split("=", 1) + assert key not in values + values[key] = value +assert set(values) == {"name", "version", "arch", "source_date_epoch"} +assert values["name"] == "loopwire" and values["version"] == sys.argv[2] and values["arch"] == "x86_64" +assert re.fullmatch(r"[0-9]+", values["source_date_epoch"]) +PY +public_release_git_head="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["releaseGitHead"])' "$proof_dir/public-release-validation.json")" +printf '%s\n' "$public_release_git_head" >"$proof_dir/public-release-git-head.txt" +cp "$release_dir/$baseline_package" "$public_release_proof/" +cp "$release_dir/loopwire-linux-x86_64.tar.gz" "$public_release_proof/" +cp "$release_dir/SHA256SUMS" "$public_release_proof/" +cp "$release_dir/SHA256SUMS.sig" "$public_release_proof/" +cp "$release_dir/release-assets.json" "$public_release_proof/" +cp packaging/release-signing-public.pem "$public_release_proof/" +cp "$proof_dir/payload-release.txt" "$public_release_proof/RELEASE" +(cd "$release_dir" && sha256sum loopwire-linux-x86_64.tar.gz) >"$proof_dir/release-payload.sha256" + +sudo dnf install -y \ + ca-certificates cpio createrepo_c curl findutils gnupg2 gzip nodejs openssl python3 \ + rpm-build rpm-sign tar xdotool xorg-x11-server-Xvfb + +# Signing material exists only inside this disposable guest. Evidence receives public keys only. +gnupg_home="$fixture_dir/gnupg" +mkdir -m 0700 "$gnupg_home" +gpg --homedir "$gnupg_home" --batch --pinentry-mode loopback --passphrase '' \ + --quick-generate-key 'Loopwire disposable Fedora repository guest fixture' rsa3072 sign 0 +fingerprint="$(gpg --homedir "$gnupg_home" --batch --with-colons --list-keys | + awk -F: '$1 == "fpr" { print $10; exit }')" +[[ "$fingerprint" =~ ^[0-9A-F]{40}$ ]] +gpg --homedir "$gnupg_home" --batch --armor --export "$fingerprint" >"$proof_dir/repository-key.asc" +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out "$fixture_dir/synthetic-release-key.pem" +openssl pkey -in "$fixture_dir/synthetic-release-key.pem" -pubout -out "$fixture_dir/synthetic-release-public.pem" +cp "$fixture_dir/synthetic-release-public.pem" "$proof_dir/synthetic-release-public.pem" + +build_fixture_release() { + local fixture_version="$1" destination="$2" package_name package_sha + mkdir -p "$destination" + SOURCE_DATE_EPOCH=0 bash scripts/build-rpm-package.sh --target fedora-44 \ + --version "$fixture_version" --arch x86_64 --release-dir "$release_dir" --output-dir "$destination" + package_name="loopwire-${fixture_version}-1.fc44.x86_64.rpm" + [ -f "$destination/$package_name" ] + [ "$(find "$destination" -maxdepth 1 -type f -name '*.rpm' | wc -l)" -eq 1 ] + package_sha="$(sha256sum "$destination/$package_name" | awk '{ print $1 }')" + printf '%s %s\n' "$package_sha" "$package_name" >"$destination/SHA256SUMS" + openssl dgst -sha256 -sign "$fixture_dir/synthetic-release-key.pem" \ + -out "$destination/SHA256SUMS.sig" "$destination/SHA256SUMS" +} + +build_fixture_release "$upgrade_version" "$fixture_dir/upgrade-release" +baseline_source_sha256="$(sha256sum "$release_dir/$baseline_package" | awk '{ print $1 }')" +upgrade_source_sha256="$(sha256sum "$fixture_dir/upgrade-release/$upgrade_package" | awk '{ print $1 }')" +printf 'baseline\t%s\t%s\nupgraded\t%s\t%s\n' \ + "$baseline_package" "$baseline_source_sha256" "$upgrade_package" "$upgrade_source_sha256" \ + >"$proof_dir/release-sources.tsv" + +python3 scripts/rpm-repository.py build --release-dir "$release_dir" --version "$version" \ + --output "$fixture_dir/initial" --signing-key "$fingerprint" --gnupg-home "$gnupg_home" +python3 scripts/rpm-repository.py build --release-dir "$fixture_dir/upgrade-release" --version "$upgrade_version" \ + --output "$fixture_dir/upgraded" --previous "$fixture_dir/initial" --signing-key "$fingerprint" \ + --gnupg-home "$gnupg_home" --release-public-key "$fixture_dir/synthetic-release-public.pem" +python3 scripts/rpm-repository.py rollback --repository "$fixture_dir/initial" \ + --output "$fixture_dir/rolled-back" --signing-key "$fingerprint" --gnupg-home "$gnupg_home" + +for repository_stage in initial upgraded rolled-back; do + python3 scripts/rpm-repository.py verify --repository "$fixture_dir/$repository_stage" \ + --public-key "$proof_dir/repository-key.asc" --fingerprint "$fingerprint" \ + >"$proof_dir/repositories/${repository_stage}-verification.json" + cp -a "$fixture_dir/$repository_stage" "$proof_dir/repositories/$repository_stage" +done +cp "$fixture_dir/initial/packages/$baseline_package" "$proof_dir/packages/" +cp "$fixture_dir/upgraded/packages/$upgrade_package" "$proof_dir/packages/" +baseline_rpm_sha256="$(sha256sum "$proof_dir/packages/$baseline_package" | awk '{ print $1 }')" +upgrade_rpm_sha256="$(sha256sum "$proof_dir/packages/$upgrade_package" | awk '{ print $1 }')" +printf 'baseline\t%s\t%s\nupgraded\t%s\t%s\n' \ + "$baseline_package" "$baseline_rpm_sha256" "$upgrade_package" "$upgrade_rpm_sha256" \ + >"$proof_dir/signed-packages.tsv" + +# Verify each distributed RPM against the repository key without changing the guest's system RPM database. +fixture_rpmdb="$fixture_dir/rpmdb" +mkdir -p "$fixture_rpmdb" +rpm --dbpath "$fixture_rpmdb" --initdb +rpm --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" +rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$baseline_package" >"$proof_dir/packages/baseline-rpm-signature.txt" +rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$upgrade_package" >"$proof_dir/packages/upgraded-rpm-signature.txt" + +# A guest-only CA exercises real TLS verification; no production trust is imported. +openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj '/CN=Loopwire disposable guest CA' \ + -keyout "$fixture_dir/ca-key.pem" -out "$fixture_dir/ca.crt" \ + -addext 'basicConstraints=critical,CA:TRUE' -addext 'keyUsage=critical,keyCertSign,cRLSign' +openssl req -newkey rsa:2048 -nodes -subj '/CN=127.0.0.1' \ + -keyout "$fixture_dir/tls-key.pem" -out "$fixture_dir/tls.csr" +printf 'subjectAltName=IP:127.0.0.1\nbasicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth\n' \ + >"$fixture_dir/tls.ext" +openssl x509 -req -in "$fixture_dir/tls.csr" -CA "$fixture_dir/ca.crt" -CAkey "$fixture_dir/ca-key.pem" \ + -CAcreateserial -days 1 -extfile "$fixture_dir/tls.ext" -out "$fixture_dir/tls.crt" +cp "$fixture_dir/ca.crt" "$proof_dir/tls-ca.crt" +cp "$fixture_dir/tls.crt" "$proof_dir/tls-server.crt" +sudo install -m 0644 "$fixture_dir/ca.crt" /etc/pki/ca-trust/source/anchors/loopwire-guest-fixture.crt +sudo update-ca-trust +mkdir -p "$fixture_dir/www/fedora/44" +ln -s "$fixture_dir/initial" "$fixture_dir/www/fedora/44/x86_64" +cat >"$fixture_dir/https-server.py" <<'PY' +import functools +import http.server +import ssl +import sys +class Handler(http.server.SimpleHTTPRequestHandler): + def do_GET(self): + # Repository revisions can share a timestamp to the second. Always serve current signed bytes. + if "If-Modified-Since" in self.headers: + del self.headers["If-Modified-Since"] + super().do_GET() +class Server(http.server.ThreadingHTTPServer): + def shutdown_request(self, request): + request.settimeout(5) + try: + request.unwrap() + except (OSError, ssl.SSLError): + request.close() +server = Server(("127.0.0.1", 8444), functools.partial(Handler, directory=sys.argv[1])) +context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) +context.load_cert_chain(sys.argv[2], sys.argv[3]) +server.socket = context.wrap_socket(server.socket, server_side=True) +server.serve_forever() +PY +python3 "$fixture_dir/https-server.py" "$fixture_dir/www" "$fixture_dir/tls.crt" "$fixture_dir/tls-key.pem" \ + >"$proof_dir/https-server.log" 2>&1 & +server_pid=$! +cleanup() { kill "$server_pid" 2>/dev/null || true; } +trap cleanup EXIT +for _attempt in $(seq 1 20); do + if curl --fail --silent --show-error "$base_url/keys/$fingerprint.asc" >"$proof_dir/https-key.asc"; then break; fi + sleep 1 +done +cmp "$proof_dir/repository-key.asc" "$proof_dir/https-key.asc" + +verify_public_stage() { + local stage="$1" + python3 scripts/verify-rpm-public.py --repository "$fixture_dir/$stage" \ + --public-key "$proof_dir/repository-key.asc" --fingerprint "$fingerprint" \ + --base-url "$base_url" --ca-file "$fixture_dir/ca.crt" \ + | tee "$proof_dir/repositories/${stage}-public-verification.json" +} + +verify_public_stage initial +sudo bash scripts/setup-fedora-repository.sh --base-url "$base_url" --fingerprint "$fingerprint" \ + >"$proof_dir/bootstrap.log" 2>&1 +cat /etc/yum.repos.d/loopwire.repo >"$proof_dir/loopwire.repo" +cat "/etc/pki/rpm-gpg/RPM-GPG-KEY-loopwire-$fingerprint" >"$proof_dir/configured-repository-key.asc" +cmp "$proof_dir/repository-key.asc" "$proof_dir/configured-repository-key.asc" +sudo dnf makecache --refresh -y >"$proof_dir/bootstrap-makecache.log" 2>&1 + +smoke_installed() { + local stage="$1" expected_version="$2" package_name="$3" stage_dir="$proof_dir/$1" + mkdir -p "$stage_dir" + rpm -q --qf '%{NAME}\t%{VERSION}-%{RELEASE}\t%{ARCH}\n' loopwire >"$stage_dir/package-metadata.tsv" + [ "$(rpm -q --qf '%{VERSION}-%{RELEASE}' loopwire)" = "$expected_version" ] + dnf repoquery --installed --qf '%{name}|%{evr}|%{arch}|%{from_repo}' loopwire >"$stage_dir/dnf-origin.txt" + grep -Fxq "loopwire|$expected_version|x86_64|loopwire" "$stage_dir/dnf-origin.txt" + dnf info --installed loopwire >"$stage_dir/dnf-info.txt" + grep -Eq '^From repository[[:space:]]*:[[:space:]]*loopwire$' "$stage_dir/dnf-info.txt" + rpm -ql loopwire | sort >"$stage_dir/package-files.txt" + while IFS= read -r installed_file; do + if [ -f "$installed_file" ]; then sha256sum "$installed_file"; fi + done <"$stage_dir/package-files.txt" >"$stage_dir/installed-files.sha256" + printf '%s %s\n' "$(sha256sum "$proof_dir/packages/$package_name" | awk '{ print $1 }')" "$package_name" \ + >"$stage_dir/signed-package.sha256" + rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$package_name" >"$stage_dir/rpm-signature.txt" + loopwire --background --help >"$stage_dir/background-help.txt" + loopwire-dsp-provider --help >"$stage_dir/dsp-provider-help.txt" + loopwire-jack-ports --help >"$stage_dir/jack-provider-help.txt" + loopwire-detect-audio --pretty >"$stage_dir/detect-audio.json" + ldd /usr/lib/loopwire/loopwire-gui >"$stage_dir/gui-ldd.txt" + if grep -Fq 'not found' "$stage_dir/gui-ldd.txt"; then + echo 'Installed GUI has unresolved shared libraries' >&2 + return 1 + fi + local gui_status=0 + # Runtime process/window variables must expand in the child shell. + # shellcheck disable=SC2016 + timeout 35s bash -c ' + stage_dir="$1" + app_pid="" + Xvfb :99 -screen 0 1280x720x24 -nolisten tcp >"$stage_dir/xvfb.log" 2>&1 & + xvfb_pid=$! + cleanup_gui() { [ -z "$app_pid" ] || kill "$app_pid" 2>/dev/null || true; kill "$xvfb_pid" 2>/dev/null || true; wait || true; } + trap cleanup_gui EXIT + sleep 1 + DISPLAY=:99 GDK_BACKEND=x11 WEBKIT_DISABLE_DMABUF_RENDERER=1 \ + /usr/lib/loopwire/loopwire-gui >"$stage_dir/gui-launch.log" 2>&1 & + app_pid=$! + for attempt in $(seq 1 20); do + kill -0 "$app_pid" 2>/dev/null || exit 1 + if DISPLAY=:99 xdotool search --name "^(Loopwire|loopwire-gui)$" >"$stage_dir/gui-window-ids.txt" 2>/dev/null; then + while read -r window_id; do DISPLAY=:99 xdotool getwindowname "$window_id"; done \ + <"$stage_dir/gui-window-ids.txt" >"$stage_dir/gui-window-names.txt" + exit 0 + fi + sleep 1 + done + exit 124 + ' bash "$stage_dir" || gui_status=$? + printf '%s\n' "$gui_status" >"$stage_dir/gui-launch-status.txt" + [ "$gui_status" -eq 0 ] + [ -s "$stage_dir/gui-window-ids.txt" ] + if grep -Eiq 'error while loading shared libraries|panic|protocol error|missing acquire timeline' "$stage_dir/gui-launch.log"; then + echo 'Installed GUI reported a startup failure' >&2 + return 1 + fi + printf '%s\t%s\tinstalled\n' "$stage" "$expected_version" >>"$proof_dir/lifecycle.tsv" +} + +sudo dnf install -y "loopwire-$baseline_package_version.x86_64" >"$proof_dir/install.log" 2>&1 +smoke_installed install "$baseline_package_version" "$baseline_package" +sudo dnf reinstall -y "loopwire-$baseline_package_version.x86_64" >"$proof_dir/reinstall.log" 2>&1 +smoke_installed reinstall "$baseline_package_version" "$baseline_package" +ln -sfn "$fixture_dir/upgraded" "$fixture_dir/www/fedora/44/x86_64" +verify_public_stage upgraded +sudo dnf makecache --refresh -y >"$proof_dir/upgrade-makecache.log" 2>&1 +sudo dnf upgrade -y loopwire >"$proof_dir/upgrade.log" 2>&1 +smoke_installed upgrade "$upgrade_package_version" "$upgrade_package" +ln -sfn "$fixture_dir/rolled-back" "$fixture_dir/www/fedora/44/x86_64" +verify_public_stage rolled-back +sudo dnf makecache --refresh -y >"$proof_dir/rollback-makecache.log" 2>&1 +sudo dnf downgrade -y "loopwire-$baseline_package_version.x86_64" >"$proof_dir/rollback.log" 2>&1 +smoke_installed rollback "$baseline_package_version" "$baseline_package" +sudo dnf remove -y loopwire >"$proof_dir/remove.log" 2>&1 +if rpm -q loopwire >/dev/null 2>&1; then + echo 'Loopwire remains registered after removal' >&2 + exit 1 +fi +for removed_file in /usr/bin/loopwire /usr/bin/loopwire-dsp-provider /usr/bin/loopwire-jack-ports \ + /usr/bin/loopwire-detect-audio /usr/lib/loopwire /usr/share/applications/loopwire.desktop \ + /usr/share/icons/hicolor/scalable/apps/loopwire.svg; do + test ! -e "$removed_file" + printf '%s\tabsent\n' "$removed_file" >>"$proof_dir/removed-files.tsv" +done +printf 'remove\t%s\tabsent\n' "$baseline_package_version" >>"$proof_dir/lifecycle.tsv" +sudo bash scripts/setup-fedora-repository.sh --remove >"$proof_dir/source-removal.log" 2>&1 +test ! -e /etc/yum.repos.d/loopwire.repo +test ! -e "/etc/pki/rpm-gpg/RPM-GPG-KEY-loopwire-$fingerprint" +sudo dnf clean all >"$proof_dir/source-removal-clean.log" 2>&1 +dnf repo list --all >"$proof_dir/source-removal-repositories.txt" +if grep -Eq '(^|[[:space:]])loopwire([[:space:]]|$)' "$proof_dir/source-removal-repositories.txt"; then + echo 'DNF still lists the removed repository' >&2 + exit 1 +fi + +{ + printf 'schema\tloopwire.fedora-repository-vm-proof.v1\n' + printf 'target\t%s\n' "$target" + printf 'git_head\t%s\n' "$git_head" + printf 'version\t%s\n' "$version" + printf 'upgrade_version\t%s\n' "$upgrade_version" + printf 'baseline_package_version\t%s\n' "$baseline_package_version" + printf 'upgrade_package_version\t%s\n' "$upgrade_package_version" + printf 'fingerprint\t%s\n' "$fingerprint" + printf 'base_url\t%s\n' "$base_url" + printf 'payload_kind\tpublic-release-baseline-with-synthetic-upgrade\n' + printf 'synthetic_upgrade\ttrue\n' + printf 'public_release_git_head\t%s\n' "$public_release_git_head" + printf 'baseline_source_sha256\t%s\n' "$baseline_source_sha256" + printf 'upgrade_source_sha256\t%s\n' "$upgrade_source_sha256" + printf 'baseline_rpm_sha256\t%s\n' "$baseline_rpm_sha256" + printf 'upgrade_rpm_sha256\t%s\n' "$upgrade_rpm_sha256" + printf 'verification_epoch\t%s\n' "$(date +%s)" +} >"$proof_dir/summary.tsv" +set +x +echo "Fedora repository lifecycle proof passed: $target" diff --git a/scripts/native-package-vm.sh b/scripts/native-package-vm.sh index 3f41104..e4ac9cf 100755 --- a/scripts/native-package-vm.sh +++ b/scripts/native-package-vm.sh @@ -25,17 +25,20 @@ Usage: native-package-vm.sh verify-all [--git-head COMMIT] native-package-vm.sh run-apt --target ubuntu-24.04|debian-13 --version VERSION --release-dir DIR native-package-vm.sh verify-apt --target ubuntu-24.04|debian-13 [--git-head COMMIT] + native-package-vm.sh run-fedora-repo --target fedora-44 --version VERSION --release-dir DIR + native-package-vm.sh verify-fedora-repo --target fedora-44 [--git-head COMMIT] Environment: LOOPWIRE_NATIVE_VM_ROOT Cache/run/evidence root (default: .vm/native-packages) LOOPWIRE_APT_VM_ROOT APT run/evidence root (default: .vm/apt-repository) + LOOPWIRE_FEDORA_VM_ROOT Fedora repository run/evidence root (default: .vm/fedora-repository) LOOPWIRE_NATIVE_VM_TARGETS Target manifest override LOOPWIRE_QEMU_IMAGE Docker QEMU tool image tag The host needs Docker, OpenSSH, /dev/kvm access, and enough disk for the official cloud images. Containers only provide QEMU tools; every proof is collected from -a separately booted guest kernel. APT proofs use verify-apt-repository-vm-proof.mjs; -the original native-package proofs use verify-native-package-vm-proof.mjs. +a separately booted guest kernel. Repository proofs use their matching strict +VM-proof verifier; the original package proofs use verify-native-package-vm-proof.mjs. USAGE } @@ -252,6 +255,14 @@ run_target() { [ -d "$release_dir" ] || fail "release directory does not exist: $release_dir" [ -f "$release_dir/loopwire-linux-x86_64.tar.gz" ] || fail "release tarball is missing" [ -f "$release_dir/SHA256SUMS" ] || fail "release checksum manifest is missing" + if [ "$proof_kind" = "fedora-repository" ]; then + for release_file in \ + "loopwire-${version}-1.fc44.x86_64.rpm" \ + SHA256SUMS.sig \ + release-assets.json; do + [ -f "$release_dir/$release_file" ] || fail "Fedora repository release artifact is missing: $release_file" + done + fi require_host require_committed_implementation download_target "$selected" @@ -271,6 +282,9 @@ run_target() { if [ "$proof_kind" = "apt" ]; then container="loopwire-apt-$id" port=$((port + 10)) + elif [ "$proof_kind" = "fedora-repository" ]; then + container="loopwire-fedora-repository-$id" + port=$((port + 20)) fi key="$(ensure_ssh_key)" public_key="$(cat "${key}.pub")" @@ -283,6 +297,11 @@ run_target() { git archive --format=tar HEAD | tar -xf - -C "$target_dir/kit" cp "$release_dir/loopwire-linux-x86_64.tar.gz" "$target_dir/kit/release/" cp "$release_dir/SHA256SUMS" "$target_dir/kit/release/" + if [ "$proof_kind" = "fedora-repository" ]; then + cp "$release_dir/loopwire-${version}-1.fc44.x86_64.rpm" "$target_dir/kit/release/" + cp "$release_dir/SHA256SUMS.sig" "$target_dir/kit/release/" + cp "$release_dir/release-assets.json" "$target_dir/kit/release/" + fi write_cloud_init "$target_dir" "$public_key" "$id" docker run --rm -v "$target_dir:/vm" "$qemu_image" \ @@ -321,6 +340,9 @@ run_target() { if [ "$proof_kind" = "apt" ]; then guest_script="packaging/vm/guest-apt-repository-smoke.sh" verifier="scripts/verify-apt-repository-vm-proof.mjs" + elif [ "$proof_kind" = "fedora-repository" ]; then + guest_script="packaging/vm/guest-fedora-repository-smoke.sh" + verifier="scripts/verify-fedora-repository-vm-proof.mjs" fi local guest_status=0 # Script paths are fixed and all client-expanded arguments are validated above. @@ -350,6 +372,7 @@ run_target() { trap - EXIT INT TERM local proof_label="native package" [ "$proof_kind" != "apt" ] || proof_label="APT repository lifecycle" + [ "$proof_kind" != "fedora-repository" ] || proof_label="Fedora repository lifecycle" echo "Verified $proof_label in matching KVM guest: $id" echo "Evidence: $evidence_dir" } @@ -360,6 +383,7 @@ verify_target() { [ -n "$git_head" ] || git_head="$(git rev-parse HEAD)" local verifier="scripts/verify-native-package-vm-proof.mjs" [ "$proof_kind" != "apt" ] || verifier="scripts/verify-apt-repository-vm-proof.mjs" + [ "$proof_kind" != "fedora-repository" ] || verifier="scripts/verify-fedora-repository-vm-proof.mjs" node "$verifier" \ --target "$selected" --evidence-dir "$vm_root/evidence/$selected/$git_head" --git-head "$git_head" } @@ -398,6 +422,13 @@ case "$command" in [ "$(realpath -m "$vm_root")" != "$(realpath -m "$image_root")" ] || fail "APT VM state must use a different root from native-package state" ;; + run-fedora-repo | verify-fedora-repo) + [ "$selected" = "fedora-44" ] || fail "$command requires the Fedora 44 target" + proof_kind="fedora-repository" + vm_root="${LOOPWIRE_FEDORA_VM_ROOT:-.vm/fedora-repository}" + [ "$(realpath -m "$vm_root")" != "$(realpath -m "$image_root")" ] || + fail "Fedora repository VM state must use a different root from native-package state" + ;; esac case "$command" in @@ -416,7 +447,7 @@ case "$command" in require_host while read -r id; do download_target "$id"; done < <(target_ids) ;; - run | run-apt) + run | run-apt | run-fedora-repo) [ -n "$selected" ] || fail "run requires --target" [ -n "$version" ] || fail "run requires --version" [ -n "$release_dir" ] || fail "run requires --release-dir" @@ -427,7 +458,7 @@ case "$command" in [ -n "$release_dir" ] || fail "run-all requires --release-dir" while read -r id; do run_target "$id" "$version" "$release_dir"; done < <(target_ids) ;; - verify | verify-apt) + verify | verify-apt | verify-fedora-repo) [ -n "$selected" ] || fail "verify requires --target" verify_target "$selected" "$git_head" ;; diff --git a/scripts/publish-fedora-workflow.sh b/scripts/publish-fedora-workflow.sh new file mode 100755 index 0000000..e9d439d --- /dev/null +++ b/scripts/publish-fedora-workflow.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +set -euo pipefail + +fail() { printf 'publish-fedora-workflow: %s\n' "$*" >&2; exit 1; } +for name in FEDORA_REPOSITORY_URL FEDORA_REPOSITORY_HOST FEDORA_REPOSITORY_ROOT FEDORA_SIGNING_FINGERPRINT \ + FEDORA_SSH_PRIVATE_KEY FEDORA_SSH_KNOWN_HOSTS FEDORA_SIGNING_KEY RUNNER_TEMP GITHUB_REPOSITORY GITHUB_SERVER_URL GITHUB_RUN_ID; do + [ -n "${!name:-}" ] || fail "missing configuration: $name" +done +operation="${OPERATION:-refresh}" +case "$operation" in + publish) [[ "${RELEASE_TAG:-}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || fail "publish requires a stable vX.Y.Z release tag" ;; + refresh) ;; + rollback) [[ "${ROLLBACK_REVISION:-}" =~ ^[a-f0-9]{64}$ ]] || fail "rollback requires a retained revision SHA-256" ;; + *) fail "unknown operation" ;; +esac +[[ "$FEDORA_SIGNING_FINGERPRINT" =~ ^[A-F0-9]{40}$ ]] || fail "signing fingerprint must be complete uppercase hexadecimal" +[[ "${FEDORA_SSH_PORT:-22}" =~ ^[0-9]+$ ]] || fail "SSH port must be numeric" +python3 - "$FEDORA_REPOSITORY_URL" <<'PY' +import runpy, sys +validate = runpy.run_path('scripts/verify-rpm-public.py')['validate_base_url'] +try: + validate(sys.argv[1]) +except ValueError as error: + sys.exit(f'publish-fedora-workflow: {error}') +PY + +work="$(mktemp -d "$RUNNER_TEMP/loopwire-fedora.XXXXXX")" +cleanup() { + gpgconf --homedir "$work/gnupg" --kill all >/dev/null 2>&1 || true + rm -rf -- "$work" +} +trap cleanup EXIT +umask 077 +mkdir "$work/gnupg" +printf '%s\n' "$FEDORA_SSH_PRIVATE_KEY" >"$work/ssh-key" +printf '%s\n' "$FEDORA_SSH_KNOWN_HOSTS" >"$work/known-hosts" +printf '%s\n' "$FEDORA_SIGNING_KEY" >"$work/signing-key.asc" +gpg --no-options --batch --homedir "$work/gnupg" --import "$work/signing-key.asc" +gpg --no-options --batch --homedir "$work/gnupg" --armor --export "$FEDORA_SIGNING_FINGERPRINT" >"$work/public-key.asc" +[ -s "$work/public-key.asc" ] || fail "configured key does not match the expected fingerprint" +sign_args=(--signing-key "$FEDORA_SIGNING_FINGERPRINT" --gnupg-home "$work/gnupg" --valid-for-days 30) +if [ -n "${FEDORA_SIGNING_PASSPHRASE:-}" ]; then + printf '%s' "$FEDORA_SIGNING_PASSPHRASE" >"$work/passphrase" + sign_args+=(--passphrase-file "$work/passphrase") +fi +unset FEDORA_SSH_PRIVATE_KEY FEDORA_SSH_KNOWN_HOSTS FEDORA_SIGNING_KEY FEDORA_SIGNING_PASSPHRASE +transport=(--root "$FEDORA_REPOSITORY_ROOT" --ssh "$FEDORA_REPOSITORY_HOST" --ssh-port "${FEDORA_SSH_PORT:-22}" + --identity-file "$work/ssh-key" --known-hosts "$work/known-hosts" + --public-key "$work/public-key.asc" --fingerprint "$FEDORA_SIGNING_FINGERPRINT") + +set +e +python3 scripts/publish-rpm-repository.py fetch "${transport[@]}" --output "$work/current" +fetch_status=$? +set -e +previous_args=() +if [ "$fetch_status" -eq 0 ]; then + expected="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["revision"])' \ + "$work/current/repository-manifest.json")" + previous_args=(--previous "$work/current") +elif [ "$fetch_status" -eq 3 ] && [ "$operation" = publish ]; then + expected=empty +else + fail "could not load the existing Fedora repository (status $fetch_status); no publication attempted" +fi + +case "$operation" in + publish) + gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" --json isDraft,isPrerelease,tagName >"$work/release.json" + python3 - "$work/release.json" "$RELEASE_TAG" <<'PY' +import json, sys +release = json.load(open(sys.argv[1])) +if release['isDraft'] or release['isPrerelease'] or release['tagName'] != sys.argv[2]: + sys.exit('Fedora publication requires the requested published stable release') +PY + mkdir "$work/release" + gh release download "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" --dir "$work/release" + release_commit="$(git rev-parse --verify "refs/tags/${RELEASE_TAG}^{commit}")" + bash scripts/verify-release-signature.sh --release-dir "$work/release" --public-key packaging/release-signing-public.pem + node scripts/release-asset-manifest.mjs verify --release-dir "$work/release" --tag "$RELEASE_TAG" \ + --git-head "$release_commit" --require-checksum --require-evidence + python3 scripts/rpm-repository.py build --release-dir "$work/release" --version "${RELEASE_TAG#v}" \ + --output "$work/candidate" "${sign_args[@]}" "${previous_args[@]}" + ;; + refresh) + python3 scripts/rpm-repository.py rollback --repository "$work/current" --output "$work/candidate" "${sign_args[@]}" + ;; + rollback) + python3 scripts/publish-rpm-repository.py fetch "${transport[@]}" --revision "$ROLLBACK_REVISION" \ + --output "$work/rollback" + python3 scripts/rpm-repository.py rollback --repository "$work/rollback" --output "$work/candidate" "${sign_args[@]}" + ;; +esac + +mkdir -p dist/fedora-publication +python3 scripts/publish-rpm-repository.py publish "${transport[@]}" --repository "$work/candidate" \ + --expected-revision "$expected" >dist/fedora-publication/publication.json +python3 scripts/verify-rpm-public.py --repository "$work/candidate" --public-key "$work/public-key.asc" \ + --fingerprint "$FEDORA_SIGNING_FINGERPRINT" --base-url "$FEDORA_REPOSITORY_URL" \ + --proof-url "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \ + --output dist/fedora-publication/fedora-channel.json +cp "$work/candidate/repository-manifest.json" dist/fedora-publication/repository-manifest.json +printf 'Fedora repository published and verified; activation record is in the workflow artifact.\n' diff --git a/scripts/publish-rpm-repository.py b/scripts/publish-rpm-repository.py new file mode 100644 index 0000000..f727b1e --- /dev/null +++ b/scripts/publish-rpm-repository.py @@ -0,0 +1,782 @@ +#!/usr/bin/env python3 +"""Publish a verified Fedora RPM repository to a POSIX origin. + +ROOT/public is the HTTP document root. The supported DNF base URL is +ROOT/public/fedora/44/x86_64. ROOT/snapshots and ROOT/state are private. +Package, key, and checksum-named repodata URLs are immutable and retained. + +RPM metadata uses a fail-closed commit protocol: repomd.xml.asc is replaced +first and repomd.xml is replaced atomically last. A client crossing that +boundary can observe a signature mismatch, but repo_gpgcheck=1 cannot accept a +partially published repository. The durable pending journal must be completed +before another revision may be published. + +The SSH transport executes this same source with Python 3 on the origin. The +client verifies OpenPGP signatures; no signing key or GPG program is sent to the +origin. Remote use requires a pinned known_hosts file and an explicit identity +file. The origin filesystem must implement flock, fsync, and same-directory +atomic rename. +""" + +import argparse +import base64 +import contextlib +import fcntl +import hashlib +import json +import os +from pathlib import Path, PurePosixPath +import re +import shlex +import shutil +import stat +import subprocess +import sys +import tarfile +import tempfile +import time + + +MANIFEST = "repository-manifest.json" +SCHEMA = "loopwire.rpm-repository.v1" +TARGET = {"distribution": "fedora", "release": "44", "architecture": "x86_64"} +PUBLIC_PREFIX = Path("fedora/44/x86_64") +REVISION = re.compile(r"[0-9a-f]{64}\Z") +FINGERPRINT = re.compile(r"(?:[A-F0-9]{40}|[A-F0-9]{64})\Z") +VERSION = r"(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)(?:\+[0-9A-Za-z]+(?:\.[0-9A-Za-z]+)*)?" +PACKAGE_PATH = re.compile(rf"packages/loopwire-{VERSION}-1\.fc44\.x86_64\.rpm\Z") +REPODATA_PATH = re.compile( + r"repodata/[0-9a-f]{64}-(?:primary|filelists|other)\.xml\.gz\Z" +) +MANIFEST_FIELDS = { + "schema", "schemaVersion", "revision", "signingFingerprint", + "createdAt", "validUntil", "target", "packages", "files", +} +PACKAGE_FIELDS = { + "name", "version", "release", "architecture", "path", + "sourceReleaseSha256", "distributedSha256", "size", +} +FILE_FIELDS = {"path", "sha256", "size", "kind"} + + +class PublicationError(Exception): + """An actionable publication failure without private transport details.""" + + +class EmptyRepository(PublicationError): + """No committed snapshot exists (distinct exit status 3).""" + + +def require(condition, message): + if not condition: + raise PublicationError(message) + + +def canonical(value): + return json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8") + + +def object_pairs(pairs): + result = {} + for key, value in pairs: + require(key not in result, "duplicate repository JSON field") + result[key] = value + return result + + +def digest(path): + with path.open("rb") as source: + if hasattr(hashlib, "file_digest"): + return hashlib.file_digest(source, "sha256").hexdigest() + result = hashlib.sha256() + for chunk in iter(lambda: source.read(1024 * 1024), b""): + result.update(chunk) + return result.hexdigest() + + +def safe_path(value): + require(isinstance(value, str) and value and "\\" not in value, + "inventory contains an invalid path") + path = PurePosixPath(value) + require(not path.is_absolute() and path.as_posix() == value + and all(part not in (".", "..") for part in path.parts), + "inventory path must be normalized and relative") + return path + + +def classify(path): + """Derive URL mutability from the protocol, never from manifest input.""" + safe_path(path) + if PACKAGE_PATH.fullmatch(path): + return "immutable" + if re.fullmatch(r"keys/(?:[A-F0-9]{40}|[A-F0-9]{64})\.asc", path): + return "immutable" + if REPODATA_PATH.fullmatch(path): + return "immutable" + if path in ("repodata/repomd.xml", "repodata/repomd.xml.asc"): + return "metadata" + raise PublicationError("inventory contains a path outside the Fedora RPM protocol") + + +def plain_path(path, directory=False, missing=False): + """Reject symlinks in every existing ancestor, including origin roots.""" + path = Path(path).absolute() + require(".." not in path.parts, "paths must not contain parent traversal") + for item in reversed((path, *path.parents)): + try: + mode = item.lstat().st_mode + except FileNotFoundError: + if missing: + continue + raise PublicationError("required path does not exist") from None + require(not stat.S_ISLNK(mode), "symlinks are forbidden in repository paths") + if item != path or directory: + require(stat.S_ISDIR(mode), "repository ancestor is not a directory") + return path + + +def tree_files(root): + root = plain_path(root, directory=True) + result = set() + for directory, dirs, files in os.walk(root, followlinks=False): + for name in dirs + files: + item = Path(directory) / name + info = item.lstat() + require(not stat.S_ISLNK(info.st_mode), "candidate contains a symlink") + if name in dirs: + require(stat.S_ISDIR(info.st_mode), "candidate contains a non-directory") + else: + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, + "candidate contains a nonregular file or hardlink") + result.add(item.relative_to(root).as_posix()) + return result + + +def read_json(path): + plain_path(path) + try: + with path.open(encoding="utf-8") as source: + return json.load(source, object_pairs_hook=object_pairs) + except (ValueError, UnicodeError) as error: + raise PublicationError("invalid repository JSON") from error + + +def inventory(root, fingerprint): + root = plain_path(root, directory=True) + actual = tree_files(root) + require(MANIFEST in actual, "candidate is missing its manifest") + manifest = read_json(root / MANIFEST) + require(isinstance(manifest, dict) and manifest.get("schema") == SCHEMA + and manifest.get("schemaVersion") == 1, "unsupported repository manifest") + require(set(manifest) == MANIFEST_FIELDS, "invalid repository manifest fields") + revision = manifest.get("revision") + require(isinstance(revision, str) and REVISION.fullmatch(revision), + "invalid candidate revision") + unsigned = {key: value for key, value in manifest.items() if key != "revision"} + require(hashlib.sha256(canonical(unsigned)).hexdigest() == revision, + "candidate revision does not match its manifest") + require(isinstance(fingerprint, str) and FINGERPRINT.fullmatch(fingerprint) + and manifest.get("signingFingerprint") == fingerprint, + "candidate signing fingerprint differs") + require(manifest.get("target") == TARGET, + "candidate must target Fedora 44 x86_64") + require(type(manifest.get("createdAt")) is int and type(manifest.get("validUntil")) is int + and manifest["validUntil"] > manifest["createdAt"], + "candidate validity interval is invalid") + require(isinstance(manifest.get("packages"), list) and manifest["packages"], + "candidate package inventory must be non-empty") + require(isinstance(manifest.get("files"), list), + "candidate file inventory must be a list") + expected = {MANIFEST} + indexed = {} + for entry in manifest["files"]: + require(isinstance(entry, dict), "invalid candidate file entry") + require(set(entry) == FILE_FIELDS, "invalid candidate file fields") + path = entry.get("path") + kind = classify(path) + require(path not in expected and entry.get("kind") == kind, + "duplicate file or incorrect inventory kind") + require(type(entry.get("size")) is int and entry["size"] >= 0, + "invalid inventory file size") + require(isinstance(entry.get("sha256"), str) and REVISION.fullmatch(entry["sha256"]), + "invalid inventory digest") + target = root / path + require(path in actual and target.stat().st_size == entry["size"] + and digest(target) == entry["sha256"], + "candidate file checksum or size differs") + expected.add(path) + indexed[path] = entry + if path.startswith("repodata/") and kind == "immutable": + require(Path(path).name.startswith(entry["sha256"] + "-"), + "repodata content filename and checksum differ") + require(actual == expected, "candidate contains unlisted files") + require("repodata/repomd.xml" in indexed and "repodata/repomd.xml.asc" in indexed, + "candidate is missing signed repository metadata") + require(f"keys/{fingerprint}.asc" in indexed, + "candidate is missing its pinned public key") + require(all(isinstance(item, dict) and set(item) == PACKAGE_FIELDS + for item in manifest["packages"]), "invalid package record fields") + package_paths = {item.get("path") for item in manifest["packages"]} + require(package_paths and all(PACKAGE_PATH.fullmatch(path or "") for path in package_paths), + "candidate has an invalid package record") + require(len(package_paths) == len(manifest["packages"]), + "candidate has duplicate package records") + require(package_paths == {path for path in indexed if PACKAGE_PATH.fullmatch(path)}, + "package records and file inventory differ") + for package in manifest["packages"]: + require(package["name"] == "loopwire" and package["release"] == "1.fc44" + and package["architecture"] == "x86_64" + and re.fullmatch(VERSION, package["version"]) + and package["path"] == ( + f"packages/loopwire-{package['version']}-1.fc44.x86_64.rpm" + ), "candidate package identity is invalid") + require(isinstance(package["sourceReleaseSha256"], str) + and REVISION.fullmatch(package["sourceReleaseSha256"]) + and isinstance(package["distributedSha256"], str) + and REVISION.fullmatch(package["distributedSha256"]) + and type(package["size"]) is int and package["size"] >= 0, + "candidate package hash or size is invalid") + entry = indexed[package["path"]] + require(entry["sha256"] == package["distributedSha256"] + and entry["size"] == package["size"], + "package record and file inventory differ") + return manifest + + +def verify_signed(root, key, fingerprint, historical=False): + manifest = inventory(root, fingerprint) + verifier = Path(__file__).resolve().with_name("rpm-repository.py") + require(verifier.is_file(), "rpm-repository.py verifier is missing") + command = [sys.executable, str(verifier), "verify", "--repository", str(root), + "--public-key", str(key), "--fingerprint", fingerprint] + if historical: + command += ["--now", str(manifest["createdAt"])] + result = subprocess.run(command, capture_output=True, text=True, check=False) + require(result.returncode == 0, + "signed RPM repository verification failed; run rpm-repository.py verify for diagnostics") + return manifest + + +def fsync_directory(path): + descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW) + try: + os.fsync(descriptor) + finally: + os.close(descriptor) + + +def make_directory(path, mode=0o755): + path = plain_path(path, directory=True, missing=True) + if path.exists(): + return + make_directory(path.parent, mode=mode) + path.mkdir(mode=mode) + descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW) + try: + os.fchmod(descriptor, mode) + os.fsync(descriptor) + finally: + os.close(descriptor) + fsync_directory(path.parent) + + +def atomic_write(target, source=None, data=None, mode=0o644, directory_mode=0o755): + plain_path(target, missing=True) + make_directory(target.parent, mode=directory_mode) + descriptor, temporary = tempfile.mkstemp(prefix=".upload-", dir=target.parent) + try: + with os.fdopen(descriptor, "wb") as output: + if source is not None: + with source.open("rb") as incoming: + shutil.copyfileobj(incoming, output, 1024 * 1024) + else: + output.write(data) + output.flush() + os.fchmod(output.fileno(), mode) + os.fsync(output.fileno()) + os.replace(temporary, target) + fsync_directory(target.parent) + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +def root_path(value): + path = Path(value) + require(path.is_absolute() and path != Path("/"), + "--root must be an absolute, non-root directory") + return plain_path(path, directory=True, missing=True) + + +def public_channel(root): + return root / "public" / PUBLIC_PREFIX + + +@contextlib.contextmanager +def locked(root, create=False): + if create: + make_directory(root) + if not root.exists(): + raise EmptyRepository("repository has no committed snapshot") + lock = root / ".publish.lock" + plain_path(lock, missing=True) + if not create and not lock.exists(): + require(not (root / "state").exists() and not public_channel(root).exists(), + "repository state exists without its publication lock") + raise EmptyRepository("repository has no committed snapshot") + flags = os.O_NOFOLLOW | (os.O_RDWR | os.O_CREAT if create else os.O_RDONLY) + descriptor = os.open(lock, flags, 0o600) + try: + info = os.fstat(descriptor) + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, + "invalid publication lock file") + try: + fcntl.flock(descriptor, + (fcntl.LOCK_EX if create else fcntl.LOCK_SH) | fcntl.LOCK_NB) + except BlockingIOError: + raise PublicationError("repository is locked by another operation; retry later") from None + yield + finally: + os.close(descriptor) + + +def state(root, name): + path = root / "state" / f"{name}.json" + plain_path(path, missing=True) + if not path.exists(): + return None + information = path.stat() + require(stat.S_ISREG(information.st_mode) and information.st_nlink == 1, + "invalid publication state file") + record = read_json(path) + require(isinstance(record, dict) and isinstance(record.get("revision"), str) + and REVISION.fullmatch(record["revision"]), "invalid publication state") + if name == "current": + require(set(record) == {"revision"}, "invalid current publication state") + elif name == "pending": + previous = record.get("previousRevision") + require(set(record) == {"revision", "previousRevision"} + and (previous == "empty" or isinstance(previous, str) + and REVISION.fullmatch(previous)), + "invalid pending publication state") + return record + + +def _checkpoint(label): + """No-op hook for process-interruption tests; never environment-controlled.""" + + +def check_public(root, manifest, immutable_only=False): + channel = public_channel(root) + for entry in manifest["files"]: + if immutable_only and entry["kind"] != "immutable": + continue + target = channel / entry["path"] + plain_path(target, missing=True) + if target.exists(): + info = target.stat() + require(stat.S_ISREG(info.st_mode) and info.st_nlink == 1, + "public target is not a standalone regular file") + require(info.st_size == entry["size"] and digest(target) == entry["sha256"], + "immutable URL collision" if immutable_only + else "committed public repository has drifted") + elif not immutable_only: + raise PublicationError("committed public repository is missing files") + if not immutable_only: + public_manifest = channel / MANIFEST + plain_path(public_manifest, missing=True) + require(public_manifest.is_file() and public_manifest.stat().st_nlink == 1 + and read_json(public_manifest) == manifest, + "committed public repository manifest has drifted") + + +def save_snapshot(root, repository, manifest): + snapshots = root / "snapshots" + make_directory(snapshots, mode=0o700) + snapshot = snapshots / manifest["revision"] + plain_path(snapshot, directory=True, missing=True) + if snapshot.exists(): + require(inventory(snapshot, manifest["signingFingerprint"]) == manifest, + "retained snapshot differs from candidate") + return snapshot + temporary = Path(tempfile.mkdtemp(prefix=".staging-", dir=snapshots)) + try: + os.chmod(temporary, 0o700) + for entry in manifest["files"]: + atomic_write(temporary / entry["path"], source=repository / entry["path"], + mode=0o600, directory_mode=0o700) + atomic_write(temporary / MANIFEST, source=repository / MANIFEST, + mode=0o600, directory_mode=0o700) + inventory(temporary, manifest["signingFingerprint"]) + fsync_directory(temporary) + os.replace(temporary, snapshot) + fsync_directory(snapshots) + finally: + if temporary.exists(): + shutil.rmtree(temporary) + return snapshot + + +def promote(root, snapshot, manifest): + """Publish immutable data, signature, then atomic repomd.xml commit.""" + channel = public_channel(root) + make_directory(channel) + devices = {path.stat().st_dev for path in + (root, channel, root / "state", root / "snapshots")} + require(len(devices) == 1, + "origin public, snapshot, and state paths must share one filesystem") + check_public(root, manifest, immutable_only=True) + for entry in manifest["files"]: + if entry["kind"] == "immutable": + target = channel / entry["path"] + if not target.exists(): + atomic_write(target, source=snapshot / entry["path"]) + _checkpoint("immutable") + signature = "repodata/repomd.xml.asc" + atomic_write(channel / signature, source=snapshot / signature) + _checkpoint("signature") + metadata = "repodata/repomd.xml" + atomic_write(channel / metadata, source=snapshot / metadata) + _checkpoint("committed") + atomic_write(channel / MANIFEST, source=snapshot / MANIFEST) + check_public(root, manifest) + _checkpoint("manifest") + atomic_write(root / "state" / "current.json", + data=canonical({"revision": manifest["revision"]}) + b"\n", mode=0o600) + _checkpoint("current") + (root / "state" / "pending.json").unlink() + fsync_directory(root / "state") + return {"status": "published", "revision": manifest["revision"], + "target": TARGET.copy()} + + +def publish_at(root, repository, fingerprint, expected): + manifest = inventory(repository, fingerprint) + with locked(root, create=True): + current = state(root, "current") + pending = state(root, "pending") + revision = current["revision"] if current else "empty" + if pending: + require(pending["revision"] == manifest["revision"], + "interrupted publication pending; recover it before publishing another revision") + require(expected == pending.get("previousRevision"), + "expected revision differs from interrupted publication") + require(revision in (pending["previousRevision"], pending["revision"]), + "current revision conflicts with pending journal") + snapshot = root / "snapshots" / pending["revision"] + require(inventory(snapshot, fingerprint) == manifest, + "pending snapshot differs from candidate") + return promote(root, snapshot, manifest) + if revision == manifest["revision"]: + check_public(root, manifest) + return {"status": "unchanged", "revision": revision, + "target": TARGET.copy()} + require(expected == revision, + "expected revision differs from current publication (compare-and-swap failed)") + if current is None: + channel = public_channel(root) + plain_path(channel, directory=True, missing=True) + require(not channel.exists() or not any(channel.iterdir()), + "refusing to adopt an unmanaged public Fedora repository") + check_public(root, manifest, immutable_only=True) + snapshot = save_snapshot(root, repository, manifest) + make_directory(root / "state", mode=0o700) + atomic_write(root / "state" / "pending.json", data=canonical({ + "revision": manifest["revision"], "previousRevision": revision, + }) + b"\n", mode=0o600) + _checkpoint("journal") + return promote(root, snapshot, manifest) + + +def recover_at(root, fingerprint, revision): + with locked(root, create=True): + pending = state(root, "pending") + require(pending is not None and pending["revision"] == revision, + "pending publication changed; fetch and verify it again before recovery") + current = state(root, "current") + require((current["revision"] if current else "empty") + in (pending.get("previousRevision"), revision), + "current revision conflicts with pending journal") + snapshot = root / "snapshots" / revision + return promote(root, snapshot, inventory(snapshot, fingerprint)) + + +@contextlib.contextmanager +def selected_snapshot(root, fingerprint, revision=None, pending_only=False): + with locked(root): + pending = state(root, "pending") + if pending_only: + if not pending: + raise EmptyRepository("repository has no pending publication") + revision = pending["revision"] + else: + require(pending is None, + "interrupted publication pending; recover before fetching snapshots") + if revision is None: + current = state(root, "current") + if not current: + raise EmptyRepository("repository has no committed snapshot") + revision = current["revision"] + snapshot = root / "snapshots" / revision + manifest = inventory(snapshot, fingerprint) + require(manifest["revision"] == revision, + "snapshot does not match selected revision") + yield snapshot, manifest + + +def write_archive(repository, output): + with tarfile.open(fileobj=output, mode="w|") as archive: + for relative in sorted(tree_files(repository)): + archive.add(repository / relative, arcname=relative, recursive=False) + + +def read_archive(source, output): + seen = set() + with tarfile.open(fileobj=source, mode="r|*") as archive: + for member in archive: + safe_path(member.name) + require(member.name == MANIFEST or classify(member.name), + "unexpected archive path") + require(member.isfile() and not member.issym() and not member.islnk() + and member.name not in seen, + "archive contains a link, special file, or duplicate") + seen.add(member.name) + target = output / member.name + make_directory(target.parent) + with archive.extractfile(member) as incoming, target.open("xb") as destination: + shutil.copyfileobj(incoming, destination, 1024 * 1024) + + +def ssh_command(args, request): + require(re.fullmatch(r"[A-Za-z0-9_][A-Za-z0-9_.-]*@[A-Za-z0-9][A-Za-z0-9_.-]*", args.ssh), + "--ssh must be USER@HOST using a hostname or IPv4 address") + require(args.known_hosts is not None and args.identity_file is not None, + "remote operations require --known-hosts and --identity-file") + known_hosts = plain_path(args.known_hosts) + identity = plain_path(args.identity_file) + require(known_hosts.is_file() and known_hosts.stat().st_nlink == 1, + "--known-hosts must name a regular file") + require(identity.is_file() and identity.stat().st_nlink == 1, + "--identity-file must name a regular file") + command = ["ssh", "-F", "/dev/null", "-T", "-o", "BatchMode=yes", + "-o", "StrictHostKeyChecking=yes", + "-o", "PasswordAuthentication=no", "-o", "KbdInteractiveAuthentication=no", + "-o", "ForwardAgent=no", "-o", "ClearAllForwardings=yes", + "-o", "ConnectTimeout=15", "-o", f"UserKnownHostsFile={known_hosts}", + "-o", "GlobalKnownHostsFile=/dev/null", "-i", str(identity), + "-o", "IdentitiesOnly=yes"] + if args.ssh_port: + command += ["-p", str(args.ssh_port)] + encoded = base64.urlsafe_b64encode(canonical(request)).decode("ascii") + source = Path(__file__).read_text(encoding="utf-8") + remote = "python3 -c " + shlex.quote(source) + " _serve " + shlex.quote(encoded) + return command + ["--", args.ssh, remote] + + +def remote_call(args, request, repository=None, output=None): + command = ssh_command(args, request) + with tempfile.TemporaryFile() as incoming, tempfile.TemporaryFile() as outgoing: + if repository: + write_archive(repository, incoming) + incoming.seek(0) + result = subprocess.run(command, stdin=incoming, stdout=outgoing, + stderr=subprocess.PIPE, check=False) + if result.returncode == 3: + raise EmptyRepository("remote repository has no selected snapshot") + if result.returncode == 1: + try: + failure = json.loads(result.stderr) + if failure.get("status") == "error" and isinstance(failure.get("message"), str): + raise PublicationError(failure["message"]) + except (ValueError, AttributeError): + pass + require(result.returncode == 0, + "SSH repository operation failed; check pinned host key, identity, connectivity, remote Python, and publisher state") + outgoing.seek(0) + if output: + read_archive(outgoing, output) + return None + try: + return json.load(outgoing) + except (ValueError, UnicodeError) as error: + raise PublicationError("SSH repository response is not valid JSON") from error + + +def serve(request): + root = root_path(request["root"]) + fingerprint = request["fingerprint"] + require(FINGERPRINT.fullmatch(fingerprint), "invalid signing fingerprint") + action = request["action"] + if action == "publish": + require(request["expected"] == "empty" or REVISION.fullmatch(request["expected"]), + "invalid expected revision") + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-upload-") as directory: + repository = Path(directory) + read_archive(sys.stdin.buffer, repository) + return publish_at(root, repository, fingerprint, request["expected"]) + if action == "recover": + require(REVISION.fullmatch(request["revision"]), "invalid recovery revision") + return recover_at(root, fingerprint, request["revision"]) + require(action in ("fetch", "fetch-pending"), "unknown remote operation") + revision = request.get("revision") + require(revision is None or REVISION.fullmatch(revision), "invalid fetch revision") + with selected_snapshot(root, fingerprint, revision, + action == "fetch-pending") as (snapshot, _): + write_archive(snapshot, sys.stdout.buffer) + return None + + +def fetch_into(args, output, pending_only=False): + if args.ssh: + remote_call(args, { + "action": "fetch-pending" if pending_only else "fetch", + "root": args.root, + "fingerprint": args.fingerprint, + "revision": getattr(args, "revision", None), + }, output=output) + else: + with selected_snapshot(root_path(args.root), args.fingerprint, + getattr(args, "revision", None), + pending_only) as (snapshot, _): + shutil.copytree(snapshot, output, dirs_exist_ok=True) + historical = not pending_only or getattr(args, "allow_expired", False) + return verify_signed(output, args.public_key, args.fingerprint, historical=historical) + + +def parser(): + result = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + actions = result.add_subparsers(dest="action", required=True) + for name in ("publish", "fetch", "recover"): + action = actions.add_parser(name) + action.add_argument("--root", required=True, + help="absolute origin root; HTTP serves ROOT/public") + action.add_argument("--public-key", required=True, type=Path) + action.add_argument("--fingerprint", required=True) + action.add_argument("--ssh", metavar="USER@HOST", + help="omit for a local POSIX origin") + action.add_argument("--ssh-port", type=int) + action.add_argument("--identity-file", type=Path, + help="required SSH private identity for remote operations") + action.add_argument("--known-hosts", type=Path, + help="required pinned known_hosts for remote operations") + if name == "publish": + action.add_argument("--repository", type=Path, required=True) + action.add_argument("--expected-revision", required=True, + help="observed revision, or literal empty for first publication") + action.add_argument("--dry-run", action="store_true", + help="verify locally; perform no origin access or upload") + elif name == "fetch": + action.add_argument("--output", type=Path, required=True, + help="new destination directory (must not exist)") + action.add_argument("--revision", + help="retained snapshot revision; defaults to committed current") + else: + action.add_argument("--dry-run", action="store_true", + help="fetch and verify pending snapshot without promotion") + action.add_argument("--allow-expired", action="store_true", + help="finish an expired signed journal, then immediately publish fresh metadata") + return result + + +def run(args): + if args.ssh: + requested_root = Path(args.root) + require(requested_root.is_absolute() and args.root != "/" + and ".." not in requested_root.parts + and requested_root.as_posix() == args.root, + "--root must be an absolute normalized non-root path") + else: + root_path(args.root) + require(FINGERPRINT.fullmatch(args.fingerprint), + "--fingerprint must be a full uppercase fingerprint") + require(args.ssh_port is None or 1 <= args.ssh_port <= 65535, + "invalid SSH port") + require(args.ssh or (args.ssh_port is None and args.known_hosts is None + and args.identity_file is None), + "SSH options require --ssh") + if args.ssh: + require(args.known_hosts is not None and args.identity_file is not None, + "remote operations require --known-hosts and --identity-file") + args.public_key = plain_path(args.public_key) + require(args.public_key.is_file() and args.public_key.stat().st_nlink == 1, + "--public-key must name a regular file") + if args.action == "publish": + require(args.expected_revision == "empty" or REVISION.fullmatch(args.expected_revision), + "invalid expected revision") + manifest = verify_signed(args.repository, args.public_key, args.fingerprint) + if args.dry_run: + return {"status": "validated", "revision": manifest["revision"], + "originChecked": False} + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-candidate-") as directory: + candidate = Path(directory) / "repository" + shutil.copytree(args.repository, candidate, symlinks=True) + require(verify_signed(candidate, args.public_key, args.fingerprint) == manifest, + "candidate changed during verification") + if args.ssh: + return remote_call(args, { + "action": "publish", "root": args.root, + "fingerprint": args.fingerprint, + "expected": args.expected_revision, + }, repository=candidate) + return publish_at(root_path(args.root), candidate, args.fingerprint, + args.expected_revision) + if args.action == "fetch": + require(args.revision is None or REVISION.fullmatch(args.revision), + "invalid selected revision") + output = plain_path(args.output, directory=True, missing=True) + require(not output.exists(), "--output must not exist") + require(output.parent.is_dir(), "--output parent must already exist") + with tempfile.TemporaryDirectory(prefix=".loopwire-rpm-fetch-", + dir=output.parent) as directory: + fetched = Path(directory) / "repository" + fetched.mkdir() + manifest = fetch_into(args, fetched) + os.rename(fetched, output) + fsync_directory(output.parent) + return {"status": "fetched", "revision": manifest["revision"]} + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-recovery-") as directory: + manifest = fetch_into(args, Path(directory), pending_only=True) + needs_refresh = manifest["validUntil"] <= int(time.time()) + if args.dry_run: + return {"status": "recovery-validated", "revision": manifest["revision"], + "requiresRefresh": needs_refresh} + if args.ssh: + result = remote_call(args, { + "action": "recover", "root": args.root, + "fingerprint": args.fingerprint, "revision": manifest["revision"], + }) + else: + result = recover_at(root_path(args.root), args.fingerprint, + manifest["revision"]) + result["requiresRefresh"] = needs_refresh + if needs_refresh: + result["nextAction"] = ( + "Immediately fetch, rebuild, sign, and publish fresh metadata; " + "the project verifier rejects the expired snapshot. DNF signature checks do not enforce this project deadline." + ) + return result + + +def main(): + try: + if len(sys.argv) == 3 and sys.argv[1] == "_serve": + result = serve(json.loads(base64.urlsafe_b64decode(sys.argv[2]))) + else: + result = run(parser().parse_args()) + if result is not None: + print(json.dumps(result, sort_keys=True)) + return 0 + except EmptyRepository: + print(json.dumps({"status": "empty", "revision": None})) + return 3 + except (PublicationError, OSError, ValueError, KeyError, TypeError, + tarfile.TarError) as error: + message = (str(error) if isinstance(error, PublicationError) + else "repository operation failed; check filesystem and inputs") + print(json.dumps({"status": "error", "message": message}), file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/rpm-repository.py b/scripts/rpm-repository.py new file mode 100644 index 0000000..3ef1fa9 --- /dev/null +++ b/scripts/rpm-repository.py @@ -0,0 +1,776 @@ +#!/usr/bin/env python3 +"""Build and verify immutable signed Loopwire Fedora repository candidates. + +Requires Python's standard library, createrepo_c, rpm, rpmkeys, rpmsign, +gpg, gpgv, and openssl. The publisher retains immutable package and metadata +objects globally, writes repomd.xml.asc first, and atomically replaces +repomd.xml as the public commit point. This module never publishes files or +changes host repository configuration. + +The explicit --date fixes repository timestamps and GnuPG signature creation +times. Byte-for-byte repeatability still requires the pinned Fedora tool image, +the same key material, and a deterministic OpenPGP algorithm; a prior package +with identical source bytes is reused rather than signed again. +""" + +import argparse +import gzip +import hashlib +import json +import os +from pathlib import Path, PurePosixPath +import re +import shlex +import shutil +import stat +import subprocess +import sys +import tempfile +import time +import xml.etree.ElementTree as ET + + +SCHEMA = "loopwire.rpm-repository.v1" +MANIFEST = "repository-manifest.json" +TARGET = {"distribution": "fedora", "release": "44", "architecture": "x86_64"} +VERSION = r"(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)(?:\+[0-9A-Za-z]+(?:\.[0-9A-Za-z]+)*)?" +HASH = r"[0-9a-f]{64}" +FINGERPRINT = r"(?:[0-9A-F]{40}|[0-9A-F]{64})" +PACKAGE_FIELDS = { + "name", "version", "release", "architecture", "path", + "sourceReleaseSha256", "distributedSha256", "size", +} +FILE_FIELDS = {"path", "sha256", "size", "kind"} +ROOT = Path(__file__).resolve().parent.parent +REPO_NS = "http://linux.duke.edu/metadata/repo" +COMMON_NS = "http://linux.duke.edu/metadata/common" + + +class RepositoryError(Exception): + """An invalid or unauthenticated RPM repository candidate.""" + + +def require(condition, message): + if not condition: + raise RepositoryError(message) + + +def run(*args, cwd=None, env=None): + try: + result = subprocess.run( + [str(value) for value in args], cwd=cwd, env=env, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, + ) + except FileNotFoundError as error: + raise RepositoryError(f"required tool is missing: {args[0]}") from error + require( + result.returncode == 0, + f"{args[0]} failed: {result.stderr.decode('utf-8', errors='replace').strip()}", + ) + return result.stdout + + +def canonical(value): + return json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8") + + +def sha256(path): + with path.open("rb") as stream: + return hashlib.file_digest(stream, "sha256").hexdigest() + + +def fingerprint(value): + require( + isinstance(value, str) and re.fullmatch(FINGERPRINT, value) is not None, + "fingerprint must be the complete uppercase OpenPGP fingerprint", + ) + return value + + +def hash_value(value, label): + require(isinstance(value, str) and re.fullmatch(HASH, value) is not None, f"invalid {label}") + return value + + +def integer(value, label, minimum=0): + require(type(value) is int and value >= minimum, f"invalid {label}") + return value + + +def exact_keys(value, expected, label): + require(isinstance(value, dict) and set(value) == set(expected), f"invalid {label} fields") + + +def object_pairs(pairs): + result = {} + for key, value in pairs: + require(key not in result, f"duplicate JSON field: {key}") + result[key] = value + return result + + +def read_json(path): + return json.loads(path.read_text(encoding="utf-8"), object_pairs_hook=object_pairs) + + +def safe_path(value): + require( + isinstance(value, str) and value + and not any(ord(character) < 33 or ord(character) > 126 for character in value), + "inventory paths must contain printable ASCII without whitespace", + ) + path = PurePosixPath(value) + require( + not path.is_absolute() and str(path) == value and ".." not in path.parts and "\\" not in value, + f"unsafe inventory path: {value}", + ) + return value + + +def classify_path(value): + safe_path(value) + if re.fullmatch(rf"keys/{FINGERPRINT}\.asc", value): + return "immutable" + if re.fullmatch(rf"packages/loopwire-{VERSION}-1\.fc44\.x86_64\.rpm", value): + return "immutable" + if re.fullmatch(rf"repodata/{HASH}-(?:primary|filelists|other)\.xml\.gz", value): + return "immutable" + if value in ("repodata/repomd.xml", "repodata/repomd.xml.asc"): + return "metadata" + raise RepositoryError(f"path is outside the Fedora repository contract: {value}") + + +def regular_file(path): + information = path.lstat() + require( + stat.S_ISREG(information.st_mode) and information.st_nlink == 1, + f"only regular files without symlinks or hardlinks are allowed: {path}", + ) + return information + + +def real_directory(path, label): + require(path.is_dir() and not path.is_symlink(), f"{label} must be a real directory") + return path + + +def tree_files(root): + real_directory(root, "repository") + files = set() + for directory, directories, names in os.walk(root, followlinks=False): + for name in directories: + path = Path(directory) / name + require(not path.is_symlink(), f"symlink directory is forbidden: {path}") + for name in names: + path = Path(directory) / name + regular_file(path) + files.add(path.relative_to(root).as_posix()) + return files + + +def key_fingerprint(key, home): + data = run( + "gpg", "--batch", "--homedir", home, "--with-colons", + "--import-options", "show-only", "--import", key, + ).decode("utf-8") + primary = [] + want_fingerprint = False + for line in data.splitlines(): + fields = line.split(":") + if fields[0] == "pub": + want_fingerprint = True + elif fields[0] == "fpr" and want_fingerprint: + primary.append(fingerprint(fields[9])) + want_fingerprint = False + elif fields[0] == "sub": + want_fingerprint = False + require(len(primary) == 1, "public key must contain exactly one primary OpenPGP key") + return primary[0] + + +def manifest_revision(manifest): + unsigned = {key: value for key, value in manifest.items() if key != "revision"} + return hashlib.sha256(canonical(unsigned)).hexdigest() + + +def rpm_identity(path): + regular_file(path) + fields = run( + "rpm", "-qp", "--queryformat", + "%{NAME}\n%{VERSION}\n%{RELEASE}\n%{ARCH}\n%{EPOCHNUM}\n", path, + ).decode("utf-8").splitlines() + require(len(fields) == 5, "RPM identity query returned unexpected fields") + name, version, release, architecture, epoch = fields + require(name == "loopwire", "repository only accepts RPM Name: loopwire") + require(re.fullmatch(VERSION, version) is not None, f"unsupported RPM version: {version}") + require(release == "1.fc44", f"repository only accepts RPM Release: 1.fc44, got {release}") + require(architecture == "x86_64", f"repository only accepts RPM Architecture: x86_64, got {architecture}") + require(epoch in ("0", "(none)"), "repository RPM must not set a nonzero epoch") + return name, version, release, architecture + + +def rpm_has_signature(path): + output = run( + "rpm", "-qp", "--queryformat", + "%{OPENPGP:pgpsig}\n%{SIGPGP:pgpsig}\n%{SIGGPG:pgpsig}\n" + "%{RSAHEADER:pgpsig}\n%{DSAHEADER:pgpsig}\n", path, + ).decode("utf-8", errors="replace") + return any(value.strip() not in ("", "(none)") for value in output.splitlines()) + + +def verify_rpm_digest(path): + run("rpmkeys", "--nosignature", "--checksig", path) + + +def verify_rpm_signature(path, public_key): + require(rpm_has_signature(path), f"RPM has no OpenPGP package signature: {path}") + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-keyring-") as temporary: + database = Path(temporary) / "rpmdb" + database.mkdir() + run("rpmkeys", "--dbpath", database, "--import", public_key) + run("rpmkeys", "--dbpath", database, "--checksig", path) + + +def package_info(root, path, source_hash, public_key): + classify_path(path) + require(path.startswith("packages/"), "RPM path is outside packages/") + package = root / path + verify_rpm_digest(package) + verify_rpm_signature(package, public_key) + name, version, release, architecture = rpm_identity(package) + require( + Path(path).name == f"{name}-{version}-{release}.{architecture}.rpm", + "RPM filename does not match its NEVRA identity", + ) + return { + "name": name, + "version": version, + "release": release, + "architecture": architecture, + "path": path, + "sourceReleaseSha256": hash_value(source_hash, "source release SHA256"), + "distributedSha256": sha256(package), + "size": package.stat().st_size, + } + + +def verify_detached_signature(data, signature, keyring, home, expected): + armored = signature.read_text(encoding="ascii") + require( + armored.startswith("-----BEGIN PGP SIGNATURE-----\n") + and armored.rstrip().endswith("-----END PGP SIGNATURE-----"), + "repomd.xml signature must be detached ASCII armor", + ) + status = run( + "gpgv", "--homedir", home, "--keyring", keyring, + "--status-fd", "1", signature, data, + ).decode("utf-8") + valid = [line.split() for line in status.splitlines() if line.startswith("[GNUPG:] VALIDSIG ")] + require( + len(valid) == 1 and valid[0][-1] == expected, + "repomd.xml signature does not match the pinned primary fingerprint", + ) + require( + not any(f"[GNUPG:] {flag}" in status for flag in ( + "EXPKEYSIG", "EXPSIG", "REVKEYSIG", "KEYREVOKED", "KEYEXPIRED", "SIGEXPIRED", + )), + "repomd.xml uses an expired or revoked signing identity", + ) + + +def load_inventory(root): + actual = tree_files(root) + require(MANIFEST in actual, "missing repository-manifest.json") + manifest = read_json(root / MANIFEST) + exact_keys( + manifest, + {"schema", "schemaVersion", "revision", "signingFingerprint", "createdAt", + "validUntil", "target", "packages", "files"}, + "repository manifest", + ) + require(manifest["schema"] == SCHEMA, "unsupported repository manifest schema") + require(type(manifest["schemaVersion"]) is int and manifest["schemaVersion"] == 1, + "unsupported repository manifest schema version") + fingerprint(manifest["signingFingerprint"]) + integer(manifest["createdAt"], "createdAt") + integer(manifest["validUntil"], "validUntil") + require(manifest["validUntil"] > manifest["createdAt"], "metadata expiry must follow creation") + exact_keys(manifest["target"], TARGET, "repository target") + require(manifest["target"] == TARGET, "repository target must be Fedora 44 x86_64") + require(manifest["revision"] == manifest_revision(manifest), "manifest revision digest mismatch") + require(isinstance(manifest["packages"], list) and manifest["packages"], "packages must be a nonempty array") + require(isinstance(manifest["files"], list), "files must be an array") + inventory = {} + for entry in manifest["files"]: + exact_keys(entry, FILE_FIELDS, "inventory entry") + path = safe_path(entry["path"]) + require(path not in inventory, f"duplicate inventory path: {path}") + require(entry["kind"] == classify_path(path), f"incorrect file classification: {path}") + integer(entry["size"], "file size") + hash_value(entry["sha256"], f"SHA256 for {path}") + require(path in actual, f"missing inventory file: {path}") + file = root / path + require( + file.stat().st_size == entry["size"] and sha256(file) == entry["sha256"], + f"inventory checksum mismatch: {path}", + ) + if path.startswith("repodata/") and entry["kind"] == "immutable": + require(Path(path).name.startswith(entry["sha256"] + "-"), + f"repodata content filename/checksum mismatch: {path}") + inventory[path] = entry + require(actual == set(inventory) | {MANIFEST}, "repository has unlisted files") + return manifest, inventory + + +def xml(root, name): + values = root.findall(f"{{{REPO_NS}}}{name}") + require(len(values) == 1, f"repomd.xml must contain exactly one {name}") + return values[0] + + +def parse_repomd(root, manifest, inventory): + path = root / "repodata/repomd.xml" + raw = path.read_bytes() + require(b" now, "repository metadata has expired; refresh and re-sign it") + regular_file(public_key) + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-verify-") as temporary: + home = Path(temporary) + require(key_fingerprint(public_key, home) == expected, "trusted public key differs from operator pin") + trusted_ring = home / "trusted.gpg" + run("gpg", "--batch", "--homedir", home, "--dearmor", "--output", trusted_ring, public_key) + key_path = f"keys/{expected}.asc" + require(key_path in inventory, "missing fingerprint-addressed repository key") + exported_key = root / key_path + require(key_fingerprint(exported_key, home) == expected, "exported key fingerprint/path mismatch") + exported_ring = home / "exported.gpg" + run("gpg", "--batch", "--homedir", home, "--dearmor", "--output", exported_ring, exported_key) + repomd = root / "repodata/repomd.xml" + signature = root / "repodata/repomd.xml.asc" + require("repodata/repomd.xml" in inventory and "repodata/repomd.xml.asc" in inventory, + "repository is missing signed repomd metadata") + verify_detached_signature(repomd, signature, trusted_ring, home, expected) + verify_detached_signature(repomd, signature, exported_ring, home, expected) + metadata, source_tags = parse_repomd(root, manifest, inventory) + indexed = primary_packages(metadata["primary"]) + require(len(indexed) == len(manifest["packages"]), "manifest/primary package count mismatch") + packages = {} + versions = set() + for entry in manifest["packages"]: + exact_keys(entry, PACKAGE_FIELDS, "package inventory entry") + path = safe_path(entry["path"]) + require(path not in packages and path in inventory and path in source_tags, + "duplicate or missing package inventory path") + require(inventory[path]["kind"] == "immutable", "RPM package must be immutable") + info = package_info(root, path, source_tags[path], public_key) + verify_rpm_signature(root / path, exported_key) + require(entry == info, f"RPM identity/hash differs from package inventory: {path}") + require(inventory[path]["sha256"] == info["distributedSha256"] + and inventory[path]["size"] == info["size"], f"file inventory differs for RPM: {path}") + require(info["version"] not in versions, "duplicate RPM version in repository") + versions.add(info["version"]) + packages[path] = info + require(set(source_tags) == set(packages), "signed source provenance does not match package inventory") + require(set(indexed) == set(packages), "primary metadata package set differs from manifest") + for path, package in packages.items(): + record = indexed[path] + require( + record == { + "name": package["name"], "version": package["version"], + "release": package["release"], "architecture": package["architecture"], + "epoch": "0", "sha256": package["distributedSha256"], "size": package["size"], + }, + f"signed primary metadata differs from RPM package: {path}", + ) + return manifest + + +def export_signer(args, directory): + expected = fingerprint(args.signing_key) + home = real_directory(args.gnupg_home, "GnuPG home") + if args.passphrase_file: + information = regular_file(args.passphrase_file) + require(information.st_mode & 0o077 == 0, "passphrase file must not be group/world accessible") + gpg = ["gpg", "--batch", "--no-tty", "--homedir", str(home)] + if args.passphrase_file: + gpg.extend(["--pinentry-mode", "loopback", "--passphrase-file", str(args.passphrase_file)]) + run(*gpg, "--list-secret-keys", expected) + key = directory / "signer.asc" + key.write_bytes(run(*gpg, "--export-options", "export-minimal", "--armor", "--export", expected)) + require(key.stat().st_size > 0, "signing public key is missing") + require(key_fingerprint(key, directory) == expected, "signing key must identify one primary key") + return gpg, key, expected + + +def signing_wrapper(directory, gpg_home, passphrase_file, date): + executable = shutil.which("gpg") + require(executable is not None, "required tool is missing: gpg") + arguments = [ + executable, "--batch", "--no-tty", "--pinentry-mode", "loopback", + "--homedir", str(gpg_home), "--faked-system-time", f"{date}!", + ] + if passphrase_file: + arguments.extend(["--passphrase-file", str(passphrase_file)]) + wrapper = directory / "rpm-gpg-wrapper" + wrapper.write_text("#!/bin/sh\nexec " + " ".join(shlex.quote(value) for value in arguments) + + ' "$@"\n', encoding="utf-8") + wrapper.chmod(0o700) + return wrapper + + +def sign_rpm(package, expected, gpg_home, passphrase_file, date, directory, public_key): + wrapper = signing_wrapper(directory, gpg_home, passphrase_file, date) + run( + "rpmsign", "--resign", + "--define", f"_openpgp_sign_id {expected}", + "--define", f"_gpg_path {gpg_home}", + "--define", f"__gpg {wrapper}", + package, + ) + verify_rpm_signature(package, public_key) + + +def rpm_version_compare(left, right): + require(re.fullmatch(VERSION, left) and re.fullmatch(VERSION, right), "invalid RPM version comparison") + expression = f"%{{lua:print(rpm.vercmp('{left}', '{right}'))}}" + value = run("rpm", "--eval", expression).decode("ascii").strip() + require(value in ("-1", "0", "1"), "rpm version comparison returned an invalid result") + return int(value) + + +def previous_repository(path, key, expected): + manifest, _inventory = load_inventory(path) + return verify_repository(path, key, expected, manifest["createdAt"]) + + +def copy_immutable(source, target, manifest): + for entry in manifest["files"]: + if entry["kind"] != "immutable": + continue + destination = target / entry["path"] + destination.parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(source / entry["path"], destination) + require( + sha256(destination) == entry["sha256"] and destination.stat().st_size == entry["size"], + "previous immutable file changed while copying the snapshot", + ) + + +def signed_release_package(args, working, packages, expected, key, staging, date): + require( + re.fullmatch(VERSION, args.version) is not None, + "Fedora publication requires X.Y.Z with optional +build metadata", + ) + release = real_directory(args.release_dir, "release directory") + checksums = release / "SHA256SUMS" + signature = release / "SHA256SUMS.sig" + regular_file(checksums) + regular_file(signature) + regular_file(args.release_public_key) + run( + "openssl", "dgst", "-sha256", "-verify", args.release_public_key, + "-signature", signature, checksums, + ) + signed = {} + for line in checksums.read_text(encoding="utf-8").splitlines(): + match = re.fullmatch(rf"({HASH}) [ *]([^/\\\s]+)", line) + require(match is not None, "invalid signed release checksum line") + checksum, name = match.groups() + require(name not in signed and name not in (".", ".."), "duplicate or invalid release checksum asset") + signed[name] = checksum + filename = f"loopwire-{args.version}-1.fc44.x86_64.rpm" + actual_rpms = {path.name for path in release.glob("*.rpm")} + known_release_rpms = {filename, f"loopwire-{args.version}-1.x86_64.rpm"} + require(filename in actual_rpms and actual_rpms <= known_release_rpms, + "release must contain the Fedora 44 RPM and no unknown RPM artifacts") + source = release / filename + regular_file(source) + source_hash = sha256(source) + require(filename in signed and signed[filename] == source_hash, + f"signed release checksum differs for {filename}") + verify_rpm_digest(source) + name, version, rpm_release, architecture = rpm_identity(source) + require(version == args.version, "RPM version differs from requested repository version") + path = f"packages/{filename}" + for previous in packages: + require(rpm_version_compare(version, previous["version"]) >= 0, + "new RPM version is lower than a published version; use explicit rollback") + matches = [entry for entry in packages if entry["version"] == version] + require(len(matches) <= 1, "previous repository has duplicate RPM versions") + if matches: + previous = matches[0] + require(previous["path"] == path and previous["sourceReleaseSha256"] == source_hash, + "same RPM version has different source release bytes") + require((working / path).is_file(), "retained RPM package is missing") + return + destination = working / path + destination.parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(source, destination) + require(sha256(destination) == source_hash, "release RPM changed while copying to repository staging") + sign_rpm(destination, expected, args.gnupg_home, args.passphrase_file, date, staging, key) + packages.append(package_info(working, path, source_hash, key)) + packages.sort(key=lambda entry: entry["path"]) + + +def generate_metadata(working, packages, date, valid_until, gpg, expected, staging): + for package in packages: + path = working / package["path"] + regular_file(path) + os.utime(path, (date, date), follow_symlinks=False) + package_list = staging / "packages.list" + package_list.write_text("".join(entry["path"] + "\n" for entry in packages), encoding="utf-8") + generated = staging / "generated" + generated.mkdir() + repo_tags = [f"loopwire-valid-until:{valid_until}"] + repo_tags.extend( + f"loopwire-source-sha256:{entry['path']}:{entry['sourceReleaseSha256']}" + for entry in packages + ) + command = [ + "createrepo_c", "--quiet", "--no-database", "--checksum", "sha256", + "--repomd-checksum", "sha256", "--general-compress-type", "gz", + "--unique-md-filenames", "--workers", "1", "--changelog-limit", "0", + "--revision", str(date), "--set-timestamp-to-revision", + "--distro", "cpe:/o:fedoraproject:fedora:44,Fedora 44", + "--content", SCHEMA, "--pkglist", package_list, "--outputdir", generated, + ] + for tag in repo_tags: + command.extend(["--repo", tag]) + command.append(working) + run(*command) + generated_repodata = generated / "repodata" + real_directory(generated_repodata, "generated repodata") + destination = working / "repodata" + destination.mkdir(parents=True, exist_ok=True) + for path in sorted(generated_repodata.iterdir()): + regular_file(path) + relative = f"repodata/{path.name}" + classify_path(relative) + target = destination / path.name + if target.exists() and path.name != "repomd.xml": + require(sha256(target) == sha256(path), f"immutable repodata collision: {path.name}") + else: + shutil.copyfile(path, target) + repomd = destination / "repomd.xml" + signature = destination / "repomd.xml.asc" + run( + *gpg, "--yes", "--faked-system-time", f"{date}!", "--digest-algo", "SHA256", + "--local-user", expected, "--armor", "--detach-sign", "--output", signature, repomd, + ) + + +def write_candidate(args, rollback=False): + output = args.output + require( + not output.exists() and not output.is_symlink(), + "output must not already exist; reuse a completed candidate for publication retries", + ) + date = int(time.time()) if args.date is None else integer(args.date, "date") + require(1 <= args.valid_for_days <= 90, "valid-for-days must be between 1 and 90") + valid_until = date + args.valid_for_days * 86400 + output.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix=f".{output.name}-", dir=output.parent) as temporary: + staging = Path(temporary) + working = staging / "repository" + working.mkdir() + gpg, key, expected = export_signer(args, staging) + previous_path = args.repository if rollback else args.previous + previous = previous_repository(previous_path, key, expected) if previous_path else None + if previous: + require(date >= previous["createdAt"], "new metadata date must not precede the previous revision") + copy_immutable(previous_path, working, previous) + packages = json.loads(json.dumps(previous["packages"])) + else: + packages = [] + if not rollback: + signed_release_package(args, working, packages, expected, key, staging, date) + require(packages, "repository package set must not be empty") + exported = working / f"keys/{expected}.asc" + exported.parent.mkdir(parents=True, exist_ok=True) + if exported.exists(): + require(exported.read_bytes() == key.read_bytes(), + "fingerprint-addressed key bytes changed; rotate trust before changing exported packets") + else: + shutil.copyfile(key, exported) + generate_metadata(working, packages, date, valid_until, gpg, expected, staging) + files = [ + { + "path": path, "sha256": sha256(working / path), + "size": (working / path).stat().st_size, "kind": classify_path(path), + } + for path in sorted(tree_files(working)) + ] + manifest = { + "schema": SCHEMA, + "schemaVersion": 1, + "signingFingerprint": expected, + "createdAt": date, + "validUntil": valid_until, + "target": TARGET, + "packages": packages, + "files": files, + } + manifest["revision"] = manifest_revision(manifest) + (working / MANIFEST).write_text(json.dumps(manifest, indent=2, sort_keys=True) + "\n", encoding="utf-8") + verify_repository(working, key, expected, date) + working.rename(output) + return manifest + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + commands = parser.add_subparsers(dest="command", required=True) + build = commands.add_parser("build", help="generate a Fedora candidate from signed native release assets") + build.add_argument("--release-dir", type=Path, required=True) + build.add_argument("--version", required=True) + build.add_argument( + "--release-public-key", type=Path, default=ROOT / "packaging/release-signing-public.pem", + help="trusted release checksum PEM (override for fixture keys)", + ) + build.add_argument("--previous", type=Path, help="verified previous snapshot whose immutable history is retained") + rollback = commands.add_parser("rollback", help="freshly sign a retained snapshot's previous package set") + rollback.add_argument("--repository", type=Path, required=True) + for command in (build, rollback): + command.add_argument("--output", type=Path, required=True) + command.add_argument("--signing-key", required=True, help="uppercase primary OpenPGP fingerprint") + command.add_argument("--gnupg-home", type=Path, required=True, + help="isolated GnuPG home containing the signing identity") + command.add_argument("--passphrase-file", type=Path, + help="protected file containing the signing-key passphrase; never pass the secret directly") + command.add_argument("--date", type=int, help="metadata/signature creation time as Unix epoch seconds") + command.add_argument("--valid-for-days", type=int, default=30) + verify = commands.add_parser("verify", help="verify the complete pinned Fedora repository trust chain") + verify.add_argument("--repository", type=Path, required=True) + verify.add_argument("--public-key", type=Path, required=True, + help="independently trusted ASCII-armored repository key") + verify.add_argument("--fingerprint", required=True, help="expected uppercase primary fingerprint") + verify.add_argument("--now", type=int, help="explicit verification time for fixtures or historical snapshots") + args = parser.parse_args() + if args.command == "verify": + manifest = verify_repository(args.repository, args.public_key, args.fingerprint, args.now) + else: + manifest = write_candidate(args, rollback=args.command == "rollback") + print(json.dumps({ + key: manifest[key] for key in + ("revision", "signingFingerprint", "createdAt", "validUntil", "target", "packages") + }, sort_keys=True)) + + +if __name__ == "__main__": + try: + main() + except (RepositoryError, ET.ParseError, OSError, ValueError, KeyError, TypeError, + EOFError, OverflowError, UnicodeError) as error: + print(f"rpm-repository: {error}", file=sys.stderr) + sys.exit(1) diff --git a/scripts/setup-fedora-repository.sh b/scripts/setup-fedora-repository.sh new file mode 100755 index 0000000..2009e52 --- /dev/null +++ b/scripts/setup-fedora-repository.sh @@ -0,0 +1,164 @@ +#!/usr/bin/env bash +set -euo pipefail + +base_url="" +fingerprint="" +install_root="/" +remove="false" +dry_run="false" + +fail() { printf 'setup-fedora-repository: %s\n' "$*" >&2; exit 1; } +usage() { + cat <<'USAGE' +Configure Loopwire's signed project repository on Fedora 44 x86_64. + +Usage: + sudo bash setup-fedora-repository.sh --base-url HTTPS_URL --fingerprint OPENPGP_FINGERPRINT + sudo bash setup-fedora-repository.sh --remove + bash setup-fedora-repository.sh --base-url HTTPS_URL --fingerprint FINGERPRINT --dry-run + +Options: + --root DIR Configure an offline filesystem tree instead of / (including its etc/os-release). + +Obtain the URL and fingerprint from the verified Loopwire channel documentation. +Requires curl, GnuPG, Python 3, and RPM. Existing unrelated DNF repositories are preserved. +This writes only the repository and key files. Run dnf makecache and dnf install yourself afterward. +USAGE +} +while [ "$#" -gt 0 ]; do + case "$1" in + --base-url) base_url="${2:?missing --base-url value}"; shift 2 ;; + --fingerprint) fingerprint="${2:?missing --fingerprint value}"; shift 2 ;; + --root) install_root="${2:?missing --root value}"; shift 2 ;; + --remove) remove="true"; shift ;; + --dry-run) dry_run="true"; shift ;; + -h|--help) usage; exit 0 ;; + *) fail "unknown option: $1" ;; + esac +done + +install_root="$(realpath -e "$install_root")" +[ -d "$install_root" ] || fail "root must be an existing directory" +repo_file="${install_root%/}/etc/yum.repos.d/loopwire.repo" +key_directory="${install_root%/}/etc/pki/rpm-gpg" +python3 - "$install_root" "$repo_file" "$key_directory" <<'PY' +import sys +from pathlib import Path +root = Path(sys.argv[1]) +for name in sys.argv[2:]: + path = Path(name) + for part in (path, *path.parents): + if part == root: + break + if part.is_symlink(): + sys.exit('setup-fedora-repository: refusing symbolic links inside the target DNF configuration tree') + if not part.is_relative_to(root): + sys.exit('setup-fedora-repository: configuration path leaves the target root') +PY +owner_marker="# Managed by Loopwire Fedora repository setup" +[ ! -L "$repo_file" ] || fail "refusing a symbolic-link repository file" +if [ -e "$repo_file" ] && ! head -n 1 "$repo_file" | grep -Fxq "$owner_marker"; then + fail "loopwire.repo already exists and is not managed by this helper" +fi +if [ "$install_root" = / ] && [ "$dry_run" != true ] && [ "$EUID" -ne 0 ]; then + fail "run with sudo to change system repository configuration, or use --dry-run" +fi + +if [ "$remove" = true ]; then + if [ "$dry_run" = true ]; then + printf 'Would remove the managed Loopwire Fedora repository and key file.\n' + exit 0 + fi + if [ -f "$repo_file" ]; then + key_name="$(sed -n 's|^gpgkey=file:///etc/pki/rpm-gpg/\(RPM-GPG-KEY-loopwire-[A-F0-9]*\)$|\1|p' "$repo_file")" + [[ "$key_name" =~ ^RPM-GPG-KEY-loopwire-[A-F0-9]{40}$ ]] || fail "managed repository has an unexpected key path" + rm -- "$repo_file" + rm -f -- "$key_directory/$key_name" + fi + printf 'Loopwire Fedora repository removed. Installed packages, RPM database keys, and other repositories are unchanged.\n' + exit 0 +fi + +for command in curl gpg python3 rpm; do + command -v "$command" >/dev/null 2>&1 || fail "$command is required" +done +fingerprint="${fingerprint^^}" +[[ "$fingerprint" =~ ^[A-F0-9]{40}$ ]] || fail "provide a complete 40-character OpenPGP fingerprint" +base_url="$(python3 - "$base_url" <<'PY' +import sys +from urllib.parse import urlsplit +value = sys.argv[1] +try: + url = urlsplit(value) + valid = (url.scheme == 'https' and url.hostname and not url.username and not url.password + and not any(char in value for char in "\\'\"`$<>?#") + and all(32 < ord(char) < 127 for char in value)) + if not valid: + raise ValueError('invalid URL') + if url.port is not None and not 1 <= url.port <= 65535: + raise ValueError('invalid port') +except ValueError: + sys.exit('setup-fedora-repository: base URL must be HTTPS without credentials, whitespace, query, or fragment') +print(value.rstrip('/')) +PY +)" +python3 - "${install_root%/}/etc/os-release" <<'PY' +import shlex +import sys +from pathlib import Path +values = {} +for line in Path(sys.argv[1]).read_text().splitlines(): + if '=' in line and not line.lstrip().startswith('#'): + key, value = line.split('=', 1) + fields = shlex.split(value) + if len(fields) == 1: + values[key] = fields[0] +if (values.get('ID'), values.get('VERSION_ID')) != ('fedora', '44'): + sys.exit('setup-fedora-repository: supported system is Fedora 44') +PY +[ "$(rpm --eval '%{_arch}')" = x86_64 ] || fail "this channel currently supports x86_64 only" +key_name="RPM-GPG-KEY-loopwire-${fingerprint}" +[ ! -L "$key_directory/$key_name" ] || fail "refusing a symbolic-link key file" +if [ "$dry_run" = true ]; then + printf 'Would verify %s/keys/%s.asc and configure Fedora 44 x86_64 with RPM and metadata signature checks.\n' \ + "$base_url" "$fingerprint" + exit 0 +fi + +temporary="$(mktemp -d)" +key_temporary="" +repo_temporary="" +cleanup() { + rm -rf -- "$temporary" + [ -z "$key_temporary" ] || rm -f -- "$key_temporary" + [ -z "$repo_temporary" ] || rm -f -- "$repo_temporary" +} +trap cleanup EXIT +mkdir -m 0700 "$temporary/gnupg" +curl --disable --fail --silent --show-error --proto '=https' --tlsv1.2 \ + --connect-timeout 10 --max-time 60 --output "$temporary/key.asc" "$base_url/keys/$fingerprint.asc" +actual="$(gpg --no-options --batch --homedir "$temporary/gnupg" --with-colons --show-keys "$temporary/key.asc" | + awk -F: '$1 == "pub" { count++ } $1 == "fpr" && !seen { print $10; seen=1 } END { if (count != 1) exit 1 }')" +[ "$actual" = "$fingerprint" ] || fail "downloaded key does not match the expected fingerprint" +cat >"$temporary/loopwire.repo" < [%w[web], %w[deploy web]], 'apps/site/package.json' => [%w[web], %w[deploy web]], 'packaging/repositories/apt-channel.json' => [%w[web], %w[deploy web]], + 'packaging/repositories/fedora-channel.json' => [%w[web], %w[deploy web]], 'apps/docs/docs/guide/install.md' => [%w[web], %w[deploy web]], 'README.md' => [%w[web], %w[web]], 'packaging/README.md' => [%w[web], %w[web]], @@ -107,7 +108,7 @@ def selected_paths(path, event, parsed) check(parsed['deploy'].dig('jobs', 'deploy-bunny', 'environment') == 'docs-production', 'preserve deployment environment') condition = parsed['deploy'].dig('jobs', 'deploy-bunny', 'if') check(condition == condition.strip, 'deployment condition must not become an always-true string with trailing whitespace') -%w[release final-release-proof publish-aur publish-apt continuous-tests].each do |name| +%w[release final-release-proof publish-aur publish-apt publish-fedora continuous-tests].each do |name| workflow = YAML.safe_load_file(File.join(ROOT, '.github/workflows', "#{name}.yml")) events = workflow['on'] || workflow[true] check(!events.key?('pull_request'), "#{name}: deliberate operator workflows must not gain PR triggers") diff --git a/scripts/test-fedora-bootstrap.py b/scripts/test-fedora-bootstrap.py new file mode 100755 index 0000000..6f2fdfc --- /dev/null +++ b/scripts/test-fedora-bootstrap.py @@ -0,0 +1,154 @@ +#!/usr/bin/env python3 +import functools +import http.server +import os +from pathlib import Path +import shutil +import ssl +import subprocess +import tempfile +import threading +import unittest + + +SCRIPT = Path(__file__).with_name("setup-fedora-repository.sh") + + +def run(*args): + return subprocess.run(args, check=True, capture_output=True, text=True) + + +class BootstrapTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.temporary = tempfile.TemporaryDirectory(prefix="loopwire-fedora-bootstrap-") + cls.work = Path(cls.temporary.name) + cls.home = cls.work / "gnupg" + cls.home.mkdir(mode=0o700) + run("gpg", "--batch", "--homedir", str(cls.home), "--pinentry-mode", "loopback", "--passphrase", "", + "--quick-generate-key", "Loopwire Fedora fixture ", "rsa2048", "sign", "1d") + listing = run("gpg", "--homedir", str(cls.home), "--with-colons", "--list-keys").stdout + cls.fingerprint = next(line.split(":")[9] for line in listing.splitlines() if line.startswith("fpr:")) + cls.public = run("gpg", "--homedir", str(cls.home), "--armor", "--export", cls.fingerprint).stdout + cls.web = cls.work / "web" + (cls.web / "keys").mkdir(parents=True) + (cls.web / "keys" / f"{cls.fingerprint}.asc").write_text(cls.public) + (cls.web / "keys" / f"{'A' * 40}.asc").write_text(cls.public) + cert, key = cls.work / "cert.pem", cls.work / "key.pem" + run("openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-days", "1", "-subj", "/CN=127.0.0.1", + "-addext", "subjectAltName=IP:127.0.0.1", "-keyout", str(key), "-out", str(cert)) + cls.requests = [] + + class Handler(http.server.SimpleHTTPRequestHandler): + def log_message(self, *_args): + pass + + def do_GET(self): + cls.requests.append(self.path) + super().do_GET() + + cls.server = http.server.ThreadingHTTPServer( + ("127.0.0.1", 0), functools.partial(Handler, directory=str(cls.web))) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain(cert, key) + cls.server.socket = context.wrap_socket(cls.server.socket, server_side=True) + threading.Thread(target=cls.server.serve_forever, daemon=True).start() + cls.url = f"https://127.0.0.1:{cls.server.server_port}" + cls.binary = cls.work / "bin" + cls.binary.mkdir() + rpm = cls.binary / "rpm" + rpm.write_text('#!/bin/sh\nprintf "%s\\n" "${TEST_ARCH:-x86_64}"\n') + rpm.chmod(0o755) + cls.environment = {**os.environ, "PATH": f"{cls.binary}:{os.environ['PATH']}", "CURL_CA_BUNDLE": str(cert)} + + @classmethod + def tearDownClass(cls): + cls.server.shutdown() + cls.server.server_close() + subprocess.run(["gpgconf", "--homedir", str(cls.home), "--kill", "all"], check=False, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + cls.temporary.cleanup() + + def setUp(self): + self.root = self.work / "root" + shutil.rmtree(self.root, ignore_errors=True) + (self.root / "etc").mkdir(parents=True) + (self.root / "etc/os-release").write_text('ID=fedora\nVERSION_ID="44"\n') + self.repo = self.root / "etc/yum.repos.d/loopwire.repo" + self.key = self.root / f"etc/pki/rpm-gpg/RPM-GPG-KEY-loopwire-{self.fingerprint}" + self.requests.clear() + + def invoke(self, *args, fingerprint=None, url=None, env=None): + return subprocess.run([ + "bash", str(SCRIPT), "--root", str(self.root), "--base-url", url or self.url, + "--fingerprint", fingerprint or self.fingerprint, *args, + ], env={**self.environment, **(env or {})}, capture_output=True, text=True) + + def test_configuration_signature_checks_and_idempotence(self): + for _ in range(2): + result = self.invoke() + self.assertEqual(result.returncode, 0, result.stderr) + text = self.repo.read_text() + for line in [f"baseurl={self.url}", "gpgcheck=1", "repo_gpgcheck=1", "sslverify=1", + f"gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-loopwire-{self.fingerprint}"]: + self.assertIn(line, text) + self.assertEqual(self.key.read_text(), self.public) + self.assertEqual(self.key.stat().st_mode & 0o777, 0o644) + + def test_dry_run_has_no_network_or_writes(self): + result = self.invoke("--dry-run") + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(self.requests, []) + self.assertFalse(self.repo.exists()) + + def test_wrong_fingerprint_preserves_existing_configuration(self): + self.assertEqual(self.invoke().returncode, 0) + before = self.repo.read_bytes() + result = self.invoke(fingerprint="A" * 40) + self.assertNotEqual(result.returncode, 0) + self.assertEqual(self.repo.read_bytes(), before) + + def test_bad_urls_and_wrong_platform_fail_before_network(self): + for url in ["http://example.invalid", "https://user:pass@example.invalid", "https://example.invalid/?", + "https://example.invalid/#", "https://example.invalid/\ninjected"]: + with self.subTest(url=url): + self.assertNotEqual(self.invoke(url=url).returncode, 0) + (self.root / "etc/os-release").write_text('ID=fedora\nVERSION_ID="43"\n') + self.assertNotEqual(self.invoke().returncode, 0) + (self.root / "etc/os-release").write_text('ID=fedora\nVERSION_ID="44"\n') + self.assertNotEqual(self.invoke(env={"TEST_ARCH": "aarch64"}).returncode, 0) + self.assertEqual(self.requests, []) + + def test_os_release_is_data(self): + sentinel = self.work / "must-not-exist" + (self.root / "etc/os-release").write_text(f'ID="$(touch {sentinel})"\nVERSION_ID=44\n') + self.assertNotEqual(self.invoke().returncode, 0) + self.assertFalse(sentinel.exists()) + + def test_unmanaged_and_symlinked_paths_are_preserved(self): + self.repo.parent.mkdir(parents=True) + self.repo.write_text("# other owner\n") + self.assertNotEqual(self.invoke().returncode, 0) + self.assertNotEqual(self.invoke("--remove").returncode, 0) + self.repo.unlink() + outside = self.work / "outside" + outside.mkdir(exist_ok=True) + (self.root / "etc/yum.repos.d").rmdir() + (self.root / "etc/yum.repos.d").symlink_to(outside, target_is_directory=True) + self.assertNotEqual(self.invoke().returncode, 0) + self.assertEqual(list(outside.iterdir()), []) + + def test_remove_is_idempotent_and_preserves_other_repositories(self): + self.assertEqual(self.invoke().returncode, 0) + other = self.repo.with_name("unrelated.repo") + other.write_text("keep") + for _ in range(2): + result = self.invoke("--remove") + self.assertEqual(result.returncode, 0, result.stderr) + self.assertFalse(self.repo.exists()) + self.assertFalse(self.key.exists()) + self.assertEqual(other.read_text(), "keep") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test-fedora-repository-vm-proof.mjs b/scripts/test-fedora-repository-vm-proof.mjs new file mode 100755 index 0000000..170d0d8 --- /dev/null +++ b/scripts/test-fedora-repository-vm-proof.mjs @@ -0,0 +1,175 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { + parseInstalledHashes, + parsePayloadRelease, + parseReleaseChecksums, + verifyReleaseAssetManifest, + verifyReleaseSignature, + verifyInstalledStage, + verifyLifecycle, + verifyPackageEntry, + verifyRpmSignature, +} from "./verify-fedora-repository-vm-proof.mjs"; + +const directory = await mkdtemp(path.join(tmpdir(), "loopwire-fedora-proof-test-")); +const baseline = "0.1.0-1.fc44"; +const upgrade = "0.1.0+dnffixture1-1.fc44"; +const packageName = `loopwire-${baseline}.x86_64.rpm`; +const packageSha256 = "b".repeat(64); +const sourceSha256 = "c".repeat(64); +const fingerprint = "1234567890ABCDEF1234567890ABCDEF12345678"; +const expectedHashes = Object.fromEntries([ + "/usr/bin/loopwire", "/usr/bin/loopwire-dsp-provider", "/usr/bin/loopwire-jack-ports", + "/usr/bin/loopwire-detect-audio", "/usr/lib/loopwire/loopwire-gui", + "/usr/share/applications/loopwire.desktop", "/usr/share/icons/hicolor/scalable/apps/loopwire.svg", +].map((name) => [name, "a".repeat(64)])); +const signature = `${packageName}:\n Header OpenPGP V4 RSA/SHA512 signature, key fingerprint: ${fingerprint.toLowerCase()}: OK\n Header SHA256 digest: OK\n Payload SHA256 digest: OK\n`; +const transitions = [ + `install\t${baseline}\tinstalled`, `reinstall\t${baseline}\tinstalled`, `upgrade\t${upgrade}\tinstalled`, + `rollback\t${baseline}\tinstalled`, `remove\t${baseline}\tabsent`, +].join("\n"); +const releaseManifest = { + schema: "loopwire.release-assets.v1", + release: { tag: "v0.1.0", version: "0.1.0", gitHead: "e".repeat(40) }, + artifacts: [ + { name: packageName, kind: "native-rpm", target: "fedora-44", architecture: "x86_64", bytes: 123, sha256: sourceSha256 }, + { name: "loopwire-linux-x86_64.tar.gz", kind: "portable-archive", target: "linux-generic", architecture: "x86_64", + bytes: 456, sha256: "d".repeat(64) }, + ], +}; +const releaseExpected = { version: "0.1.0", rpmName: packageName, rpmBytes: 123, rpmSha256: sourceSha256, + tarBytes: 456, tarSha256: "d".repeat(64) }; +let passed = 0; + +async function test(name, action) { + await action(); + passed += 1; + console.log(`PASS ${name}`); +} +async function stageFixture() { + await rm(path.join(directory, "install"), { recursive: true, force: true }); + await mkdir(path.join(directory, "install"), { recursive: true }); + const files = { + "package-metadata.tsv": `loopwire\t${baseline}\tx86_64\n`, + "dnf-origin.txt": `loopwire|${baseline}|x86_64|loopwire\n`, + "dnf-info.txt": `Installed packages\nName : loopwire\nVersion : 0.1.0\nRelease : 1.fc44\nArchitecture : x86_64\nFrom repository : loopwire\n`, + "package-files.txt": `${Object.keys(expectedHashes).join("\n")}\n`, + "installed-files.sha256": `${Object.entries(expectedHashes).map(([name, hash]) => `${hash} ${name}`).join("\n")}\n`, + "signed-package.sha256": `${packageSha256} ${packageName}\n`, + "rpm-signature.txt": signature, + "background-help.txt": "Usage: Loopwire background restore\n", + "dsp-provider-help.txt": "Usage: Loopwire DSP provider\n", + "jack-provider-help.txt": "Usage: Loopwire JACK provider\n", + "detect-audio.json": "{\"backends\":[]}\n", + "gui-ldd.txt": "libgtk-3.so.0 => /lib64/libgtk-3.so.0\nlibwebkit2gtk-4.1.so.0 => /lib64/libwebkit2gtk-4.1.so.0\n", + "gui-launch-status.txt": "0\n", "gui-window-ids.txt": "1234\n", "gui-window-names.txt": "Loopwire\n", + "gui-launch.log": "", "xvfb.log": "", + }; + for (const [name, content] of Object.entries(files)) await writeFile(path.join(directory, "install", name), content); +} +async function verifyStage() { + await verifyInstalledStage(directory, "install", baseline, fingerprint, packageName, packageSha256, expectedHashes); +} +async function change(name, transform) { + const file = path.join(directory, "install", name); + await writeFile(file, transform(await readFile(file, "utf8"))); +} + +try { + await test("complete ordered lifecycle accepted", () => verifyLifecycle(transitions, baseline, upgrade)); + await test("missing reinstall rejected", () => assert.throws(() => verifyLifecycle( + transitions.split("\n").filter((line) => !line.startsWith("reinstall")).join("\n"), baseline, upgrade), /lifecycle transitions/)); + await test("rollback remaining at new version rejected", () => assert.throws(() => verifyLifecycle( + transitions.replace(`rollback\t${baseline}`, `rollback\t${upgrade}`), baseline, upgrade), /lifecycle transitions/)); + await test("fabricated summary pass is insufficient", () => assert.throws(() => verifyLifecycle("pass", baseline, upgrade), /lifecycle transitions/)); + await test("duplicate file hash rejected", () => assert.throws(() => parseInstalledHashes( + `${"a".repeat(64)} /usr/bin/loopwire\n${"b".repeat(64)} /usr/bin/loopwire\n`), /duplicate/)); + await test("parent traversal hash rejected", () => assert.throws(() => parseInstalledHashes( + `${"a".repeat(64)} /usr/../etc/passwd\n`), /invalid/)); + await test("selective signed release checksums accepted", () => assert.deepEqual(parseReleaseChecksums( + `${sourceSha256} ${packageName}\n${"d".repeat(64)} loopwire-linux-x86_64.tar.gz\n${"e".repeat(64)} release-assets.json\n`), + new Map([[packageName, sourceSha256], ["loopwire-linux-x86_64.tar.gz", "d".repeat(64)], + ["release-assets.json", "e".repeat(64)]]))); + await test("duplicate signed release checksum rejected", () => assert.throws(() => parseReleaseChecksums( + `${sourceSha256} ${packageName}\n${sourceSha256} ${packageName}\n`), /duplicate/)); + await test("valid public release signature accepted and tamper rejected", async () => { + const signing = path.join(directory, "release-signing"); + await mkdir(signing); + const privateKey = path.join(signing, "private.pem"); + const publicKey = path.join(signing, "public.pem"); + const checksums = path.join(signing, "SHA256SUMS"); + const signatureFile = path.join(signing, "SHA256SUMS.sig"); + await writeFile(checksums, `${sourceSha256} ${packageName}\n`); + for (const args of [["genpkey", "-algorithm", "RSA", "-pkeyopt", "rsa_keygen_bits:2048", "-out", privateKey], + ["pkey", "-in", privateKey, "-pubout", "-out", publicKey], + ["dgst", "-sha256", "-sign", privateKey, "-out", signatureFile, checksums]]) { + const result = spawnSync("openssl", args, { encoding: "utf8" }); + assert.equal(result.status, 0, result.stderr); + } + verifyReleaseSignature(checksums, signatureFile, publicKey); + await writeFile(checksums, `${"0".repeat(64)} ${packageName}\n`); + assert.throws(() => verifyReleaseSignature(checksums, signatureFile, publicKey), /openssl verification failed/); + }); + await test("exact public release manifest accepted", () => verifyReleaseAssetManifest( + JSON.stringify(releaseManifest), releaseExpected)); + await test("wrong Fedora manifest target rejected", () => assert.throws(() => verifyReleaseAssetManifest( + JSON.stringify({ ...releaseManifest, artifacts: releaseManifest.artifacts.map((entry, index) => index ? entry : { ...entry, target: "opensuse-tumbleweed" }) }), + releaseExpected), /artifact count/)); + await test("wrong Fedora manifest hash rejected", () => assert.throws(() => verifyReleaseAssetManifest( + JSON.stringify({ ...releaseManifest, artifacts: releaseManifest.artifacts.map((entry, index) => index ? entry : { ...entry, sha256: "0".repeat(64) }) }), + releaseExpected), /Fedora release artifact/)); + await test("wrong public release version rejected", () => assert.throws(() => verifyReleaseAssetManifest( + JSON.stringify({ ...releaseManifest, release: { ...releaseManifest.release, version: "0.2.0" } }), releaseExpected), /public release version/)); + const releaseText = "name=loopwire\nversion=0.1.0\narch=x86_64\nsource_date_epoch=1788115521\n"; + await test("exact portable RELEASE accepted", () => parsePayloadRelease(releaseText, "0.1.0")); + await test("wrong portable RELEASE version rejected", () => assert.throws(() => parsePayloadRelease( + releaseText.replace("version=0.1.0", "version=0.2.0"), "0.1.0"), /RELEASE version/)); + await test("valid RPM signature accepted", () => verifyRpmSignature(signature, fingerprint, packageName)); + await test("RPM NOKEY status rejected", () => assert.throws(() => verifyRpmSignature( + signature.replace(": OK", ": NOKEY"), fingerprint, packageName), /failed/)); + await test("RPM signed by wrong key rejected", () => assert.throws(() => verifyRpmSignature( + signature.replace(fingerprint.toLowerCase(), "0".repeat(40)), fingerprint, packageName), /wrong key/)); + await test("unsigned RPM output rejected", () => assert.throws(() => verifyRpmSignature( + `${packageName}: digests OK\n`, fingerprint, packageName), /lacks/)); + const packageEntry = { + name: "loopwire", version: "0.1.0", release: "1.fc44", architecture: "x86_64", + path: `packages/${packageName}`, sourceReleaseSha256: sourceSha256, + distributedSha256: packageSha256, size: 123, + }; + await test("exact signed package manifest entry accepted", () => verifyPackageEntry( + packageEntry, { version: "0.1.0", name: packageName }, packageSha256, sourceSha256, "fixture")); + await test("public baseline source hash substitution rejected", () => assert.throws(() => verifyPackageEntry( + { ...packageEntry, sourceReleaseSha256: "d".repeat(64) }, { version: "0.1.0", name: packageName }, + packageSha256, sourceSha256, "fixture"), /source release hash/)); + await test("distributed RPM substitution rejected", () => assert.throws(() => verifyPackageEntry( + { ...packageEntry, distributedSha256: "d".repeat(64) }, { version: "0.1.0", name: packageName }, + packageSha256, sourceSha256, "fixture"), /distributed RPM hash/)); + + await stageFixture(); + await test("complete Fedora installed stage accepted", verifyStage); + for (const [name, file, mutation, pattern] of [ + ["changed installed bytes rejected", "installed-files.sha256", (value) => value.replace("a".repeat(64), "d".repeat(64)), /signed repository RPM payload/], + ["wrong installed version rejected", "package-metadata.tsv", (value) => value.replace(baseline, upgrade), /package metadata/], + ["local RPM installation without repository origin rejected", "dnf-origin.txt", (value) => value.replace("|loopwire\n", "|@commandline\n"), /repository origin/], + ["wrong signed RPM digest rejected", "signed-package.sha256", (value) => value.replace(packageSha256, "d".repeat(64)), /signed RPM digest/], + ["failed RPM signature rejected", "rpm-signature.txt", (value) => value.replace(": OK", ": NOT OK"), /signature verification failed/], + ["unresolved GUI dependency rejected", "gui-ldd.txt", (value) => `${value}libmissing.so => not found\n`, /linkage/], + ["unrelated X11 window rejected", "gui-window-names.txt", () => "xterm\n", /application window/], + ["failed GUI process rejected", "gui-launch-status.txt", () => "124\n", /GUI launch/], + ["GUI panic rejected", "gui-launch.log", () => "thread main panicked\n", /fatal GUI log/], + ["empty provider output rejected", "dsp-provider-help.txt", () => "", /empty evidence/], + ["missing installed helper rejected", "package-files.txt", (value) => value.replace("/usr/bin/loopwire-dsp-provider\n", ""), /missing/], + ]) { + await stageFixture(); + await change(file, mutation); + await test(name, () => assert.rejects(verifyStage, pattern)); + } + console.log(`Fedora VM proof verifier tests passed: ${passed}`); +} finally { + await rm(directory, { recursive: true, force: true }); +} diff --git a/scripts/test-fedora-workflow-preflight.py b/scripts/test-fedora-workflow-preflight.py new file mode 100755 index 0000000..539a63e --- /dev/null +++ b/scripts/test-fedora-workflow-preflight.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python3 +import os +from pathlib import Path +import subprocess +import tempfile +import unittest + + +SCRIPT = Path(__file__).with_name("publish-fedora-workflow.sh").resolve() + + +class PreflightTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory(prefix="loopwire-fedora-preflight-") + self.root = Path(self.temp.name) + self.addCleanup(self.temp.cleanup) + binary = self.root / "bin" + binary.mkdir() + gpg = binary / "gpg" + gpg.write_text('#!/bin/sh\nprintf called > "$FEDORA_TEST_MARKER"\nexit 1\n') + gpg.chmod(0o755) + self.marker = self.root / "used-key" + self.env = { + **os.environ, "PATH": f"{binary}:{os.environ['PATH']}", "FEDORA_TEST_MARKER": str(self.marker), + "FEDORA_REPOSITORY_URL": "https://packages.example.invalid/fedora/44/x86_64", + "FEDORA_REPOSITORY_HOST": "publisher@example.invalid", "FEDORA_REPOSITORY_ROOT": "/srv/loopwire-rpm", + "FEDORA_SIGNING_FINGERPRINT": "A" * 40, "FEDORA_SSH_PRIVATE_KEY": "private-ssh-fixture", + "FEDORA_SSH_KNOWN_HOSTS": "known-hosts-fixture", "FEDORA_SIGNING_KEY": "private-gpg-fixture", + "RUNNER_TEMP": str(self.root), "GITHUB_REPOSITORY": "sandwichfarm/loopwire", + "GITHUB_SERVER_URL": "https://github.com", "GITHUB_RUN_ID": "123", + "OPERATION": "publish", "RELEASE_TAG": "v1.2.3", + } + + def rejected(self, values, message): + result = subprocess.run(["bash", str(SCRIPT)], env={**self.env, **values}, capture_output=True, text=True) + self.assertNotEqual(result.returncode, 0) + self.assertIn(message, result.stderr) + self.assertFalse(self.marker.exists(), "invalid input reached signing-key operations") + self.assertNotIn("private-ssh-fixture", result.stdout + result.stderr) + self.assertNotIn("private-gpg-fixture", result.stdout + result.stderr) + + def test_https_url_rejected_before_keys_or_origin_access(self): + for url in ["http://example.invalid", "https://user:pass@example.invalid", "https://example.invalid/?", + "https://example.invalid/#", "https://example.invalid/\ninjected"]: + with self.subTest(url=url): + self.rejected({"FEDORA_REPOSITORY_URL": url}, "base URL") + + def test_missing_configuration(self): + self.rejected({"FEDORA_SIGNING_KEY": ""}, "missing configuration: FEDORA_SIGNING_KEY") + + def test_tag_rollback_and_fingerprint_validation(self): + self.rejected({"RELEASE_TAG": "v1.2.3; unexpected"}, "stable vX.Y.Z") + self.rejected({"OPERATION": "rollback", "ROLLBACK_REVISION": "HEAD"}, "revision SHA-256") + self.rejected({"FEDORA_SIGNING_FINGERPRINT": "short"}, "fingerprint") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test-fedora-workflow.rb b/scripts/test-fedora-workflow.rb new file mode 100755 index 0000000..dd1026a --- /dev/null +++ b/scripts/test-fedora-workflow.rb @@ -0,0 +1,41 @@ +#!/usr/bin/env ruby +require 'yaml' + +root = File.expand_path('..', __dir__) +workflow = YAML.safe_load_file(File.join(root, '.github/workflows/publish-fedora.yml')) +release = YAML.safe_load_file(File.join(root, '.github/workflows/release.yml')) +events = workflow['on'] || workflow[true] +check = ->(condition, message) { raise message unless condition } +check.call(!events.key?('push') && !events.key?('pull_request'), 'Fedora publication must not run on arbitrary source changes') +check.call(events.key?('workflow_call') && events.key?('workflow_dispatch'), 'release and operator entrypoints required') +check.call(events.fetch('schedule').any? { |item| item['cron'] == '53 5 * * 1' }, 'weekly metadata refresh required') +check.call(events.dig('workflow_dispatch', 'inputs', 'operation', 'options') == %w[publish refresh rollback], 'operator operations drifted') +check.call(workflow.dig('permissions', 'contents') == 'read', 'GitHub access must stay read-only') +check.call(workflow.dig('concurrency', 'cancel-in-progress') == false, 'active metadata promotion must not be cancelled') +job = workflow.dig('jobs', 'publish') +check.call(job['environment'] == 'packages-production', 'production secrets must be environment-scoped') +check.call(job['if'].include?("vars.FEDORA_REPOSITORY_ENABLED == 'true'"), 'explicit repository enablement required') +check.call(job['if'].include?('github.event.repository.default_branch'), 'operator runs must use reviewed default-branch code') +check.call(job['if'] == job['if'].strip, 'job condition has literal trailing whitespace') +check.call(job.dig('container', 'image').match?(/^fedora:44@sha256:[a-f0-9]{64}$/), 'workflow must pin its Fedora toolchain') +publisher = job.fetch('steps').find { |step| step['run'] == 'bash scripts/publish-fedora-workflow.sh' } +check.call(publisher, 'workflow must use the reviewed publisher entrypoint') +check.call(publisher.dig('env', 'OPERATION') == "${{ inputs.operation || 'refresh' }}", 'scheduled runs must refresh') +%w[FEDORA_SIGNING_KEY FEDORA_SSH_PRIVATE_KEY FEDORA_SSH_KNOWN_HOSTS FEDORA_SIGNING_PASSPHRASE].each do |name| + check.call(publisher.dig('env', name) == "${{ secrets.#{name} }}", "#{name} must come from secrets") +end +caller = release.dig('jobs', 'publish-fedora') +check.call(caller['needs'] == 'publish-release', 'Fedora publication must wait for existing release gates') +check.call(caller['uses'] == './.github/workflows/publish-fedora.yml', 'release must reuse the reviewed workflow') +check.call(caller.dig('with', 'tag') == '${{ needs.publish-release.outputs.tag }}', 'use only verified release tag output') + +script = File.read(File.join(root, 'scripts/publish-fedora-workflow.sh')) +publish_index = script.index('scripts/publish-rpm-repository.py publish') +%w[verify-release-signature.sh release-asset-manifest.mjs].each do |gate| + check.call(script.index(gate) && script.index(gate) < publish_index, "#{gate} must precede publication") +end +check.call(script.include?('--require-checksum --require-evidence'), 'public release inventory/evidence verification required') +check.call(script.include?('--expected-revision "$expected"'), 'origin publication must use revision CAS') +check.call(script.index('python3 scripts/verify-rpm-public.py') > publish_index, 'activation requires verification of served bytes') +check.call(script.include?('trap cleanup EXIT') && script.include?('unset FEDORA_SSH_PRIVATE_KEY'), 'private files/environment need cleanup') +puts 'Fedora workflow contract passed: pinned toolchain, protected release ordering, secret transport, refresh and public proof.' diff --git a/scripts/test-publish-rpm-repository.py b/scripts/test-publish-rpm-repository.py new file mode 100644 index 0000000..9915166 --- /dev/null +++ b/scripts/test-publish-rpm-repository.py @@ -0,0 +1,794 @@ +#!/usr/bin/env python3 +"""Regression tests for the Fedora RPM repository publisher. + +Run locally with Python's standard library: + python3 scripts/test-publish-rpm-repository.py + +Pass --with-ssh inside the pinned RPM tools container to start a disposable +loopback-only sshd and exercise publish/fetch/recover with generated client and +host keys. No production host, credentials, keyring, or repository is touched. +""" + +import argparse +import base64 +import fcntl +import hashlib +import importlib.util +import io +import json +import os +from pathlib import Path +import shutil +import socket +import stat +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +import urllib.error +import urllib.request +from unittest import mock + + +SCRIPT = Path(__file__).with_name("publish-rpm-repository.py") +WITH_SSH = False +FPR = "A" * 40 + + +def load(name, path): + specification = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(specification) + specification.loader.exec_module(module) + return module + + +publisher = load("rpm_publisher", SCRIPT) + + +def write_manifest(root, manifest): + manifest.pop("revision", None) + manifest["revision"] = hashlib.sha256(publisher.canonical(manifest)).hexdigest() + (root / publisher.MANIFEST).write_text(json.dumps(manifest), encoding="utf-8") + return manifest + + +def fixture(root, version="1.0.0", created=1, package_bytes=None): + root.mkdir() + package = f"packages/loopwire-{version}-1.fc44.x86_64.rpm" + files = { + package: package_bytes or f"rpm package {version}".encode(), + f"keys/{FPR}.asc": b"synthetic public key", + "repodata/repomd.xml": f"repomd {version} {created}".encode(), + "repodata/repomd.xml.asc": f"signature {version} {created}".encode(), + } + for kind in ("primary", "filelists", "other"): + data = f"{kind} {version} {created}".encode() + files[f"repodata/{hashlib.sha256(data).hexdigest()}-{kind}.xml.gz"] = data + entries = [] + for path, data in sorted(files.items()): + target = root / path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(data) + entries.append({ + "path": path, + "kind": publisher.classify(path), + "size": len(data), + "sha256": hashlib.sha256(data).hexdigest(), + }) + package_data = files[package] + manifest = { + "schema": publisher.SCHEMA, + "schemaVersion": 1, + "createdAt": created, + "validUntil": created + 2592000, + "signingFingerprint": FPR, + "target": publisher.TARGET.copy(), + "packages": [{ + "name": "loopwire", + "version": version, + "release": "1.fc44", + "architecture": "x86_64", + "path": package, + "sourceReleaseSha256": "1" * 64, + "distributedSha256": hashlib.sha256(package_data).hexdigest(), + "size": len(package_data), + }], + "files": entries, + } + return json.loads(json.dumps(write_manifest(root, manifest))) + + +class PublicationTests(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory(prefix="loopwire-rpm-publisher-tests-") + self.directory = Path(self.temporary.name) + self.root = self.directory / "origin" + self.first = self.directory / "first" + self.second = self.directory / "second" + self.one = fixture(self.first) + self.two = fixture(self.second, "1.1.0", 2) + + def tearDown(self): + self.temporary.cleanup() + + @property + def channel(self): + return publisher.public_channel(self.root) + + def publish_first(self): + return publisher.publish_at(self.root, self.first, FPR, "empty") + + def assert_current(self, revision): + self.assertEqual(publisher.state(self.root, "current"), {"revision": revision}) + + def test_publish_idempotence_cas_retention_and_fetch(self): + self.assertEqual(self.publish_first()["status"], "published") + self.assertEqual(self.publish_first()["status"], "unchanged") + with self.assertRaisesRegex(publisher.PublicationError, "compare-and-swap"): + publisher.publish_at(self.root, self.second, FPR, "empty") + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assert_current(self.two["revision"]) + old_package = self.one["packages"][0]["path"] + self.assertEqual((self.channel / old_package).read_bytes(), + (self.first / old_package).read_bytes()) + self.assertTrue((self.root / "snapshots" / self.one["revision"]).is_dir()) + with publisher.selected_snapshot(self.root, FPR, self.one["revision"]) as (snapshot, manifest): + self.assertEqual(snapshot.name, self.one["revision"]) + self.assertEqual(manifest, self.one) + + def test_restrictive_umask_sets_public_and_private_permissions(self): + previous_umask = os.umask(0o077) + try: + self.publish_first() + + def permissions(path): + return stat.S_IMODE(path.stat().st_mode) + + self.assertEqual(permissions(self.root), 0o755) + public = self.root / "public" + for path in (public, *public.rglob("*")): + self.assertEqual(permissions(path), 0o755 if path.is_dir() else 0o644, + str(path)) + for private in (self.root / "snapshots", self.root / "state"): + for path in (private, *private.rglob("*")): + self.assertEqual(permissions(path), 0o700 if path.is_dir() else 0o600, + str(path)) + self.assertEqual(permissions(self.root / ".publish.lock"), 0o600) + finally: + os.umask(previous_umask) + + def test_existing_operator_root_permissions_are_preserved(self): + self.root.mkdir(mode=0o750) + (self.root / "public").mkdir(mode=0o750) + self.root.chmod(0o750) + (self.root / "public").chmod(0o750) + self.publish_first() + self.assertEqual(stat.S_IMODE(self.root.stat().st_mode), 0o750) + self.assertEqual(stat.S_IMODE((self.root / "public").stat().st_mode), 0o750) + + def test_immutable_collision_fails_before_journal_or_snapshot(self): + self.publish_first() + collision = self.directory / "collision" + changed = fixture(collision, "1.0.0", 3, b"different bytes at immutable URL") + with self.assertRaisesRegex(publisher.PublicationError, "immutable URL collision"): + publisher.publish_at(self.root, collision, FPR, self.one["revision"]) + self.assert_current(self.one["revision"]) + self.assertIsNone(publisher.state(self.root, "pending")) + self.assertFalse((self.root / "snapshots" / changed["revision"]).exists()) + + def test_commit_order_is_signature_then_atomic_repomd(self): + self.publish_first() + events = [] + + def inspect(label): + events.append(label) + public_xml = (self.channel / "repodata/repomd.xml").read_bytes() + public_signature = (self.channel / "repodata/repomd.xml.asc").read_bytes() + if label == "immutable": + self.assertEqual(public_xml, (self.first / "repodata/repomd.xml").read_bytes()) + self.assertEqual(public_signature, + (self.first / "repodata/repomd.xml.asc").read_bytes()) + elif label == "signature": + self.assertEqual(public_xml, (self.first / "repodata/repomd.xml").read_bytes()) + self.assertEqual(public_signature, + (self.second / "repodata/repomd.xml.asc").read_bytes()) + elif label == "committed": + self.assertEqual(public_xml, (self.second / "repodata/repomd.xml").read_bytes()) + self.assertEqual(public_signature, + (self.second / "repodata/repomd.xml.asc").read_bytes()) + + with mock.patch.object(publisher, "_checkpoint", side_effect=inspect): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assertLess(events.index("immutable"), events.index("signature")) + self.assertLess(events.index("signature"), events.index("committed")) + + def test_every_promotion_checkpoint_is_recoverable(self): + for index, checkpoint in enumerate( + ("journal", "immutable", "signature", "committed", "manifest", "current")): + with self.subTest(checkpoint=checkpoint): + root = self.directory / f"checkpoint-{index}" + + def interrupt(label): + if label == checkpoint: + raise InterruptedError("simulated interruption") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(root, self.first, FPR, "empty") + self.assertEqual(publisher.state(root, "pending")["revision"], + self.one["revision"]) + publisher.recover_at(root, FPR, self.one["revision"]) + self.assertEqual(publisher.state(root, "current")["revision"], + self.one["revision"]) + self.assertIsNone(publisher.state(root, "pending")) + + def test_killed_process_leaves_durable_journal_and_blocks_competitors(self): + self.publish_first() + code = ( + "import importlib.util,os,sys; from pathlib import Path; " + "s=importlib.util.spec_from_file_location('p',sys.argv[1]); " + "p=importlib.util.module_from_spec(s); s.loader.exec_module(p); " + "p._checkpoint=lambda label: os._exit(97) if label=='signature' else None; " + "p.publish_at(Path(sys.argv[2]),Path(sys.argv[3]),sys.argv[4],sys.argv[5])" + ) + process = subprocess.run([sys.executable, "-c", code, str(SCRIPT), str(self.root), + str(self.second), FPR, self.one["revision"]], check=False) + self.assertEqual(process.returncode, 97) + self.assert_current(self.one["revision"]) + self.assertEqual(publisher.state(self.root, "pending")["revision"], self.two["revision"]) + with self.assertRaisesRegex(publisher.PublicationError, "recover"): + with publisher.selected_snapshot(self.root, FPR): + pass + with self.assertRaisesRegex(publisher.PublicationError, "recover"): + publisher.publish_at(self.root, self.first, FPR, self.one["revision"]) + publisher.recover_at(self.root, FPR, self.two["revision"]) + self.assert_current(self.two["revision"]) + + def test_exclusive_flock_blocks_publish_fetch_and_recovery(self): + self.publish_first() + descriptor = os.open(self.root / ".publish.lock", os.O_RDONLY | os.O_NOFOLLOW) + fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB) + try: + with self.assertRaisesRegex(publisher.PublicationError, "locked"): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + with self.assertRaisesRegex(publisher.PublicationError, "locked"): + with publisher.selected_snapshot(self.root, FPR): + pass + finally: + os.close(descriptor) + + def test_separate_process_lock_blocks_concurrent_cas_writer(self): + self.publish_first() + code = ( + "import fcntl,os,sys; " + "fd=os.open(sys.argv[1],os.O_RDONLY|os.O_NOFOLLOW); " + "fcntl.flock(fd,fcntl.LOCK_EX); print('locked',flush=True); " + "sys.stdin.readline(); os.close(fd)" + ) + holder = subprocess.Popen( + [sys.executable, "-c", code, str(self.root / ".publish.lock")], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True, + ) + try: + self.assertEqual(holder.stdout.readline().strip(), "locked") + with self.assertRaisesRegex(publisher.PublicationError, "locked"): + publisher.publish_at(self.root, self.second, FPR, self.one["revision"]) + self.assert_current(self.one["revision"]) + finally: + holder.stdin.write("release\n") + holder.stdin.flush() + holder.wait(timeout=10) + holder.stdin.close() + holder.stdout.close() + + def test_empty_fetch_has_no_filesystem_side_effect(self): + with self.assertRaises(publisher.EmptyRepository): + with publisher.selected_snapshot(self.root, FPR): + pass + self.assertFalse(self.root.exists()) + + def test_malicious_paths_kind_target_and_revision_fail_before_write(self): + for index, bad in enumerate(("../../escape", "/etc/passwd", "repodata/../escape", + "state/current.json", "packages//x.rpm")): + with self.subTest(path=bad): + candidate = self.directory / f"bad-path-{index}" + manifest = fixture(candidate, f"2.0.{index}") + manifest["files"][0]["path"] = bad + write_manifest(candidate, manifest) + with self.assertRaises(publisher.PublicationError): + publisher.publish_at(self.root, candidate, FPR, "empty") + self.assertFalse(self.root.exists()) + self.one["target"]["release"] = "45" + write_manifest(self.first, self.one) + with self.assertRaisesRegex(publisher.PublicationError, "Fedora 44"): + self.publish_first() + + def test_candidate_symlinks_hardlinks_and_unlisted_files_are_rejected(self): + target = self.first / "unlisted" + target.symlink_to(self.second / publisher.MANIFEST) + with self.assertRaisesRegex(publisher.PublicationError, "symlink"): + self.publish_first() + target.unlink() + os.link(self.first / publisher.MANIFEST, target) + with self.assertRaisesRegex(publisher.PublicationError, "hardlink"): + self.publish_first() + target.unlink() + target.write_text("extra", encoding="utf-8") + with self.assertRaisesRegex(publisher.PublicationError, "unlisted"): + self.publish_first() + self.assertFalse(self.root.exists()) + + def test_origin_and_public_symlinks_and_drift_are_rejected(self): + elsewhere = self.directory / "elsewhere" + elsewhere.mkdir() + self.root.symlink_to(elsewhere, target_is_directory=True) + with self.assertRaisesRegex(publisher.PublicationError, "symlink"): + self.publish_first() + self.root.unlink() + (publisher.public_channel(self.root)).mkdir(parents=True) + (publisher.public_channel(self.root) / "foreign").write_text("unmanaged") + with self.assertRaisesRegex(publisher.PublicationError, "unmanaged"): + self.publish_first() + shutil.rmtree(self.root) + self.publish_first() + target = self.channel / self.one["packages"][0]["path"] + target.unlink() + target.symlink_to(self.first / self.one["packages"][0]["path"]) + with self.assertRaisesRegex(publisher.PublicationError, "symlink"): + self.publish_first() + target.unlink() + target.write_bytes(b"drift") + with self.assertRaisesRegex(publisher.PublicationError, "drifted"): + self.publish_first() + + def test_archive_rejects_traversal_links_specials_and_duplicates(self): + for kind in ("traversal", "symlink", "hardlink", "duplicate"): + with self.subTest(kind=kind): + archive = io.BytesIO() + with tarfile.open(fileobj=archive, mode="w") as output: + name = "../escape" if kind == "traversal" else publisher.MANIFEST + member = tarfile.TarInfo(name) + member.size = 2 + if kind in ("symlink", "hardlink"): + member.type = tarfile.SYMTYPE if kind == "symlink" else tarfile.LNKTYPE + member.linkname = "/etc/passwd" + output.addfile(member, io.BytesIO(b"{}")) + if kind == "duplicate": + output.addfile(member, io.BytesIO(b"{}")) + archive.seek(0) + destination = self.directory / f"archive-{kind}" + destination.mkdir() + with self.assertRaises(publisher.PublicationError): + publisher.read_archive(archive, destination) + + def test_remote_arguments_are_data_and_pinned_credentials_are_required(self): + known = self.directory / "known_hosts" + identity = self.directory / "identity" + known.write_text("host ssh-ed25519 AAAA", encoding="utf-8") + identity.write_text("identity", encoding="utf-8") + args = argparse.Namespace(ssh="publisher@example.invalid", ssh_port=2222, + known_hosts=known, identity_file=identity) + request = {"root": "/tmp/path with 'quotes';$(touch /tmp/unsafe)", "action": "fetch"} + command = publisher.ssh_command(args, request) + self.assertEqual(command[1:3], ["-F", "/dev/null"]) + self.assertIn("StrictHostKeyChecking=yes", command) + self.assertIn("ForwardAgent=no", command) + self.assertIn("GlobalKnownHostsFile=/dev/null", command) + remote = __import__("shlex").split(command[-1]) + self.assertEqual(remote[:2], ["python3", "-c"]) + self.assertEqual(json.loads(base64.urlsafe_b64decode(remote[-1])), request) + args.identity_file = None + with self.assertRaisesRegex(publisher.PublicationError, "identity"): + publisher.ssh_command(args, request) + args.identity_file = identity + args.ssh = "publisher@host;touch" + with self.assertRaises(publisher.PublicationError): + publisher.ssh_command(args, request) + + def test_expired_recovery_requires_explicit_historical_verification(self): + expired = self.directory / "expired" + manifest = fixture(expired, "2.0.0", int(time.time()) - 2592001) + + def interrupt(label): + if label == "journal": + raise InterruptedError("expired pending journal") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(self.root, expired, FPR, "empty") + key = self.directory / "key.asc" + key.write_text("synthetic", encoding="utf-8") + base = argparse.Namespace(root=str(self.root), public_key=key, fingerprint=FPR, + ssh=None, ssh_port=None, identity_file=None, + known_hosts=None, action="recover", dry_run=True, + allow_expired=False) + + def verify(_root, _key, _fingerprint, historical=False): + if not historical: + raise publisher.PublicationError("expired repository") + return manifest + + with mock.patch.object(publisher, "verify_signed", side_effect=verify): + with self.assertRaisesRegex(publisher.PublicationError, "expired"): + publisher.run(base) + base.allow_expired = True + checked = publisher.run(base) + self.assertTrue(checked["requiresRefresh"]) + base.dry_run = False + completed = publisher.run(base) + self.assertTrue(completed["requiresRefresh"]) + self.assertIn("Immediately", completed["nextAction"]) + + +class SignedDnfCommitTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + if not WITH_SSH: + raise unittest.SkipTest("runs with --with-ssh in the pinned RPM tools container") + required = ("createrepo_c", "dnf", "gpg", "openssl", "rpmbuild", "rpmkeys", "rpmsign") + missing = [tool for tool in required if shutil.which(tool) is None] + if missing: + raise unittest.SkipTest("missing RPM tools: " + ", ".join(missing)) + cls.temporary = tempfile.TemporaryDirectory(prefix="loopwire-rpm-dnf-tests-") + cls.directory = Path(cls.temporary.name) + cls.gnupg = cls.directory / "gnupg" + cls.gnupg.mkdir(mode=0o700) + cls.date = int(time.time()) - 120 + cls.shell( + "gpg", "--homedir", cls.gnupg, "--batch", "--pinentry-mode", "loopback", + "--passphrase", "", "--faked-system-time", f"{cls.date - 60}!", + "--quick-generate-key", "Loopwire Fedora publisher fixture", "rsa2048", "sign", "1y", + ) + listing = cls.shell("gpg", "--homedir", cls.gnupg, "--with-colons", "--list-keys").stdout + cls.fingerprint = next(line.split(":")[9] for line in listing.splitlines() + if line.startswith("fpr:")) + cls.public_key = cls.directory / "repository-key.asc" + cls.public_key.write_text(cls.shell( + "gpg", "--homedir", cls.gnupg, "--armor", "--export", cls.fingerprint, + ).stdout, encoding="utf-8") + cls.release_private = cls.directory / "release-private.pem" + cls.release_public = cls.directory / "release-public.pem" + cls.shell("openssl", "genpkey", "-algorithm", "RSA", "-pkeyopt", "rsa_keygen_bits:2048", + "-out", cls.release_private) + cls.shell("openssl", "pkey", "-in", cls.release_private, "-pubout", + "-out", cls.release_public) + cls.first = cls.build_candidate("1.0.0", cls.date) + cls.second = cls.build_candidate("1.1.0", cls.date + 1, cls.first) + cls.one = json.loads((cls.first / publisher.MANIFEST).read_text(encoding="utf-8")) + cls.two = json.loads((cls.second / publisher.MANIFEST).read_text(encoding="utf-8")) + + @classmethod + def tearDownClass(cls): + if hasattr(cls, "gnupg"): + subprocess.run(["gpgconf", "--homedir", str(cls.gnupg), "--kill", "gpg-agent"], + check=False, capture_output=True) + if hasattr(cls, "temporary"): + cls.temporary.cleanup() + + @classmethod + def shell(cls, *command, cwd=None, ok=True): + result = subprocess.run(list(map(str, command)), cwd=cwd, capture_output=True, + text=True, check=False) + if ok and result.returncode != 0: + raise AssertionError("command failed: " + " ".join(map(str, command)) + + "\n" + result.stdout + result.stderr) + return result + + @classmethod + def build_rpm(cls, version, release): + top = cls.directory / f"rpmbuild-{version}" + for child in ("BUILD", "BUILDROOT", "RPMS", "SOURCES", "SPECS", "SRPMS"): + (top / child).mkdir(parents=True) + spec = top / "SPECS/loopwire.spec" + spec.write_text( + "Name: loopwire\n" + f"Version: {version}\n" + "Release: 1.fc44\nSummary: DNF publisher fixture\n" + "License: MIT\nBuildArch: x86_64\n\n" + "%description\nDNF publisher fixture.\n\n" + "%prep\n\n%build\n\n" + "%install\nmkdir -p %{buildroot}/usr/bin\n" + f"printf '#!/bin/sh\\necho {version}\\n' > %{{buildroot}}/usr/bin/loopwire\n" + "chmod 0755 %{buildroot}/usr/bin/loopwire\n\n" + "%files\n/usr/bin/loopwire\n", + encoding="utf-8", + ) + cls.shell("rpmbuild", "-bb", "--define", f"_topdir {top}", spec) + built = top / f"RPMS/x86_64/loopwire-{version}-1.fc44.x86_64.rpm" + release.mkdir() + target = release / built.name + shutil.copyfile(built, target) + checksums = release / "SHA256SUMS" + checksums.write_text(f"{publisher.digest(target)} {target.name}\n", encoding="utf-8") + cls.shell("openssl", "dgst", "-sha256", "-sign", cls.release_private, + "-out", release / "SHA256SUMS.sig", checksums) + + @classmethod + def build_candidate(cls, version, date, previous=None): + release = cls.directory / f"release-{version}" + cls.build_rpm(version, release) + candidate = cls.directory / f"candidate-{version}" + command = [ + sys.executable, SCRIPT.with_name("rpm-repository.py"), "build", + "--release-dir", release, "--version", version, "--output", candidate, + "--signing-key", cls.fingerprint, "--gnupg-home", cls.gnupg, + "--date", str(date), "--valid-for-days", "30", + "--release-public-key", cls.release_public, + ] + if previous: + command += ["--previous", previous] + cls.shell(*command) + return candidate + + def setUp(self): + self.origin = self.directory / self.id().split(".")[-1] + + def publisher_cli(self, action, *extra, ok=True): + root = self.origin / "origin" + result = self.shell( + sys.executable, SCRIPT, action, "--root", root, + "--public-key", self.public_key, "--fingerprint", self.fingerprint, + *extra, ok=False, + ) + if ok: + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + return result + + def dnf_query(self, label): + repos = self.origin / f"repos-{label}" + cache = self.origin / f"cache-{label}" + persist = self.origin / f"persist-{label}" + repos.mkdir(parents=True) + channel = publisher.public_channel(self.origin / "origin") + (repos / "loopwire.repo").write_text( + "[loopwire]\nname=Loopwire test\nenabled=1\n" + f"baseurl=file://{channel}/\n" + "gpgcheck=1\nrepo_gpgcheck=1\n" + f"gpgkey=file://{channel}/keys/{self.fingerprint}.asc\n" + "metadata_expire=0\n", + encoding="utf-8", + ) + return self.shell( + "dnf", "-y", "--setopt", f"reposdir={repos}", + "--setopt", f"cachedir={cache}", "--setopt", f"persistdir={persist}", + "--disablerepo=*", "--enablerepo=loopwire", "repoquery", "loopwire", + ok=False, + ) + + def test_real_dnf_rejects_interrupted_signature_xml_pair_and_accepts_recovery(self): + root = self.origin / "origin" + publisher.publish_at(root, self.first, self.fingerprint, "empty") + initial = self.dnf_query("initial") + self.assertEqual(initial.returncode, 0, initial.stdout + initial.stderr) + self.assertIn("loopwire-0:", initial.stdout) + + def interrupt(label): + if label == "signature": + raise InterruptedError("leave new signature with old repomd.xml") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(root, self.second, self.fingerprint, self.one["revision"]) + mixed = self.dnf_query("mixed") + # DNF5 currently exits zero after excluding a repository whose metadata + # signature failed, so package absence plus its explicit verification + # diagnostic is the acceptance boundary. + self.assertIn("Bad PGP signature", mixed.stdout + mixed.stderr) + self.assertNotIn("loopwire-0:", mixed.stdout) + publisher.recover_at(root, self.fingerprint, self.two["revision"]) + recovered = self.dnf_query("recovered") + self.assertEqual(recovered.returncode, 0, recovered.stdout + recovered.stderr) + self.assertIn("loopwire-0:", recovered.stdout) + + def test_signed_cli_publish_fetch_idempotence_and_retained_revision(self): + dry = self.publisher_cli( + "publish", "--repository", self.first, "--expected-revision", "empty", + "--dry-run", + ) + self.assertEqual(json.loads(dry.stdout)["status"], "validated") + self.assertFalse((self.origin / "origin").exists()) + first = self.publisher_cli( + "publish", "--repository", self.first, "--expected-revision", "empty", + ) + self.assertEqual(json.loads(first.stdout)["revision"], self.one["revision"]) + unchanged = self.publisher_cli( + "publish", "--repository", self.first, "--expected-revision", "empty", + ) + self.assertEqual(json.loads(unchanged.stdout)["status"], "unchanged") + self.publisher_cli( + "publish", "--repository", self.second, + "--expected-revision", self.one["revision"], + ) + current = self.origin / "current" + fetched = self.publisher_cli("fetch", "--output", current) + self.assertEqual(json.loads(fetched.stdout)["revision"], self.two["revision"]) + retained = self.origin / "retained" + fetched = self.publisher_cli( + "fetch", "--output", retained, "--revision", self.one["revision"], + ) + self.assertEqual(json.loads(fetched.stdout)["revision"], self.one["revision"]) + self.assertEqual((retained / publisher.MANIFEST).read_bytes(), + (self.first / publisher.MANIFEST).read_bytes()) + + +class SshPublicationTests(unittest.TestCase): + def test_actual_ssh_publish_fetch_and_recover_without_remote_gpg(self): + if not WITH_SSH: + self.skipTest("use --with-ssh inside the pinned disposable RPM tools container") + self.assertTrue(Path("/.dockerenv").exists() and os.geteuid() == 0, + "--with-ssh is restricted to root in a disposable container") + sshd = shutil.which("sshd") + self.assertIsNotNone(sshd, "openssh-server is required") + # Fedora's container root account is locked by default. Unlock it only + # inside this disposable container; sshd still permits public-key auth + # exclusively and listens on a random loopback port. + unlocked = subprocess.run(["passwd", "-d", "root"], capture_output=True, + text=True, check=False) + self.assertEqual(unlocked.returncode, 0, unlocked.stderr) + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-ssh-tests-") as temporary: + directory = Path(temporary) + first = directory / "first" + second = directory / "second" + one = fixture(first) + two = fixture(second, "1.1.0", 2) + origin = directory / "origin" + identity = directory / "identity" + host_key = directory / "host-key" + for key in (identity, host_key): + subprocess.run(["ssh-keygen", "-q", "-t", "ed25519", "-N", "", + "-f", str(key)], check=True) + with socket.socket() as listener: + listener.bind(("127.0.0.1", 0)) + port = listener.getsockname()[1] + known = directory / "known_hosts" + known.write_text(f"[127.0.0.1]:{port} " + host_key.with_suffix(".pub").read_text()) + remote_bin = directory / "remote-bin" + remote_bin.mkdir() + (remote_bin / "python3").symlink_to(sys.executable) + configuration = directory / "sshd.conf" + configuration.write_text( + f"Port {port}\nListenAddress 127.0.0.1\nHostKey {host_key}\n" + f"PidFile {directory / 'sshd.pid'}\nAuthorizedKeysFile {identity}.pub\n" + "PermitRootLogin prohibit-password\nPasswordAuthentication no\n" + "KbdInteractiveAuthentication no\nUsePAM no\nStrictModes no\nAllowUsers root\n" + f"LogLevel ERROR\nSetEnv PATH={remote_bin}\n") + Path("/run/sshd").mkdir(exist_ok=True) + with (directory / "sshd.log").open("wb") as log: + server = subprocess.Popen([sshd, "-D", "-e", "-f", str(configuration)], + stdout=log, stderr=log) + try: + for _ in range(100): + self.assertIsNone(server.poll(), "temporary sshd exited") + try: + with socket.create_connection(("127.0.0.1", port), timeout=0.1): + break + except OSError: + time.sleep(0.05) + connection = argparse.Namespace( + ssh="root@127.0.0.1", ssh_port=port, + identity_file=identity, known_hosts=known, + ) + probe = publisher.ssh_command(connection, {}) + probe[-1] = "python3 -c 'import shutil; assert shutil.which(\"gpg\") is None'" + result = subprocess.run(probe, capture_output=True, text=True, check=False) + self.assertEqual(result.returncode, 0, result.stderr) + result = publisher.remote_call(connection, { + "action": "publish", "root": str(origin), "fingerprint": FPR, + "expected": "empty", + }, repository=first) + self.assertEqual(result["revision"], one["revision"]) + fetched = directory / "fetched" + fetched.mkdir() + publisher.remote_call(connection, { + "action": "fetch", "root": str(origin), "fingerprint": FPR, + "revision": None, + }, output=fetched) + self.assertEqual((fetched / publisher.MANIFEST).read_bytes(), + (first / publisher.MANIFEST).read_bytes()) + def interrupt(label): + if label == "signature": + raise InterruptedError("simulate remote process loss") + + with mock.patch.object(publisher, "_checkpoint", side_effect=interrupt): + with self.assertRaises(InterruptedError): + publisher.publish_at(origin, second, FPR, one["revision"]) + pending = directory / "pending" + pending.mkdir() + publisher.remote_call(connection, { + "action": "fetch-pending", "root": str(origin), + "fingerprint": FPR, "revision": None, + }, output=pending) + self.assertEqual((pending / publisher.MANIFEST).read_bytes(), + (second / publisher.MANIFEST).read_bytes()) + result = publisher.remote_call(connection, { + "action": "recover", "root": str(origin), "fingerprint": FPR, + "revision": two["revision"], + }) + self.assertEqual(result["revision"], two["revision"]) + self.assertIsNone(publisher.state(origin, "pending")) + known.write_text("", encoding="utf-8") + with self.assertRaisesRegex(publisher.PublicationError, "SSH"): + publisher.remote_call(connection, { + "action": "fetch", "root": str(origin), "fingerprint": FPR, + "revision": None, + }, output=directory / "untrusted") + finally: + server.terminate() + server.wait(timeout=10) + + +class NginxPublicTests(unittest.TestCase): + def test_syntax_and_live_cache_headers(self): + if not WITH_SSH: + self.skipTest("runs with --with-ssh in the pinned RPM tools container") + nginx = shutil.which("nginx") + self.assertIsNotNone(nginx, "nginx is required") + snippet_path = (SCRIPT.parent.parent / "packaging/repositories/nginx-rpm.conf") + with tempfile.TemporaryDirectory(prefix="loopwire-rpm-nginx-tests-") as temporary: + directory = Path(temporary) + origin = directory / "origin" + candidate = directory / "candidate" + manifest = fixture(candidate) + publisher.publish_at(origin, candidate, FPR, "empty") + snippet = directory / "nginx-rpm.conf" + snippet.write_text( + snippet_path.read_text(encoding="utf-8").replace( + "/srv/loopwire-rpm", str(origin)), encoding="utf-8") + with socket.socket() as listener: + listener.bind(("127.0.0.1", 0)) + port = listener.getsockname()[1] + config = directory / "nginx.conf" + config.write_text( + "user root;\nworker_processes 1;\nerror_log stderr notice;\n" + f"pid {directory / 'nginx.pid'};\nevents {{ worker_connections 64; }}\n" + "http { access_log off; server { " + f"listen 127.0.0.1:{port}; include {snippet};" + " } }\n", encoding="utf-8") + checked = subprocess.run([nginx, "-t", "-c", str(config)], + capture_output=True, text=True, check=False) + self.assertEqual(checked.returncode, 0, checked.stderr) + server = subprocess.Popen([nginx, "-c", str(config), "-g", "daemon off;"], + stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, + text=True) + try: + base = f"http://127.0.0.1:{port}/fedora/44/x86_64/" + for _ in range(100): + try: + urllib.request.urlopen(base + "repodata/repomd.xml", timeout=0.1).close() + break + except (OSError, urllib.error.URLError): + if server.poll() is not None: + self.fail(server.stderr.read()) + time.sleep(0.02) + + def headers(path): + with urllib.request.urlopen(base + path, timeout=2) as response: + self.assertEqual(response.status, 200) + return response.headers + + self.assertIn("no-store", headers("repodata/repomd.xml")["Cache-Control"]) + self.assertIn("no-store", headers("repodata/repomd.xml.asc")["Cache-Control"]) + package = manifest["packages"][0]["path"] + self.assertIn("immutable", headers(package)["Cache-Control"]) + hashed = next(entry["path"] for entry in manifest["files"] + if entry["path"].endswith("primary.xml.gz")) + self.assertIn("immutable", headers(hashed)["Cache-Control"]) + with self.assertRaises(urllib.error.HTTPError) as missing: + urllib.request.urlopen(base + + "packages/loopwire-9.9.9-1.fc44.x86_64.rpm", timeout=2) + self.assertEqual(missing.exception.code, 404) + self.assertIn("no-store", missing.exception.headers["Cache-Control"]) + missing.exception.close() + finally: + server.terminate() + server.wait(timeout=10) + server.stderr.close() + + +if __name__ == "__main__": + if "--with-ssh" in sys.argv: + WITH_SSH = True + sys.argv.remove("--with-ssh") + unittest.main(verbosity=2) diff --git a/scripts/test-rpm-public.py b/scripts/test-rpm-public.py new file mode 100755 index 0000000..1009ab5 --- /dev/null +++ b/scripts/test-rpm-public.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +import contextlib +import functools +import hashlib +import http.server +import importlib.util +import io +import json +from pathlib import Path +import ssl +import subprocess +import sys +import tempfile +import threading +import unittest +from unittest.mock import patch + + +SCRIPT = Path(__file__).with_name("verify-rpm-public.py") +spec = importlib.util.spec_from_file_location("rpm_public", SCRIPT) +module = importlib.util.module_from_spec(spec) +spec.loader.exec_module(module) + + +class PublicTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.temporary = tempfile.TemporaryDirectory(prefix="loopwire-rpm-public-") + cls.root = Path(cls.temporary.name) + cls.web = cls.root / "web" + cls.web.mkdir() + cls.cert, key = cls.root / "cert.pem", cls.root / "key.pem" + subprocess.run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-days", "1", + "-subj", "/CN=127.0.0.1", "-addext", "subjectAltName=IP:127.0.0.1", + "-keyout", str(key), "-out", str(cls.cert)], check=True, capture_output=True) + + class Handler(http.server.SimpleHTTPRequestHandler): + def log_message(self, *_args): + pass + + cls.server = http.server.ThreadingHTTPServer( + ("127.0.0.1", 0), functools.partial(Handler, directory=str(cls.web))) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain(cls.cert, key) + cls.server.socket = context.wrap_socket(cls.server.socket, server_side=True) + threading.Thread(target=cls.server.serve_forever, daemon=True).start() + cls.url = f"https://127.0.0.1:{cls.server.server_port}" + + @classmethod + def tearDownClass(cls): + cls.server.shutdown() + cls.server.server_close() + cls.temporary.cleanup() + + def setUp(self): + payload = b"signed rpm bytes" + (self.web / "packages").mkdir(exist_ok=True) + (self.web / "packages/loopwire.rpm").write_bytes(payload) + manifest = json.dumps({"target": {"distribution": "fedora", "release": "44", "architecture": "x86_64"}, + "revision": "a" * 64, "files": [{ + "path": "packages/loopwire.rpm", "size": len(payload), "sha256": hashlib.sha256(payload).hexdigest()}]}) + (self.root / "repository-manifest.json").write_text(manifest) + (self.web / "repository-manifest.json").write_text(manifest) + self.output = self.root / "channel.json" + self.output.unlink(missing_ok=True) + + def invoke(self, *extra, verifier_error=None): + args = [str(SCRIPT), "--repository", str(self.root), "--public-key", str(self.root / "key.asc"), + "--fingerprint", "A" * 40, "--base-url", self.url] + if "--output" not in extra: + args += ["--ca-file", str(self.cert)] + args += extra + trust = ssl.create_default_context(cafile=str(self.cert)) + with patch.object(sys, "argv", args), patch.object(module.subprocess, "run") as verifier, \ + patch.object(module.ssl, "create_default_context", return_value=trust), \ + contextlib.redirect_stdout(io.StringIO()) as output: + if verifier_error: + verifier.side_effect = verifier_error + module.main() + verifier.assert_called_once() + self.assertTrue(verifier.call_args.kwargs["check"]) + return json.loads(output.getvalue()) + + def test_exact_public_bytes_produce_fedora_record(self): + result = self.invoke("--output", str(self.output), "--proof-url", + "https://github.com/sandwichfarm/loopwire/actions/runs/123") + self.assertEqual(result["files"], 2) + record = json.loads(self.output.read_text()) + self.assertEqual(record["target"], "fedora-44") + self.assertEqual(record["baseUrl"], self.url) + + def test_package_or_manifest_tamper_fails(self): + for path in [self.web / "packages/loopwire.rpm", self.web / "repository-manifest.json"]: + with self.subTest(path=path.name): + before = path.read_bytes() + path.write_bytes(b"tampered") + with self.assertRaises(ValueError): + self.invoke() + path.write_bytes(before) + + def test_local_signature_failure_prevents_network(self): + with self.assertRaises(subprocess.CalledProcessError): + self.invoke(verifier_error=subprocess.CalledProcessError(1, "verify")) + + def test_custom_ca_cannot_activate(self): + with self.assertRaisesRegex(ValueError, "custom-CA fixture"): + self.invoke("--output", str(self.output), "--ca-file", str(self.cert), "--proof-url", + "https://github.com/sandwichfarm/loopwire/actions/runs/123") + + def test_invalid_url_fingerprint_and_proof_fail(self): + for args in [("--base-url", "http://example.invalid"), ("--fingerprint", "short"), + ("--output", str(self.output), "--proof-url", "https://example.invalid/run/1")]: + with self.subTest(args=args), self.assertRaises(ValueError): + self.invoke(*args) + + def test_wrong_target_fails(self): + manifest = json.loads((self.root / "repository-manifest.json").read_text()) + manifest["target"] = {"distribution": "opensuse", "release": "tumbleweed", "architecture": "x86_64"} + (self.root / "repository-manifest.json").write_text(json.dumps(manifest)) + with self.assertRaisesRegex(ValueError, "Fedora"): + self.invoke() + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test-rpm-repository.py b/scripts/test-rpm-repository.py new file mode 100644 index 0000000..750f9fb --- /dev/null +++ b/scripts/test-rpm-repository.py @@ -0,0 +1,459 @@ +#!/usr/bin/env python3 +"""Credential-free Fedora repository tests with real RPM, GPG, createrepo, and DNF. + +Run in the pinned tools image built from packaging/repositories/Dockerfile.rpm-tools. +All keys, packages, repositories, web servers, DNF state, and RPM databases are +temporary. No host repository configuration or production credentials are used. +""" + +import functools +import hashlib +import http.server +import json +import os +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import threading +import time +import unittest + + +SCRIPT = Path(__file__).with_name("rpm-repository.py") + + +def run(*args, ok=True, **kwargs): + result = subprocess.run([str(argument) for argument in args], capture_output=True, text=True, **kwargs) + if ok and result.returncode: + raise AssertionError(f"{args!r}\n{result.stdout}\n{result.stderr}") + return result + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +class QuietHandler(http.server.SimpleHTTPRequestHandler): + def log_message(self, *_args): + pass + + +class RepositoryTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + for tool in ( + "createrepo_c", "dnf", "gpg", "gpgv", "openssl", "rpm", "rpmbuild", "rpmkeys", "rpmsign", + ): + if not shutil.which(tool): + raise RuntimeError(f"{tool} required; run tests in Dockerfile.rpm-tools") + cls.temporary = tempfile.TemporaryDirectory(prefix="loopwire-rpm-tests-") + cls.root = Path(cls.temporary.name) + cls.root.chmod(0o755) + cls.gnupg = cls.root / "gnupg" + cls.gnupg.mkdir(mode=0o700) + cls.fingerprints = [] + for identity in ("Loopwire RPM Test", "Wrong RPM Test"): + run( + "gpg", "--homedir", cls.gnupg, "--batch", "--pinentry-mode", "loopback", + "--passphrase", "", "--quick-generate-key", identity, "rsa2048", "sign", "0", + ) + listing = run("gpg", "--homedir", cls.gnupg, "--with-colons", "--list-keys", identity).stdout + cls.fingerprints.append(next( + line.split(":")[9] for line in listing.splitlines() if line.startswith("fpr:") + )) + cls.fingerprint, cls.wrong_fingerprint = cls.fingerprints + cls.key = cls.root / "repository.asc" + cls.key.write_text(run( + "gpg", "--homedir", cls.gnupg, "--armor", "--export", cls.fingerprint, + ).stdout) + cls.wrong_key = cls.root / "wrong.asc" + cls.wrong_key.write_text(run( + "gpg", "--homedir", cls.gnupg, "--armor", "--export", cls.wrong_fingerprint, + ).stdout) + cls.release_private = cls.root / "release-private.pem" + cls.release_public = cls.root / "release-public.pem" + run( + "openssl", "genpkey", "-algorithm", "RSA", "-pkeyopt", "rsa_keygen_bits:2048", + "-out", cls.release_private, + ) + run("openssl", "pkey", "-in", cls.release_private, "-pubout", "-out", cls.release_public) + cls.date = int(time.time()) + cls.release1 = cls.make_release("1.0.0") + cls.release2 = cls.make_release("1.1.0") + cls.base = cls.root / "base" + cls.build(cls.release1, "1.0.0", cls.base) + + @classmethod + def tearDownClass(cls): + run("gpgconf", "--homedir", cls.gnupg, "--kill", "gpg-agent", ok=False) + cls.temporary.cleanup() + + @classmethod + def make_rpm( + cls, version, *, name="loopwire", release="1.fc44", architecture="x86_64", suffix="", + ): + fixture = cls.root / f"rpm-{version}-{name}-{release}-{architecture}{suffix}" + top = fixture / "rpmbuild" + for directory in ("BUILD", "BUILDROOT", "RPMS", "SOURCES", "SPECS", "SRPMS"): + (top / directory).mkdir(parents=True) + spec = top / "SPECS/fixture.spec" + spec.write_text( + f"Name: {name}\nVersion: {version}\nRelease: {release}\n" + "Summary: Loopwire repository test fixture\nLicense: MIT\n" + f"BuildArch: {architecture}\n\n" + "%description\nRepository fixture.\n\n%prep\n\n%build\n\n" + "%install\nmkdir -p %{buildroot}/usr/share/loopwire\n" + f"printf '%s\\n' '{version}{suffix}' > %{{buildroot}}/usr/share/loopwire/fixture\n\n" + "%files\n/usr/share/loopwire/fixture\n", + ) + run( + "rpmbuild", "--define", f"_topdir {top}", "--define", "_buildhost fixture.invalid", + "--define", f"_source_date_epoch {cls.date}", "--define", "use_source_date_epoch_as_buildtime 1", + "-bb", spec, + ) + packages = list((top / "RPMS").glob("**/*.rpm")) + if len(packages) != 1: + raise AssertionError(f"expected one RPM fixture, got {packages}") + return packages[0] + + @classmethod + def make_release( + cls, version, *, name="loopwire", release="1.fc44", architecture="x86_64", suffix="", + ): + directory = cls.root / f"release-{version}-{name}-{release}-{architecture}{suffix}" + directory.mkdir() + built = cls.make_rpm( + version, name=name, release=release, architecture=architecture, suffix=suffix, + ) + filename = f"loopwire-{version}-1.fc44.x86_64.rpm" + shutil.copyfile(built, directory / filename) + cls.sign_release(directory) + return directory + + @classmethod + def sign_release(cls, release): + packages = sorted(release.glob("*.rpm")) + (release / "SHA256SUMS").write_text("".join( + f"{digest(package)} {package.name}\n" for package in packages + )) + run( + "openssl", "dgst", "-sha256", "-sign", cls.release_private, + "-out", release / "SHA256SUMS.sig", release / "SHA256SUMS", + ) + + @classmethod + def build(cls, release, version, output, *extra, ok=True): + return run( + sys.executable, SCRIPT, "build", "--release-dir", release, "--version", version, + "--output", output, "--signing-key", cls.fingerprint, "--gnupg-home", cls.gnupg, + "--release-public-key", cls.release_public, "--date", cls.date, *extra, ok=ok, + ) + + def setUp(self): + self.case_dir = self.root / self.id().split(".")[-1] + self.case_dir.mkdir(mode=0o755) + self.repo = self.case_dir / "repository" + shutil.copytree(self.base, self.repo) + + def verify(self, *extra, ok=True, key=None, fingerprint=None): + return run( + sys.executable, SCRIPT, "verify", "--repository", self.repo, + "--public-key", key or self.key, "--fingerprint", fingerprint or self.fingerprint, + *extra, ok=ok, + ) + + def rewrite_manifest(self, update_packages=False): + path = self.repo / "repository-manifest.json" + manifest = json.loads(path.read_text()) + for entry in manifest["files"]: + file = self.repo / entry["path"] + if file.is_file(): + entry.update(sha256=digest(file), size=file.stat().st_size) + if update_packages: + for package in manifest["packages"]: + file = self.repo / package["path"] + package.update(distributedSha256=digest(file), size=file.stat().st_size) + manifest.pop("revision", None) + encoded = json.dumps(manifest, sort_keys=True, separators=(",", ":")).encode() + manifest["revision"] = hashlib.sha256(encoded).hexdigest() + path.write_text(json.dumps(manifest)) + + def dnf_download(self, package="loopwire", *, key_url=None): + handler = functools.partial(QuietHandler, directory=str(self.repo)) + server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + state = self.case_dir / f"dnf-{time.time_ns()}" + repos = state / "repos" + downloads = state / "downloads" + for directory in (state, repos, downloads, state / "cache", state / "persist", state / "log"): + directory.mkdir(exist_ok=True) + base = f"http://127.0.0.1:{server.server_port}" + (repos / "loopwire.repo").write_text( + "[loopwire]\nname=Loopwire test\nenabled=1\n" + f"baseurl={base}/\ngpgkey={key_url or base + f'/keys/{self.fingerprint}.asc'}\n" + "gpgcheck=1\nrepo_gpgcheck=1\nmetadata_expire=0\nsslverify=1\n" + ) + command = [ + "dnf", "--assumeyes", "--setopt", f"reposdir={repos}", + "--setopt", f"cachedir={state / 'cache'}", "--setopt", f"persistdir={state / 'persist'}", + "--setopt", f"logdir={state / 'log'}", "--setopt", "optional_metadata_types=", + "--repo", "loopwire", "download", "--from-repo", "loopwire", + "--destdir", downloads, package, + ] + try: + return run(*command, ok=False), downloads + finally: + server.shutdown() + server.server_close() + thread.join() + + def test_signed_chain_and_real_dnf_download(self): + source = self.release1 / "loopwire-1.0.0-1.fc44.x86_64.rpm" + source_before = digest(source) + summary = json.loads(self.verify().stdout) + self.assertEqual(summary["signingFingerprint"], self.fingerprint) + self.assertEqual(summary["target"], { + "distribution": "fedora", "release": "44", "architecture": "x86_64", + }) + package = summary["packages"][0] + self.assertEqual(package["sourceReleaseSha256"], source_before) + self.assertNotEqual(package["distributedSha256"], source_before) + self.assertEqual(digest(source), source_before, "generator mutated the source release RPM") + result, downloads = self.dnf_download("loopwire-1.0.0-1.fc44.x86_64") + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + downloaded = downloads / "loopwire-1.0.0-1.fc44.x86_64.rpm" + self.assertEqual(digest(downloaded), package["distributedSha256"]) + + def test_retention_version_order_and_fresh_rollback(self): + upgraded = self.case_dir / "upgraded" + self.build(self.release2, "1.1.0", upgraded, "--previous", self.base) + old = json.loads((self.base / "repository-manifest.json").read_text()) + new = json.loads((upgraded / "repository-manifest.json").read_text()) + for entry in old["files"]: + if entry["kind"] == "immutable": + self.assertEqual(digest(upgraded / entry["path"]), entry["sha256"]) + self.assertEqual([package["version"] for package in new["packages"]], ["1.0.0", "1.1.0"]) + failed = self.build( + self.release1, "1.0.0", self.case_dir / "downgrade", "--previous", upgraded, ok=False, + ) + self.assertNotEqual(failed.returncode, 0) + rollback = self.case_dir / "rollback" + run( + sys.executable, SCRIPT, "rollback", "--repository", self.base, "--output", rollback, + "--signing-key", self.fingerprint, "--gnupg-home", self.gnupg, + "--date", self.date + 60, + ) + rolled = json.loads((rollback / "repository-manifest.json").read_text()) + self.assertEqual(rolled["packages"], old["packages"]) + self.assertEqual(rolled["createdAt"], self.date + 60) + self.assertNotEqual(rolled["revision"], old["revision"]) + run( + sys.executable, SCRIPT, "verify", "--repository", rollback, + "--public-key", self.key, "--fingerprint", self.fingerprint, "--now", self.date + 60, + ) + + def test_release_signature_checksum_and_extra_rpm_rejected(self): + release = self.case_dir / "release" + shutil.copytree(self.release1, release) + (release / "SHA256SUMS.sig").write_bytes(b"invalid") + self.assertNotEqual(self.build(release, "1.0.0", self.case_dir / "bad", ok=False).returncode, 0) + self.sign_release(release) + checksums = release / "SHA256SUMS" + checksums.write_text(checksums.read_text() * 2) + run( + "openssl", "dgst", "-sha256", "-sign", self.release_private, + "-out", release / "SHA256SUMS.sig", checksums, + ) + self.assertNotEqual(self.build( + release, "1.0.0", self.case_dir / "duplicate", ok=False, + ).returncode, 0) + self.sign_release(release) + shutil.copyfile(next(release.glob("*.rpm")), release / "other.rpm") + self.sign_release(release) + self.assertNotEqual(self.build( + release, "1.0.0", self.case_dir / "extra", ok=False, + ).returncode, 0) + (release / "other.rpm").unlink() + shutil.copyfile(next(release.glob("loopwire-1.0.0-1.fc44.x86_64.rpm")), + release / "loopwire-1.0.0-1.x86_64.rpm") + self.sign_release(release) + self.build(release, "1.0.0", self.case_dir / "known-sibling") + + def test_deterministic_candidate_and_build_metadata_upgrade(self): + second = self.case_dir / "second" + self.build(self.release1, "1.0.0", second) + self.assertEqual( + (second / "repository-manifest.json").read_bytes(), + (self.base / "repository-manifest.json").read_bytes(), + ) + for entry in json.loads((self.base / "repository-manifest.json").read_text())["files"]: + self.assertEqual(digest(second / entry["path"]), entry["sha256"]) + release = self.make_release("1.0.0+rpmfixture1") + upgraded = self.case_dir / "upgraded" + self.build(release, "1.0.0+rpmfixture1", upgraded, "--previous", self.base) + self.repo = upgraded + result, downloads = self.dnf_download("loopwire") + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertTrue((downloads / "loopwire-1.0.0+rpmfixture1-1.fc44.x86_64.rpm").is_file()) + + def test_encrypted_signing_key_uses_protected_passphrase_file(self): + home = self.root / "encrypted-gnupg" + home.mkdir(mode=0o700) + passphrase = self.case_dir / "passphrase" + passphrase.write_text("fixture passphrase with spaces\n") + passphrase.chmod(0o600) + try: + run( + "gpg", "--homedir", home, "--batch", "--pinentry-mode", "loopback", + "--passphrase-file", passphrase, "--quick-generate-key", + "Encrypted RPM Fixture", "rsa2048", "sign", "0", + ) + listing = run("gpg", "--homedir", home, "--with-colons", "--list-keys").stdout + identity = next(line.split(":")[9] for line in listing.splitlines() if line.startswith("fpr:")) + run("gpgconf", "--homedir", home, "--kill", "gpg-agent") + output = self.case_dir / "encrypted" + self.build( + self.release1, "1.0.0", output, "--gnupg-home", home, + "--signing-key", identity, "--passphrase-file", passphrase, + "--date", int(time.time()) + 1, + ) + run( + sys.executable, SCRIPT, "verify", "--repository", output, + "--public-key", output / f"keys/{identity}.asc", "--fingerprint", identity, + ) + passphrase.chmod(0o644) + self.assertNotEqual(self.build( + self.release2, "1.1.0", self.case_dir / "weak-secret", + "--gnupg-home", home, "--signing-key", identity, + "--passphrase-file", passphrase, ok=False, + ).returncode, 0) + finally: + run("gpgconf", "--homedir", home, "--kill", "gpg-agent", ok=False) + + def test_package_name_version_architecture_release_and_repack_rejected(self): + variants = [ + (dict(name="other"), "other"), + (dict(architecture="noarch"), "architecture"), + (dict(release="1.fc43"), "release"), + ] + for options, suffix in variants: + release = self.make_release("1.2.0", suffix=suffix, **options) + self.assertNotEqual(self.build( + release, "1.2.0", self.case_dir / suffix, ok=False, + ).returncode, 0) + self.assertNotEqual(self.build( + self.release1, "1.0.1", self.case_dir / "version-mismatch", ok=False, + ).returncode, 0) + repack = self.make_release("1.0.0", suffix="repack") + self.assertNotEqual(self.build( + repack, "1.0.0", self.case_dir / "repack", "--previous", self.base, ok=False, + ).returncode, 0) + + def test_wrong_repository_signer_rejected_by_verifier_and_dnf(self): + self.assertNotEqual(self.verify( + ok=False, fingerprint=self.wrong_fingerprint, + ).returncode, 0) + self.assertNotEqual(self.verify(ok=False, key=self.wrong_key).returncode, 0) + result, _downloads = self.dnf_download( + key_url=self.wrong_key.resolve().as_uri(), + ) + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + + def test_repomd_tampering_and_missing_signature_rejected(self): + repomd = self.repo / "repodata/repomd.xml" + repomd.write_text(repomd.read_text().replace("Fedora 44", "Forged 44")) + self.rewrite_manifest() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + result, _downloads = self.dnf_download() + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + (self.repo / "repodata/repomd.xml.asc").unlink() + result, _downloads = self.dnf_download() + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + + def test_modified_or_unsigned_rpm_rejected_by_verifier_and_dnf(self): + package = next(self.repo.glob("packages/*.rpm")) + package.write_bytes(package.read_bytes() + b"tampered") + self.rewrite_manifest(update_packages=True) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + result, _downloads = self.dnf_download("loopwire-1.0.0-1.fc44.x86_64") + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + shutil.copytree(self.base, self.repo, dirs_exist_ok=True) + package = next(self.repo.glob("packages/*.rpm")) + run("rpmsign", "--delsign", package) + self.rewrite_manifest(update_packages=True) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + + def test_tampered_or_incomplete_metadata_and_previous_snapshot_rejected(self): + primary = next(self.repo.glob("repodata/*-primary.xml.gz")) + primary.write_bytes(primary.read_bytes() + b"tampered") + self.rewrite_manifest() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + self.assertNotEqual(self.build( + self.release2, "1.1.0", self.case_dir / "invalid-previous", + "--previous", self.repo, ok=False, + ).returncode, 0) + shutil.copytree(self.base, self.repo, dirs_exist_ok=True) + next(self.repo.glob("repodata/*-filelists.xml.gz")).unlink() + self.assertNotEqual(self.verify(ok=False).returncode, 0) + + def test_expired_future_and_forged_validity_rejected(self): + self.assertNotEqual(self.verify("--now", self.date + 31 * 86400, ok=False).returncode, 0) + self.assertNotEqual(self.verify("--now", self.date - 60, ok=False).returncode, 0) + manifest = json.loads((self.repo / "repository-manifest.json").read_text()) + manifest["validUntil"] += 365 * 86400 + manifest.pop("revision") + manifest["revision"] = hashlib.sha256(json.dumps( + manifest, sort_keys=True, separators=(",", ":"), + ).encode()).hexdigest() + (self.repo / "repository-manifest.json").write_text(json.dumps(manifest)) + self.assertNotEqual(self.verify("--now", self.date + 31 * 86400, ok=False).returncode, 0) + + def test_path_classification_extra_file_and_manifest_duplicates_rejected(self): + path = self.repo / "repository-manifest.json" + original = path.read_text() + for replacement in ("../../outside", "/absolute", "repodata/../secret", "packages//double"): + manifest = json.loads(original) + manifest["files"][0]["path"] = replacement + path.write_text(json.dumps(manifest)) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + path.write_text(original) + manifest = json.loads(original) + manifest["files"][0]["kind"] = "metadata" + manifest.pop("revision") + manifest["revision"] = hashlib.sha256(json.dumps( + manifest, sort_keys=True, separators=(",", ":"), + ).encode()).hexdigest() + path.write_text(json.dumps(manifest)) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + path.write_text(original) + (self.repo / "unadvertised").write_text("extra") + self.assertNotEqual(self.verify(ok=False).returncode, 0) + (self.repo / "unadvertised").unlink() + path.write_text(original.replace( + '"schema": "loopwire.rpm-repository.v1",', + '"schema": "duplicate",\n "schema": "loopwire.rpm-repository.v1",', + )) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + + def test_symlink_hardlink_and_output_reuse_rejected(self): + package = next(self.repo.glob("packages/*.rpm")) + original = package.read_bytes() + package.unlink() + package.symlink_to(self.base / package.relative_to(self.repo)) + self.assertNotEqual(self.verify(ok=False).returncode, 0) + package.unlink() + package.write_bytes(original) + os.link(package, self.case_dir / "hardlink") + self.assertNotEqual(self.verify(ok=False).returncode, 0) + output = self.case_dir / "exists" + output.mkdir() + self.assertNotEqual(self.build( + self.release2, "1.1.0", output, ok=False, + ).returncode, 0) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/scripts/verify-docs.sh b/scripts/verify-docs.sh index 713dae8..298475c 100644 --- a/scripts/verify-docs.sh +++ b/scripts/verify-docs.sh @@ -7,6 +7,7 @@ required_files=( "apps/docs/docs/index.md" "apps/docs/docs/guide/install.md" "apps/docs/docs/guide/apt-repository.md" + "apps/docs/docs/guide/fedora-repository.md" "apps/docs/docs/guide/basic-usage.md" "apps/docs/docs/guide/start-on-boot.md" "apps/docs/docs/guide/backends.md" @@ -19,6 +20,7 @@ required_files=( "apps/docs/docs/developer/vm-matrix.md" "apps/docs/docs/developer/release.md" "apps/docs/docs/developer/apt-repository.md" + "apps/docs/docs/developer/fedora-repository.md" "apps/docs/docs/developer/release-notes.md" "apps/docs/docs/release-notes/0.1.0.md" "apps/docs/docs/release-notes/unreleased.md" @@ -67,10 +69,16 @@ assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/troubleshooting" assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/basic-usage" assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/apt-repository" assert_contains "apps/docs/docs/.vitepress/config.ts" "/developer/apt-repository" +assert_contains "apps/docs/docs/.vitepress/config.ts" "/guide/fedora-repository" +assert_contains "apps/docs/docs/.vitepress/config.ts" "/developer/fedora-repository" assert_contains "apps/docs/docs/guide/apt-repository.md" "Signed-By" assert_contains "apps/docs/docs/guide/apt-repository.md" "--allow-downgrades" assert_contains "apps/docs/docs/developer/apt-repository.md" "APT_REPOSITORY_ENABLED" assert_contains "apps/docs/docs/developer/apt-repository.md" "Final activation is a human operation" +assert_contains "apps/docs/docs/guide/fedora-repository.md" "repo_gpgcheck=1" +assert_contains "apps/docs/docs/guide/fedora-repository.md" "sudo dnf downgrade loopwire" +assert_contains "apps/docs/docs/developer/fedora-repository.md" "FEDORA_REPOSITORY_ENABLED" +assert_contains "apps/docs/docs/developer/fedora-repository.md" "Final activation is a human operation" assert_contains "apps/docs/docs/.vitepress/config.ts" 'base: "/docs/"' assert_contains "apps/docs/docs/.vitepress/config.ts" "/developer/release-notes" assert_contains "apps/docs/docs/.vitepress/config.ts" "/release-notes/0.1.0" diff --git a/scripts/verify-fedora-repository-vm-proof.mjs b/scripts/verify-fedora-repository-vm-proof.mjs new file mode 100755 index 0000000..04e9111 --- /dev/null +++ b/scripts/verify-fedora-repository-vm-proof.mjs @@ -0,0 +1,456 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { lstat, mkdtemp, readFile, readdir, realpath, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const repositoryRoot = path.dirname(path.dirname(fileURLToPath(import.meta.url))); +const requiredPaths = [ + "/usr/bin/loopwire", "/usr/bin/loopwire-dsp-provider", "/usr/bin/loopwire-jack-ports", + "/usr/bin/loopwire-detect-audio", "/usr/lib/loopwire/loopwire-gui", + "/usr/share/applications/loopwire.desktop", "/usr/share/icons/hicolor/scalable/apps/loopwire.svg", +]; + +function requireThat(condition, message) { + if (!condition) throw new Error(message); +} +async function bytes(directory, name) { + const file = path.join(directory, name); + const stat = await lstat(file); + requireThat(stat.isFile() && !stat.isSymbolicLink(), `evidence must be a regular file: ${name}`); + return readFile(file); +} +async function text(directory, name, nonempty = true) { + const result = (await bytes(directory, name)).toString("utf8"); + requireThat(!nonempty || result.trim(), `empty evidence: ${name}`); + return result; +} +function tsvMap(value, label) { + const map = new Map(); + for (const line of value.trimEnd().split("\n")) { + const separator = line.indexOf("\t"); + requireThat(separator > 0, `${label} must contain key/value TSV`); + const key = line.slice(0, separator); + requireThat(!map.has(key), `${label} repeats ${key}`); + map.set(key, line.slice(separator + 1)); + } + return map; +} +function stageTable(value, label) { + const result = new Map(); + for (const line of value.trimEnd().split("\n")) { + const fields = line.split("\t"); + requireThat(fields.length === 3 && ["baseline", "upgraded"].includes(fields[0]), `invalid ${label} row`); + requireThat(!result.has(fields[0]), `${label} repeats ${fields[0]}`); + requireThat(/^loopwire-[0-9A-Za-z.+~_-]+-1\.fc44\.x86_64\.rpm$/.test(fields[1]), `invalid ${label} package name`); + requireThat(/^[a-f0-9]{64}$/.test(fields[2]), `invalid ${label} SHA-256`); + result.set(fields[0], { name: fields[1], sha256: fields[2] }); + } + requireThat(result.size === 2, `${label} must contain baseline and upgraded rows`); + return result; +} +function equal(actual, expected, label) { + requireThat(actual === expected, `${label}: expected ${JSON.stringify(expected)}, got ${JSON.stringify(actual)}`); +} +function command(program, args) { + const result = spawnSync(program, args, { encoding: "utf8", maxBuffer: 32 * 1024 * 1024 }); + requireThat(!result.error && result.status === 0, + `${program} verification failed: ${result.error?.message ?? result.stderr ?? result.stdout}`); + return result.stdout; +} +function sha256(buffer) { return createHash("sha256").update(buffer).digest("hex"); } + +export function parseReleaseChecksums(value) { + const entries = new Map(); + for (const [index, line] of value.trimEnd().split("\n").entries()) { + const match = /^([a-f0-9]{64}) {2}([^/\\\s]+)$/.exec(line); + requireThat(match && !entries.has(match[2]), `invalid or duplicate signed checksum entry at line ${index + 1}`); + entries.set(match[2], match[1]); + } + return entries; +} + +export function verifyReleaseSignature(checksumsFile, signatureFile, publicKeyFile) { + command("openssl", ["dgst", "-sha256", "-verify", publicKeyFile, "-signature", signatureFile, checksumsFile]); +} + +export function verifyReleaseAssetManifest(value, expected) { + const manifest = JSON.parse(value); + assert.deepEqual(Object.keys(manifest).sort(), ["artifacts", "release", "schema"], "release manifest fields"); + equal(manifest.schema, "loopwire.release-assets.v1", "release manifest schema"); + assert.deepEqual(Object.keys(manifest.release).sort(), ["gitHead", "tag", "version"], "release identity fields"); + equal(manifest.release.tag, `v${expected.version}`, "public release tag"); + equal(manifest.release.version, expected.version, "public release version"); + requireThat(/^[a-f0-9]{40}$/.test(manifest.release.gitHead), "public release commit must be a full lowercase hash"); + requireThat(Array.isArray(manifest.artifacts), "release artifacts must be an array"); + const fedora = manifest.artifacts.filter((entry) => entry && entry.target === "fedora-44"); + equal(fedora.length, 1, "Fedora release artifact count"); + assert.deepEqual(Object.keys(fedora[0]).sort(), ["architecture", "bytes", "kind", "name", "sha256", "target"], + "Fedora release artifact fields"); + assert.deepEqual(fedora[0], { name: expected.rpmName, kind: "native-rpm", target: "fedora-44", + architecture: "x86_64", bytes: expected.rpmBytes, sha256: expected.rpmSha256 }, "Fedora release artifact"); + const portable = manifest.artifacts.filter((entry) => entry?.name === "loopwire-linux-x86_64.tar.gz"); + equal(portable.length, 1, "x86_64 portable release artifact count"); + for (const [key, wanted] of Object.entries({ kind: "portable-archive", target: "linux-generic", architecture: "x86_64", + bytes: expected.tarBytes, sha256: expected.tarSha256 })) equal(portable[0][key], wanted, `portable release artifact ${key}`); + return manifest; +} + +export function parsePayloadRelease(value, expectedVersion) { + const fields = new Map(); + for (const line of value.trimEnd().split("\n")) { + const separator = line.indexOf("="); + requireThat(separator > 0, "invalid RELEASE field"); + const key = line.slice(0, separator); + requireThat(!fields.has(key), `duplicate RELEASE field: ${key}`); + fields.set(key, line.slice(separator + 1)); + } + assert.deepEqual([...fields.keys()].sort(), ["arch", "name", "source_date_epoch", "version"], "RELEASE fields"); + equal(fields.get("name"), "loopwire", "RELEASE name"); + equal(fields.get("version"), expectedVersion, "RELEASE version"); + equal(fields.get("arch"), "x86_64", "RELEASE architecture"); + requireThat(/^[0-9]+$/.test(fields.get("source_date_epoch") ?? ""), "RELEASE source date epoch"); + return fields; +} + +export function parseInstalledHashes(value) { + const result = {}; + for (const line of value.trimEnd().split("\n")) { + const match = /^([a-f0-9]{64}) {2}(\/usr\/[^\r\n]+)$/.exec(line); + requireThat(match && !match[2].split("/").includes(".."), "invalid installed-file checksum record"); + requireThat(!Object.hasOwn(result, match[2]), `duplicate installed path: ${match[2]}`); + result[match[2]] = match[1]; + } + return result; +} + +export function verifyRpmSignature(value, fingerprint, packageName) { + requireThat(value.includes(packageName), "RPM signature output names the wrong package"); + requireThat(!/NOT OK|NOKEY|NOTTRUSTED|BAD|FAILED|UNSIGNED/i.test(value), "RPM signature verification failed"); + const normalized = value.toLowerCase(); + const fullFingerprint = fingerprint.toLowerCase(); + const keyId = fingerprint.slice(-16).toLowerCase(); + const shortKeyId = fingerprint.slice(-8).toLowerCase(); + requireThat(/(?:OpenPGP[^\n]* )?RSA\/SHA(?:256|512) signature/i.test(value), + "RPM lacks an RSA OpenPGP signature"); + requireThat(normalized.includes(`key fingerprint: ${fullFingerprint}`) || + normalized.includes(`key id ${keyId}`) || normalized.includes(`key id ${shortKeyId}`), + "RPM signature uses the wrong key"); + requireThat(/Signature[^\n]*:\s*OK/i.test(value), "RPM signature was not accepted"); +} + +export async function rpmPayload(packageFile) { + const bsdtar = spawnSync("bsdtar", ["--version"], { encoding: "utf8" }); + if (!bsdtar.error && bsdtar.status === 0) { + const listed = command("bsdtar", ["-tf", packageFile]).trimEnd().split("\n"); + for (const name of listed) { + const normalized = name.replace(/^\.\//, ""); + requireThat(normalized.startsWith("usr/") && !normalized.split("/").includes(".."), "unsafe RPM payload path"); + } + const temporary = await mkdtemp(path.join(tmpdir(), "loopwire-rpm-payload-")); + try { + command("bsdtar", ["--no-same-owner", "--no-same-permissions", "-xf", packageFile, "-C", temporary]); + const files = {}; + async function collect(directory) { + for (const entry of await readdir(directory, { withFileTypes: true })) { + const item = path.join(directory, entry.name); + if (entry.isDirectory()) { + await collect(item); + continue; + } + if (!entry.isFile() && !entry.isSymbolicLink()) continue; + const resolved = await realpath(item); + requireThat(resolved.startsWith(`${temporary}${path.sep}`), "RPM payload link escapes extraction root"); + if (!(await lstat(resolved)).isFile()) continue; + const installed = `/${path.relative(temporary, item).split(path.sep).join("/")}`; + requireThat(!Object.hasOwn(files, installed), `duplicate RPM payload path: ${installed}`); + files[installed] = sha256(await readFile(item)); + } + } + await collect(path.join(temporary, "usr")); + return files; + } finally { + await rm(temporary, { recursive: true, force: true }); + } + } + const mount = path.dirname(packageFile); + return JSON.parse(command("bash", [path.join(repositoryRoot, "scripts/with-rpm-tools.sh"), + "--read-only-path", mount, "python3", "-c", String.raw` +import hashlib, json, pathlib, subprocess, sys, tempfile +package = pathlib.Path(sys.argv[1]) +listed = subprocess.run(['rpm', '-qlp', str(package)], check=True, capture_output=True, text=True).stdout.splitlines() +for name in listed: + pure = pathlib.PurePosixPath(name) + assert pure.is_absolute() and pure.parts[:2] == ('/', 'usr') and '..' not in pure.parts, 'unsafe package path' +with tempfile.TemporaryDirectory() as temporary: + root = pathlib.Path(temporary) + first = subprocess.Popen(['rpm2cpio', str(package)], stdout=subprocess.PIPE) + extracted = subprocess.run(['cpio', '--quiet', '-idm', '--no-absolute-filenames'], cwd=root, + stdin=first.stdout, capture_output=True, text=False) + first.stdout.close() + assert first.wait() == 0 and extracted.returncode == 0, extracted.stderr.decode(errors='replace') + files = {} + for name in listed: + item = root / name.lstrip('/') + if not item.is_file(): + continue + resolved = item.resolve(strict=True) + assert resolved.is_relative_to(root), 'package link escapes extraction root' + assert name not in files, 'duplicate package path' + files[name] = hashlib.sha256(item.read_bytes()).hexdigest() + print(json.dumps(files, sort_keys=True)) +`, packageFile])); +} + +export function verifyLifecycle(value, baselineVersion, upgradeVersion) { + equal(value.trimEnd(), [ + `install\t${baselineVersion}\tinstalled`, `reinstall\t${baselineVersion}\tinstalled`, + `upgrade\t${upgradeVersion}\tinstalled`, `rollback\t${baselineVersion}\tinstalled`, + `remove\t${baselineVersion}\tabsent`, + ].join("\n"), "lifecycle transitions"); +} + +export function verifyPackageEntry(entry, expected, packageSha256, sourceSha256, label) { + requireThat(entry && typeof entry === "object" && !Array.isArray(entry), `${label} package entry missing`); + equal(entry.name, "loopwire", `${label} package name`); + equal(entry.version, expected.version, `${label} package version`); + equal(entry.release, "1.fc44", `${label} package release`); + equal(entry.architecture, "x86_64", `${label} package architecture`); + equal(entry.path, `packages/${expected.name}`, `${label} package path`); + equal(entry.sourceReleaseSha256, sourceSha256, `${label} source release hash`); + equal(entry.distributedSha256, packageSha256, `${label} distributed RPM hash`); + requireThat(Number.isSafeInteger(entry.size) && entry.size > 0, `${label} package size missing`); +} + +export async function verifyInstalledStage(directory, stage, version, fingerprint, packageName, packageSha256, payloadHashes) { + const prefix = `${stage}/`; + equal((await text(directory, `${prefix}package-metadata.tsv`)).trim(), + `loopwire\t${version}\tx86_64`, `${stage} package metadata`); + equal((await text(directory, `${prefix}dnf-origin.txt`)).trim(), + `loopwire|${version}|x86_64|loopwire`, `${stage} repository origin`); + const info = await text(directory, `${prefix}dnf-info.txt`); + requireThat(/^From repository\s*:\s*loopwire$/m.test(info), `${stage} lacks DNF repository origin`); + const files = (await text(directory, `${prefix}package-files.txt`)).trim().split("\n"); + for (const installed of requiredPaths) requireThat(files.includes(installed), `${stage} missing ${installed}`); + const hashes = parseInstalledHashes(await text(directory, `${prefix}installed-files.sha256`)); + assert.deepEqual(hashes, payloadHashes, `${stage} installed bytes must equal the signed repository RPM payload`); + equal((await text(directory, `${prefix}signed-package.sha256`)).trim(), `${packageSha256} ${packageName}`, + `${stage} signed RPM digest`); + verifyRpmSignature(await text(directory, `${prefix}rpm-signature.txt`), fingerprint, packageName); + for (const help of ["background-help.txt", "dsp-provider-help.txt", "jack-provider-help.txt"]) { + await text(directory, `${prefix}${help}`); + } + const detection = JSON.parse(await text(directory, `${prefix}detect-audio.json`)); + requireThat(detection && typeof detection === "object", `${stage} detection must be JSON object/array`); + const linkage = await text(directory, `${prefix}gui-ldd.txt`); + requireThat(!linkage.includes("not found") && /libgtk-3/.test(linkage) && /libwebkit2gtk/.test(linkage), + `${stage} GUI linkage missing or unresolved`); + equal((await text(directory, `${prefix}gui-launch-status.txt`)).trim(), "0", `${stage} GUI launch`); + requireThat(/^\d+(?:\n\d+)*\n?$/.test(await text(directory, `${prefix}gui-window-ids.txt`)), `${stage} lacks X11 window ids`); + const names = (await text(directory, `${prefix}gui-window-names.txt`)).trim().split("\n"); + requireThat(names.every((name) => /^(Loopwire|loopwire-gui)$/.test(name)), `${stage} lacks Loopwire application window`); + const launch = await text(directory, `${prefix}gui-launch.log`, false); + requireThat(!/error while loading shared libraries|panic|protocol error|missing acquire timeline/i.test(launch), `${stage} fatal GUI log`); + await text(directory, `${prefix}xvfb.log`, false); +} + +export async function verifyEvidence({ target, evidenceDir, gitHead }) { + equal(target, "fedora-44", "supported target"); + requireThat(/^[a-f0-9]{40}$/.test(gitHead ?? ""), "--git-head must be a full lowercase commit hash"); + const summary = tsvMap(await text(evidenceDir, "summary.tsv"), "summary"); + equal(summary.get("schema"), "loopwire.fedora-repository-vm-proof.v1", "schema"); + equal(summary.get("target"), target, "target"); + equal(summary.get("git_head"), gitHead, "summary commit"); + equal((await text(evidenceDir, "git-head.txt")).trim(), gitHead, "evidence commit"); + equal(summary.get("payload_kind"), "public-release-baseline-with-synthetic-upgrade", "payload provenance kind"); + equal(summary.get("synthetic_upgrade"), "true", "synthetic fixture disclosure"); + const version = summary.get("version"); + requireThat(/^(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)(?:\+[0-9A-Za-z]+(?:\.[0-9A-Za-z]+)*)?$/.test(version ?? ""), "invalid baseline version"); + const upgradedVersion = `${version}${version.includes("+") ? "." : "+"}dnffixture1`; + equal(summary.get("upgrade_version"), upgradedVersion, "synthetic fixture version"); + const baselineVersion = `${version}-1.fc44`; + const upgradeVersion = `${upgradedVersion}-1.fc44`; + equal(summary.get("baseline_package_version"), baselineVersion, "baseline package version"); + equal(summary.get("upgrade_package_version"), upgradeVersion, "upgrade package version"); + const fingerprint = summary.get("fingerprint"); + requireThat(/^[A-F0-9]{40}$/.test(fingerprint ?? ""), "invalid signing fingerprint"); + const baseUrl = summary.get("base_url"); + equal(baseUrl, "https://127.0.0.1:8444/fedora/44/x86_64", "guest-only HTTPS origin"); + const epoch = summary.get("verification_epoch"); + requireThat(/^[0-9]{10}$/.test(epoch ?? "") && Number(epoch) <= Date.now() / 1000 + 300, "invalid proof timestamp"); + const os = new Map((await text(evidenceDir, "os-release")).split("\n").filter((line) => /^[A-Z_]+=/.test(line)).map((line) => { + const separator = line.indexOf("="); + return [line.slice(0, separator), line.slice(separator + 1).replace(/^"|"$/g, "")]; + })); + equal(os.get("ID"), "fedora", "guest OS"); + equal(os.get("VERSION_ID"), "44", "guest OS version"); + requireThat(["kvm", "qemu"].includes((await text(evidenceDir, "virtualization.txt")).trim()), "not a VM proof"); + requireThat((await text(evidenceDir, "uname.txt")).includes("Linux"), "guest kernel evidence missing"); + await text(evidenceDir, "console.log"); + const targetRows = (await readFile(path.join(repositoryRoot, "packaging/vm/native-package-targets.tsv"), "utf8")) + .split("\n").map((line) => line.split("\t")).filter((row) => row[0] === target); + requireThat(targetRows.length === 1, "target missing or duplicate in image manifest"); + const row = targetRows[0]; + const image = tsvMap(await text(evidenceDir, "image.tsv"), "image"); + for (const [key, expected] of Object.entries({ schema: "loopwire.native-package-image.v1", target, + distro: row[1], url: row[4], checksum_algorithm: row[5], checksum: row[6], actual_checksum: row[6], firmware: row[8] })) { + equal(image.get(key), expected, `image ${key}`); + } + equal((await text(evidenceDir, "initial-package-status.txt")).trim(), "absent", "clean guest status"); + requireThat(/^[a-f0-9]{64} {2}loopwire-linux-x86_64\.tar\.gz\n?$/.test(await text(evidenceDir, "release-payload.sha256")), + "missing original payload digest"); + const publicDirectory = path.join(evidenceDir, "public-release"); + const publicInventory = (await readdir(publicDirectory)).sort(); + assert.deepEqual(publicInventory, ["RELEASE", "SHA256SUMS", "SHA256SUMS.sig", + `loopwire-${baselineVersion}.x86_64.rpm`, "loopwire-linux-x86_64.tar.gz", "release-assets.json", + "release-signing-public.pem"].sort(), "public release evidence inventory"); + equal(await text(publicDirectory, "release-signing-public.pem"), + await readFile(path.join(repositoryRoot, "packaging/release-signing-public.pem"), "utf8"), "committed release signing key"); + verifyReleaseSignature(path.join(publicDirectory, "SHA256SUMS"), path.join(publicDirectory, "SHA256SUMS.sig"), + path.join(repositoryRoot, "packaging/release-signing-public.pem")); + const releaseChecksums = parseReleaseChecksums(await text(publicDirectory, "SHA256SUMS")); + const publicRpmName = `loopwire-${baselineVersion}.x86_64.rpm`; + const publicRpm = await bytes(publicDirectory, publicRpmName); + const publicTar = await bytes(publicDirectory, "loopwire-linux-x86_64.tar.gz"); + const publicManifest = await bytes(publicDirectory, "release-assets.json"); + for (const [name, content] of [[publicRpmName, publicRpm], ["loopwire-linux-x86_64.tar.gz", publicTar], + ["release-assets.json", publicManifest]]) { + requireThat(releaseChecksums.has(name), `signed checksums lack ${name}`); + equal(releaseChecksums.get(name), sha256(content), `signed public release hash for ${name}`); + } + const releaseManifest = verifyReleaseAssetManifest(publicManifest.toString("utf8"), { + version, rpmName: publicRpmName, rpmBytes: publicRpm.length, rpmSha256: sha256(publicRpm), + tarBytes: publicTar.length, tarSha256: sha256(publicTar), + }); + parsePayloadRelease(await text(publicDirectory, "RELEASE"), version); + equal(await text(evidenceDir, "payload-release.txt"), await text(publicDirectory, "RELEASE"), "captured RELEASE data"); + equal(summary.get("public_release_git_head"), releaseManifest.release.gitHead, "summary public release commit"); + equal((await text(evidenceDir, "public-release-git-head.txt")).trim(), releaseManifest.release.gitHead, + "captured public release commit"); + + const source = await text(evidenceDir, "loopwire.repo"); + for (const line of ["[loopwire]", `baseurl=${baseUrl}`, "enabled=1", "gpgcheck=1", "repo_gpgcheck=1", "sslverify=1", + `gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-loopwire-${fingerprint}`]) { + requireThat(source.split("\n").includes(line), `DNF source lacks ${line}`); + } + requireThat(!/sslverify\s*=\s*0|gpgcheck\s*=\s*0|repo_gpgcheck\s*=\s*0|skip_if_unavailable\s*=\s*True/i.test(source), + "DNF proof bypasses repository authentication or availability failures"); + equal(await text(evidenceDir, "https-key.asc"), await text(evidenceDir, "repository-key.asc"), "HTTPS public key"); + equal(await text(evidenceDir, "configured-repository-key.asc"), await text(evidenceDir, "repository-key.asc"), "configured public key"); + command("openssl", ["verify", "-attime", epoch, "-CAfile", path.join(evidenceDir, "tls-ca.crt"), + "-verify_ip", "127.0.0.1", path.join(evidenceDir, "tls-server.crt")]); + + const sources = stageTable(await text(evidenceDir, "release-sources.tsv"), "release sources"); + const signed = stageTable(await text(evidenceDir, "signed-packages.tsv"), "signed packages"); + const expectedNames = { + baseline: `loopwire-${baselineVersion}.x86_64.rpm`, + upgraded: `loopwire-${upgradeVersion}.x86_64.rpm`, + }; + equal(sources.get("baseline").sha256, sha256(publicRpm), "baseline source must be the public release RPM"); + equal(summary.get("baseline_source_sha256"), sha256(publicRpm), "summary public baseline source hash"); + for (const stage of ["baseline", "upgraded"]) { + equal(sources.get(stage).name, expectedNames[stage], `${stage} source RPM name`); + equal(signed.get(stage).name, expectedNames[stage], `${stage} signed RPM name`); + equal(summary.get(`${stage === "upgraded" ? "upgrade" : stage}_source_sha256`), sources.get(stage).sha256, + `${stage} summary source hash`); + equal(summary.get(`${stage === "upgraded" ? "upgrade" : stage}_rpm_sha256`), signed.get(stage).sha256, + `${stage} summary signed hash`); + equal(sha256(await bytes(evidenceDir, `packages/${expectedNames[stage]}`)), signed.get(stage).sha256, + `${stage} signed RPM evidence hash`); + } + const packageNames = (await readdir(path.join(evidenceDir, "packages"))).sort(); + assert.deepEqual(packageNames, ["baseline-rpm-signature.txt", expectedNames.baseline, + expectedNames.upgraded, "upgraded-rpm-signature.txt"].sort(), "package evidence inventory"); + verifyRpmSignature(await text(evidenceDir, "packages/baseline-rpm-signature.txt"), fingerprint, expectedNames.baseline); + verifyRpmSignature(await text(evidenceDir, "packages/upgraded-rpm-signature.txt"), fingerprint, expectedNames.upgraded); + const payloadHashes = { + [baselineVersion]: await rpmPayload(path.join(evidenceDir, "packages", expectedNames.baseline)), + [upgradeVersion]: await rpmPayload(path.join(evidenceDir, "packages", expectedNames.upgraded)), + }; + assert.deepEqual(await rpmPayload(path.join(publicDirectory, publicRpmName)), payloadHashes[baselineVersion], + "repository signing must preserve the public release RPM payload"); + + for (const stage of ["initial", "upgraded", "rolled-back"]) { + const directory = path.join(evidenceDir, "repositories", stage); + const result = command("bash", [path.join(repositoryRoot, "scripts/with-rpm-tools.sh"), "--read-only-path", evidenceDir, + "python3", path.join(repositoryRoot, "scripts/rpm-repository.py"), "verify", "--repository", directory, + "--public-key", path.join(evidenceDir, "repository-key.asc"), "--fingerprint", fingerprint, "--now", epoch]); + JSON.parse(result); + JSON.parse(await text(evidenceDir, `repositories/${stage}-verification.json`)); + const manifest = JSON.parse(await text(directory, "repository-manifest.json")); + equal(manifest.schema, "loopwire.rpm-repository.v1", `${stage} repository schema`); + equal(manifest.schemaVersion, 1, `${stage} repository schema version`); + assert.deepEqual(manifest.target, { distribution: "fedora", release: "44", architecture: "x86_64" }, + `${stage} repository target`); + const served = JSON.parse(await text(evidenceDir, `repositories/${stage}-public-verification.json`)); + equal(served.status, "verified", `${stage} HTTPS verification`); + equal(served.revision, manifest.revision, `${stage} HTTPS revision`); + equal(served.files, manifest.files.length + 1, `${stage} HTTPS file count including manifest`); + const expectedVersions = stage === "upgraded" ? [baselineVersion, upgradeVersion] : [baselineVersion]; + equal(manifest.packages.length, expectedVersions.length, `${stage} package count`); + for (const expectedVersion of expectedVersions) { + const fixtureStage = expectedVersion === baselineVersion ? "baseline" : "upgraded"; + const matches = manifest.packages.filter((entry) => entry.name === "loopwire" && + `${entry.version}-${entry.release}` === expectedVersion); + requireThat(matches.length === 1, `${stage} must contain exactly one ${expectedVersion} package`); + verifyPackageEntry(matches[0], { version: expectedVersion.replace(/-1\.fc44$/, ""), name: expectedNames[fixtureStage] }, + signed.get(fixtureStage).sha256, sources.get(fixtureStage).sha256, `${stage} ${expectedVersion}`); + } + if (stage !== "upgraded") requireThat(!manifest.packages.some((entry) => entry.version === upgradedVersion), + `${stage} unexpectedly advertises upgrade`); + } + + verifyLifecycle(await text(evidenceDir, "lifecycle.tsv"), baselineVersion, upgradeVersion); + for (const [stage, expectedVersion] of Object.entries({ + install: baselineVersion, reinstall: baselineVersion, upgrade: upgradeVersion, rollback: baselineVersion, + })) { + const fixtureStage = expectedVersion === baselineVersion ? "baseline" : "upgraded"; + await verifyInstalledStage(evidenceDir, stage, expectedVersion, fingerprint, expectedNames[fixtureStage], + signed.get(fixtureStage).sha256, payloadHashes[expectedVersion]); + const log = await text(evidenceDir, `${stage}.log`); + requireThat(log.includes("loopwire") && log.includes(expectedVersion), `${stage} lacks DNF operation/version log`); + } + const commands = await text(evidenceDir, "commands.log"); + for (const needle of ["dnf install -y loopwire-", "dnf reinstall -y", "dnf upgrade -y loopwire", + "dnf downgrade -y", "dnf remove -y loopwire", " -Kv ", "smoke_installed", "xdotool"]) { + requireThat(commands.includes(needle), `missing executed command: ${needle}`); + } + for (const file of ["bootstrap.log", "bootstrap-makecache.log", "upgrade-makecache.log", "rollback-makecache.log", + "remove.log", "source-removal.log", "source-removal-clean.log"]) await text(evidenceDir, file); + const requests = await text(evidenceDir, "https-server.log"); + for (const requested of [`/fedora/44/x86_64/keys/${fingerprint}.asc`, "/fedora/44/x86_64/repodata/repomd.xml", + "/fedora/44/x86_64/repodata/repomd.xml.asc", `/fedora/44/x86_64/packages/${expectedNames.baseline}`, + `/fedora/44/x86_64/packages/${expectedNames.upgraded}`]) { + requireThat(requests.includes(requested), `missing real HTTPS request: ${requested}`); + } + const removal = tsvMap(await text(evidenceDir, "removed-files.tsv"), "removed paths"); + for (const removed of [...requiredPaths.filter((value) => value !== "/usr/lib/loopwire/loopwire-gui"), "/usr/lib/loopwire"]) { + equal(removal.get(removed), "absent", `removed ${removed}`); + } + requireThat(!/(^|\s)loopwire(\s|$)/m.test(await text(evidenceDir, "source-removal-repositories.txt")), + "removed DNF source remains active"); + return { target, gitHead, baselineVersion, upgradeVersion, fingerprint, + packageHashes: { [baselineVersion]: signed.get("baseline").sha256, [upgradeVersion]: signed.get("upgraded").sha256 } }; +} + +if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { + const args = {}; + for (let index = 2; index < process.argv.length; index += 2) { + const option = process.argv[index]; + requireThat(["--target", "--evidence-dir", "--git-head"].includes(option) && process.argv[index + 1], `invalid option: ${option}`); + requireThat(!Object.hasOwn(args, option), `duplicate option: ${option}`); + args[option] = process.argv[index + 1]; + } + requireThat(args["--evidence-dir"], "--evidence-dir is required"); + const result = await verifyEvidence({ target: args["--target"], evidenceDir: path.resolve(args["--evidence-dir"]), gitHead: args["--git-head"] }); + console.log(`Fedora repository VM proof verified: ${result.target}`); + console.log(JSON.stringify(result)); + } catch (error) { + console.error(`verify-fedora-repository-vm-proof: ${error.message}`); + process.exitCode = 1; + } +} diff --git a/scripts/verify-github-workflows.sh b/scripts/verify-github-workflows.sh index 2c072d5..51e2f6a 100755 --- a/scripts/verify-github-workflows.sh +++ b/scripts/verify-github-workflows.sh @@ -96,6 +96,7 @@ fi workflows=( ".github/workflows/ci.yml" ".github/workflows/publish-apt.yml" + ".github/workflows/publish-fedora.yml" ".github/workflows/web.yml" ".github/workflows/aur.yml" ".github/workflows/workflow-checks.yml" @@ -337,6 +338,7 @@ assert_contains ".github/workflows/vm-matrix.yml" "apps/docs/docs/guide/support- ruby "$root/scripts/test-ci-impact.rb" ruby "$root/scripts/test-ci-workflow-paths.rb" ruby "$root/scripts/test-apt-workflow.rb" +ruby "$root/scripts/test-fedora-workflow.rb" node "$root/scripts/test-native-package-proof-snapshot.mjs" echo "GitHub workflow contract verification passed." diff --git a/scripts/verify-requirements.sh b/scripts/verify-requirements.sh index 7dce409..dbe75a9 100644 --- a/scripts/verify-requirements.sh +++ b/scripts/verify-requirements.sh @@ -124,8 +124,9 @@ done assert_script "package.json" "check" "pnpm check:verify && pnpm lint && pnpm typecheck && pnpm test && pnpm build && pnpm verify:site" assert_script "package.json" "check:verify" \ - "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri && pnpm verify:apt" + "pnpm verify:requirements && pnpm verify:docs && pnpm test:setup-github && pnpm verify:scripts && pnpm verify:workflows && pnpm verify:runtime && pnpm verify:tauri && pnpm verify:apt && pnpm verify:rpm-repository" assert_script "package.json" "verify:apt" "bash scripts/verify-apt-repository.sh" +assert_script "package.json" "verify:rpm-repository" "bash scripts/verify-rpm-repository.sh" assert_script "package.json" "verify:requirements" "bash scripts/verify-requirements.sh" assert_script "package.json" "setup:github" "node scripts/setup-github-actions.mjs" assert_script "package.json" "test:setup-github" "node scripts/test-setup-github-actions.mjs" diff --git a/scripts/verify-rpm-public.py b/scripts/verify-rpm-public.py new file mode 100755 index 0000000..6a167a0 --- /dev/null +++ b/scripts/verify-rpm-public.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python3 +"""Verify a served Fedora repository before producing its website activation record.""" +import argparse +from datetime import datetime, timezone +import hashlib +import json +from pathlib import Path +import re +import ssl +import subprocess +import sys +import tempfile +import urllib.error +import urllib.parse +import urllib.request + + +class NoRedirects(urllib.request.HTTPRedirectHandler): + def redirect_request(self, request, response, code, message, headers, new_url): + response.close() + raise ValueError("repository verification does not follow redirects; use the canonical HTTPS URL") + + +def validate_base_url(value): + url = urllib.parse.urlsplit(value) + if (url.scheme != "https" or not url.hostname or url.username or url.password + or any(char in value for char in "\\'\"`$<>?#") + or any(ord(char) <= 32 or ord(char) >= 127 for char in value)): + raise ValueError("base URL must be HTTPS without credentials, whitespace, query, fragment, or shell metacharacters") + if url.port is not None and not 1 <= url.port <= 65535: + raise ValueError("invalid HTTPS port in base URL") + return value.rstrip("/") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repository", required=True, type=Path) + parser.add_argument("--public-key", required=True, type=Path) + parser.add_argument("--fingerprint", required=True) + parser.add_argument("--base-url", required=True) + parser.add_argument("--ca-file", type=Path, help="custom CA for isolated test servers") + parser.add_argument("--proof-url", help="GitHub Actions run URL required for --output") + parser.add_argument("--output", type=Path, help="write verified channel configuration after all checks") + args = parser.parse_args() + base_url = validate_base_url(args.base_url) + if args.output and args.ca_file: + raise ValueError("custom-CA fixture checks cannot produce public activation records") + if args.output and (not args.proof_url or not re.fullmatch( + r"https://github\.com/sandwichfarm/loopwire/actions/runs/[1-9][0-9]*", args.proof_url)): + raise ValueError("activation output requires the verifying project GitHub Actions run URL") + fingerprint = args.fingerprint.upper() + if not re.fullmatch(r"[A-F0-9]{40}", fingerprint): + raise ValueError("a complete OpenPGP fingerprint is required") + subprocess.run([ + sys.executable, str(Path(__file__).with_name("rpm-repository.py")), "verify", + "--repository", str(args.repository), "--public-key", str(args.public_key), + "--fingerprint", fingerprint, + ], check=True, stdout=subprocess.PIPE) + manifest_path = args.repository / "repository-manifest.json" + manifest = json.loads(manifest_path.read_text()) + if manifest.get("target") != {"distribution": "fedora", "release": "44", "architecture": "x86_64"}: + raise ValueError("candidate does not target Fedora 44 x86_64") + manifest_bytes = manifest_path.read_bytes() + entries = [*manifest["files"], { + "path": "repository-manifest.json", "size": len(manifest_bytes), + "sha256": hashlib.sha256(manifest_bytes).hexdigest(), + }] + context = ssl.create_default_context(cafile=str(args.ca_file) if args.ca_file else None) + opener = urllib.request.build_opener(NoRedirects(), urllib.request.HTTPSHandler(context=context)) + for entry in entries: + url = base_url + "/" + urllib.parse.quote(entry["path"], safe="/+") + request = urllib.request.Request(url, headers={"Cache-Control": "no-cache", "User-Agent": "Loopwire-RPM-Proof/1"}) + try: + response = opener.open(request, timeout=30) + except urllib.error.HTTPError as error: + status = error.code + error.close() + raise ValueError(f"repository returned HTTP {status} for {entry['path']}") from None + digest, size = hashlib.sha256(), 0 + with response: + while chunk := response.read(1024 * 1024): + size += len(chunk) + if size > entry["size"]: + raise ValueError(f"public file is larger than expected: {entry['path']}") + digest.update(chunk) + if size != entry["size"] or digest.hexdigest() != entry["sha256"]: + raise ValueError(f"public file differs from the verified candidate: {entry['path']}") + record = { + "schemaVersion": 1, "status": "verified", "target": "fedora-44", + "baseUrl": base_url, "signingFingerprint": fingerprint, "revision": manifest["revision"], + "verifiedAt": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"), "proofUrl": args.proof_url, + } + if args.output: + args.output.parent.mkdir(parents=True, exist_ok=True) + with tempfile.NamedTemporaryFile(mode="w", dir=args.output.parent, delete=False) as temporary: + json.dump(record, temporary, indent=2) + temporary.write("\n") + temporary_path = Path(temporary.name) + temporary_path.replace(args.output) + print(json.dumps({"status": "verified", "revision": manifest["revision"], "files": len(entries)})) + + +if __name__ == "__main__": + try: + main() + except (OSError, ValueError, KeyError, subprocess.CalledProcessError, urllib.error.URLError) as error: + print(f"verify-rpm-public: {error}", file=sys.stderr) + sys.exit(1) diff --git a/scripts/verify-rpm-repository.sh b/scripts/verify-rpm-repository.sh new file mode 100755 index 0000000..9dec45d --- /dev/null +++ b/scripts/verify-rpm-repository.sh @@ -0,0 +1,20 @@ +#!/usr/bin/env bash +set -euo pipefail +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +if [ "${1:-}" != --inside ]; then + exec bash "$root/scripts/with-rpm-tools.sh" --container bash "$root/scripts/verify-rpm-repository.sh" --inside +fi +cd "$root" +export PYTHONDONTWRITEBYTECODE=1 +bash -n scripts/setup-fedora-repository.sh scripts/publish-fedora-workflow.sh \ + scripts/with-rpm-tools.sh packaging/vm/guest-fedora-repository-smoke.sh +node --check scripts/verify-fedora-repository-vm-proof.mjs +node --check scripts/test-fedora-repository-vm-proof.mjs +python3 scripts/test-rpm-repository.py +python3 scripts/test-publish-rpm-repository.py --with-ssh +python3 scripts/test-fedora-bootstrap.py +python3 scripts/test-rpm-public.py +python3 scripts/test-fedora-workflow-preflight.py +node scripts/test-fedora-repository-vm-proof.mjs +node --test apps/site/src/lib/rpmChannel.test.mjs +echo 'Fedora repository development verification passed.' diff --git a/scripts/verify-scripts.sh b/scripts/verify-scripts.sh index bc29593..270f81d 100755 --- a/scripts/verify-scripts.sh +++ b/scripts/verify-scripts.sh @@ -103,6 +103,10 @@ node --check scripts/verify-native-package-vm-proof.mjs node --check scripts/verify-native-package-proof-snapshot.mjs node --check scripts/verify-apt-repository-vm-proof.mjs node --check scripts/test-apt-repository-vm-proof.mjs +node --check scripts/verify-fedora-repository-vm-proof.mjs +node --check scripts/test-fedora-repository-vm-proof.mjs +node scripts/test-fedora-repository-vm-proof.mjs +bash -n packaging/vm/guest-fedora-repository-smoke.sh bash scripts/build-portable-linux-binary.sh -- --help | grep -Fq -- "--output FILE" || { echo "verify-scripts: portable builder does not accept the package-script separator" >&2 exit 1 diff --git a/scripts/with-rpm-tools.sh b/scripts/with-rpm-tools.sh new file mode 100755 index 0000000..31ce37d --- /dev/null +++ b/scripts/with-rpm-tools.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +set -euo pipefail + +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +image="${LOOPWIRE_RPM_TOOLS_IMAGE:-loopwire-rpm-tools:fedora-44}" +force_container=false +mounts=() +while [ "$#" -gt 0 ]; do + case "$1" in + --container) force_container=true; shift ;; + --read-only-path) + input="$(realpath -e "${2:?missing --read-only-path value}")" + mounts+=(--volume "$input:$input:ro") + shift 2 + ;; + *) break ;; + esac +done +[ "$#" -gt 0 ] || { echo 'Usage: with-rpm-tools.sh [--container] [--read-only-path PATH] COMMAND [ARG ...]' >&2; exit 2; } +available=true +for command in createrepo_c dnf gpg gpgv openssl python3 rpm rpmkeys rpmsign; do + command -v "$command" >/dev/null 2>&1 || available=false +done +export PYTHONDONTWRITEBYTECODE=1 +if [ "$available" = true ] && [ "$force_container" = false ]; then + exec "$@" +fi +command -v docker >/dev/null 2>&1 || { echo 'RPM repository tools or Docker are required; see the Fedora repository guide.' >&2; exit 1; } +if ! docker image inspect "$image" >/dev/null 2>&1; then + docker build --file "$root/packaging/repositories/Dockerfile.rpm-tools" --tag "$image" "$root" >&2 +fi +exec docker run --rm --network none --env PYTHONDONTWRITEBYTECODE=1 \ + --volume "$root:$root:ro" "${mounts[@]}" --workdir "$root" "$image" "$@" From 3e95cfeb41c17227bcca1b3dd4471edb8618020d Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:49:07 +0200 Subject: [PATCH 3/7] Keep Fedora signature proof isolated under RPM 6 RPM 6 requires root ownership for its database lock even when the database is outside the system path. Run only the disposable fixture database operations through sudo while preserving the host RPM database and proof output. Constraint: Signature verification must not import fixture keys into the guest system RPM database Confidence: high Scope-risk: narrow Tested: Guest Bash syntax, ShellCheck, 34 proof-verifier regressions and whitespace Not-tested: Clean Fedora KVM lifecycle rerun follows this commit --- packaging/vm/guest-fedora-repository-smoke.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/packaging/vm/guest-fedora-repository-smoke.sh b/packaging/vm/guest-fedora-repository-smoke.sh index 99999dc..a82e241 100755 --- a/packaging/vm/guest-fedora-repository-smoke.sh +++ b/packaging/vm/guest-fedora-repository-smoke.sh @@ -163,11 +163,11 @@ printf 'baseline\t%s\t%s\nupgraded\t%s\t%s\n' \ # Verify each distributed RPM against the repository key without changing the guest's system RPM database. fixture_rpmdb="$fixture_dir/rpmdb" -mkdir -p "$fixture_rpmdb" -rpm --dbpath "$fixture_rpmdb" --initdb -rpm --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" -rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$baseline_package" >"$proof_dir/packages/baseline-rpm-signature.txt" -rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$upgrade_package" >"$proof_dir/packages/upgraded-rpm-signature.txt" +sudo mkdir -p "$fixture_rpmdb" +sudo rpm --dbpath "$fixture_rpmdb" --initdb +sudo rpm --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" +sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$baseline_package" >"$proof_dir/packages/baseline-rpm-signature.txt" +sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$upgrade_package" >"$proof_dir/packages/upgraded-rpm-signature.txt" # A guest-only CA exercises real TLS verification; no production trust is imported. openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj '/CN=Loopwire disposable guest CA' \ From d6ff447b59c83a4e5398289a7126a5e04842162b Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:51:30 +0200 Subject: [PATCH 4/7] Place disposable RPM verification state under Fedora's allowed temp path Fedora's RPM policy rejects database locks below a user's home even for a root process. Use a unique root-owned /var/tmp database for the two offline signature checks and remove it immediately, without touching the system RPM DB. Constraint: Fedora RPM and SELinux policy controls usable database paths Confidence: high Scope-risk: narrow Directive: Keep fixture key imports isolated from the system RPM database Tested: Guest Bash syntax, ShellCheck, 34 proof mutations and whitespace Not-tested: Clean Fedora KVM lifecycle rerun follows this commit --- packaging/vm/guest-fedora-repository-smoke.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/packaging/vm/guest-fedora-repository-smoke.sh b/packaging/vm/guest-fedora-repository-smoke.sh index a82e241..82dd8a6 100755 --- a/packaging/vm/guest-fedora-repository-smoke.sh +++ b/packaging/vm/guest-fedora-repository-smoke.sh @@ -162,12 +162,14 @@ printf 'baseline\t%s\t%s\nupgraded\t%s\t%s\n' \ >"$proof_dir/signed-packages.tsv" # Verify each distributed RPM against the repository key without changing the guest's system RPM database. -fixture_rpmdb="$fixture_dir/rpmdb" -sudo mkdir -p "$fixture_rpmdb" +fixture_rpmdb="/var/tmp/loopwire-fedora-proof-rpmdb-${git_head}" +sudo rm -rf -- "$fixture_rpmdb" +sudo install -d -m 0700 "$fixture_rpmdb" sudo rpm --dbpath "$fixture_rpmdb" --initdb sudo rpm --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$baseline_package" >"$proof_dir/packages/baseline-rpm-signature.txt" sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$upgrade_package" >"$proof_dir/packages/upgraded-rpm-signature.txt" +sudo rm -rf -- "$fixture_rpmdb" # A guest-only CA exercises real TLS verification; no production trust is imported. openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj '/CN=Loopwire disposable guest CA' \ From 5b9c1c831b082c724fd873893b8498a4d7324cf7 Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:53:15 +0200 Subject: [PATCH 5/7] Let RPM 6 initialize its isolated verification database RPM 6 rejects a manually pre-created database directory under the Fedora guest policy. Import the disposable public key directly through rpmkeys so RPM owns initialization, verify both repository packages, and delete the database. Constraint: RPM 6 owns initialization semantics for alternate database paths Confidence: high Scope-risk: narrow Directive: Never point fixture rpmkeys operations at the system RPM database Tested: Direct Fedora 44 rpmkeys alternate-db import; ShellCheck; 34 proof mutations Not-tested: Clean Fedora KVM lifecycle rerun follows this commit --- packaging/vm/guest-fedora-repository-smoke.sh | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/packaging/vm/guest-fedora-repository-smoke.sh b/packaging/vm/guest-fedora-repository-smoke.sh index 82dd8a6..091b534 100755 --- a/packaging/vm/guest-fedora-repository-smoke.sh +++ b/packaging/vm/guest-fedora-repository-smoke.sh @@ -164,9 +164,7 @@ printf 'baseline\t%s\t%s\nupgraded\t%s\t%s\n' \ # Verify each distributed RPM against the repository key without changing the guest's system RPM database. fixture_rpmdb="/var/tmp/loopwire-fedora-proof-rpmdb-${git_head}" sudo rm -rf -- "$fixture_rpmdb" -sudo install -d -m 0700 "$fixture_rpmdb" -sudo rpm --dbpath "$fixture_rpmdb" --initdb -sudo rpm --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" +sudo rpmkeys --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$baseline_package" >"$proof_dir/packages/baseline-rpm-signature.txt" sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$upgrade_package" >"$proof_dir/packages/upgraded-rpm-signature.txt" sudo rm -rf -- "$fixture_rpmdb" From e73c5bb32da6ff74c7953bd0acc71376c98a65ea Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 16:55:53 +0200 Subject: [PATCH 6/7] Retain isolated RPM trust only for the guest lifecycle Lifecycle-stage signature checks reuse the disposable RPM database. Keep it until the guest finishes and delete it from the exit trap together with the HTTPS fixture, including on failures. Constraint: Every install stage rechecks the exact repository RPM signature Confidence: high Scope-risk: narrow Directive: Cleanup must remove the alternate database on every guest exit Tested: Bash syntax, ShellCheck, 34 proof mutations and whitespace Not-tested: Clean Fedora KVM lifecycle rerun follows this commit --- packaging/vm/guest-fedora-repository-smoke.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/packaging/vm/guest-fedora-repository-smoke.sh b/packaging/vm/guest-fedora-repository-smoke.sh index 091b534..b0f6250 100755 --- a/packaging/vm/guest-fedora-repository-smoke.sh +++ b/packaging/vm/guest-fedora-repository-smoke.sh @@ -163,11 +163,16 @@ printf 'baseline\t%s\t%s\nupgraded\t%s\t%s\n' \ # Verify each distributed RPM against the repository key without changing the guest's system RPM database. fixture_rpmdb="/var/tmp/loopwire-fedora-proof-rpmdb-${git_head}" +server_pid="" +cleanup() { + [ -z "$server_pid" ] || kill "$server_pid" 2>/dev/null || true + sudo rm -rf -- "$fixture_rpmdb" +} +trap cleanup EXIT sudo rm -rf -- "$fixture_rpmdb" sudo rpmkeys --dbpath "$fixture_rpmdb" --import "$proof_dir/repository-key.asc" sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$baseline_package" >"$proof_dir/packages/baseline-rpm-signature.txt" sudo rpm --dbpath "$fixture_rpmdb" -Kv "$proof_dir/packages/$upgrade_package" >"$proof_dir/packages/upgraded-rpm-signature.txt" -sudo rm -rf -- "$fixture_rpmdb" # A guest-only CA exercises real TLS verification; no production trust is imported. openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj '/CN=Loopwire disposable guest CA' \ @@ -212,8 +217,6 @@ PY python3 "$fixture_dir/https-server.py" "$fixture_dir/www" "$fixture_dir/tls.crt" "$fixture_dir/tls-key.pem" \ >"$proof_dir/https-server.log" 2>&1 & server_pid=$! -cleanup() { kill "$server_pid" 2>/dev/null || true; } -trap cleanup EXIT for _attempt in $(seq 1 20); do if curl --fail --silent --show-error "$base_url/keys/$fingerprint.asc" >"$proof_dir/https-key.asc"; then break; fi sleep 1 From 471e39dec8b0bab1c2e1a7d800ea0cfd5572432d Mon Sep 17 00:00:00 2001 From: sandwich Date: Sat, 5 Sep 2026 17:04:24 +0200 Subject: [PATCH 7/7] Preserve Fedora repository evidence and production boundary Map all nine development requirements to the signed package, publication, clean-guest, workflow and documentation evidence. Keep the five production operations explicitly human-owned before public activation. Constraint: The public release RPM is baseline proof; fixture signing and upgrade are not production material Confidence: high Scope-risk: narrow Tested: Final pnpm check and Fedora KVM lifecycle proof at e73c5bb Not-tested: Production origin and activation remain the issue's human tasks --- .planning/STATE.md | 5 +- .../260905-mhp-SUMMARY.md | 88 +++++++++++++++++++ 2 files changed, 91 insertions(+), 2 deletions(-) create mode 100644 .planning/quick/260905-mhp-signed-fedora/260905-mhp-SUMMARY.md diff --git a/.planning/STATE.md b/.planning/STATE.md index 5466819..9edbec6 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,7 +3,7 @@ gsd_state_version: 1.0 milestone: v0.5 milestone_name: GitHub Operator Setup status: Ready for Review -last_updated: "2026-09-05T14:08:23Z" +last_updated: "2026-09-05T15:03:31Z" last_activity: 2026-09-05 progress: total_phases: 1 @@ -27,7 +27,7 @@ See: .planning/PROJECT.md (updated 2026-07-03) Phase: 19 of 19 complete Plan: 19.1 — Hardened GitHub Actions Setup Status: Ready for review in PR #9 -Last activity: 2026-09-05 - completed quick task 260905-kyo: signed APT repository development and clean-guest proof +Last activity: 2026-09-05 - completed quick task 260905-mhp: signed Fedora repository development and clean-guest proof ## Blockers / Concerns @@ -94,6 +94,7 @@ Last activity: 2026-09-05 - completed quick task 260905-kyo: signed APT reposito | 260905-hia | Minimal landing identity, GSAP reactions and screenshot proofs in PR #40 | 2026-09-05 | 6e84a9a | [260905-hia-landing-identity](./quick/260905-hia-landing-identity/) | | 260905-i4l | Scope CI to affected files while retaining release validation | 2026-09-05 | f9bc0d8 | [260905-i4l-scope-ci](./quick/260905-i4l-scope-ci/) | | 260905-kyo | Signed APT repository development and clean-guest lifecycle proof | 2026-09-05 | 639cbbb | [260905-kyo-signed-apt](./quick/260905-kyo-signed-apt/) | +| 260905-mhp | Signed Fedora repository development and clean-guest lifecycle proof | 2026-09-05 | e73c5bb | [260905-mhp-signed-fedora](./quick/260905-mhp-signed-fedora/) | ## Accumulated Context diff --git a/.planning/quick/260905-mhp-signed-fedora/260905-mhp-SUMMARY.md b/.planning/quick/260905-mhp-signed-fedora/260905-mhp-SUMMARY.md new file mode 100644 index 0000000..e59efd4 --- /dev/null +++ b/.planning/quick/260905-mhp-signed-fedora/260905-mhp-SUMMARY.md @@ -0,0 +1,88 @@ +--- +status: complete +issue: 36 +depends_on: 45 +--- + +# Signed Fedora repository development + +Issue: https://github.com/sandwichfarm/loopwire/issues/36 + +## Result and stack + +The Fedora 44 x86_64 development work is complete. This branch intentionally stacks on #35/PR #45 to reuse its +reviewed SSH/POSIX publication, protected environment, activation gate, and KVM harness. The dedicated Fedora channel +record remains `pending`; the separate Human operational tasks still own production hosting, signing identity, +GitHub configuration, first public publication, and website activation. Until then the existing signed direct RPM and +automatic installer remain visible. + +## Development checklist evidence + +1. **Provider evaluation:** the maintainer runbook compares COPR and project-owned delivery across output provenance, + signing, Fedora targeting, promotion, proof, retention, and rollback. Project-owned was selected because it consumes + the exact project-authenticated release RPM, signs only a staged copy, controls publication and produces local/CI + proof. COPR output would be a separate provider build needing provider-specific evidence. +2. **Package path:** the generator accepts only Fedora 44 x86_64 `loopwire-VERSION-1.fc44.x86_64.rpm`, while allowing + the known signed openSUSE sibling in the real GitHub Release. It verifies the OpenSSL release signature, signed + checksum, RPM digest, NEVRA and public release manifest before repository processing. Source release and distributed + hashes are recorded separately; the source GitHub RPM is never mutated. +3. **Signing:** `rpmsign` applies an RSA/SHA-256 OpenPGP package signature to the staged repository copy. + `repodata/repomd.xml.asc` separately authenticates SHA-256 metadata objects. Both are verified using isolated RPM + and GnuPG databases pinned to the expected primary fingerprint. Passphrases travel only through protected files. +4. **Publication/rollback:** the Fedora publisher retains RPMs, fingerprint keys, checksum-named repodata, and private + snapshots indefinitely in v1. It validates before writes, locks the origin, requires revision CAS, rejects + immutable collisions, writes the new signature then atomically commits `repomd.xml`, journals every checkpoint, + and recovers interrupted or explicitly reviewed expired transactions. Real DNF fails closed during the brief mixed + signature/XML state and succeeds after recovery. Rollback freshly signs the retained package set. +5. **Protected automation:** Publish Fedora Repository uses a checksum-pinned Fedora 44 container and supports a + release call, manual publish/refresh/rollback, and weekly refresh. `FEDORA_REPOSITORY_ENABLED=true` plus the + `packages-production` environment gates all writes. Stable publication waits for the existing GitHub Release and + evidence gates, re-downloads public assets, publishes over pinned SSH, and verifies every HTTPS-served byte before + producing a reviewable activation record. +6. **Bootstrap:** the repeat-safe helper targets Fedora 44 x86_64, verifies the HTTPS key's full fingerprint, writes + only the managed `.repo` and fingerprint key file, and requires `gpgcheck=1`, `repo_gpgcheck=1`, `sslverify=1`, + `skip_if_unavailable=False`. Dry-run has no network/writes; removal preserves other repositories, installed packages + and RPM-database trust. Docs explain inspecting/removing previously accepted RPM keys during rotation or compromise. +7. **Regression tests:** real DNF accepts only valid package and repodata signatures and rejects wrong signers, + unsigned/tampered RPMs and changed metadata. Tests cover name/version/release/architecture, real release sibling + assets, version order, retention, fresh rollback, deterministic fixed-date candidates, encrypted keys, unsafe paths, + concurrent locks, CAS, every interruption checkpoint, permissions, actual SSH without remote GPG, Nginx cache/404 + headers and real DNF recovery from a mismatched signature/XML transition. +8. **Clean Fedora lifecycle:** a checksum-pinned Fedora 44 KVM guest consumes the actual public v0.1.0 Fedora RPM, + authenticated through the project release key, SHA256SUMS, release-assets manifest, public release commit and tar + RELEASE metadata. The repository-signed baseline is installed/reinstalled through DNF, upgraded to the explicitly + synthetic `+dnffixture1`, downgraded to the public baseline, removed, and its repository removed. Proof binds DNF + origin, embedded signatures, source/distributed hashes, installed `/usr` bytes, providers, backend JSON, GUI linkage + and a real X11 window. Fixture keys use an isolated RPM database that is removed on every exit. +9. **Docs/UI:** Fedora homepage, install guide, support matrix, release guide, user guide, operator runbook, packaging + docs, navigation and release notes are updated. Pending preserves the existing authenticated local-RPM path; + verified-fixture browser proof switches only Fedora to `sudo dnf install loopwire` and its separate setup link. + DNF's custom-deadline replay limitation is explicit. Production activation remains human-owned and the repository + is never described as a default Fedora repository. + +## Verification + +- Final `pnpm check` passed: project verification, types, 295 workspace tests, 22 Rust tests, native/package gates, + production builds, static-site verification, and both APT/Fedora repository suites. +- `pnpm verify:rpm-repository` passed in the pinned Fedora 44 image: 13 generator, 20 publisher, 7 bootstrap, + 6 public HTTPS, 3 workflow preflight, 34 raw proof-verifier and 4 channel-gate cases. +- The publisher suite exercised actual SSH and DNF5. DNF rejected the intentionally interrupted signature/XML pair; + recovery restored a valid repository. Nginx syntax and live cache/404 headers passed. +- The Fedora 44 KVM lifecycle produced 122 evidence files at `e73c5bb` and passed the independent verifier. Baseline + source SHA-256 is `5a163db0acd1d2f8c73f8cff1b4cc05f12a3d811bfedaf681ea46f460d4d1fb3`, matching the public release manifest. +- Pending and activated-fixture Chromium runs passed the nine platform panels, commands, keyboard/copy failure and + recovery, no-JavaScript fallback, responsive widths, signal motion/reduced-motion/visibility checks, and Fedora + setup link/guide command. The checked-in channel was restored to pending. +- Workflow contracts, actionlint, ShellCheck, Python/Node/Ruby/Bash syntax, docs/build checks and whitespace passed. + A comprehensive review's public-release provenance finding was fixed and approved on targeted re-review. + +## Boundaries + +The issue's five Human operational tasks remain unchecked. No production host, TLS/DNS, account, OpenPGP identity, +SSH credential, GitHub environment value, repository publication, or website activation was created or changed. +The project verifier enforces the custom signed metadata deadline; DNF itself verifies signatures but does not enforce +that project-specific expiry tag, so HTTPS/origin control and monitoring remain part of operations. + +Power-loss and network filesystems were not exercised; production requires local POSIX flock/fsync/atomic-rename +semantics. The KVM repository signer, TLS CA and upgrade version are disposable fixture material. Only the baseline +source RPM is the actual published v0.1.0 Fedora artifact.