Skip to content

release: add mport package installation to DVD images - #503

Merged
laffer1 merged 2 commits into
masterfrom
bsdinstall-mport-packages
Sep 5, 2026
Merged

release: add mport package installation to DVD images#503
laffer1 merged 2 commits into
masterfrom
bsdinstall-mport-packages

Conversation

@laffer1

@laffer1 laffer1 commented Sep 5, 2026

Copy link
Copy Markdown
Member

Summary

  • stage a verified dependency closure from a blessed mport repository on dvd1.iso while retaining the full repository index
  • add a bsdinstall package menu for curated DVD packages and full-catalog network searches
  • install selections with mport, seed /var/db/mport/index.db, and mount installation media at /packages
  • use the canonical /usr/mports path throughout the affected release tooling
  • include the kmod.mk parser fix left after the legacy-architecture cleanup
  • document release configuration and keep future system package support out of the application-package path

Validation

  • sh -n on all modified shell scripts
  • git diff --check
  • bmake -C release -V MPORT_REPOSITORY -V DVD_PACKAGE_ROOTS
  • bmake -C usr.sbin/bsdinstall/scripts
  • synthetic blessed-index test covering dependency cycles, missing dependencies, empty descriptions, package subset staging, and system-package exclusion
  • mandoc -Tlint usr.sbin/bsdinstall/bsdinstall.8 reports only existing diagnostics outside the new text

A complete dvd1.iso build and installer boot test were not run.

AI contribution checklist

  • Human maintainer has manually reviewed every AI-assisted change
  • No proprietary or copyleft code was introduced
  • Final maintainer style review completed

AI-Assisted-by: OpenAI Codex (GPT-5)

Summary by Sourcery

Enable verified application package installation from DVD media while retaining network catalog access and adopting the canonical mports layout.

New Features:

  • Add curated mport package installation and full-catalog search to the bsdinstall workflow.
  • Stage verified application package dependency closures and the complete repository index on DVD images.

Bug Fixes:

  • Fix the kmod.mk conditional structure left after legacy-architecture cleanup.

Enhancements:

  • Use /usr/mports consistently throughout release tooling and switch package operations to mport.
  • Mount installation media packages at /packages, seed the target mport index, and preserve network installation for packages not present on the DVD.

Build:

  • Extend release configuration with blessed mport repository and DVD package-root settings, limited to dvd1.iso package staging.

Documentation:

  • Document blessed repository requirements, DVD package staging, installer behavior, and the separation of application packages from future system packages.

Tests:

  • Add validation coverage for package dependency closure staging, repository integrity checks, and installer package handling.

Stage a verified package dependency closure and the full blessed mport index on DVD images. Add a bsdinstall package menu that installs curated or searched packages into the target with mport and seeds its repository index.

AI-Assisted-by: OpenAI Codex (GPT-5)
Signed-off-by: Lucas Holt <luke@foolishgames.com>
The legacy architecture removal deleted the PowerPC and MIPS conditional blocks but left one closing directive behind. Remove it so kmod.mk parses correctly.

AI-Assisted-by: OpenAI Codex (GPT-5)
Signed-off-by: Lucas Holt <luke@foolishgames.com>
Copilot AI lite review requested due to automatic review settings September 5, 2026 15:52
@sourcery-ai

sourcery-ai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR introduces verified mport application installation from DVD images: release builds stage dependency-complete package selections alongside the full blessed catalog, bsdinstall offers curated and searchable installation flows using /packages and mport, and release tooling is standardized on /usr/mports with related configuration and parser cleanup.

Sequence diagram for bsdinstall package installation

sequenceDiagram
    participant User
    participant BSDInstall
    participant Media as DVD_/packages
    participant Mport
    participant Target as Target_system
    User->>BSDInstall: packages
    BSDInstall->>Media: Read installer.tsv and catalog.tsv
    BSDInstall->>Target: Mount DVD packages at /packages
    User->>BSDInstall: Select package
    BSDInstall->>Target: Seed /var/db/mport/index.db if absent
    BSDInstall->>Mport: install -U -c BSDINSTALL_CHROOT install -y package-name
    Mport->>Media: Use verified .mport bundles
    Mport->>Target: Install package and automatic dependencies
Loading

Flow diagram for verified DVD package staging

flowchart TD
    A["Blessed mport repository"] --> B["Read complete index.db"]
    B --> C["Validate configured package roots"]
    C --> D["Resolve dependency closure"]
    D --> E["Verify bundle filenames and SHA-256 hashes"]
    E --> F["Stage packages, installer.tsv, catalog.tsv, and index.db"]
    F --> G["Build dvd1.iso with /packages"]
Loading

File-Level Changes

Change Details Files
Stage a verified, size-bounded application package subset and complete repository metadata for DVD installation media.
  • Add configurable blessed repository and curated package-root settings.
  • Resolve recursive dependency closures from the SQLite index, including shared dependencies and cycles.
  • Validate index uniqueness, application package types, bundle filenames, presence, and SHA-256 hashes before staging.
  • Copy the full index plus installer and searchable catalog metadata, while retaining only selected package bundles and dependencies.
release/Makefile
release/packages.md
release/release.conf.sample
release/scripts/pkg-stage.sh
Add bsdinstall support for selecting and installing packages from DVD media or network repositories.
  • Add a package-menu script with curated defaults and full-catalog search results.
  • Mount DVD package media at /packages and unmount it through installer lifecycle handling.
  • Seed /var/db/mport/index.db when needed and invoke mport for selected application packages.
  • Keep system-package entries available in metadata but exclude them from the application workflow.
usr.sbin/bsdinstall/bsdinstall.8
usr.sbin/bsdinstall/scripts/Makefile
usr.sbin/bsdinstall/scripts/auto
usr.sbin/bsdinstall/scripts/mount
usr.sbin/bsdinstall/scripts/packages
usr.sbin/bsdinstall/scripts/umount
Modernize release tooling for mports and propagate DVD package configuration through release builds.
  • Use /usr/mports consistently for tree updates, package builds, embedded ports, and GCE fetches.
  • Pass repository and package-root overrides into release make invocations.
  • Replace legacy pkg-based staging/setup paths with mport where applicable.
  • Limit package staging to dvd1.iso while preserving existing non-DVD image behavior.
release/Makefile
release/release.conf.sample
release/release.sh
release/tools/gce.conf
Repair the kmod.mk conditional structure after legacy-architecture removal.
  • Remove the unmatched closing conditional that prevented correct makefile parsing.
sys/conf/kmod.mk

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@laffer1
laffer1 merged commit 5c55f84 into master Sep 5, 2026
7 of 11 checks passed
@laffer1
laffer1 deleted the bsdinstall-mport-packages branch September 5, 2026 15:54
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Claude finished @laffer1's task in 2m 51s —— View job


Code Review: PR #503 — mport package installation on DVD images

This PR is already merged; below is a post-merge review focused on correctness and security, per the repo's review guidelines.

Summary of what I checked

  • release/scripts/pkg-stage.sh: builds SQL queries by string interpolation, but every value (root, version, pkg from the recursive closure, bundle) is passed through validate_atom/character-class checks (*[!A-Za-z0-9_.,+~:-]*) before being embedded, which excludes quote characters — no SQL injection vector found.
  • usr.sbin/bsdinstall/scripts/packages: cross-checked mport(8)'s actual CLI against usage here (contrib/mport/mport/mport.c):
    • -Uno-index ('U' longopt), matches "don't clobber the just-copied DVD index."
    • install -y, info -e, clean subcommands all exist with the flags used.
    • ${debugFile#+} sentinel matches the existing convention used in scripts/script, scripts/config, and bsdinstall itself — not a bug.
    • Field layouts between pkg-stage.sh's installer.tsv/catalog.tsv output and the read/awk -F "$TAB" consumers in packages line up correctly.
  • mount/umount: mount target changed from $BSDINSTALL_CHROOT/dist/packages to $BSDINSTALL_CHROOT/packages (read-only nullfs); umount now detects the mount via mount | awk ... instead of a directory-existence check, and still removes the mountpoint directory afterward — no leftover directories, logic is consistent.
  • sys/conf/kmod.mk: the stray .endif removal correctly repairs the file left over from the prior MIPS/PowerPC/sparc64 removal (Remove MIPS, PowerPC, and sparc64 support #500); confirmed the remaining .endif/.if nesting is now balanced around that block.
  • Shell style: no bashisms introduced ($(( )), case glob negation with !, no local/[[ in the new/changed lines) — consistent with the POSIX sh requirement in AI_POLICY.md.
  • No assembly or contrib/mksh files touched, so no policy-restricted paths are involved.

Notes (non-blocking)

  • release/tools/gce.conf and the git/pkg fallback path in release.sh still reference pkg info -o -q -a / pkg install in one spot (cloud-image extra-package fetch) — pre-existing behavior outside this PR's core mport path, just flagged for awareness since the rest of the tree moved to /usr/mports.
  • install_package() in usr.sbin/bsdinstall/scripts/packages falls through to mport install even when prepare_resolver can't confirm real network reachability (only checks for a non-empty resolv.conf) — acceptable as a best-effort UX gate, real failures still surface via the existing error dialog.

No correctness bugs, malloc/free issues (N/A — no C changes), or bounds/security issues found in the diff.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new bsdinstall package UI and GCE tooling include verified cases where user actions can silently do nothing and where mports tarball naming is inconsistent, breaking intended behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds offline-capable mport application installation support to dvd1.iso builds by staging a blessed repository subset onto the DVD, integrating a new bsdinstall packages workflow, and standardizing release tooling on /usr/mports.

Changes:

  • Stage a verified dependency closure of selected application packages (plus full index.db) into dvd/packages during dvd1.iso builds.
  • Add a new bsdinstall packages script and hook it into the automated and “Final Configuration” installer flows, mounting media at /packages.
  • Fix an unmatched conditional in sys/conf/kmod.mk and align release tooling paths from /usr/ports to /usr/mports.
File summaries
File Description
usr.sbin/bsdinstall/scripts/umount Unmount and clean up the new package-media mountpoint under the target root.
usr.sbin/bsdinstall/scripts/packages New installer workflow for curated + searchable package installation via mport.
usr.sbin/bsdinstall/scripts/mount Mount DVD package media read-only at the mport media path in the target root.
usr.sbin/bsdinstall/scripts/Makefile Install the new packages script as part of bsdinstall scripts.
usr.sbin/bsdinstall/scripts/auto Invoke package installation when DVD package metadata is present; add “Packages” to final menu.
usr.sbin/bsdinstall/bsdinstall.8 Document the new bsdinstall packages subcommand behavior.
sys/conf/kmod.mk Remove an extra .endif to fix conditional structure.
release/tools/gce.conf Update mports fetch path for installed-package sources in GCE image tooling.
release/scripts/pkg-stage.sh Replace pkg(8)-based staging with mport index validation + bundle copying.
release/release.sh Pass staging configuration through to make release and use /usr/mports in the chroot.
release/release.conf.sample Document MPORT_REPOSITORY and DVD_PACKAGE_ROOTS configuration knobs.
release/packages.md New documentation for DVD package staging and installer behavior.
release/Makefile Add staging variables and run pkg-stage.sh to populate dvd/packages for dvd1.iso.
Review details

Suppressed comments (1)

usr.sbin/bsdinstall/scripts/packages:174

  • If a search result row has missing/invalid bundle metadata, install_package will return immediately (valid_bundle fails) with no user-visible error. This makes the menu selection appear to have no effect.
	[ "$result" -eq 0 ] || return 0
	bundle=$(awk -F "$TAB" -v name="$choice" '$1 == name { print $4; exit }' "$results")
	install_package "$choice" "$bundle"
  • Files reviewed: 13/13 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread release/tools/gce.conf
for PACKAGE in ${_INSTALLED_PACKAGES}; do
chroot ${DESTDIR} \
make -C /usr/ports/${PACKAGE} fetch
make -C /usr/mports/${PACKAGE} fetch
Comment on lines +122 to +125
while IFS= read -r pkg; do
bundle=$(awk -F "$TAB" -v name="$pkg" '$1 == name { print $4; exit }' "$CATALOG")
install_package "$pkg" "$bundle"
done < "$selection"

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants