Skip to content

ci: refactor the pipeline into a modular structure - #61

Merged
pratyush618 merged 7 commits into
masterfrom
ci/modular-pipeline
Aug 6, 2026
Merged

pratyush618 merged 7 commits into
masterfrom
ci/modular-pipeline

Conversation

@kartikeya-27

@kartikeya-27 kartikeya-27 commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

Builds on #58 and #60, both now merged.

Two changes: flatten the Python source layout, then refactor CI into a modular structure modelled on the sibling dagron and taskito repos.

1. py_src/ removed

py_src/reclink/ → reclink/ via git mv (22 files, all recorded as renames), and python-source dropped from [tool.maturin] since a root-level package is the default. All 8 references updated: pyproject.toml, .pre-commit-config.yaml, README.md (×3), ci.yml (×3), and docs/docs/getting-started/installation.mdx (×2).

No compiled .so or __pycache__ entered the index — verified across the whole commit range.

No import shadowing. maturin writes _core.cpython-*.so directly into ./reclink/, so import reclink from the repo root resolves to the source tree and finds the compiled _core there. The real hazard — an sdist that installs but imports the uncompiled source tree — is now guarded by the new verify-sdist action, which asserts site-packages is on the import path.

2. Modular CI

A thin ci.yml runs a dorny/paths-filter changes job and dispatches to reusable per-suite workflows, with FORCE on non-PR events so master always gets a full run regardless of the filter.

Before After
ci.yml lint → cargo fmt, clippy ci-rust.yml → lint
ci.yml lint → ruff check/format, mypy ci-python.yml → lint
ci.yml rust-test ci-rust.yml → test
ci.yml test matrix (6 cells) ci-python.yml → test (same 6 pairs)
docs.yml build on PR ci-docs.yml → build
docs.yml build+Pages+docs-dist on push docs.yml (unchanged behaviour)
release.yml, cleanup.yml unchanged, moved onto shared composites
— ci-release.yml, actionlint.yml, ci-status (new)

Shared setup lives in composite actions under .github/actions/ (setup-rust, setup-python, setup-node, build-docs, verify-sdist), including the save-if cache discipline so only one job per namespace writes the Cargo cache.

No check was dropped. Every run:/uses: line in the old four workflows was diffed against the new set; the only absent lines have exact 1:1 replacements (py_src/→reclink/, cd docs-site &&→working-directory:, a literal --out-dir→$WASM_OUT_DIR, and added quoting on the mypy subshell that shellcheck flagged).

Deliberate additions: fail-fast: false on the pytest matrix, timeout-minutes throughout, verify-sdist in the sdist job, and Dependabot now scanning /.github/actions/* — without that the composites' pinned actions would never get updates.

Action versions

The bumps from #58 landed on the flat workflow files this PR deletes or rewrites, so a plain merge would have silently dropped them. Each was re-applied at its new home: actions/checkout@v7 in all six workflows that check out; actions/setup-python@v7 in the setup-python and verify-sdist composites; actions/setup-node@v7 in setup-node; actions/upload-pages-artifact@v5 in docs.yml; astral-sh/setup-uv@v9.0.0 in both Python composites — each carrying a comment recording why it is an exact tag, so nobody "fixes" it back to @v9 (v8 stopped publishing major/minor tags).

Verification

grep docs-site / py_src / npm-in-.github          0 hits each
actionlint 1.7.12                                 exit 0
YAML parse, 15 files                              0 failures
uses: ./... refs, 9 distinct                      all resolve
cargo fmt --all --check                           exit 0
cargo clippy --all-targets -- -D warnings         exit 0
cargo test --workspace                            exit 0
uv run ruff check reclink/ tests/                 exit 0
uv run ruff format --check reclink/ tests/        exit 0
uv run mypy reclink/ tests/python/                exit 0, 40 files
uv run maturin develop --release                  exit 0
uv run pytest tests/python/                       exit 0, 432 passed
wasm-pack build crates/reclink-wasm               exit 0
cd docs && pnpm install --frozen-lockfile         exit 0
cd docs && pnpm build                             exit 0
cd docs && pnpm typecheck                         exit 0

Known issues — please read before merging

  1. Branch protection must be repointed at CI status. Every suite is skippable via the paths filter, and a skipped job never reports, so the old per-job required checks (lint, rust-test, test) no longer exist and would block merges forever if still required. ci-status is the single always-running gate.
  2. docs.yml no longer runs on pull_request. Intentional — PR docs builds moved to ci-docs.yml so a docs PR doesn't build the site twice — but it changes which workflow name appears on a docs PR, and PRs no longer produce a docs-dist artifact (only master pushes do).
  3. Pages deploy guard loosened from event_name == 'push' && ref == master to just ref == master, so workflow_dispatch on master can now deploy. Pages concurrency is now pages with cancel-in-progress: false.
  4. actionlint does not cover the composite actions. Upstream parses .github/actions/*/action.yml as workflows and emits spurious "on" section is missing errors (reproduced on both 1.7.7 and 1.7.12). The composites are covered by JSON-schema validation plus the uses:-resolution and YAML-parse checks instead. A follow-up could add check-jsonschema --builtin-schema vendor.github-actions.
  5. pnpm typecheck still isn't run in CI. It wasn't before this stack either, so nothing regressed — but docs/.npmrc's node-linker=hoisted exists specifically so typecheck can resolve @types/react, and that guarantee is currently only exercised locally. Cheap to add to ci-docs.yml.

Pre-existing issues surfaced but deliberately not fixed here

  • reclink.__init__.__version__ is "0.1.0" while pyproject.toml says 0.1.1. dagron's check_versions.py gate was deliberately not ported, because it would make this branch red on arrival. Worth a separate fix.
  • uv sync --extra dev fails on a default Python 3.14 (PyO3 0.24 caps at 3.13). CI pins 3.12/3.13 so it isn't hit there, but local dev on a 3.14 default breaks.
  • .pre-commit-config.yaml's ruff format --check . fails on README.md, and the hook's uv env fails to build the maturin extension. Both reproduce on master; CI is unaffected because it scans reclink/ tests/.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

@pratyush618, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fb630ba0-b5ab-423d-b215-b5d2790dc98b

📥 Commits

Reviewing files that changed from the base of the PR and between b32f330 and 89a76fc.

📒 Files selected for processing (41)
  • .github/actions/build-docs/action.yml
  • .github/actions/setup-node/action.yml
  • .github/actions/setup-python/action.yml
  • .github/actions/setup-rust/action.yml
  • .github/actions/verify-sdist/action.yml
  • .github/dependabot.yml
  • .github/workflows/actionlint.yml
  • .github/workflows/ci-docs.yml
  • .github/workflows/ci-python.yml
  • .github/workflows/ci-release.yml
  • .github/workflows/ci-rust.yml
  • .github/workflows/ci.yml
  • .github/workflows/cleanup.yml
  • .github/workflows/docs.yml
  • .github/workflows/release.yml
  • .pre-commit-config.yaml
  • README.md
  • docs/docs/getting-started/installation.mdx
  • pyproject.toml
  • reclink/__init__.py
  • reclink/__main__.py
  • reclink/_core.pyi
  • reclink/_pandas_accessor.py
  • reclink/_pandas_compat.py
  • reclink/_polars_accessor.py
  • reclink/_polars_plugin.py
  • reclink/active_learning.py
  • reclink/async_api.py
  • reclink/benchmark.py
  • reclink/cli.py
  • reclink/evaluation.py
  • reclink/export.py
  • reclink/fuzzy_join.py
  • reclink/metrics.py
  • reclink/phonetic.py
  • reclink/pipeline.py
  • reclink/presets.py
  • reclink/py.typed
  • reclink/streaming.py
  • reclink/threshold_optimizer.py
  • reclink/utils.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Base automatically changed from docs/rebrand-and-restructure to master August 6, 2026 03:03
maturin defaults to a root-level Python source dir, so the wrapper only added a path prefix to every lint invocation.
Toolchain setup was copy-pasted across every job; centralising it also gives the Cargo cache a single place to enforce its save-if discipline.
A paths-filter gate keeps PRs from running suites they cannot affect, and the always-running ci-status job gives branch protection one check that skipped suites cannot defeat.
docs.yml drops its pull_request trigger now that ci-docs.yml builds the site as a PR check, so a docs PR no longer builds twice.
The bumps landed on master's flat files, which this branch replaced.
setup-uv is pinned to an exact tag because it no longer ships majors.
Mirrors the pattern the docs branch put in docs.yml, so the composite
actions build the site the same way the deploy workflow used to.
@pratyush618
pratyush618 force-pushed the ci/modular-pipeline branch from bf0e2b8 to 89a76fc Compare August 6, 2026 03:04
@pratyush618
pratyush618 merged commit b5a5e13 into master Aug 6, 2026
16 checks passed
@pratyush618
pratyush618 deleted the ci/modular-pipeline branch August 6, 2026 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants