Skip to content

perf: cut the slowest test, add targeted local runs, drop a CI job - #177

Merged
LKSNDRTMLKV merged 3 commits into
mainfrom
perf/test-and-ci-speedup
Aug 23, 2026
Merged

perf: cut the slowest test, add targeted local runs, drop a CI job#177
LKSNDRTMLKV merged 3 commits into
mainfrom
perf/test-and-ci-speedup

Conversation

@LKSNDRTMLKV

Copy link
Copy Markdown
Member

Three of the four items from the CI/test-speed research. The fourth — sharing the Postgres container — is the big one and gets its own branch.

Where the time was

Measured before changing anything, on this workspace:

  • 2,486 test-seconds total. 171 tests (16%) accounted for 86% of it.
  • One test accounted for 50 of those seconds on its own.
  • Engine CI: 16.1m end-to-end, longest single job 5.2m.

perf(resolver) — prove the bucket bound against a small cap

bucket_map_stays_bounded_under_fresh_ip_flood drove 55,000 iterations through
the rate limiter in a debug build to assert the map never exceeds its cap. It
took 50.259s — the slowest test in the workspace by roughly 500x, and a
serial long pole in an otherwise 14-second suite.

The property is "the map never exceeds max_buckets". That holds at any cap, so
it is now proven against a cap of 50: 0.019s, a 2,600x cut.

max_buckets becomes a field, defaulted to MAX_BUCKETS by the only
constructor a binary calls; with_max_buckets is #[cfg(test)]. The sweep
threshold that was hardcoded at 10_000 is now max_buckets / SWEEP_DIVISOR,
which evaluates to exactly 10,000 — production behaviour is unchanged, and the
two numbers can no longer drift apart.

The production value is pinned separately. the_default_limiter_uses_the_production_cap
asserts the default constructor uses MAX_BUCKETS and that the derived sweep
threshold is still 10,000. Splitting them means shrinking the cap in the
property test cannot quietly shrink it in the binary — the property test proves
the bound holds, this one proves the bound is the one we intend.

Effect on the whole suite: 64.5s → 28.8s wall clock, from this one test.

feat(just)test-changed

Maps changed files to their crates, then hands nextest an rdeps() filterset —
every test in those crates and everything depending on them. Measured
selectivity against 1,041 tests:

Edit in Selected Cut
dpp-resolver 67 94%
dpp-integrator 247 76%
dpp-vault 471 55%
dpp-dal 511 51%

Verified end to end: a resolver-only edit runs rdeps(dpp-resolver) — 68 tests
across 3 binaries, 65 binaries skipped, 13.8s.

It reads committed and uncommitted changes, because the point is to be useful
mid-edit rather than only after a commit.

It falls back to the full suite whenever the blast radius is not a crate — a
manifest, a migration under ops/, CI config, .config/, the justfile, or any
path it cannot attribute. Erring toward running everything is the only safe
direction for a tool whose job is deciding what to skip. Verified: a justfile
edit runs all 1,043.

Stated plainly in the recipe docs and the script header: this is an iteration
aid, not a gate.
It reasons about crate boundaries, not behaviour — a change
that alters a runtime contract without touching the dependent crate's source is
invisible to it, as is anything reached only through a trait object. just check
before pushing, regardless.

One known limit worth recording: rdeps(dpp-domain) selects zero, because
the core crates come from crates.io rather than the workspace, so nextest cannot
see that edge. Acceptable — a core repin should run everything anyway — but it
means this tool is blind to the dependency that changes most consequentially.

ci: — lint with the integration features instead of a second compile

Feature-gated code compiles was a 2.6m job doing cargo check with the
integration-tests features on, existing because those suites are otherwise
compiled nowhere except the Docker tiers. Clippy now runs with the same feature
list and does both jobs — and lints that code, which cargo check did not.

Safe to fold, and checked rather than assumed: features are additive, and
the workspace has zero cfg(not(feature = "integration-tests")) code, so
nothing the default build compiles is skipped by this one. The comment says so,
and says what has to change if that stops being true. The explicit per-package
feature list is preserved with its original reasoning — --all-features would
drag in cli/desktop and wasm-fixture-tests, neither of which is the gap.

One job removed, ~2.6m of compute per run, no coverage lost.

Not in this branch

Sharing the Postgres container, which is where the remaining 86% lives: 61
start_pg() call sites each start a fresh postgres:17, costing 12–16s apiece.

The design hinges on a fact I confirmed empirically rather than assumed —
nextest runs each test in its own process (two tests in one binary printed
two distinct PIDs). So no in-process sharing is possible; the container has to
be started outside the test process and discovered through the environment, with
per-test isolation coming from a template database rather than a fresh server.

That is a change to how every DB-backed suite gets its database, so it gets its
own branch and its own review.

Verification

just check green: 842 tests (the +1 is the new production-cap assertion), all
gates, cargo audit. The folded clippy invocation was run locally with the exact
CI feature list and is clean.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results
Complexity 0
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@LKSNDRTMLKV
LKSNDRTMLKV merged commit 11ba940 into main Aug 23, 2026
22 checks passed
@LKSNDRTMLKV
LKSNDRTMLKV deleted the perf/test-and-ci-speedup branch August 23, 2026 01:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant