perf: cut the slowest test, add targeted local runs, drop a CI job - #177
Merged
Conversation
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three of the four items from the CI/test-speed research. The fourth — sharing the Postgres container — is the big one and gets its own branch.
Where the time was
Measured before changing anything, on this workspace:
perf(resolver)— prove the bucket bound against a small capbucket_map_stays_bounded_under_fresh_ip_flooddrove 55,000 iterations throughthe rate limiter in a debug build to assert the map never exceeds its cap. It
took 50.259s — the slowest test in the workspace by roughly 500x, and a
serial long pole in an otherwise 14-second suite.
The property is "the map never exceeds
max_buckets". That holds at any cap, soit is now proven against a cap of 50: 0.019s, a 2,600x cut.
max_bucketsbecomes a field, defaulted toMAX_BUCKETSby the onlyconstructor a binary calls;
with_max_bucketsis#[cfg(test)]. The sweepthreshold that was hardcoded at
10_000is nowmax_buckets / SWEEP_DIVISOR,which evaluates to exactly 10,000 — production behaviour is unchanged, and the
two numbers can no longer drift apart.
The production value is pinned separately.
the_default_limiter_uses_the_production_capasserts the default constructor uses
MAX_BUCKETSand that the derived sweepthreshold is still 10,000. Splitting them means shrinking the cap in the
property test cannot quietly shrink it in the binary — the property test proves
the bound holds, this one proves the bound is the one we intend.
Effect on the whole suite: 64.5s → 28.8s wall clock, from this one test.
feat(just)—test-changedMaps changed files to their crates, then hands nextest an
rdeps()filterset —every test in those crates and everything depending on them. Measured
selectivity against 1,041 tests:
dpp-resolverdpp-integratordpp-vaultdpp-dalVerified end to end: a resolver-only edit runs
rdeps(dpp-resolver)— 68 testsacross 3 binaries, 65 binaries skipped, 13.8s.
It reads committed and uncommitted changes, because the point is to be useful
mid-edit rather than only after a commit.
It falls back to the full suite whenever the blast radius is not a crate — a
manifest, a migration under
ops/, CI config,.config/, the justfile, or anypath it cannot attribute. Erring toward running everything is the only safe
direction for a tool whose job is deciding what to skip. Verified: a justfile
edit runs all 1,043.
Stated plainly in the recipe docs and the script header: this is an iteration
aid, not a gate. It reasons about crate boundaries, not behaviour — a change
that alters a runtime contract without touching the dependent crate's source is
invisible to it, as is anything reached only through a trait object.
just checkbefore pushing, regardless.
One known limit worth recording:
rdeps(dpp-domain)selects zero, becausethe core crates come from crates.io rather than the workspace, so nextest cannot
see that edge. Acceptable — a core repin should run everything anyway — but it
means this tool is blind to the dependency that changes most consequentially.
ci:— lint with the integration features instead of a second compileFeature-gated code compileswas a 2.6m job doingcargo checkwith theintegration-testsfeatures on, existing because those suites are otherwisecompiled nowhere except the Docker tiers. Clippy now runs with the same feature
list and does both jobs — and lints that code, which
cargo checkdid not.Safe to fold, and checked rather than assumed: features are additive, and
the workspace has zero
cfg(not(feature = "integration-tests"))code, sonothing the default build compiles is skipped by this one. The comment says so,
and says what has to change if that stops being true. The explicit per-package
feature list is preserved with its original reasoning —
--all-featureswoulddrag in
cli/desktopandwasm-fixture-tests, neither of which is the gap.One job removed, ~2.6m of compute per run, no coverage lost.
Not in this branch
Sharing the Postgres container, which is where the remaining 86% lives: 61
start_pg()call sites each start a freshpostgres:17, costing 12–16s apiece.The design hinges on a fact I confirmed empirically rather than assumed —
nextest runs each test in its own process (two tests in one binary printed
two distinct PIDs). So no in-process sharing is possible; the container has to
be started outside the test process and discovered through the environment, with
per-test isolation coming from a template database rather than a fresh server.
That is a change to how every DB-backed suite gets its database, so it gets its
own branch and its own review.
Verification
just checkgreen: 842 tests (the +1 is the new production-cap assertion), allgates,
cargo audit. The folded clippy invocation was run locally with the exactCI feature list and is clean.