Skip to content

chore(test): cap stage-1 test concurrency instead of taking every core - #299

Merged
jbr-sekoia merged 1 commit into
mainfrom
chore/test-concurrency
Sep 18, 2026
Merged

jbr-sekoia merged 1 commit into
mainfrom
chore/test-concurrency

Conversation

@jbr-sekoia

Copy link
Copy Markdown
Collaborator

Why

npm test runs stage 1 with node --test over every test file and no --test-concurrency, so node applies its own default: os.availableParallelism() workers. On a 14-core workstation that is a dozen-plus test processes at once.

Two costs, both measured on this machine:

  • The developer's machine stops being usable. Load average reached 15.87 on 14 cores with the suite running, and that was the suite alone.
  • It starves the one suite that cannot afford to be starved. test/trigger-watcher.test.js uses real timers and real fs.watch against wall-clock budgets. scripts/run-tests.js already isolates it into its own serial stage for exactly this reason (issue (test): trigger-watcher timing tests fail under host load and block the pre-commit hook #260), but stage 1 handing every core to itself is the same contention one step earlier.

What

Stage 1 runs at 4 workers, bounded by what the machine actually has:

const DEFAULT_CONCURRENCY = 4;
Math.max(1, Math.min(DEFAULT_CONCURRENCY, os.availableParallelism()))

SWITCHBOARD_TEST_CONCURRENCY overrides it. A value that is not a positive integer is ignored with a message on stderr and the cap applies, so a typo slows a run down rather than failing it.

CI is unaffected: GitHub's ubuntu-latest and windows-2022 runners have 4 cores or fewer, so min(4, available) is what they were already getting.

Stage 2 is untouched — it was already --test-concurrency=1.

The runner now executes only when it is the entry point (require.main === module), so stageOneConcurrency can be imported and tested rather than inferred from the spawn.

Not in scope

  • Any change to what the tests assert, or to stage 2's isolation.
  • taskset and friends: capping the whole process tree is the right tool when several agents run the suite at once, and it is documented in .ai/agent-practices.md rather than wired into the runner. Node's availableParallelism() honours the affinity mask, so taskset -c 0-3 npm test reduces the number of workers rather than crowding them onto fewer cores — measured: taskset -c 0-3 node -e 'os.availableParallelism()' reports 4 on this 14-core box.

Acceptance

  • npm test with no environment set spawns at most 4 stage-1 workers.
  • SWITCHBOARD_TEST_CONCURRENCY=12 npm test spawns 12.
  • SWITCHBOARD_TEST_CONCURRENCY=banana warns and falls back to 4.
  • A 2-core machine gets 2, not 4.
  • Mutation: removing the cap so the default is availableParallelism() turns unset: caps at DEFAULT_CONCURRENCY rather than taking every core red on any machine with more than 4 cores.

Verification

npx eslint . → 0 errors, 317 warnings (unchanged).

npm test under the new cap → stage 1 tests 1601 / pass 1598 / fail 1 / skipped 2, stage 2 tests 120 / pass 119 / fail 0 / skipped 1. The single failure is test/ipc-path-validator.test.js "allows files under ~/.claude/", pre-existing and environmental on this machine: ~/.claude/CLAUDE.md is a symlink out of ~/.claude and the validator resolves on disk. This branch touches neither that test nor ipc-path-validator.js.

trigger-watcher.test.js passed 119/119 under the cap, which was the risk worth checking — fewer cores could in principle have made its wall-clock assertions flakier rather than steadier.

Node defaults a multi-file --test run to os.availableParallelism() workers.
On a 14-core workstation that is a dozen-plus test processes at once: it
saturates the machine the developer is working on, and it starves
trigger-watcher.test.js, whose wall-clock budgets are the suite's most
contention-sensitive assertions.

Stage 1 now runs at 4 workers, bounded by what the machine has, and
SWITCHBOARD_TEST_CONCURRENCY overrides it. CI runners have 4 cores or
fewer, so their runtime is unchanged.

The runner only executes when it is the entry point, so the helper can be
imported and tested.
@jbr-sekoia
jbr-sekoia merged commit 65d8aca into main Sep 18, 2026
10 checks passed
@jbr-sekoia
jbr-sekoia deleted the chore/test-concurrency branch September 18, 2026 08:38

@devsuitup devsuitup left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed e350123 against main bd7ce49; formula, env parsing, argv assembly, the new test file and docs read; one stage-1 run measured with and without the cap on the 8-core / 16 GB desktop that runs the pre-commit hook every day.

Measured cost, not disclosed. Stage 1 over the real 144-file list: 66.1 s with node's default (8 workers here) vs 86.6 s with --test-concurrency=4 — +20.5 s, +31 %, on every commit through husky. The body quantifies the load-average problem on the 14-core box and says CI (≤ 4 cores) is unaffected, both true, but a flat ceiling of 4 discards half the cores on the 5–8-core class, which is where this hook actually runs most. Either state that trade-off with the number in the body and in .ai/agent-practices.md "Test concurrency", or let the cap scale — e.g. min(available, max(4, available - 2)) with a ceiling of 8 gives 2→2, 4→4, 8→6, 14→8, keeps two cores free for the UI and the serial stage, and still stops the dozen-plus-process case. Your call which; the measurement should appear either way.

Smaller:

  • README.md:123 says "4 workers"; the formula is min(4, available) (your own 2-core acceptance line). .ai/agent-practices.md has it right.
  • scripts/run-tests.js:23-26 is three lines of rationale plus the pointer; the ceiling here is the pointer alone (PRs #127/#130).
  • test/run-tests-concurrency.test.js: the bad-value cases assert the fallback but not the warning; the acceptance line says "warns and falls back". A muted console.error passes today. Capture stderr or drop "warns".

Checked, clean: formula yields 1/2/4/4/4/4 for 1/2/4/8/16/64 cores; availableParallelism with the cpus().length fallback (dead under engines >=20, harmless); Number() + isInteger && > 0 rejects NaN/0/negative/decimal/text, empty string treated as unset; --test-concurrency=N is a real flag and stage 2 stays at 1; the test imports the real export, spawns nothing; section renumbering in agent-practices has no stale cross-reference; no attribution trailers; CI green on e350123.

@jbr-sekoia jbr-sekoia mentioned this pull request Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants