chore(test): cap stage-1 test concurrency instead of taking every core - #299
Conversation
Node defaults a multi-file --test run to os.availableParallelism() workers. On a 14-core workstation that is a dozen-plus test processes at once: it saturates the machine the developer is working on, and it starves trigger-watcher.test.js, whose wall-clock budgets are the suite's most contention-sensitive assertions. Stage 1 now runs at 4 workers, bounded by what the machine has, and SWITCHBOARD_TEST_CONCURRENCY overrides it. CI runners have 4 cores or fewer, so their runtime is unchanged. The runner only executes when it is the entry point, so the helper can be imported and tested.
devsuitup
left a comment
There was a problem hiding this comment.
Reviewed e350123 against main bd7ce49; formula, env parsing, argv assembly, the new test file and docs read; one stage-1 run measured with and without the cap on the 8-core / 16 GB desktop that runs the pre-commit hook every day.
Measured cost, not disclosed. Stage 1 over the real 144-file list: 66.1 s with node's default (8 workers here) vs 86.6 s with --test-concurrency=4 — +20.5 s, +31 %, on every commit through husky. The body quantifies the load-average problem on the 14-core box and says CI (≤ 4 cores) is unaffected, both true, but a flat ceiling of 4 discards half the cores on the 5–8-core class, which is where this hook actually runs most. Either state that trade-off with the number in the body and in .ai/agent-practices.md "Test concurrency", or let the cap scale — e.g. min(available, max(4, available - 2)) with a ceiling of 8 gives 2→2, 4→4, 8→6, 14→8, keeps two cores free for the UI and the serial stage, and still stops the dozen-plus-process case. Your call which; the measurement should appear either way.
Smaller:
README.md:123says "4 workers"; the formula ismin(4, available)(your own 2-core acceptance line)..ai/agent-practices.mdhas it right.scripts/run-tests.js:23-26is three lines of rationale plus the pointer; the ceiling here is the pointer alone (PRs #127/#130).test/run-tests-concurrency.test.js: the bad-value cases assert the fallback but not the warning; the acceptance line says "warns and falls back". A mutedconsole.errorpasses today. Capture stderr or drop "warns".
Checked, clean: formula yields 1/2/4/4/4/4 for 1/2/4/8/16/64 cores; availableParallelism with the cpus().length fallback (dead under engines >=20, harmless); Number() + isInteger && > 0 rejects NaN/0/negative/decimal/text, empty string treated as unset; --test-concurrency=N is a real flag and stage 2 stays at 1; the test imports the real export, spawns nothing; section renumbering in agent-practices has no stale cross-reference; no attribution trailers; CI green on e350123.
Why
npm testruns stage 1 withnode --testover every test file and no--test-concurrency, so node applies its own default:os.availableParallelism()workers. On a 14-core workstation that is a dozen-plus test processes at once.Two costs, both measured on this machine:
test/trigger-watcher.test.jsuses real timers and realfs.watchagainst wall-clock budgets.scripts/run-tests.jsalready isolates it into its own serial stage for exactly this reason (issue (test): trigger-watcher timing tests fail under host load and block the pre-commit hook #260), but stage 1 handing every core to itself is the same contention one step earlier.What
Stage 1 runs at 4 workers, bounded by what the machine actually has:
SWITCHBOARD_TEST_CONCURRENCYoverrides it. A value that is not a positive integer is ignored with a message on stderr and the cap applies, so a typo slows a run down rather than failing it.CI is unaffected: GitHub's
ubuntu-latestandwindows-2022runners have 4 cores or fewer, somin(4, available)is what they were already getting.Stage 2 is untouched — it was already
--test-concurrency=1.The runner now executes only when it is the entry point (
require.main === module), sostageOneConcurrencycan be imported and tested rather than inferred from the spawn.Not in scope
tasksetand friends: capping the whole process tree is the right tool when several agents run the suite at once, and it is documented in.ai/agent-practices.mdrather than wired into the runner. Node'savailableParallelism()honours the affinity mask, sotaskset -c 0-3 npm testreduces the number of workers rather than crowding them onto fewer cores — measured:taskset -c 0-3 node -e 'os.availableParallelism()'reports 4 on this 14-core box.Acceptance
npm testwith no environment set spawns at most 4 stage-1 workers.SWITCHBOARD_TEST_CONCURRENCY=12 npm testspawns 12.SWITCHBOARD_TEST_CONCURRENCY=bananawarns and falls back to 4.availableParallelism()turnsunset: caps at DEFAULT_CONCURRENCY rather than taking every corered on any machine with more than 4 cores.Verification
npx eslint .→ 0 errors, 317 warnings (unchanged).npm testunder the new cap → stage 1tests 1601 / pass 1598 / fail 1 / skipped 2, stage 2tests 120 / pass 119 / fail 0 / skipped 1. The single failure istest/ipc-path-validator.test.js"allows files under ~/.claude/", pre-existing and environmental on this machine:~/.claude/CLAUDE.mdis a symlink out of~/.claudeand the validator resolves on disk. This branch touches neither that test noripc-path-validator.js.trigger-watcher.test.jspassed 119/119 under the cap, which was the risk worth checking — fewer cores could in principle have made its wall-clock assertions flakier rather than steadier.