Skip to content

perf: use SsoHashSet for small module chunk memberships - #15731

Merged
SyMind merged 6 commits into
mainfrom
codex/chunk-graph-module-sso-set
Sep 17, 2026
Merged

SyMind merged 6 commits into
mainfrom
codex/chunk-graph-module-sso-set

Conversation

@SyMind

@SyMind SyMind commented Sep 16, 2026

Copy link
Copy Markdown
Member

Motivation

ChunkGraphModule::chunks allocates a hash table even when a module belongs to only one or a few chunks. Inspired by rustc's SsoHashSet, keep these small memberships inline.

In the main CodSpeed baseline, connect_chunk_and_module accounts for 7.14% of the rust@build_chunk_graph simulation profile; chunk-set insertion accounts for 3.85%, including 2.99% in hash-table growth.

Changes

  • Introduce reusable rspack_collections::SsoHashSet<T>, storing up to four elements inline and promoting to FxHashSet on the fifth distinct insertion. Chunk graph and plugin consumers use SsoHashSet<ChunkUkey> directly, without a ModuleChunks alias. Large sets retain their allocation on removal/clear to avoid repeated promotion.
  • Support non-Copy, non-Clone elements by moving them during promotion. Add borrowed lookup/removal, Extend, and FromIterator.
  • Preserve membership checks, deduplication, removal, subset checks, and borrowed/owned iteration. Iteration order remains unspecified.
  • Retain the same representation in module-concatenation and SplitChunks snapshots. Convert to a hash set only when preparing a chunk combination. Clone is needed to preserve memberships while mutating the graph and to retain independent concatenation snapshots; small clones do not allocate.
  • Adapt the split-chunks and ESM-library consumers without changing JavaScript APIs or configuration.
  • Cover 1/4/5/9 memberships, promotion, and removal/reconnection through SplitChunks in the existing config-case harness. The case uses shared.js directly as its entry and validates memberships in compilation hooks, without a separate runtime test entry.

Validation of the generic implementation (latest alias removal additionally rebuilt the binding and passed focused Clippy, formatting, 2 membership and 202 SplitChunks regression tests):

  • Rebuilt the development native binding; JavaScript packages were already built and are unchanged.
  • On the current 64-bit target, SsoHashSet<ChunkUkey> remains 32 bytes and ChunkGraphModule remains 96 bytes.
  • New regression case: 2 harness tests passed (8 configurations in each runtime mode).
  • SplitChunks config cases: 202 passed.
  • Scope-hoisting config cases: 20 passed.
  • Chunk HMR cases: 77 passed.
  • Rust formatting and focused Clippy with all targets and warnings denied passed.

Remote validation for 0372199 completed successfully: PR CI, dispatched simulation and walltime CI, and the CodSpeed performance check passed.

Latest CodSpeed comparison against main:

Benchmark Mode Base → head Result
rust@build_chunk_graph CPU simulation 26.2 → 25.2 ms +3.58% efficiency in the PR report
rust@split_chunks CPU simulation 1.6 → 1.6 ms (rounded) About -1%; below the 2% threshold, classified untouched
rust@create_concatenate_module CPU simulation 13.7 → 13.5 ms About +1%; classified untouched
bundle@threejs-10x-development Walltime 199.3 → 197.5 ms Classified untouched
bundle@threejs-10x-production-sourcemap Walltime 468.2 → 460.7 ms Classified untouched

The latest PR simulation report contains 1 improvement, 49 untouched benchmarks, 47 skipped benchmarks, and no detected regressions. The combined simulation/walltime comparison contains 1 improvement and 96 untouched benchmarks, with no detected regressions. Simulation values are simulated timings, not elapsed walltime; the 3.58% improvement applies to the chunk-graph benchmark, not total bundling time.

Compared with the previous specialized implementation at 77e7d7c, all 97 benchmarks are classified untouched. Generalizing the collection and removing the aliases introduced no detected significant regression.

CodSpeed flags different runtime environments for the comparison against main and identifies the self-hosted walltime environment as unknown, so small differences should be interpreted cautiously. The main comparison uses d4cd073 because f077888 has no benchmark run; their diff contains website files only.

@SyMind
SyMind requested a review from JSerFeng as a code owner September 16, 2026 23:26
@SyMind
SyMind marked this pull request as draft September 16, 2026 23:27
@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

📦 Binary Size-limit

Comparing 8eb5682 to test: migrate expectation files to ESM (#15726) by Jiahan Chen

❌ Size increased by 20.00KB from 68.77MB to 68.79MB (⬆️0.03%)

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Rsdoctor Bundle Diff Analysis

⚠️ Note: The latest commit (f07788809426ce8ad36c435207ce7b0fd5943d45) does not have baseline artifacts. Using commit d4cd073db21483ed7c524dedf59ad1d8bf400b8e for baseline comparison instead. If this seems incorrect, please wait a few minutes and try rerunning the workflow.

Found 5 projects in monorepo, 0 projects with changes.

📊 Quick Summary
Project Total Size Gzip Size Change Gzip Change
popular-libs 1.7 MB 529.8 KB 0 0
react-10k 5.4 MB 1.3 MB 0 0
react-1k 827.7 KB 226.6 KB 0 0
react-5k 2.6 MB 670.9 KB 0 0
ui-components 5.0 MB 1.4 MB 0 0

Generated by Rsdoctor GitHub Action

@codspeed

codspeed Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by 3.58%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 49 untouched benchmarks
⏩ 47 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
rust@build_chunk_graph 26.2 ms 25.2 ms +3.58%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/chunk-graph-module-sso-set (8eb5682) with main (d4cd073)2

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (f077888) during the generation of this report, so d4cd073 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@SyMind SyMind changed the title perf: inline small module chunk memberships perf: use SsoHashSet for small module chunk memberships Sep 17, 2026
@SyMind
SyMind marked this pull request as ready for review September 17, 2026 02:09
@JSerFeng

Copy link
Copy Markdown
Contributor

@codspeedbot explain why performance improved

@codspeed

codspeed Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

@codspeedbot explain why performance improved

Why rust@build_chunk_graph got faster

I compared the Simulation flamegraphs of the base (d4cd073, 26.2 ms) and head (8eb5682, 25.2 ms) runs. The whole −1.04 ms is concentrated in one function: the chunk-membership insert you changed.

Subtree BASE HEAD Δ
ChunkGraph::connect_chunk_and_module 1.87 ms (7.14%) 0.99 ms (3.93%) −0.88 ms
HashMap<ChunkUkey, ()>::insert 1.00 ms (3.85%) not in profile (<1%) −1.00 ms
↳ ↳ RawTable::reserve_rehash (table alloc + growth) 781 µs (2.99%) −781 µs
CodeSplitter::add_and_enter_module (its parent) 11.7 ms 10.8 ms −0.9 ms
CodeSplitter::split 16.3 ms 15.4 ms −0.9 ms

That −0.88 ms is ~85% of the total benchmark delta, and it is exactly the cost the PR targets: for the common case of a module belonging to only a few chunks, no hashbrown table is allocated and no rehash/growth happens — the ukeys stay inline. The residual growth cost that remains is on the ChunkGraphChunk side (chunk → modules), which the PR does not change.

Everything else is flat, which is what makes the attribution solid rather than a codegen or allocator artifact:

  • memcpy self time 993.7 µs in both runs (identical to the tenth of a µs), probe_seq 1.1 ms both
  • prepare_module_connection_map 5.6 ms both, process_block 7.6 ms both, get_block_modules 4.3 ms both, enter_module 8.7 ms both
  • ChunkGraph::add_module 765 → 752 µs, allocator totals unchanged (mi_malloc_aligned 2.8 → 2.7 ms) — no allocation was pushed elsewhere

Stability check: comparing head against your branch's own first implementation commit (33fadc5) shows rust@build_chunk_graph as untouched, so the 25.2 ms value reproduces across the branch and is not run-to-run jitter. This matters because CodSpeed flagged different runner CPUs (EPYC 7763 → 9V74) between base and head; a cross-CPU shift would move memcpy/library subtrees too, and here they are byte-identical.

Scope of the win: it applies where chunk memberships are built. The bundle walltime cases are classified untouched, so this is a chunk-graph phase improvement, not a ~3.6% reduction in overall bundling time.

Summary
  • Warming up the performance wizard...
  • Analyze why rust@build_chunk_graph improved — Compared base (d4cd073) and head (8eb5682) Simulation flamegraphs; delta localized to connect_chunk_and_module
  • Check the result is stable, not runner noise — Cross-checked the branch's own earlier run (33fadc5): build_chunk_graph unchanged across branch commits
  • Handling comment — Comment handling finished successfully

Learnings updated. See what CodSpeed remembers about this repository.

Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.

@SyMind
SyMind merged commit 1a2b4cc into main Sep 17, 2026
48 checks passed
@SyMind
SyMind deleted the codex/chunk-graph-module-sso-set branch September 17, 2026 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants