Skip to content

perf: centralize chunk ownership and use array-backed storage - #15730

Draft
SyMind wants to merge 2 commits into
mainfrom
codex/chunk-graph-array-storage
Draft

SyMind wants to merge 2 commits into
mainfrom
codex/chunk-graph-array-storage

Conversation

@SyMind

@SyMind SyMind commented Sep 16, 2026

Copy link
Copy Markdown
Member

Motivation

Chunk allocation currently uses process-wide keys and separate hash tables for Chunk values and graph relationships. Centralizing ownership makes graph-local array indexing possible while preserving identity across deletion, slot reuse, and incremental snapshot restoration.

Changes

  • Make ChunkGraph own and allocate Chunks and topology together. Route creation, removal, splitting, plugin hooks, HMR, and temporary module-execution graphs through the shared ownership model.
  • Represent ChunkUkey as a reusable slot index plus a checked, process-wide nonzero identity. Use array-backed ChunkSlotMap tables, validate complete keys, and preserve identities when cloning graph snapshots.
  • Add preallocated ChunkMap / ChunkSet working containers for CodeSplitter masks, SplitChunks indices, chunk-combination lookup, runtime requirements, and current-pass worksets. Keep historical caches and sparse collections hash-based; preserve explicit semantic ordering.
  • Preserve JavaScript Compilation ownership checks and use lossless process-local diagnostic references. Update architecture documentation and add dedicated Rust integration tests.
  • Adapt existing benchmarks to the ownership API without adding benchmark cases.

Validation

  • Built the native binding and dedicated Rust test artifacts before running tests.
  • cargo test -p rspack_core_test: 13 passed.
  • Existing JS suites: chunks/watch 539, hash 192, cache 70, chunk IDs 16, and import-module 92 passed.
  • cargo fmt --all --check, git diff --check, and commit formatting/spelling hooks passed.
  • Existing local CodSpeed walltime comparisons are available here. These isolate the latest working-table migration, not the full PR against main. React build, SplitChunks, and full hash showed no significant change. Runtime requirements varied from 16.29 ms to 15.59 ms against a 15.92 ms baseline on repeat, so no stable gain is claimed. PR CodSpeed results will be used to evaluate the full change and guide further optimization.

Tradeoffs

Keys grow to eight bytes. Array traversal and allocation scale with the slot high-water mark, including holes; sparse and historical collections retain HashMap/HashSet storage. The identity counter deliberately fails on exhaustion rather than wrapping. Incremental pass and cache invalidation rules remain in effect.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying rspack with  Cloudflare Pages  Cloudflare Pages

Latest commit: e19ef49
Status: ✅  Deploy successful!
Preview URL: https://0869b0f8.rspack-v2.pages.dev
Branch Preview URL: https://codex-chunk-graph-array-stor.rspack-v2.pages.dev

View logs

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

📦 Binary Size-limit

Comparing 8385bb9 to test: migrate expectation files to ESM (#15726) by Jiahan Chen

❌ Size increased by 24.00KB from 68.77MB to 68.79MB (⬆️0.03%)

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Rsdoctor Bundle Diff Analysis

⚠️ Note: The latest commit (f07788809426ce8ad36c435207ce7b0fd5943d45) does not have baseline artifacts. Using commit d4cd073db21483ed7c524dedf59ad1d8bf400b8e for baseline comparison instead. If this seems incorrect, please wait a few minutes and try rerunning the workflow.

Found 5 projects in monorepo, 0 projects with changes.

📊 Quick Summary
Project Total Size Gzip Size Change Gzip Change
popular-libs 1.7 MB 529.8 KB 0 0
react-10k 5.4 MB 1.3 MB 0 0
react-1k 827.7 KB 226.6 KB 0 0
react-5k 2.6 MB 670.9 KB 0 0
ui-components 5.0 MB 1.4 MB 0 0

Generated by Rsdoctor GitHub Action

@codspeed

codspeed Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 50 untouched benchmarks
⏩ 47 skipped benchmarks1


Comparing codex/chunk-graph-array-storage (8385bb9) with main (d4cd073)2

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (f077888) during the generation of this report, so d4cd073 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@SyMind
SyMind marked this pull request as draft September 16, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant