Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .agents/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,37 @@ Manages modules and their relationships through dependencies and connections.

### Chunk system

`ChunkGraph` owns both Chunk values and their relationships. Allocate through
`ChunkGraph::create_chunk`; named creation and removal go through
`BuildChunkGraphArtifact`. `ChunkSlotMap<Chunk>` exposes value access, not independent
insertion or removal. `CompilationChunkIds` receives `&mut Compilation`.

A `ChunkUkey` combines an array slot with a process-wide allocation identity.
Deleted slots may be reused, but every access checks the full key. Graph snapshots
copy values and the graph's free-slot stack without copying the identity allocator.
Identities are `NonZeroU32`. Following the layout used by `slotmap::SecondaryMap`,
each table stores `Vec<Option<(ChunkUkey, T)>>`: the identity supplies a niche for
the empty state, without manual initialization or destruction. Only `ChunkGraph`
allocates identities and recycles slots; value tables carry no allocation state.
Sequential insertion uses `Vec::push`, and `with_capacity` reserves storage without
initializing slots. Traversal still scans the slot high-water mark, so sparse or
historical collections should retain HashMap/HashSet storage.
`ChunkMap<T>` reuses that storage for per-Chunk working data, such as CodeSplitter's
module masks. Its insertion API requires the current owning Chunk table and checks
membership before replacing a slot. Reserve using `slot_count()` (plus expected
new Chunks while building), not `len()`. A map can hold one identity per slot and
must not replace caches that need multiple historical identities.
`ChunkSet` uses the same storage for current-graph membership, retaining the full
identity instead of only a slot bit. Per-pass tables for split-chunk indices,
chunk-combination lookup, and chunk runtime requirements reserve the slot
high-water mark up front. Algorithms that expose processing order sort by their
semantic order or the full key; they must not derive order from reusable slots.
The `as_u32()`
projection is for tooling identities, never array indexing; `as_u64()` preserves
both fields for process-local diagnostics. These handles are not persistent cache
identities. Graph algorithms access their owned Chunk table directly; topology
and values can be borrowed separately when an algorithm needs both.

Chunks are groups of modules bundled together.

**Chunk Types:**
Expand Down
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 22 additions & 1 deletion crates/rspack_binding_api/src/chunk.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,27 @@ pub struct Chunk {
}

impl Chunk {
// Incremental snapshots preserve keys across compilations. Native identity
// does not extend the lifetime of a JavaScript handle from an earlier build.
pub(crate) fn belongs_to(&self, compilation: &Compilation) -> bool {
self.compilation_id == compilation.id()
&& compilation
.build_chunk_graph_artifact
.chunk_graph
.chunks
.contains(&self.chunk_ukey)
}

pub(crate) fn validate_compilation(&self, compilation: &Compilation) -> napi::Result<()> {
if self.belongs_to(compilation) {
Ok(())
} else {
Err(napi::Error::from_reason(
"Chunk does not belong to this Compilation or has been removed",
))
}
}

fn with_compilation<R>(
&self,
f: impl FnOnce(&Compilation) -> napi::Result<R>,
Expand All @@ -37,7 +58,7 @@ impl Chunk {
self.with_compilation(|compilation| {
if let Some(chunk) = compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph.chunks
.get(&self.chunk_ukey)
{
f(compilation, chunk)
Expand Down
7 changes: 6 additions & 1 deletion crates/rspack_binding_api/src/chunk_graph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ impl ChunkGraph {
#[napi(ts_return_type = "boolean")]
pub fn has_chunk_entry_dependent_chunks(&self, chunk: &Chunk) -> Result<bool> {
self.with_compilation(|compilation| {
chunk.validate_compilation(compilation)?;
Ok(
compilation
.build_chunk_graph_artifact
Expand All @@ -61,6 +62,7 @@ impl ChunkGraph {
#[napi(ts_return_type = "Module[]")]
pub fn get_chunk_modules(&self, chunk: &Chunk) -> Result<Vec<ModuleObject>> {
self.with_compilation(|compilation| {
chunk.validate_compilation(compilation)?;
let module_graph = compilation.get_module_graph();
let modules = compilation
.build_chunk_graph_artifact
Expand All @@ -84,6 +86,7 @@ impl ChunkGraph {
#[napi(ts_return_type = "Iterable<Module>")]
pub fn get_chunk_entry_modules_iterable(&self, chunk: &Chunk) -> Result<Vec<ModuleObject>> {
self.with_compilation(|compilation| {
chunk.validate_compilation(compilation)?;
let modules = compilation
.build_chunk_graph_artifact
.chunk_graph
Expand All @@ -102,6 +105,7 @@ impl ChunkGraph {
#[napi(ts_return_type = "number")]
pub fn get_number_of_entry_modules(&self, chunk: &Chunk) -> Result<u32> {
self.with_compilation(|compilation| {
chunk.validate_compilation(compilation)?;
Ok(
compilation
.build_chunk_graph_artifact
Expand All @@ -117,12 +121,12 @@ impl ChunkGraph {
chunk: &Chunk,
) -> Result<Vec<ChunkWrapper>> {
self.with_compilation(|compilation| {
chunk.validate_compilation(compilation)?;
let chunks = compilation
.build_chunk_graph_artifact
.chunk_graph
.get_chunk_entry_dependent_chunks_iterable(
&chunk.chunk_ukey,
&compilation.build_chunk_graph_artifact.chunk_by_ukey,
&compilation.build_chunk_graph_artifact.chunk_group_by_ukey,
);

Expand All @@ -142,6 +146,7 @@ impl ChunkGraph {
source_type: String,
) -> Result<Vec<ModuleObject>> {
self.with_compilation(|compilation| {
chunk.validate_compilation(compilation)?;
let module_graph = compilation.get_module_graph();

let chunk_modules = compilation
Expand Down
3 changes: 2 additions & 1 deletion crates/rspack_binding_api/src/chunk_group.rs
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,8 @@ impl ChunkGroup {
.filter_map(|chunk_ukey| {
compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph
.chunks
.get(chunk_ukey)
.map(|chunk| chunk.files().iter())
})
Expand Down
18 changes: 10 additions & 8 deletions crates/rspack_binding_api/src/compilation/chunks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,13 @@ impl Chunks {
#[napi(getter)]
pub fn size(&self) -> napi::Result<u32> {
let compilation = self.as_ref()?;
Ok(compilation.build_chunk_graph_artifact.chunk_by_ukey.len() as u32)
Ok(
compilation
.build_chunk_graph_artifact
.chunk_graph
.chunks
.len() as u32,
)
}

#[napi(js_name = "_values", ts_return_type = "Chunk[]")]
Expand All @@ -49,7 +55,8 @@ impl Chunks {
Ok(
compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph
.chunks
.keys()
.map(|chunk_ukey| ChunkWrapper::new(*chunk_ukey, compilation))
.collect::<Vec<_>>(),
Expand All @@ -59,11 +66,6 @@ impl Chunks {
#[napi(js_name = "_has")]
pub fn has(&self, chunk: &Chunk) -> napi::Result<bool> {
let compilation = self.as_ref()?;
Ok(
compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.contains(&chunk.chunk_ukey),
)
Ok(chunk.belongs_to(compilation))
}
}
3 changes: 2 additions & 1 deletion crates/rspack_binding_api/src/compilation/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,8 @@ impl JsCompilation {
.and_then(|c| {
compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph
.chunks
.get(c)
.map(|chunk| ChunkWrapper::new(chunk.ukey(), compilation))
}),
Expand Down
2 changes: 1 addition & 1 deletion crates/rspack_binding_api/src/path_data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ impl JsPathData {
Some(
compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph.chunks
.get(&chunk_ukey)
.ok_or_else(|| {
napi::Error::from_reason(format!(
Expand Down
28 changes: 22 additions & 6 deletions crates/rspack_core/src/artifacts/build_chunk_graph_artifact.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,14 @@ use rustc_hash::FxHashMap as HashMap;
use tracing::instrument;

use crate::{
ArtifactExt, ChunkByUkey, ChunkGraph, ChunkGroupByUkey, ChunkGroupUkey, ChunkUkey, Compilation,
Logger,
ArtifactExt, ChunkGraph, ChunkGroupByUkey, ChunkGroupUkey, ChunkUkey, Compilation, Logger,
build_chunk_graph::code_splitter::CodeSplitter,
fast_set,
incremental::{IncrementalPasses, Mutation},
};

#[derive(Debug, Default)]
pub struct BuildChunkGraphArtifact {
pub chunk_by_ukey: ChunkByUkey,
pub chunk_graph: ChunkGraph,
pub chunk_group_by_ukey: ChunkGroupByUkey,
pub entrypoints: FxIndexMap<String, ChunkGroupUkey>,
Expand All @@ -29,6 +27,26 @@ pub struct BuildChunkGraphArtifact {
}

impl BuildChunkGraphArtifact {
pub fn add_named_chunk(&mut self, name: String) -> rspack_error::Result<(ChunkUkey, bool)> {
if let Some(key) = self.named_chunks.get(&name).copied() {
assert!(self.chunk_graph.chunks.contains(&key));
return Ok((key, false));
}
let key = self
.chunk_graph
.create_chunk(Some(name.clone()), crate::ChunkKind::Normal)?;
self.named_chunks.insert(name, key);
Ok((key, true))
}

pub fn remove_chunk(&mut self, key: &ChunkUkey) -> Option<crate::Chunk> {
let chunk = self
.chunk_graph
.remove_chunk(key, &mut self.chunk_group_by_ukey)?;
self.named_chunks.retain(|_, stored| stored != key);
Some(chunk)
}

pub(crate) fn set_code_splitter(&mut self, code_splitter: CodeSplitter) {
fast_set(&mut self.code_splitter, code_splitter);
}
Expand Down Expand Up @@ -104,13 +122,12 @@ impl BuildChunkGraphArtifact {
/// cached chunks across incremental compilations, so we need to restore the
/// same state before running the next sealing/rendering pipeline.
fn reset_chunk_rendered_state(&mut self) {
for chunk in self.chunk_by_ukey.values_mut() {
for chunk in self.chunk_graph.chunks.values_mut() {
chunk.set_rendered(false);
}
}

fn reset_for_rebuild(&mut self) {
self.chunk_by_ukey = Default::default();
self.chunk_graph = Default::default();
self.chunk_group_by_ukey = Default::default();
self.entrypoints.clear();
Expand Down Expand Up @@ -188,7 +205,6 @@ impl ArtifactExt for BuildChunkGraphArtifact {
fn recover(_incremental: &crate::incremental::Incremental, new: &mut Self, old: &mut Self) {
new.code_splitter = mem::take(&mut old.code_splitter);
rayon::scope(|s| {
s.spawn(|_| new.chunk_by_ukey.clone_from(&old.chunk_by_ukey));
s.spawn(|_| new.chunk_graph.clone_from(&old.chunk_graph));
s.spawn(|_| new.chunk_group_by_ukey.clone_from(&old.chunk_group_by_ukey));

Expand Down
16 changes: 6 additions & 10 deletions crates/rspack_core/src/chunk.rs
Original file line number Diff line number Diff line change
Expand Up @@ -310,12 +310,12 @@ impl Chunk {
}

impl Chunk {
pub fn new(name: Option<String>, kind: ChunkKind) -> Self {
pub(crate) fn new(ukey: ChunkUkey, name: Option<String>, kind: ChunkKind) -> Self {
Self {
name,
filename_template: None,
css_filename_template: None,
ukey: ChunkUkey::new(),
ukey,
id: None,
id_name_hints: Default::default(),
prevent_integration: false,
Expand Down Expand Up @@ -378,12 +378,6 @@ impl Chunk {
split_data.add_runtime(&self.runtime);
}

pub fn split(&mut self, new_chunk: &mut Chunk, chunk_group_by_ukey: &mut ChunkGroupByUkey) {
let mut split_data = ChunkSplitData::with_capacity(self.groups.len(), self.id_name_hints.len());
self.split_collect_new_chunk_data(new_chunk.ukey, chunk_group_by_ukey, &mut split_data);
split_data.apply_to(new_chunk);
}

pub fn can_be_initial(&self, chunk_group_by_ukey: &ChunkGroupByUkey) -> bool {
self
.groups
Expand Down Expand Up @@ -921,7 +915,8 @@ impl Chunk {
if filter_fn(chunk_ukey, compilation)
&& let Some(chunk_id) = compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph
.chunks
.expect_get(chunk_ukey)
.id()
.cloned()
Expand Down Expand Up @@ -952,7 +947,8 @@ impl Chunk {
) -> Option<(ChunkId, Vec<ChunkId>)> {
let chunk = compilation
.build_chunk_graph_artifact
.chunk_by_ukey
.chunk_graph
.chunks
.expect_get(chunk_ukey);
if let (Some(chunk_id), Some(child_chunk_ids)) = (
chunk.id().cloned(),
Expand Down
Loading
Loading