Skip to content

refactor(loader): separate reusable metadata from execution state - #15602

Closed
intellild wants to merge 5 commits into
mainfrom
codex/loader-metadata
Closed

intellild wants to merge 5 commits into
mainfrom
codex/loader-metadata

Conversation

@intellild

@intellild intellild commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Motivation

Resolved loader metadata should be reusable across runs, while pitch data and execution flags belong to each run. The JavaScript bridge also used to combine both in JsLoaderItem, converting unchanged loader requests, types and cache flags back to Rust on each return.

For example, with JavaScript loader -> builtin:swc-loader -> JavaScript loader, each JavaScript invocation can update dependencies and pitch data and return { loaderData, state } without repeating loader metadata. Each loader retains its own pitch data for its normal stage, while content, source maps and additional data pass along the pipeline as one current value each. A rebuild reuses the resolved native metadata but starts with fresh execution flags and pitch data.

Changes

  • Store Loaders directly on the normal module, backed by Vec<ResolvedLoader> and lazily initialized immutable LoaderItem metadata. Access it through LoaderRunnerContext::loaders().
  • Create fresh loader_data and LoaderItemState collections for each native run, indexed by loader position. Keep each loader's pitch data separate from execution flags, and mirror the split in the binding with loaderData, JsLoaderMetadata and JsLoaderItemState.
  • Pass an owned #[napi(object)] JsLoaderContextState and a separate loaderData array to JavaScript. The existing loader-context getters/setters update them, and the runner returns both through a single #[napi(object)] JsLoaderResult for Rust to merge.
  • Adapt parallel-loader payloads, pitch data writeback and loader-cache access to the separated data and state. Preserve the public module loader snapshot shape, existing Buffer/source-map transfers, hook timing and native context ownership.
  • Mark finish_called only for loaders whose normal stage has executed. Previously, an unrelated JavaScript return could mark a pending builtin loader as finished and incorrectly preserve its input when it returned no output.
  • Cover return conversion without metadata, independent pitch data across builtin loaders, pitch data replacement in a worker, binary/map/additional-data roundtrips and a native loader that returns no output; document the boundary in English and Chinese.

Stack

  1. refactor(loader): replace should-yield hooks with execution kind #15592 — execution kind and removal of the should-yield hook (base: main).
  2. This PR — reusable metadata and per-run state, including the JavaScript bridge.
  3. perf(loader): schedule loader chains as execution units #15401 — LoaderChain scheduling and Rust-side chain caching.

The class/boxed-context ownership protocol, BoxSource transfer and cross-invocation JavaScript wrapper reuse remain separate follow-up work.

Validation

  • pnpm run build:cli:dev — passed.
  • cargo clippy -p rspack_binding_api -p rspack_loader_runner --all-targets --offline -- -D warnings — passed.
  • cargo test -p rspack_loader_runner --offline — 10 passed.
  • pnpm --dir tests/rspack-test run test --project base -t loader — 567 passed, including parallel loaders, cache/rebuild cases and the new state regressions.
  • pnpm --dir tests/rspack-test run test --project base Config -t source-map — 184 passed.
  • Rust formatting, JavaScript lint/format, documentation spelling and git diff --check — passed.
  • Binding declarations regenerated from the Rust N-API definitions.

by OpenAI Codex

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 10, 2026

Copy link
Copy Markdown

Deploying rspack with  Cloudflare Pages  Cloudflare Pages

Latest commit: b67ed7a
Status: ✅  Deploy successful!
Preview URL: https://f279ecf5.rspack-v2.pages.dev
Branch Preview URL: https://codex-loader-metadata.rspack-v2.pages.dev

View logs

@intellild
intellild added this pull request to stack #15603 September 10, 2026 02:56
@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

📦 Binary Size-limit

Comparing b67ed7a to test: use rspack.sources in test cases (#15734) by Jiahan Chen

🎉 Size decreased by 12.00KB from 68.79MB to 68.78MB (⬇️0.02%)

@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Rsdoctor Bundle Diff Analysis

Found 5 projects in monorepo, 0 projects with changes.

📊 Quick Summary
Project Total Size Gzip Size Change Gzip Change
popular-libs 1.7 MB 529.8 KB 0 0
react-10k 5.4 MB 1.3 MB 0 0
react-1k 827.7 KB 226.6 KB 0 0
ui-components 5.0 MB 1.4 MB 0 0
react-5k 2.6 MB 670.9 KB 0 0

Generated by Rsdoctor GitHub Action

@codspeed

codspeed Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 0.03%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 48 untouched benchmarks
⏩ 47 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
rust@create_full_hash 2.3 ms 2.4 ms -2.18%
rust@create_module_hashes 7.6 ms 7.4 ms +2.17%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/loader-metadata (b67ed7a) with main (7163c74)

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@intellild

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-16T08:17:44.194816Z 933ff2f Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 292af28c37

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@intellild
intellild force-pushed the codex/loader-execution-kind branch from 55b3464 to 241276d Compare September 14, 2026 02:41
Base automatically changed from codex/loader-execution-kind to main September 14, 2026 05:43
@intellild
intellild force-pushed the codex/loader-metadata branch 2 times, most recently from 3280b36 to f1bc654 Compare September 14, 2026 06:54
@intellild

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: f1bc6544e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@intellild
intellild marked this pull request as ready for review September 14, 2026 09:05
@intellild
intellild requested a review from hardfist as a code owner September 14, 2026 09:05
@intellild
intellild requested a review from SyMind September 14, 2026 12:03
@intellild
intellild marked this pull request as draft September 15, 2026 07:31
@intellild
intellild force-pushed the codex/loader-metadata branch 2 times, most recently from 32dd6cc to d5cf27b Compare September 16, 2026 04:45
@intellild

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 933ff2fb04

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@intellild
intellild force-pushed the codex/loader-metadata branch from 037fbe4 to b67ed7a Compare September 17, 2026 04:38
@intellild intellild closed this Sep 17, 2026
@intellild
intellild removed this pull request from stack #15603 September 17, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant