Skip to content

Use ModelingToolkit's generated initialization maps in the kernel - #541

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/use-mtk-generated-init-maps
Sep 13, 2026
Merged

ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/use-mtk-generated-init-maps

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

What changed and why

ModelingToolkit now emits the state and parameter initialization maps as isbits
RuntimeGeneratedFunctions under FullSpecialize
(#5043,
#5045), so EnsembleGPUKernel can
evaluate them on the device as they are. This deletes the host-side symbolic tracing that
reconstructed them as gather recipes — the approach in the closed
#516 — and uses the maps directly.

ext/ModelingToolkitBaseExt.jl goes from 314 to 108 lines. The tracing could not express
map entries computed from the solved initialization system at all; the generated maps
handle them for free.

A problem qualifies when both maps are isbits and the state map builds an isbits u0.
Reaching that needs three things together, and each missing one now gets its own error
naming what to change:

  1. SciMLBase.FullSpecialize, or the maps are host closures that cannot be uploaded.
  2. Static u0_constructor/p_constructor, because the maps rebuild u0/p inside the
    kernel, which cannot allocate. ModelingToolkit fixes the container when the problem is
    built, so this cannot be corrected afterwards.
  3. Out-of-place. Not an independent choice: an MVector is a mutable struct and so is not
    isbits, which rules it out of a device array; an in-place problem cannot write into an
    immutable SVector. Out-of-place plus SVector is the only self-consistent pairing.

make_parameter_compatible additionally rejects parameters whose contents are not isbits
— an interpolation object, a callable closing over an array, a type — naming the offending
MTKParameters portions. Converting buffer storage to SArray cannot rescue those, and
they previously produced a silently non-isbits problem that failed deep in the kernel.

Conversion hook

The extension's static_parameter_storage was a duplicate of make_static_storage in
src/utils.jl, so it is gone. The survivor is documented and @public as the conversion
hook, so a package owning a non-isbits type can make it work without a change here:

# In the package that owns `MyInterpolation`
function DiffEqGPU.make_static_storage(itp::MyInterpolation)
    return MyStaticInterpolation(
        DiffEqGPU.make_static_storage(itp.t), DiffEqGPU.make_static_storage(itp.u)
    )
end

Both u0 and the parameters route through it. Unconverted values fall through the identity
method and are then caught by the isbits check, so the failure mode is an error naming the
value rather than a kernel crash.

Verification

Julia 1.12.7, GROUP=JLArrays. The full ModelingToolkit DAE suite, all 18 testsets:

GPURosenbrock23 DAE                            | 3  3
GPURodas4 DAE                                  | 3  3
GPURodas5P DAE                                 | 3  3
GPUKvaerno3 DAE                                | 3  3
GPUKvaerno5 DAE                                | 3  3
Structured MTKParameters storage               | 7  7
Underdetermined initialization                 | 2  2
Overdetermined initialization                  | 2  2
Bounded initialization                         | 6  6
Immutable SCC initialization                   | 3  3
SCC initialization rejects non-triangular      | 1  1
Stateless all-linear SCC initialization        | 3  3
Host symbolic setter with trivial init         | 5  5
MTK Pendulum DAE with initialization           | 21 21
Computed initialization maps                   | 4  4
Initialization maps must be kernel-compatible  | 4  4
Non-isbits parameters are rejected             | 4  4
make_static_storage is the conversion hook     | 7  7

Zero failures, exit 0. GROUP=QA: Extensions loaded 6/6, Quality Assurance 21/21, exit
0 — all six backend extensions load, so ExplicitImports sees the real module set. Runic,
typos over the diff and git diff --check are clean.

The Computed initialization maps testset is the case closed #516 existed to fix: my is
an observed variable of the torn initialization system, so the state map computes it rather
than copying it. Host reference and kernel agree exactly (u0 = [3.0, 1.0]). The old
tracing threw on this shape.

What I did not verify

  • No CUDA run. This machine has no GPU, so everything above is JLArrays, which runs
    kernels on the CPU. That matters here specifically: a heap allocation inside a kernel
    passes under JLArrays and fails on a real device. That is why the isbits checks are
    explicit host-side assertions rather than something inferred from a green suite — but a
    CUDA run is still the real confirmation.
  • The registry does not yet have the ModelingToolkit side. ModelingToolkitBase
    v1.71.3 predates Fix FullSpecialize initialization map buffer types ModelingToolkit.jl#5145, which fixes the
    map buffer types this depends on. Everything above was run against ModelingToolkit
    master. CI will be red until a ModelingToolkitBase release carrying #5145 exists,
    at which point the [compat] lower bound should be raised to it; I have not guessed a
    version number.
  • Only the JLArrays and QA groups were run. CPU, CPU 32-bit, OpenCL and the docs
    build were not.

Anything a reviewer should push back on

  • This narrows what is accepted: a problem that previously went through tracing and worked
    now errors unless it is out-of-place with static constructors. The tutorial already
    pre-announced requiring FullSpecialize, but not the other two. Worth deciding whether
    that warrants a version bump.
  • The predicate is semantic — is the map isbits and does it build an isbits u0 — not a
    check on the specialization level. A small enough system can satisfy it at
    AutoDespecialize and is then accepted, which I think is right but is a judgement call.

Ignore this draft until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code (model: claude-opus-5[1m])

https://claude.ai/code/session_01GdSpCLd7NBZuuePJmcDzU7

ModelingToolkit now emits the state and parameter initialization maps as
isbits `RuntimeGeneratedFunction`s under `FullSpecialize` (#5043), so
`EnsembleGPUKernel` can evaluate them on the device as they are. Delete the
host-side symbolic tracing that reconstructed them as gather recipes: 232
lines, and it could not express entries computed from the solved
initialization system at all.

A problem qualifies when both maps are isbits and the state map builds an
isbits `u0`. Reaching that needs `FullSpecialize`, static `u0_constructor` /
`p_constructor`, and out-of-place. The last is not an independent choice: an
`MVector` is a mutable struct and so is not isbits, while an in-place problem
cannot write into an immutable `SVector`, which leaves out-of-place plus
`SVector` as the only self-consistent pairing. Each missing piece gets its own
error naming what to change, rather than failing inside the kernel.

`make_parameter_compatible` now also rejects parameters whose contents are not
isbits — an interpolation object, a callable closing over an array, a type —
naming the offending `MTKParameters` portions. Converting buffer storage to
`SArray` cannot rescue those, and previously they produced a silently
non-isbits problem that failed deep in the kernel.

The extension's `static_parameter_storage` was a duplicate of
`make_static_storage` in core, so it is gone. The survivor is documented and
`@public` as the conversion hook: a package owning a non-isbits type adds a
method giving it an isbits stand-in, and both `u0` and the parameters pick it
up without DiffEqGPU knowing the type.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Agent-Harness: Claude Code 2.0.14
Agent-Model: claude-opus-5[1m]
Agent-Session: https://claude.ai/code/session_01GdSpCLd7NBZuuePJmcDzU7
Claude-Session: https://claude.ai/code/session_01GdSpCLd7NBZuuePJmcDzU7
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review September 13, 2026 20:50
@ChrisRackauckas
ChrisRackauckas merged commit 81f600a into SciML:master Sep 13, 2026
15 of 27 checks passed
ChrisRackauckas added a commit that referenced this pull request Sep 14, 2026
- Use ModelingToolkit's generated initialization maps in the kernel (#541)
- fix: restore Enzyme gradients through kernel ensembles (#532)



Agent-Harness: Claude Code
Agent-Model: claude-opus-5[1m]
Claude-Session: https://claude.ai/code/session_014FEzNTLFutCmTEAZ3zBg5R

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants