munari

Munari seamlessly bridges HTML with WebGL, unlocking a new frontier of visual expression on the web.

Munari enables you to seamlessly 'lift' any single or group of HTML elements, including whole pages, into a WebGL context, and back, on demand. Seamlessness is the key and focus of Munari. Here's what happens when an element is lifted into WebGL and returned back to the DOM.

The hard part is the swap. Hide the page and show the scene on different frames and you get a flash, a jump, or a frame of nothing at all. So the scene draws its copy underneath first, same size, same place, invisible, and the page keeps holding until that copy proves it has painted. When the two trade places they are identical, so there is nothing to see.

In the air it is still the same element. You can type in it, select its text, click things inside it. The page holds its old spot open the whole time, so sending it back drops it where it started and it goes on being ordinary DOM.

One <Surface> declares both copies, and its view prop says which renderer should be holding the pixels. The protocol underneath is in packages/core.

I'm continually surprised at what this simple technique can unlock, and I'm often adding new examples in the labs.

The Flight demo is a good example. It's an ordinary drag and drop card stack. But what if the cards really behaved like actual paper? Not a lot of options there. You could build your app in WebGL, add a landing loading bar, and have max flexibility. But then you lose all the benefits of the DOM. There are some hacks that might work like the <foreignObject> trick, but it's limited and brittle.

Munari is built upon ThreeJS and the experimental HTML-in-Canvas API in Chrome. Today, this means it's only visible to an infinitesimally small number of design engineering nerds, like myself, who happen to have this Chrome flag enabled.

Munari is a bet on the future of web UI. The HTML-in-Canvas API is a big deal. It's like Core Animation for the web. Coveted effects like liquid glass, depth of field, real progressive blur, and other shader-driven effects are all unlocked. Because of this, I believe HTML-in-Canvas will get the momentum it needs to become a standard. When that day arrives, I want Munari to be one of the first things you reach for when building a new UI.

While we all wait, I intend to make Munari easy to use as a progressive enhancement with a clear fallback path. Apparently, you can also register a token with Google that enables the experimental API for your users automatically! That's over a billion people to treat to the impossible, with an easy fallback.

Why 'Munari'

Bruno Munari was a playful Italian designer, artist, and inventor. Sometimes he'd mount gauze, torn film, and scraps of plastic in slide frames and throw them across a wall. He called them proiezioni dirette, direct projections: the material itself, making them larger and immersive with light. This library brings the same energy to the web. The real DOM (layout, focus, accessibility, scrolling, selectable text) is the source of truth, and can now project into WebGL, coming alive, while still the DOM.

Requirements

The library is built on Chrome's HTML-in-canvas origin trial (drawElementImage). Chrome needs --enable-features=CanvasDrawElement, or a registered origin-trial token.

Without that capability a Surface has nothing to rasterize, so it stays on its page copy and never presents in WebGL. Your DOM is still there and still works. See When the trial is absent for the one thing you have to handle yourself.

three and @react-three/fiber are peer dependencies. three uses instanceof internally; two copies in one dependency graph fail without an error. Your app owns the single copy.

Install

npm install @petepetrash/munari three @react-three/fiber

Your first Surface

A <Surface> names one piece of content and declares the two copies of it: <Surface.DOM> is the page copy, <Surface.WebGL> is the mesh. <SurfaceCanvas> is the r3f Canvas that hosts them. Set view to 'webgl' and the page copy is released the frame the mesh proves it has drawn; set it back to 'dom' and the page takes the hold again.

The button below is still live DOM in both places: click it on the mesh and its React state updates normally.

import { useState } from 'react'
import {
  Surface,
  SurfaceCanvas,
  useSupportsDOMSurfaces,
  useSurfaceView,
} from '@petepetrash/munari'
import '@petepetrash/munari/style.css'
import './app.css'

function Panel({ count, onPress }: { count: number; onPress: () => void }) {
  return (
    <div className="surface-panel">
      <p>Ordinary React, rendered as matter.</p>
      <button type="button" onClick={onPress}>Pressed {count} times</button>
    </div>
  )
}

export function App() {
  const supported = useSupportsDOMSurfaces()
  const [count, setCount] = useState(0)
  const { surface, view, show } = useSurfaceView('panel')
  const panel = <Panel count={count} onPress={() => setCount((value) => value + 1)} />

  if (!supported) {
    return <main className="munari-demo">{panel}</main>
  }

  return (
    <main className="munari-demo">
      <SurfaceCanvas camera={{ position: [0, 0, 6], fov: 45 }}>
        <ambientLight intensity={2} />
      </SurfaceCanvas>

      <Surface surface={surface} source={panel} view={view}>
        <Surface.DOM>{panel}</Surface.DOM>
        <Surface.WebGL alpha="source" pointerEvents="content" />
      </Surface>

      <button type="button" onClick={() => show(view === 'webgl' ? 'dom' : 'webgl')}>
        {view === 'webgl' ? 'Bring it back' : 'Hand it over'}
      </button>
    </main>
  )
}

html,
body,
#root,
.munari-demo {
  width: 100%;
  height: 100%;
  margin: 0;
}

.munari-demo {
  background: #171612;
}

.surface-panel {
  box-sizing: border-box;
  display: grid;
  width: 400px;
  height: 300px;
  place-content: center;
  gap: 16px;
  padding: 32px;
  border-radius: 24px;
  color: #171612;
  background: #f4efdf;
  font: 16px/1.4 system-ui, sans-serif;
  text-align: center;
}

source is what Munari captures; the same element rendered inside <Surface.DOM> is what the user sees and tabs to while the page holds it. The two are separate React renders of the same component, so any state they share is held above the Surface — that is why count lives in App.

By default <Surface.WebGL> stands exactly where the page copy stands, at the page copy's size, so you do not size or place the mesh yourself. Pass placement="manual" and your own geometry when you want to put it somewhere else.

A detached element

When the content is not React — markup you built by hand, or a subtree another system owns — hand the element to adopt instead of source. Munari takes ownership of it; do not also mount it in the page.

import { useEffect, useState } from 'react'
import { Surface } from '@petepetrash/munari'

export function StaticPanel() {
  const [element, setElement] = useState<HTMLElement>()

  useEffect(() => {
    const node = document.createElement('article')
    node.style.cssText =
      'box-sizing:border-box;width:400px;height:300px;padding:32px;background:#f4efdf;color:#171612'
    node.innerHTML = '<h1>Static HTML</h1><p>This is still a live DOM subtree.</p>'
    setElement(node)
  }, [])

  return (
    <Surface name="static" adopt={element} view="webgl">
      <Surface.WebGL />
    </Surface>
  )
}

innerHTML is your call to make, so use it only for trusted markup.

Either way, the content root must declare its own pixel size. Chrome rasterizes that element at its layout box. A zero-sized root produces an empty texture without an error. Import the package stylesheet once, then read its short header for the hover, active, focus, and floating-layer CSS contract.

When the trial is absent

Most browsers do not have the trial, and that is a supported state rather than a failure. Every <Surface> keeps rendering its page copy, presentedView stays 'dom', and munari reports the reason through onError — or to the console if you have not passed one.

Ask before you branch:

import { useSupportsDOMSurfaces } from '@petepetrash/munari'

function Workspace() {
  const supported = useSupportsDOMSurfaces()
  return supported ? <WorkspaceScene /> : <WorkspaceDOM />
}

The hook answers false on the server and through hydration, then the real answer. Reading the capability directly during render instead — a useMemo, a module constant — disagrees with server markup on exactly the machines that do have the trial. supportsDOMSurfaces() is the same question without the hook, for events, effects and diagnostics.

The one thing that does not degrade by itself

Content degrades on its own. Gestures do not. If a pointer handler puts the scene into a state that only the renderer can leave, and the renderer never arrives, no further input can leave it either:

// Wrong without the trial: `flying` is set and nothing ever clears it.
const onPointerDown = (id) => {
  setFlying(id)
  setView('webgl')
}

Branch at the gesture, not only at the scene:

const onPointerDown = (id) => {
  if (!supported) return carryWithCss(id)
  setFlying(id)
  setView('webgl')
}

Prefer deriving that state from the Surface over keeping your own copy of it. useSurfaceState(handle) reports presentedView, isChanging and supported, and none of them can strand you, because munari never claims a hold it cannot take.

Run the lab locally

The repo uses Node 24 and npm 11. The local launcher starts Vite, opens an isolated Chrome with CanvasDrawElement enabled, and stops the server when you close that Chrome window:

npm ci
npm run lab

Set CHROME_PATH if Chrome is installed somewhere unusual. npm run dev still starts only Vite for a browser that already has the flag enabled.

Go further

A Surface with page and WebGL presentations but no view is a Twin: both copies present at once and the page copy is never released. A Surface can also supply a capture without either presentation. That source-only path lets a material sample live page content for a reflection without drawing a WebGL copy of the page.

A Surface can be split into named parts with <Surface.Part>. All of its parts transfer together or not at all, so a multi-piece object cannot be caught half in the air. <Surface.Anchor name="…"> stands a scene object on a box inside the source that is marked data-munari-anchor, in the geometry's own coordinates.

useSurfaceView('card') is what a scene reaches for when its content changes hands and changes back. It gives you the handle, the view to pass to <Surface>, a show(view) to ask with, and mounted — true for exactly as long as the WebGL side should be in the tree, including the linger after it lands. show('webgl') does nothing on a browser without the trial, so view can never name a renderer that will not arrive.

useSurface('card') and createSurface('card') give you the handle alone — content identity independent of the trees presenting it. Both take just a name, and the name is optional. view, timing and the callbacks are props of the <Surface> that presents the handle, so one declaration owns them. useSurfaceProgress and useSurfaceDriver are how a scene scales its own motion by the crossing.

For a custom shader, pass your own material to <Surface.WebGL> and read the texture with useSurfaceTexture(). In that material slot, the hook returns a configured texture; it is not nullable. To sample another Surface by handle, use useSurfaceTextureOf(handle), which returns null until that source has a texture. DOM textures are premultiplied; apply masks to the full vec4 and blend with ONE / ONE_MINUS_SRC_ALPHA. SURFACE_RADIUS_GLSL is the GLSL half of the corner mask.

The advanced entry

@petepetrash/munari/advanced is the second, deliberate doorway. It re-exports the whole of the renderer-agnostic core — the crossing law, paint accounting, chrome measurement, the plane/screen math — plus FrameSurface, which wears a canvas you already render yourself. createCanvasFrameSource publishes into one: write the complete frame, then call publish(). Presentation receipts are available there when another renderer must not release its pixels until the named frame reaches the screen.

Names behind /advanced move with the kernel rather than with the component API. If a scene only needs a Surface, it should never import from it.

Read the authoring contract before you capture an existing component system. Coding agents can start at llms.txt and the shipped Munari skill.

Working with an agent

In a repository checkout, start with the task-to-owner guide. The system model explains the linked abstractions and the difference between requested view, readiness, presentation, and release. The Revision 3 proposal and compound sketches are historical design material, not current API references.

For an installed package, use its README, skill, index.d.ts, and advanced.d.ts. The current package does not include the full repository docs or registry. Repository-relative links outside those shipped files need source matched to that release; GitHub main can describe a different API. The version-local documentation work is planned, not shipped.

Repo shape

path	what it is
packages/core	the renderer-agnostic core; no dependencies, bundled into the package
packages/react	the @petepetrash/munari package: React/three components over core
registry/	source you copy into your project (nothing published)
apps/lab	the demo and development app
instruments/	browser probes and CI gates
tests/conformance/	the test suites that define core's behavior

Dependencies point one way: apps depend on packages/react, which depends on packages/core. tests/boundary.test.ts checks the actual imports.

See AGENTS.md for the working rules, docs/decisions.md for the numbered design decisions, docs/platform.md for what the platform is measured to do, docs/authoring.md for how to write markup a Surface can draw, and docs/focus.md for the focus and spatial-navigation contract.

Development

npm ci
npm run lab              # Vite + a compatible local Chrome
npm run dev              # Vite only
npm run check:origin-trial
npm run typecheck
npm test
npm run lint
npm run gate:idle-zero   # browser gate: idle Surfaces cost 0 paints/s

package.json lists available gate commands; the CI workflow selects the gates run on each push. The instrument guide gives each check's scope and limits. Run GPU gates in series. A capability skip is not a passing behavior check; use STRICT_CAPABILITY=1 where HTML-in-canvas must be present.

npm run build stages the package with core bundled in and peers left external. The staged package includes the canonical root README, license, changelog, llms.txt, and Munari skill. Inspect it, then publish from the staged directory:

npm run build
npm pack --dry-run packages/react/dist
npm publish packages/react/dist