The question
The receiver decodes with WebCodecs and draws to a <canvas> inside WebView2. The alternative is to present natively — a child window with a D3D11 swap chain, fed by Media Foundation or by WebCodecs frames handed across — bypassing the browser compositor entirely.
That is a large piece of work in the most user-visible code in the project. It should not begin until one number says it is worth it, and the instrumentation to produce that number now exists.
What is already known
The obvious wins are already taken. The canvas context is created with desynchronized: true, which on Windows gives it a swap chain of its own (Chromium M82+) rather than a compositor round trip, and alpha: false, which avoids a blend on every frame.
What remains unmeasured is the wait between drawImage returning and the compositor actually picking the frame up. That is the to screen row on the HUD, added in #64, and it is precisely the cost a native surface would remove.
The decision rule
Get a real reading first — see the measurement issue.
- A couple of milliseconds: the compositor is not the problem. Close this, and record the number so nobody proposes it again from intuition.
- Tens of milliseconds: there is a real case, and the target to beat is known rather than hoped for.
Either outcome is a result. Closing this on evidence is as valuable as building it.
If it turns out to be worth doing
The parts that make it hard, so nobody starts underestimating it:
- Two windows, one app. A native surface means a child HWND alongside the WebView, and every piece of UI — the overlay, the HUD, the connect card — either moves with it or has to be composited over it. The overlay states are already a tested state machine (
components/window.ts); that is the contract to preserve.
- Getting frames out of WebCodecs. A
VideoFrame can be copied, but copying per frame is exactly the cost being removed. The interesting version keeps decode in Media Foundation and never involves the WebView in video at all — which means reimplementing the decode path, not wrapping it.
- Input. Pointer lock, the edge-roaming handoff and
F8 forwarding are all DOM-level today.
- The fallback stays. Not every machine will do this, and the canvas path has to remain the thing that always works.
Non-goals
Rewriting the frontend. The receiver's interface is settled; this is about where video pixels land, and nothing else.
The question
The receiver decodes with WebCodecs and draws to a
<canvas>inside WebView2. The alternative is to present natively — a child window with a D3D11 swap chain, fed by Media Foundation or by WebCodecs frames handed across — bypassing the browser compositor entirely.That is a large piece of work in the most user-visible code in the project. It should not begin until one number says it is worth it, and the instrumentation to produce that number now exists.
What is already known
The obvious wins are already taken. The canvas context is created with
desynchronized: true, which on Windows gives it a swap chain of its own (Chromium M82+) rather than a compositor round trip, andalpha: false, which avoids a blend on every frame.What remains unmeasured is the wait between
drawImagereturning and the compositor actually picking the frame up. That is theto screenrow on the HUD, added in #64, and it is precisely the cost a native surface would remove.The decision rule
Get a real reading first — see the measurement issue.
Either outcome is a result. Closing this on evidence is as valuable as building it.
If it turns out to be worth doing
The parts that make it hard, so nobody starts underestimating it:
components/window.ts); that is the contract to preserve.VideoFramecan be copied, but copying per frame is exactly the cost being removed. The interesting version keeps decode in Media Foundation and never involves the WebView in video at all — which means reimplementing the decode path, not wrapping it.F8forwarding are all DOM-level today.Non-goals
Rewriting the frontend. The receiver's interface is settled; this is about where video pixels land, and nothing else.