Delta Chat Desktop — screen sharing & camera effects

A runtime patch that adds screen sharing, virtual backgrounds / blur / color filters, and face-tracked avatars (2D photo puppets and 3D VRM models) to Delta Chat Desktop's 1:1 calls — all on the sender side, so the person you call needs no changes and no new version.

Tested against Delta Chat Desktop 2.53.0 through 2.59.0 (Electron 41.x), on Windows. This is an unofficial community patch, not affiliated with Delta Chat.

Screenshots

The Effects panel in a call — avatar, VRM presets, background, and filters:

Received on the other end with nothing installed — a photo background and a 3D VRM avatar showing up on a stock phone (Ubuntu Touch):

The screen-share source picker (whole screen or a single window, with a layout choice):

What it does

• Screen sharing — share a whole screen or a single window, with your camera composited in (picture-in-picture / side-by-side / screen-only). On Windows, whole-screen shares also carry desktop audio.
• Background — blur, or replace with any image.
• Filters — B&W, sepia, warm, cool, vivid, invert. Combine with any background.
• 2D photo puppet — turn any picture into an avatar. It tracks your head; for human-face photos it also blinks and moves the mouth (a landmark-driven mesh warp). Tip: generate a character with any AI image tool and load it here.
• 3D VRM avatar — a rigged VTuber-style model driven by your face (head pose + blink + mouth + smile). Ships five CC0 presets; load any .vrm.

Everything shows in a live self-preview (the app's own top-right self-view is repointed at the processed video), and everything is applied to the outgoing track, so stock receivers on any platform just see your video change.

Why it works without touching the other end

Delta Chat's call signalling is a single offer/answer exchanged over e-mail; the webapp has no negotiationneeded handler, so tracks can't be added mid-call. But calls-webapp always creates its transceivers up front:

peerConnection.addTransceiver("video", { direction: "sendrecv" })
peerConnection.addTransceiver("audio", { direction: "sendrecv" })

A video sender therefore always exists. Every effect composites onto a <canvas>, and canvas.captureStream()'s track is swapped into that existing sender with replaceTrack() — no renegotiation, invisible to the peer.

Easiest: one-file installer (Windows)

For non-technical users there's a single-file installer, DeltaChat-Effects-Setup.exe — double-click it and it does everything: closes Delta Chat, installs the effect files to %LOCALAPPDATA%\DeltaChat-Effects, and patches the app. Run it again after any Delta Chat update; run it with --remove to uninstall. Windows SmartScreen may warn about an unknown publisher (it's unsigned) — choose More info → Run anyway.

Build it yourself with installer.py:

pip install pyinstaller
pyinstaller --onefile --name DeltaChat-Effects-Setup --add-data "screenshare_inject.js;." --add-data "assets;assets" installer.py

Install on a new device (Windows)

Everything the patch needs — the face-tracking models, 3D libraries, and the five avatar presets — ships in this repository, so a fresh download is a complete install kit. No other downloads are needed.

1. Install Delta Chat Desktop and run it once.
2. Install Python 3 (tick Add to PATH).
3. Get this repo, either way works:

   git clone https://github.com/Rabs9/deltachat-desktop-effects.git
   

   or download the ZIP (green Code button → Download ZIP, or this direct link) and extract it anywhere.
4. Fully quit Delta Chat (it minimises to the tray — right-click the tray icon and quit), then in the extracted folder run:

   python patch_deltachat.py
   

Usage

python patch_deltachat.py            # apply (idempotent, re-runnable)
python patch_deltachat.py --revert   # restore the original app.asar
python verify.py                     # sanity-check the repacked archive

Close Delta Chat first (fully quit — it minimises to the tray). Re-run after every Delta Chat update, since an update replaces app.asar. The pristine original is always kept at resources/app.asar.orig.

In a call, an Effects bar appears top-left (Delta Chat owns the bottom-centre controls and the top-right self-view). First use of a segmentation/tracking feature briefly loads its model.

How it's built

The patch edits the Electron main process bundle (/bundle_out/index.js) inside app.asar to:

1. Widen the calls-webapp CSP (connect-src 'self', img-src data:, wasm-unsafe-eval, worker-src) for the injected UI, WASM, and modules.
2. Add supportFetchAPI to the calls-webapp scheme (upstream sets it on the webxdc scheme but not this one, so fetch() otherwise fails).
3. Allow the display-capture permission and register setDisplayMediaRequestHandler (loopback audio gated to whole screens).
4. Extend the calls-webapp protocol handler to inject the client script, serve the screen-share source list, and serve vendored assets from /dcss-assets (models, WASM, three.js, VRMs), plus an injected import map.
5. Mirror the call window's console into %LOCALAPPDATA%\DeltaChat\logs for diagnosis.

All client logic lives in screenshare_inject.js, embedded base64 into the bundle and injected as a classic <script> (it runs before the app's deferred module bundle, so its WebRTC hooks install in time).

Face tracking & segmentation: MediaPipe Tasks Vision. 3D avatars: three.js + @pixiv/three-vrm. See assets/ and vendor_addons.py for how the libraries were vendored.

Requirements / caveats

• The patcher refuses to run unless every code anchor matches exactly once, so a Delta Chat version bump fails loudly rather than corrupting the archive. If that happens, the anchors need re-targeting for the new version.
• Relies on enable_embedded_asar_integrity_validation being disabled in the Electron fuse table (it is in this build).
• Effects apply to the single video track, so avatar / screen / camera are mutually exclusive at a time; the compositor is what shows camera + screen together.
• Desktop / sender only. getDisplayMedia() and the tracking stack are desktop features; sending from mobile isn't possible (mobile WebViews lack the APIs).
• 1:1 calls only — Delta Chat has no group calls yet.

Related

Upstream feature request for official screen sharing: deltachat-desktop#6587.

Licensing

Our own code (patch_deltachat.py, screenshare_inject.js, verify.py, vendor_addons.py) is MIT — see LICENSE.

Bundled third-party assets under assets/ retain their own licenses (three.js and three-vrm MIT, MediaPipe Apache-2.0, the VRoid sample avatars CC0). Full notices in THIRD_PARTY.md.

This patch modifies Delta Chat Desktop (GPL-3.0) at runtime on your own machine; it does not redistribute Delta Chat's code.