Skip to content

Add MXL (Media eXchange Layer) input/output support - #32

Closed
matiaspl wants to merge 5 commits into
mxl-integrationfrom
mxl-develop
Closed

matiaspl wants to merge 5 commits into
mxl-integrationfrom
mxl-develop

Conversation

@matiaspl

Copy link
Copy Markdown
Collaborator

Continuation of #30 after the head branch was renamed from
mxl-support to mxl-develop. The rename via GitHub's API kept the
commits intact but closed PR #30 automatically, so this replaces it
with the same content.

Summary

Adds MXL — the dmf-mxl shared-memory
uncompressed-media transport — as a first-class in/out option for
avplumber, without introducing MXL-specific node types.

  • deps/ffmpeg-patches/0008-avformat-libmxl-demuxer-muxer.patch — squashed from 32 upstream commits on cbcrc/FFmpeg branch dmf-mxl/master (pin 5c5d593), rebased onto n7.1.5 on top of the existing 0001-0007 stack. Provenance preserved via Co-authored-by trailers.
  • deps/ffmpeg-patches/{Dockerfile.mkpatch,mkpatch-*.sh} — Docker-based generator that reproduces the patch from the fork; works on macOS + colima and Linux. verify.sh bumped to 8 patches; new post-apply tree hash f4eebb6a….
  • demos/mxl/ — round-trip demo built on avplumber's generic input / output nodes with format="mxl". Writer: lavfi testsrc → v210 encode → MXL. Reader: MXL → mpeg4 → fragmented mp4.
  • demos/mixer/Dockerfile — folded the MXL runtime (libmxl SDK v1.1.0 via vcpkg + gcc-13 + Rust, modernized cmake, FFmpeg configure gets --enable-libmxl --enable-{demuxer,muxer,protocol}=mxl) into the shared mixer image. One image now covers mixer/playlist/replay/mxl.

Verified end-to-end on colima: writer publishes an MXL flow visible in mxl-info -l, reader consumes it and produces a fragmented mp4 with 16,753 mpeg4 packets under v1.1.0. Full CUDA composition suite (overlay_many_cuda, transition_cuda, …) still passes assertions in the folded image.

Known gaps (documented in demos/mxl/README.md)

  • options={"blocking":"1"} doesn't currently reach the MXL demuxer's private AVOptions through avplumber's Input node; worked around with auto_restart:"group" on the reader input. Root-cause belongs in src/nodes/input.cpp.
  • Reader has no realtime pacer between decode and encode, so it runs at wall-clock max. Adding a RealtimeVideoFrame node between them would cap it to the source frame rate.

Test plan

  • deps/ffmpeg-patches/verify.sh /path/to/FFmpeg — full stack applies to n7.1.5 and reproduces f4eebb6a255d1f4b76eb795885102d1cfea8cd08.
  • docker build -f demos/mixer/Dockerfile -t avplumber-mixer:local . — assertions inside the image check both overlay_many_cuda and the MXL demuxer/muxer are present.
  • MXL round-trip on the target NVIDIA host — see demos/mxl/README.md.
  • Existing demos regression: run demos/mixer and demos/playlist against the folded image; no MXL-related changes should affect their behavior.

🤖 Generated with Claude Code

matiaspl and others added 5 commits September 9, 2026 15:06
Adds 0008-avformat-libmxl-demuxer-muxer.patch (squashed from 32 upstream
commits on cbcrc/FFmpeg branch dmf-mxl/master, pinned at 5c5d593), plus
a Docker-based generator that rebuilds the patch from the fork onto
n7.1.5 for reproducibility. verify.sh is bumped to expect 8 patches and
the new post-apply tree hash f4eebb6a255d1f4b76eb795885102d1cfea8cd08.

The generator lives at deps/ffmpeg-patches/Dockerfile.mkpatch and picks
MXL commits by (a) files under libavformat/mxl* and (b) commit subjects
containing "mxl", then cherry-picks with -X theirs onto n7.1.5 + the
existing 0001-0007 stack. mkpatch-finish.sh squashes the picks into a
single provenance-preserving patch with Co-authored-by trailers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds demos/mxl/, a demo that publishes into and reads back from an MXL
shared-memory domain through avplumber's generic input/output nodes
(format="mxl"), with no MXL-specific node types.

The Dockerfile mirrors demos/mixer/Dockerfile: bases on
nvidia/cuda:11.7.1-devel-ubuntu22.04, installs gcc-13 (required by the
MXL SDK) alongside the distro gcc-11, brings up vcpkg + Rust 1.88.0,
builds libmxl v1.1.0-beta-1, applies the full 8-patch stack, and
configures FFmpeg with the CUDA composition suite plus
--enable-libmxl --enable-demuxer=mxl --enable-muxer=mxl
--enable-protocol=mxl. avplumber's python_module is built with
HAVE_CUDA=1 HAVE_NVCC=1 so the same image also runs mixer-class
workloads on real NVIDIA hosts. The demo runs without a GPU too — CUDA
init fails silently, MXL round-trip works.

mxl_demo.py builds two graphs in one process:
  writer: lavfi testsrc -> decode -> rescale(yuv422p10le) ->
          AssumeVideoFormat -> enc(v210) -> mux -> Output(format="mxl")
  reader: Input(format="mxl") -> demux -> dec(v210) -> rescale(yuv420p)
          -> AssumeVideoFormat -> enc(mpeg4) -> mux -> fragmented mp4

Verified end-to-end on colima (aarch64, no GPU): the reader produces
28k+ mpeg4 packets of the testsrc pattern in a 10-second run. The
fragmented-mp4 output stays playable on SIGINT.

Known gaps (documented in demos/mxl/README.md):
  * options={"blocking":"1"} does not currently reach the MXL demuxer's
    private AVOptions through avplumber's input node; worked around
    with auto_restart:"group" on the reader input.
  * Reader runs at wall-clock max — a RealtimeVideoFrame node between
    decode and encode would cap it to the source frame rate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Deletes demos/mxl/Dockerfile and merges its libmxl SDK build, vcpkg /
Rust / gcc-13 setup, cmake upgrade, tests/Makefile cleanups, MXL
FFmpeg configure flags, and MXL demuxer/muxer assertions into
demos/mixer/Dockerfile. The mixer image is now a superset of what the
mixer, playlist, replay, and mxl demos need — one image, one build.

Also COPYs demos/mxl into the image so the MXL demo can be run through
the shared tag by overriding the entrypoint:

  docker run --rm --ipc=host --entrypoint python3 \
      -v /dev/shm/mxl:/dev/shm/mxl \
      -v "$PWD/demos/mxl/test-media:/media" \
      avplumber-mixer:local /build/demos/mxl/mxl_demo.py

Verified end-to-end: the folded image still exposes overlay_many_cuda
(patch 0002), transition_cuda, the CUDA composition suite, plus the
MXL demuxer/muxer (patch 0008), and pyplumber loads cleanly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
v1.1.0 was cut on 2026-09-09, 125 commits ahead of the beta pin we
shipped. Notable changes in that window: audio RMA samples are packed
in channel-major order, fabrics/OFI got batch grain drain and
configurable CQ depth, the audio testsrc no longer busy-spins at
100% CPU, and Rust deps were bumped to fix RUSTSEC-2026-0204.

Verified end-to-end: MXL demuxer/muxer still register in FFmpeg, the
CUDA composition suite still passes assertions, and the mxl_demo
round-trip publishes into and reads back from the v1.1.0 SDK,
producing 16,753 mpeg4 packets in a short run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`input` and `input_rec` forward the JSON `options` dictionary to
`avformat_open_input`, which strips out any AVOptions it recognizes
and leaves unrecognized entries behind. Nothing was inspecting those
leftovers, so misnamed options or ones targeting the wrong scope
were silently ignored. Log each remaining entry as a warning.

Verified against the MXL round-trip demo that `blocking=0/1` and
`grain_index_init=head` are consumed by the MXL demuxer as advertised
(`blocking=0` produces the expected EAGAIN + auto_restart storm,
`blocking=1` matches the auto default for video-only). Drop the
"known gap" bullet in `demos/mxl/README.md` and the matching inline
comment in `mxl_demo.py`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jpietek

jpietek commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Superseded by #36, which carries the newer MXL work (GPU zero-copy reads, hardware verification) rebased onto develop after the FFmpeg 8 series and the HDR mixer landed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants