Conversation
Coalesce pending requests across cuts, RTCP feedback, and periodic forcing without delaying video frames. Add the mixer CLI option, native node tests, and a 16-input NVENC rapid-cut/fade regression. Validated 133 mixer/timing tests and 18 native limiter tests, plus isolated 60 fps NVENC A/B runs.
Headerless packed v210 frames become P210 CUDA frames via pinned staging and a GPU unpack kernel, with fixture generator and tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d fixtures cuda_rect_overlay resolves one per-format descriptor (sample bytes, storage shift, chroma footprint) and passes it to the existing kernels: P210, planar 10-bit and 444 layouts flow through the same region, clear, blit, scale and fused RGB(A) paths as NV12. Clears are depth/range-aware 16-bit memsets (alpha opaque is 1023, not 255<<2); the redundant NV12 plane-region special case folds into the generic descriptor branch. FFmpeg patch 0008 gives transition_cuda a word-sample kernel whose wipe boundary moves per chroma pixel, so U/V can never split; legacy 8-bit paths keep their exact arithmetic (float blend, truncate on store) while new 10-bit results round to nearest. MixerGraphBuilder and the demo gain a working_format knob defaulting to nv12. Fixtures add the native HLG BT.2020 and SDR-promoted BT.709 families with color manifests and transfer checkpoints (E = 0, 1/12, 1 -> Y 64, 502, 940). smoke_mixer_10bit drives two v210 sources through A/B composition and fade/wipe transitions and compares raw P210/444 output against an exact CPU reference at binary-fraction alphas. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uard All 14 cases pass on the T4: sdr8/HLG P210 and planar-444 runs through fade endpoints, exact binary-fraction blends and wipes, plus 4x4 grids of 16 simultaneous sources, each compared bit-exactly against the CPU reference. The transition node needs defer_preliminary_init and an explicit dst_frame_rate, matching the production builder. Finite sequences carry tail-guard frames because the dual-input filter's EOF flush drops a variable number of in-flight trailing frames (recorded follow-up; the single-input flush drains correctly). The new gradient fixture family makes 10-bit visually checkable: a shallow dark ramp whose top half is quantized to 8-bit codes - under a 4x contrast stretch the halves must band at different widths, and identical halves expose an 8-bit bottleneck anywhere in the chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Programmatic callers and the test suite drive mixer.py through the GraphOptions dataclass, which must mirror every CLI option it reaches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t program With working_format p210le the H.264 outputs (file and Janus) receive NV12 from one GPU scale_cuda conversion; the program edge keeps the 10-bit working format for raw taps and future Main10/MXL branches. Janus and its config are untouched - the wire stays 8-bit H.264. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sixteen full-HD v210 sources (gradient/ramp/HLG/SDR-promoted) compose as a 4x4 P210 grid and encode zero-copy to 8-bit H.264 and HEVC Main10; a fullscreen static-gradient pass with a 4x contrast stretch before the 8-bit encode makes the 10-bit half visibly out-resolve the 8-bit half. Looping InputRec passes rely on container exit: stopping a looping InputRec -> v210_to_cuda chain mid-stream wedges shutdown (recorded follow-up); finite-input passes shut down cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
define_scene appends one explicit scale_cuda format conversion to every scene graph string (and the per-source fallback scaler) when the working format is not NV12, so NVDEC-decoded sources promote to P210 at their crop/scale stage instead of being rejected by the compositor. The NV12 output conversion becomes a shared helper applied once on both the flag and config code paths, covering renditions as well. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
10-bit 4:2:2/4:2:0/4:4:4 mixer operation becomes another config field: canvas.working_format (default nv12) selects the compositor/transition sw_format, validated against the qualified layout list. The config value is authoritative on the --config path and drives both the builder and the encoded-output NV12 conversion, so an HLG/10-bit run reuses the full mixer, its scene library and the webui control surface unchanged - only the JSON differs, exactly like the 30/60 fps variants. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cuda_rect_overlay gains a fused NV12->P210 promotion path alongside the existing RGB->YUV one: convert_scale_plane bilinear-scales a lower-depth semiplanar source and stores at the canvas depth (codes x4 for 8->10, 16->64/235->940), resampling the chroma footprint, so an 8-bit clip (Big Buck Bunny via NVDEC) mixes onto a 10-bit HLG program with no separate convert node. The compositor previously rejected NV12 input to a P210 canvas outright, which is why the full demo produced no program. Adds a "v210" config source kind (build_v210_input) for true 10-bit inputs: packed v210 unpacked on the GPU to the working format with its declared HLG/BT.2020 color contract. Kernel unit test covers the promotion path exactly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The config-path v210 source builder needs it; without the registration the loader raised AttributeError at startup for any config with a v210 source. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mooth HLG content Janus/WebRTC renditions now default to hevc_nvenc: current Safari and Chrome negotiate HEVC over WebRTC on hardware-decode machines, it is ~2x more efficient than H.264 at the same bitrate, and Main10 carries the program's 10-bit depth. Each rendition converts the P210 program to its encoder format in one GPU pass (P010 for HEVC Main10, keeping 10-bit; NV12 for H.264) instead of the blanket NV12 downconvert, and the encoder signals the canvas color contract (canvas.color_* -> VUI) so an HLG program is tagged BT.2020/arib-std-b67. hlg_content.py generates smooth, seamlessly-looping HLG demo content (phase 2*pi*i/N): a full 0->peak luminance sweep, saturated BT.2020 wide-gamut bars, and specular highlights orbiting against near-black - a genuine HDR/WCG showcase rather than the short correctness loop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a smooth-gradient comparison pattern (no specular, dark-to-mid range where 8-bit contours worst) written as a matched 10-bit and 8-bit-quantised pair, for a side-by-side scene that makes the bit-depth difference directly visible. quantize8 rounds each code to a multiple of four (8-bit precision inside the 10-bit container). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…bit display The previous comparison was invisible over an 8-bit path: a display quantises the 10-bit half down to 8-bit, so both halves matched. Now the comparison is a shallow ramp in a narrow code window, quantised (8-bit half) and then stretched to full range, so the 8-bit half's coarse steps amplify into wide visible bands while the 10-bit half stays fine - demonstrable on any display, not just a 10-bit one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CUDA port of FFmpeg n8.1 libavfilter/opencl/tonemap.cl + colorspace_common.cl: HLG or PQ BT.2020 -> BT.709 SDR, static peak, operators direct/linear/clip/reinhard/hable/mobius. tonemap_reference.py is a numpy oracle of the same math; the kernel matches it for HLG and PQ within 2 codes on the T4. This is the body of the planned tonemap_cuda FFmpeg filter (patch 0009) for the SDR output rendition; no Vulkan, pure CUDA. HDR10/PQ metadata and the FFmpeg filter wrapping follow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CUDA HDR->SDR tone mapper (HLG/PQ BT.2020 P010 -> BT.709 SDR NV12), zero-copy, ported from vf_tonemap_opencl; the SDR output/neural-preproc filter. Series is now 9 patches; pinned tree updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A rendition with tonemap=<operator> converts the HDR program to BT.709 SDR 8-bit via scale_cuda->P010 + tonemap_cuda before its encoder (e.g. an SDR H.264 output), while tonemap="" keeps HDR passthrough. Single output, no split; the transfer is taken from the canvas color contract (arib-std-b67->hlg, smpte2084->pq). smoke_tonemap.py exercises the filter end to end (HLG v210 -> P210 -> P010 -> tonemap_cuda -> NV12, in-range SDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Composes a flat-colour source into each semiplanar canvas (NV12 420/8, P010 420/10, P210 422/10) and checks the downloaded canvas exactly: flat colour is invariant through copy, 8->10 promotion (x4) and 420<->422 chroma resampling, so every combination has an exact expected value. Covers same-format copy, same-subsampling depth promote (NV12->P010), and cross-subsampling promote-in (NV12->P210, P010->P210). All six pass on the T4, qualifying P010 as a canvas and the cross-subsampling path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Profiling (nsys) showed the mixer is launch/occupancy-bound, not compute-saturated, and Tonemap_Cuda is the heaviest per-frame kernel (~279us). Swap the transcendentals in the tonemap device math to CUDA fast intrinsics (__powf/__expf/__log10f) in both tests/cuda/tonemap.cu and the filter (patch 0009); output still matches the numpy oracle within 4 codes (test tolerance widened 2->4). Series tree re-pinned. Adds measure_unpack.py / measure_compose.py profiling harnesses (unpack is only ~62us/launch, so no unpack optimization needed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FFmpeg tonemap's desat=0.5 default mixes bright pixels toward grey, desaturating the SDR output. Expose it as a per-rendition option defaulting to 0 (keep saturation); configs can raise it if highlight clipping needs taming. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract shared _init_avp + _make_builder used by both the --input and --config build paths (working_format/builder config now set in one place, not duplicated). Default the rendition/janus codec by program depth (H.264 for 8-bit, HEVC Main10 for 10-bit) instead of forcing HEVC everywhere - keeps 8-bit output universal while HDR gets HEVC. Only add a rendition scale/convert node when it actually resizes or changes format, dropping the no-op pass on same-format renditions. Full mixer pytest suite passes (the two graph tests my HEVC-default change had broken are green again). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
render_mixer_10bit.py, hlg_content.py, measure_compose.py and measure_unpack.py were one-off tools for producing demo content and nsys profiling runs, not CI tests. Removed to keep the test set to what guards behaviour: kernel unit tests, fixture tests, the mixer smokes, the interop matrix and the tonemap oracle. Recoverable from history. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ibute Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keep the 10-bit mixer PR to code and tests; docs (NODES.md, the v210 node note, the ffmpeg patch README note) will land separately. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…a-tests - _harness.py: the AVPlumber setup, packed-v210 ingest chain, plane extraction and frame-drain loop the four smokes each repeated. - smoke_tonemap now checks the HLG fixture's transfer checkpoints against the numpy oracle through the real tonemap_cuda filter, replacing the standalone tonemap.cu copy of the device math and its unit test. - test_v210_fixture folded to one HLG-checkpoint assert in the v210 smoke (packing is already checked against FFmpeg's CPU v210 decoder there). - v210_fixture: drop the gradient family, manifest and CLI, which only the removed render harness used. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Replace the per-version directories with a single deps/ffmpeg/8/ series that applies to both upstream n8.0 and n8.1. The composition-suite sources are new files (no base context), and the remaining patches are regenerated with minimal context; the only real 8.0/8.1 difference is the libnpp configure check 8.1 added that fails on CUDA 13, which the shared apply.sh rewrites to the stream-context probe (a no-op on 8.0). verify.sh checks either base against a pinned tree in 8/bases.env. Remove the 7.1.5 series and the triplicated per-version verify/README/ base.env; the three demo Dockerfiles call apply.sh and default to n8.1. Net: -5.1k lines of duplicated patch text, one README instead of three. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds 8/0011-avformat-libmxl-demuxer-muxer.patch, squashed from the two MXL commits on cbcrc/FFmpeg branch dmf-mxl/8.1 (pinned at 9eddb90). That branch forks from the same n8.1 commit deps/ffmpeg/8/bases.env pins, so both picks apply without conflicts — unlike the earlier 7.1.5-based version of this patch, which force-fitted an 8.x codebase onto 7.1.5 with -X theirs and dragged fork test scaffolding along. The configure hunks are re-anchored so one copy serves both bases: the fork puts its `require_pkg_config libmxl` next to libmpeghdec, which only exists in 8.1, and carries a whitespace-only reindent of the mmal check. The generator moves the require next to the mxl_* deps lines and drops the reindent; verify.sh now reproduces the pinned tree for both n8.0 and n8.1 with patch_count=11. The generator lives at deps/ffmpeg/Dockerfile.mkpatch and needs no compiler — it replays git history only, so it runs on macOS hosts too. MXL_REMOTE_REF/MXL_PIN/FFMPEG_TAG move it to a future base (the fork already carries a dmf-mxl/9.0 branch). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds demos/mxl/, a demo that publishes into and reads back from an MXL
shared-memory domain through avplumber's generic input/output nodes
(format="mxl"), with no MXL-specific node types.
The Dockerfile mirrors demos/mixer/Dockerfile: bases on
nvidia/cuda:11.7.1-devel-ubuntu22.04, installs gcc-13 (required by the
MXL SDK) alongside the distro gcc-11, brings up vcpkg + Rust 1.88.0,
builds libmxl v1.1.0-beta-1, applies the FFmpeg patch series, and
configures FFmpeg with the CUDA composition suite plus
--enable-libmxl --enable-demuxer=mxl --enable-muxer=mxl
--enable-protocol=mxl. avplumber's python_module is built with
HAVE_CUDA=1 HAVE_NVCC=1 so the same image also runs mixer-class
workloads on real NVIDIA hosts. The demo runs without a GPU too — CUDA
init fails silently, MXL round-trip works.
mxl_demo.py builds two graphs in one process:
writer: lavfi testsrc -> decode -> rescale(yuv422p10le) ->
AssumeVideoFormat -> enc(v210) -> mux -> Output(format="mxl")
reader: Input(format="mxl") -> demux -> dec(v210) -> rescale(yuv420p)
-> AssumeVideoFormat -> enc(mpeg4) -> mux -> fragmented mp4
Verified end-to-end on colima (aarch64, no GPU): the reader produces
28k+ mpeg4 packets of the testsrc pattern in a 10-second run. The
fragmented-mp4 output stays playable on SIGINT.
Known gaps (documented in demos/mxl/README.md):
* options={"blocking":"1"} does not currently reach the MXL demuxer's
private AVOptions through avplumber's input node; worked around
with auto_restart:"group" on the reader input.
* Reader runs at wall-clock max — a RealtimeVideoFrame node between
decode and encode would cap it to the source frame rate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Deletes demos/mxl/Dockerfile and merges its libmxl SDK build, vcpkg /
Rust / gcc-13 setup, cmake upgrade, MXL FFmpeg configure flags, and MXL
demuxer/muxer assertions into demos/mixer/Dockerfile. The mixer image is
now a superset of what the mixer, playlist, replay, and mxl demos need
— one image, one build.
FFmpeg comes from deps/ffmpeg/apply.sh like the other demo images. The
7.1.5-era tests/Makefile and fate/mxl.mak cleanups are gone: the 8.1
patch carries only MXL files plus configure/Makefile/allformats glue, so
there is no fork scaffolding left to strip.
Also COPYs demos/mxl into the image so the MXL demo can be run through
the shared tag by overriding the entrypoint:
docker run --rm --ipc=host --entrypoint python3 \
-v /dev/shm/mxl:/dev/shm/mxl \
-v "$PWD/demos/mxl/test-media:/media" \
avplumber-mixer:local /build/demos/mxl/mxl_demo.py
demos/mxl/README.md is retargeted at the FFmpeg 8.x series (patch
8/0011, deps/ffmpeg/verify.sh, the dmf-mxl/8.1 fork branch) and notes
that the end-to-end run predates the 8.1 migration. It also records
where this is heading: the demuxer emits one packed v210 frame per
grain, which is exactly what the v210_to_cuda node consumes, so the read
path can drop swscale and the CPU v210 decoder entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v1.1.0 was cut on 2026-09-09, 125 commits ahead of the beta pin we shipped. Notable changes in that window: audio RMA samples are packed in channel-major order, fabrics/OFI got batch grain drain and configurable CQ depth, the audio testsrc no longer busy-spins at 100% CPU, and Rust deps were bumped to fix RUSTSEC-2026-0204. Verified end-to-end: MXL demuxer/muxer still register in FFmpeg, the CUDA composition suite still passes assertions, and the mxl_demo round-trip publishes into and reads back from the v1.1.0 SDK, producing 16,753 mpeg4 packets in a short run. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`input` and `input_rec` forward the JSON `options` dictionary to `avformat_open_input`, which strips out any AVOptions it recognizes and leaves unrecognized entries behind. Nothing was inspecting those leftovers, so misnamed options or ones targeting the wrong scope were silently ignored. Log each remaining entry as a warning. Verified against the MXL round-trip demo that `blocking=0/1` and `grain_index_init=head` are consumed by the MXL demuxer as advertised (`blocking=0` produces the expected EAGAIN + auto_restart storm, `blocking=1` matches the auto default for video-only). Drop the "known gap" bullet in `demos/mxl/README.md` and the matching inline comment in `mxl_demo.py`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The MXL demuxer hands out one packed v210 frame per grain, which is exactly what v210_to_cuda consumes, so the reader no longer needs the CPU v210 decoder or swscale: the grain is staged once through pinned memory, unpacked to CUDA p210 by the PTX kernel, and finished by scale_cuda + h264_nvenc. NVENC also drops the mpeg4 fallback the image needed for lack of libx264. Selected by --gpu-unpack, defaulting to auto: on when /dev/nvidiactl is mapped in, CPU path otherwise, so the GPU-less runs the demo was verified with keep working. Also enable the demuxer's zero_copy by default, which points each AVPacket at the MXL ring in /dev/shm instead of copying the grain out. Nothing holds a reference on the grain, so the packet queue is planned down to one frame and the documented history_duration goes from 100 ms (~2 grains) to 1 s; --no-zero-copy trades the memcpy back for immunity. The geometry is now a CLI contract (--width/--height/--fps) that the writer rescales to, since packed v210 carries no dimensions and the reader derives the row stride from the width. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three fixes found while running the MXL round-trip on an RTX 4000 Ada with FFmpeg 8.1: The MXL muxer ignores PTS and increments its own grain index per packet, so an unpaced lavfi source published ~2300 grains/s, ran the flow into the future and left every reader "too late". Insert realtime(set_pts) + force_fps ahead of the v210 encoder. The demuxer picks its grain index during avformat_open_input's probe, i.e. seconds before the group starts, with CUDA and NVENC init in between; by then it is behind the ring tail and the EAGAIN is fatal to input.cpp. on_too_late=reset re-derives it, and reset_on_drop rebases the timestamps so the output does not open with a multi-second hole. The reader's mp4 was frequently 28 bytes: fragmented mp4 only helps if the data leaves the muxer's 256 KiB avio buffer, which a low-bitrate run killed after 25 s never fills. Add flush_packets=1, plus a one-second NVENC GOP so fragments are cut at a useful rate. README records what was actually verified on the hardware and the SIGINT teardown hang, which reproduces on the CPU path too and so is not from this wiring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Trim the MXL demo README to the facts a reader needs: fold the duplicated build recipe into what is MXL-specific about the shared image, merge the two teardown-hang gaps into one, and drop the in-image verification snippet the Dockerfile already asserts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 18, 2026
Collaborator
|
Reopening is blocked by GitHub after the base-branch deletion and the rebase force-push. Continued as the PR opened from the same branch against develop (see the newest open MXL PR). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds MXL (Media eXchange Layer) shared-memory transport support on top of the FFmpeg 8.x patch series, plus a round-trip demo that exercises it end-to-end.
What's here
8/0011-avformat-libmxl-demuxer-muxer.patch— libmxl demuxer, muxer, URI parser, JSON/diagnostic helpers, FATE coverage and--enable-libmxlglue, squashed fromcbcrc/FFmpegbranchdmf-mxl/8.1(pinned at9eddb90).patch_countin8/bases.envbumped to 11.deps/ffmpeg/Dockerfile.mkpatch,mkpatch-0011-mxl.shandmkpatch-finish.shreplay the fork's history in a container (no compiler, works on Linux and macOS hosts) so the squashed patch can be re-cut against a newer fork branch or FFmpeg base.demos/mixer/Dockerfilebuildsdmf-mxl/mxl@v1.1.0with gcc-13 (the SDK needs C++20; FFmpeg and avplumber keep gcc-11) via vcpkg, and configures FFmpeg with--enable-libmxl --enable-demuxer=mxl --enable-muxer=mxl --enable-protocol=mxl. The build asserts the mxl demuxer and muxer are registered.demos/mxl— writer (lavfi testsrcor a looped file → v210 → MXL) and reader (MXL → v210 unpack → mp4) graphs in one process, using avplumber's genericinput/outputnodes withformat="mxl". No new node types. The reader unpacks straight onto the GPU withv210_to_cuda+ NVENC when a CUDA device is present, and falls back to the CPU v210 decoder otherwise. Grains are taken zero-copy out of the shared-memory ring by default.inputandinput_recnow log a warning for input options libavformat did not consume. Silently dropped options cost real debugging time on this demo (blocking=1was being swallowed); this is the only change undersrc/.Validation
Verified end-to-end on x86_64 Fedora with an RTX 4000 Ada (driver 615.71, Docker + NVIDIA Container Toolkit) against FFmpeg
n8.1-12-g93aafbbfrom this stack: GPU zero-copy, GPU with--no-zero-copy,--gpu-unpack off, from bothlavfi testsrcand a looped file, at 320x240p25 and 640x480p25. All runs started at PTS 0, reported 25.0 fps (25.06 on NVENC) and decoded back to the source pattern.Known gaps (Ctrl-C hanging in teardown, the unpaced reader, the write path still going through swscale and the CPU v210 encoder) are documented in
demos/mxl/README.md.🤖 Generated with Claude Code