Skip to content

10-bit HDR mixer: SDR/HLG/PQ conversion, native 4:2:2, HDR10 outputs, shutdown fixes - #33

Merged
jpietek merged 1 commit into
developfrom
ffmpeg8
Sep 18, 2026
Merged

jpietek merged 1 commit into
developfrom
ffmpeg8

Conversation

@jpietek

@jpietek jpietek commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

10-bit HDR mixer: per-source SDR/HLG/PQ conversion, native 4:2:2 canvases, HDR10 outputs, and the shutdown fixes.

Stacked on #34 (FFmpeg 8.0/8.1 series); merge #34 first, then this PR (squash).

What it does

  • Colour: every source is normalised to the canvas contract (sdr, hlg, pq) on the GPU by the new tonemap_cuda filter (auto per-frame resolution, untagged files are BT.709 SDR); renditions convert back with clip/mobius/hable, a knee (tonemap_param) keeps SDR content exact inside an HDR canvas.
  • 10-bit / 4:2:2: P010 and P210 canvases, 8-bit sources promoted in the compositor, v210 sources (packed 10-bit 4:2:2) stay 4:2:2 through conversion and compositing; tonemap_cuda reads and writes NV12/P010/NV16/P210 and resamples in one pass.
  • Outputs: HEVC Main10 with HLG/PQ VUI over Janus and files; PQ carries HDR10 static metadata (enc_video hdr_metadata, values from pyplumber.mixer.color). Needs nv-codec-headers 13 (driver ≥ 570), both Dockerfiles updated.
  • Runtime fixes: filter node no longer drops queued frames when an EOF marker arrives mid-wait (multi-input transitions lost their tail); Python shutdown deadlock and stopped-queue peeks fixed (from codex/recorder-shutdown, folded in), with the seek-flush latch cleared so seeks/speed changes keep flowing; per-node control bindings release the GIL.
  • Code layout: pure code motion plus libav/avcpp reuse. Compositor: src/nodes/hwaccel/cuda_rect_draw.{hpp,cpp} next to a ~700-line cuda_rect_overlay.cpp. Mixer control: src/mixer/orchestrator/ (MixerOrchestrator.hpp, core.cpp, one file per transition kind), src/mixer/primitives/ (CamelCase component headers: TickGrid over av::Rational, Cadence, MonotonicClock, CutLatency, Snapshot, MixerState, ...; snake_case free-function modules: pixel_layout, compositor_layers, compositor_geometry), plus routing.hpp, graph_ops, TransitionScheduler, Playout.hpp. Hand-rolled rescales, plane maths, HDR side-data sizes and colour ints replaced by av_rescale*, av_inv_q, av_image_get_linesize, av_pix_fmt_get_chroma_sub_sample, the libavutil metadata allocators and AVColorTransferCharacteristic; AGENTS.md now asks for that. Fix: source_switcher no longer queries the timeline at AV_NOPTS_VALUE. Layout table in doc/mixer.md; new CPU unit tests for pixel layout, HDR10 metadata serialisation, canvas.latency_ms, and an SDR media wipe on an HLG P210 canvas; one cpp_binary fixture builds the standalone C++ tests.
  • Python package: avpmixer/ becomes pyplumber/mixer/, so images ship one Python package; the package root imports the native module lazily so configuration, control and color helpers load without it.
  • Demo tooling: make_config.py emits HDR shows (--color, --working-format, --sdr-port, colour-tagged and v210 sources); tests/sdr_patterns.py renders SDR pattern clips. README shows the 16-source 1080p60 HDR example.
  • Config/CLI: working_format, color, latency_ms, v210 sources, rendition color/tonemap*/max_cll/max_fall; --input-color, --working-format; only semiplanar working formats; 32-source cap enforced at load; docs/config.md opens with a complete loader-validated example, plus a shipped HDR example.

Known limitations (documented in demos/mixer/docs/config.md)

32 sources per show (compositor pad mask), all sources always running, no runtime source add/remove, --input >32 files needs homogeneous inputs, NVENC output is 4:2:0, HLG/PQ need a 10-bit canvas.

Verification (all on this head or its immediate predecessors, see commits)

  • Unit: mixer 114, colour 22, replay 76, playlist 73, playout 25, dmabuf 5; after the module split and libav reuse: 135 CPU tests, the four GPU smokes, check_transition_recovery (plain and --prewarm), and the live 16-source HDR show (cuts, fades, media wipes on the HLG canvas, both Janus legs fresh) all green on the final head.
  • GPU smokes (T4): v210, interop matrix, 10-bit mixer incl. 16-source grids, transfers matrix (3×3 transfers × depths, round trips, rejections), output qualification (HEVC Main10 HLG VUI, PQ + HDR10 SEIs, SDR round trip within 7 codes), shutdown smoke, live takes (24 cuts/fades on a mixed 8/10-bit show, both Janus legs fresh).
  • Demos: cuda-overlay matrix 12/12, replay integration 24 fps module 37 passed, playlist scripted run, dmabuf-browser unit tests.
  • Downstream: eka-recorder e2e PASS ×2 (tests/downstream/eka-recorder-smoke.sh, 0 panics; the Sep-15 baseline on the same variant panicked); sports-live-reframer legacy smoke passes (v2 variant has a pre-existing, unrelated node-factory failure, identical on Sep 15).
  • Regression found and fixed during review: replay speed-continuity failure caused by the latched queue-stop flags after a seek (A/B confirmed), fixed in graph_core.hpp.

Follow-ups (not in this PR)

Router-first topology (per-slot conversion, runtime source add/remove, lifts the 32-source cap); reframer/recorder repos need the FFmpeg 8 migration (patch dir, avcpp pin, band-blur patch port); Dockerfile.fedora is broken on develop already (no pkg-config for FFmpeg headers).

🤖 Generated with Claude Code

@jpietek jpietek changed the title Support FFmpeg 8.1 CUDA pipelines and live recorder reconnects 10-bit 4:2:2 HDR mixer: P210 compositing, HLG/PQ sources, HEVC Main10 and tonemapped SDR outputs Sep 16, 2026
@jpietek
jpietek changed the base branch from develop to ffmpeg81-support September 16, 2026 16:03
@jpietek jpietek changed the title 10-bit 4:2:2 HDR mixer: P210 compositing, HLG/PQ sources, HEVC Main10 and tonemapped SDR outputs 10bit mixer Sep 16, 2026
@jpietek
jpietek force-pushed the ffmpeg8 branch 2 times, most recently from bda4927 to 7a04557 Compare September 16, 2026 16:09
@jpietek jpietek changed the title 10bit mixer 10-bit mixer with per-source SDR, HLG and PQ conversion Sep 16, 2026
@jpietek
jpietek changed the base branch from ffmpeg81-support to develop September 17, 2026 13:13
@jpietek jpietek changed the title 10-bit mixer with per-source SDR, HLG and PQ conversion 10-bit HDR mixer: SDR/HLG/PQ conversion, native 4:2:2, HDR10 outputs, shutdown fixes Sep 17, 2026
… shutdown fixes

Every source is normalised to the canvas contract (sdr, hlg, pq) on the GPU by
the new tonemap_cuda filter; P010/P210 canvases with 8-bit sources promoted in
the compositor; v210 sources stay 4:2:2; HEVC Main10 with HLG/PQ signalling and
HDR10 static metadata on outputs; filter EOF-tail, Python shutdown and seek
latch fixes; per-node GIL release. Compositor and orchestrator split into
modules under src/mixer (primitives/, orchestrator/), libav/avcpp reused
where the code re-implemented them; the mixer library ships as
pyplumber.mixer. Show files gain working_format, color, latency_ms, v210
sources, tone-mapped renditions and HDR10 metadata; docs and examples follow.

Squash of PR #33 (78 commits) onto develop after #34.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jpietek
jpietek merged commit 70d3220 into develop Sep 18, 2026
0 of 3 checks passed
@jpietek
jpietek deleted the ffmpeg8 branch September 18, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant