Skip to content

Work out whether HEVC is worth the risk it carries #86

Description

@nbkdoesntknowcoding

The appeal

HEVC carries roughly the same quality at 30–50% less bitrate than H.264. On a congested link that is the difference between a sharp picture and a soft one, and every Mac this runs on encodes it in hardware already.

It was considered during the latency research and rejected, on two grounds. This issue exists to test whether those grounds still hold, because if they don't the gain is real.

Why it was rejected

Decoder availability is not guaranteed. WebCodecs will report HEVC support and still fail on some Windows hardware, and the failure is not clean — it happens at decode time, after a session looks healthy. Where H.264 decode is essentially universal, HEVC depends on GPU, driver and on Windows sometimes a paid codec extension from the Store.

hev1 versus hvc1 is a trap. The two differ in whether parameter sets travel in-band or in the codec description, and WebCodecs is strict about it. This project already learned that lesson in H.264: the description field must be omitted or the decoder silently never produces frames. HEVC has more ways to get the same class of thing wrong.

What would settle it

Not an implementation. A decision, with evidence behind it:

  1. How common is working hardware HEVC decode across Windows machines people actually own? Not what VideoDecoder.isConfigSupported() claims — what genuinely decodes. Those are different, and the difference is the whole issue.
  2. Does it pay off at this bitrate? The claimed savings are measured on long-form video. Screen content is mostly static with sudden large changes, which is a different rate-distortion problem, and the sender already runs well under the link's capacity much of the time. A 40% saving on a stream that is not bandwidth-limited buys nothing.
  3. What does it cost in latency? HEVC encode is more expensive, and this project trades quality for latency by design everywhere else. If it adds milliseconds to encode it is the wrong trade regardless of the bitrate win.
  4. How does fallback work? Both codecs have to be negotiated, and a receiver whose HEVC decode fails mid-session has to fall back without the user watching a black screen while it happens.

Where the ground has already been prepared

H264Encoder reads its emitted profile back out of the SPS rather than trusting the settings, after low-latency mode turned out to select a different encoder entirely. Any HEVC work should do the same from the start — read back what came out, do not trust what was asked for.

The honest default

If the answer to (1) is "most machines, reliably", this is worth building. If it is "most machines, usually", it is not — a codec that works for four people in five is worse than one that works for everybody, because the fifth cannot tell a broken app from an unsupported one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    deep-diveSubstantial work — needs real expertise in one areaperformanceLatency, throughput, frame pacingresearchAnswer the question first; the code may not follow

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions