peter/siftlode
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(plex): don't tonemap 10-bit SDR; identity-guard the HW→SW fallback

Browse Source 
Deep-review round 1 on feature/plex-hdr-hw-tonemap:

- F1 (correctness): _select_encode gated tonemap on bit-depth alone, so a
  10-bit *SDR* (bt709) source got the HDR hable tone curve applied, visibly
  altering an SDR picture where only a 10->8-bit downconvert was needed. Split
  the ffprobe gate into a 3-way classifier (_probe_video_kind: hwsafe / hdr /
  sdr10); only true-HDR (PQ/HLG) takes the OpenCL tonemap graph, 10-bit SDR
  drops to software libx264 (an exact downconvert). Applies to both auto and on.

- F2 (race): the runtime HW->SW fallback popped+killed _sessions[key] with no
  identity check. Because wait_for blocks OUTSIDE _lock, a concurrent same-key
  _spawn (rapid seek-restart) or _enforce_cap cull could kill this session; the
  fallback then respawned software over the NEWER session and breached the
  transcode cap. Guard the respawn: only fall back if _sessions[key] is still
  this session, else yield the currently-registered one.

- tests: 3-way classifier (incl. HLG + empty-probe), real _opencl_present glob,
  real wait_for proc fast-fail branch, tonemap-variant fallback, the F1
  regressions (auto + on), the supersede/no-respawn race, and the on/opencl
  matrix cell. Backend gate green (ruff + full pytest).
This commit is contained in:
peter
2026-08-30 15:02:41 +02:00
parent 4827aebb7f
commit a2a8f5681d
2 changed files with 275 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/plex/stream.py
+68 -41
View File
@@ -162,23 +162,35 @@ _OPENCL_ICD_DIR = Path("/etc/OpenCL/vendors")
# full-GPU VAAPI pipeline. A source outside this set (AV1, MPEG-4 ASP/XviD…) falls back to software even
# when the render node is present, because the GPU can't decode it to feed its own encoder.
_HW_DECODABLE = {"h264", "hevc", "h265", "vp9", "vc1", "mpeg2video"}
# Color-transfer values that mark an HDR source (PQ / HLG). A HW-decodable HDR/10-bit source can't take
# the plain scale_vaapi=format=nv12 path (Gen 9.5 VPP refuses it) — it needs the OpenCL tonemap graph
# (_venc_opts tonemap branch) when the OpenCL runtime is present, else software libx264. See _probe_video_hwsafe.
# Color-transfer values that mark a true-HDR source (PQ / HLG). A HW-decodable HDR source can't take the
# plain scale_vaapi=format=nv12 path (Gen 9.5 VPP refuses it) — it needs the OpenCL tonemap graph
# (_venc_opts tonemap branch) when the OpenCL runtime is present, else software libx264. A 10-bit source
# WITHOUT one of these transfers is SDR, not HDR, and must not be tone-mapped. See _probe_video_kind.
_HDR_TRANSFERS = {"smpte2084", "arib-std-b67"}
def _probe_video_hwsafe(src: Path) -> bool:
"""Whether the source is safe for the PLAIN VAAPI transcode pipeline — i.e. 8-bit SDR. A 10-bit
(p010) or HDR source breaks `scale_vaapi=format=nv12` on Gen 9.5 VPP (`VAProfile not supported` →
ffmpeg emits zero segments and the item never plays), so it takes the OpenCL tonemap graph instead
(when the OpenCL runtime is present) or software libx264 (which CPU-decodes and downconverts to 8-bit
yuv420p cleanly). Ground truth from ffprobe (bit depth via pix_fmt / bits_per_raw_sample, HDR via
color_transfer); the Plex mirror doesn't store these.
# The three source classes that drive the transcode-pipeline choice (see _probe_video_kind).
_VK_HWSAFE = "hwsafe" # 8-bit SDR — the plain scale_vaapi=format=nv12 GPU path
_VK_HDR = "hdr" # PQ/HLG HDR — needs the OpenCL tonemap graph (or software if no runtime)
_VK_SDR10 = "sdr10" # 10-bit *SDR* — a plain 10→8-bit downconvert, NEVER the HDR tone curve
Returns True (assume HW-safe) on any probe failure: the runtime HW→SW fallback in start_session is
the backstop for a HW session that still produces nothing, so a probe miss only costs the rare
wasted first attempt, never a stranded item."""
def _probe_video_kind(src: Path) -> str:
"""Classify the source for the transcode pipeline — one of `_VK_HWSAFE` / `_VK_HDR` / `_VK_SDR10`:
* `_VK_HWSAFE` — 8-bit SDR: safe for the plain VAAPI path (`scale_vaapi=format=nv12` + h264_vaapi).
* `_VK_HDR` — PQ/HLG HDR (any depth): the plain scale path breaks on Gen 9.5 VPP, so it needs the
OpenCL tonemap graph (`tonemap_opencl=hable`) when the runtime is present, else software libx264.
* `_VK_SDR10` — 10-bit but *SDR* (bt709 etc.): NOT HDR. Applying the HDR hable tone curve here would
visibly alter an SDR picture (crushed highlights/midtones) where only a 10→8-bit downconvert was
needed, so it must NOT take the tonemap graph — it goes to software libx264, which CPU-decodes and
downconverts to 8-bit yuv420p exactly. (Kept off the plain scale_vaapi 10-bit path too: that path
is unverified on Gen 9.5 VPP, and software is the known-correct downconvert.)
Ground truth from ffprobe (bit depth via pix_fmt / bits_per_raw_sample, HDR via color_transfer); the
Plex mirror doesn't store these. Returns `_VK_HWSAFE` on any probe failure: the runtime HW→SW fallback
in start_session backstops a HW session that still produces nothing, so a probe miss only costs the
rare wasted first attempt, never a stranded item."""
try:
out = subprocess.run(
[
@@ -198,7 +210,7 @@ def _probe_video_hwsafe(src: Path) -> bool:
timeout=10,
)
except (subprocess.SubprocessError, OSError):
return True
return _VK_HWSAFE
pix_fmt = bits = transfer = ""
for line in out.stdout.splitlines():
k, _, v = line.strip().partition("=")
@@ -210,14 +222,12 @@ def _probe_video_hwsafe(src: Path) -> bool:
elif k == "color_transfer":
transfer = v
if transfer in _HDR_TRANSFERS:
return False # HDR (PQ/HLG) — VPP can't tonemap it to nv12 as configured
if bits.isdigit() and int(bits) > 8:
return False
# pix_fmt bit-depth suffix (yuv420p10le, p010le, yuv444p12le…) signals >8-bit even when
# bits_per_raw_sample is absent (some containers don't set it).
if re.search(r"(?:10|12|16)(?:le|be)$", pix_fmt):
return False
return True
return _VK_HDR # PQ/HLG — needs a real HDR→SDR tonemap, not a plain downconvert
# >8-bit but not HDR = 10-bit SDR. bits_per_raw_sample OR the pix_fmt bit-depth suffix
# (yuv420p10le, p010le, yuv444p12le…) signals >8-bit even when the other is absent.
if (bits.isdigit() and int(bits) > 8) or re.search(r"(?:10|12|16)(?:le|be)$", pix_fmt):
return _VK_SDR10
return _VK_HWSAFE
def _opencl_present() -> bool:
@@ -333,33 +343,42 @@ def _select_encode(db: DbSession, item: PlexItem, transcode: bool, src: Path) ->
* `(False, False)` → software libx264 (the fallback: no render node, non-decodable source, or a
10-bit/HDR source with no OpenCL runtime to tonemap on the GPU).
* `(True, False)` → full-GPU VAAPI (`scale_vaapi=format=nv12` + `h264_vaapi`) — an 8-bit SDR source.
* `(True, True)` → VAAPI decode → OpenCL tonemap → VAAPI encode — a 10-bit/HDR source, when the
OpenCL runtime is present (Gen 9.5 VPP can't tonemap natively).
* `(True, True)` → VAAPI decode → OpenCL tonemap → VAAPI encode — a true-HDR (PQ/HLG) source, when
the OpenCL runtime is present (Gen 9.5 VPP can't tonemap natively).
A 10-bit *SDR* source is deliberately NOT a `(True, True)` case: it needs a plain 10→8-bit downconvert,
not the HDR tone curve (which would visibly darken/flatten an SDR picture), so it drops to software.
Honours the admin `plex_transcode_hw` policy: "off" → always software; "on" → HW wherever the render
node exists (trusting the admin, skipping the decodability gate — the runtime HW→SW fallback still
backstops a session that produces nothing — but still selecting the tonemap graph for a 10-bit/HDR
source so it doesn't hit the scale_vaapi wall); "auto" (default) → HW only when the render node is
present AND the iGPU can hardware-decode the source, with tonemap chosen for a 10-bit/HDR source that
has an OpenCL runtime (else that source drops to software)."""
backstops a session that produces nothing — but still routing a true-HDR source through the tonemap
graph so it doesn't hit the scale_vaapi wall); "auto" (default) → HW only when the render node is
present AND the iGPU can hardware-decode the source, with tonemap chosen only for a true-HDR (PQ/HLG)
source that has an OpenCL runtime, while a 10-bit SDR source drops to software."""
if not transcode:
return (False, False)
mode = (sysconfig.get_str(db, "plex_transcode_hw") or "auto").lower()
if mode == "off" or not _render_node_present():
return (False, False)
if mode == "on":
# A 10-bit/HDR source still needs the tonemap graph (scale_vaapi can't handle it); use it when
# OpenCL is present, else plain HW and let the runtime backstop catch a miss. The probe is only
# consulted when OpenCL exists (short-circuit), so "on" stays probe-free where it can't matter.
tonemap = _opencl_present() and not _probe_video_hwsafe(src)
# Trust the admin: HW wherever the render node exists, skipping the decodability gate (the runtime
# backstop catches a miss). Only a true-HDR (PQ/HLG) source needs the tonemap graph — a 10-bit SDR
# source takes the plain scale path here, NOT the HDR tone curve. Probe only when OpenCL exists
# (short-circuit), so "on" stays probe-free where tonemap can't apply anyway.
tonemap = _opencl_present() and _probe_video_kind(src) == _VK_HDR
return (True, tonemap)
# auto
if (item.codec_video or "").lower() not in _HW_DECODABLE:
return (False, False)
if _probe_video_hwsafe(src):
kind = _probe_video_kind(src)
if kind == _VK_HWSAFE:
return (True, False) # 8-bit SDR → full-GPU scale path
# 10-bit / HDR, HW-decodable: HW OpenCL tonemap when the runtime is present, else software.
return (True, True) if _opencl_present() else (False, False)
if kind == _VK_HDR:
# PQ/HLG: HW OpenCL tonemap when the runtime is present, else software libx264.
return (True, True) if _opencl_present() else (False, False)
# _VK_SDR10 — 10-bit SDR: a plain 10→8-bit downconvert, done exactly on the CPU. Never the HDR tone
# curve (it would alter an SDR picture), and off the unverified scale_vaapi 10-bit path — software.
return (False, False)
def _want_hw(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> bool:
@@ -626,12 +645,20 @@ def start_session(
# other sessions). Pass the proc so a HW session that dies at VAAPI init fails fast (not after 25s).
ready = wait_for(_seg(0), timeout=25.0, proc=s.proc)
if not ready and s.hw:
# Runtime HW→SW fallback: the VAAPI session produced no segments (a VAProfile/init/tonemap
# failure the _select_encode gates didn't catch — e.g. an admin-forced "on" on a source the GPU
# can't decode, or an unforeseen driver refusal). Respawn on libx264 (no HW, no tonemap) so a
# failed GPU session never strands playback; the software path CPU-decodes and downconverts every
# source. s.hw is now False, so the route honestly reports "CPU" and the player's Preparing-stream
# state stays correct.
# We waited on _seg(0) OUTSIDE _lock, so `s.proc` dying is ambiguous: it may have failed on its
# own (VAProfile/init/tonemap refusal), OR a concurrent same-key _spawn (a rapid seek-restart) or
# _enforce_cap may have killed it to install/keep a newer session. Only fall back if THIS session
# is still the registered one — otherwise respawning would tear down the newer session and (via
# _spawn's supersede + _enforce_cap) breach the transcode cap. If superseded, yield to whoever is
# registered now. (A tiny window remains between this check and _spawn's own lock; that just makes
# the fallback lose to an even-newer seek, which is the correct seek-restart outcome anyway.)
with _lock:
superseded = _sessions.get(key) is not s
if superseded:
return current_session(key)
# Runtime HW→SW fallback: respawn on libx264 (no HW, no tonemap) so a failed GPU session never
# strands playback; the software path CPU-decodes and downconverts every source. s.hw is now
# False, so the route honestly reports "CPU" and the player's Preparing-stream state stays correct.
log.warning(
"plex transcode: HW session produced no segments — falling back to software key=%s", key
)