feat(plex): HW HDR→SDR tonemap for 10-bit/HDR transcode (OpenCL on the iGPU)
Upgrade mode-3's 10-bit/HDR branch from "route to software libx264" (BUG 1's behaviour) to a full-GPU pipeline: VAAPI decode → OpenCL tonemap (hable, HDR10/HLG → bt709 SDR nv12) → h264_vaapi. Gen 9.5 (UHD 630) can't tonemap in the VAAPI VPP (tonemap_vaapi fails), so OpenCL is the only HW route — proven on the prod iGPU at ~4.2x realtime, ~1 CPU core. - stream.py: `_select_encode` returns (hw, tonemap); a 10-bit/HDR HW-decodable source picks the tonemap graph when the OpenCL runtime is present, else drops to software. `_opencl_present` gates on a registered ICD. `_venc_opts` / `_hwaccel_in_opts` gain the tonemap variant (named vaapi+opencl devices, `-filter_hw_device ocl`, hwmap→tonemap_opencl→hwmap reverse). `_want_hw` kept as the hw-half wrapper. The runtime HW→SW fallback still backstops a session that emits no segment. - Dockerfile: add the Intel NEO OpenCL runtime (intel-opencl-icd). It was removed from Debian trixie, so it's pulled from bookworm (22.43 still supports Gen 9.5; the .deb runs on the trixie base — verified live). Harmless without /dev/dri. - route + FE: thread `tonemap` through the session response; the stats overlay mode row reads "transcode · GPU · HDR→SDR" for the tonemap path. - tests: tonemap command branch, the (hw, tonemap) selection matrix (auto/on × 8-bit/10-bit × opencl), start_session tonemap wiring, and the fmtMode tag.
This commit is contained in:
+114
-33
@@ -96,12 +96,14 @@ class HlsSession:
|
||||
entry: str,
|
||||
transcode: bool = False,
|
||||
hw: bool = False,
|
||||
tonemap: bool = False,
|
||||
):
|
||||
self.key = key
|
||||
self.dir = directory
|
||||
self.proc = proc
|
||||
self.transcode = transcode # video re-encoded (non-h264 source), not stream-copied
|
||||
self.hw = hw # transcode ran on the VAAPI hardware encoder (else libx264 software)
|
||||
self.tonemap = tonemap # HW HDR/10-bit path: OpenCL tonemap between VAAPI decode and encode
|
||||
self.start_s = start_s # the REQUESTED seek offset
|
||||
# The REAL media start: with `-ss X -c:v copy` ffmpeg can only cut at the nearest earlier
|
||||
# keyframe K<=X, and hls.js zero-bases the timeline to K (so video.currentTime=0 is K, not X).
|
||||
@@ -153,21 +155,26 @@ _HLS_TAIL = [
|
||||
# Everything else — the `-copyts`/`_HLS_TAIL`/segment/seek-restart model and the AAC audio path — is
|
||||
# unchanged, so hls.js, subtitles, multi-audio and seeking keep working exactly as for remux.
|
||||
_VAAPI_DEVICE = "/dev/dri/renderD128"
|
||||
# Where an OpenCL ICD registers its driver. The Intel NEO runtime (intel-opencl-icd) drops an .icd file
|
||||
# here; its presence is the precondition for the HW HDR→SDR tonemap graph (see _opencl_present).
|
||||
_OPENCL_ICD_DIR = Path("/etc/OpenCL/vendors")
|
||||
# Codecs the prod iGPU (UHD 630 / Quick Sync Gen 9.5) can hardware-DECODE — the precondition for the
|
||||
# full-GPU VAAPI pipeline. A source outside this set (AV1, MPEG-4 ASP/XviD…) falls back to software even
|
||||
# when the render node is present, because the GPU can't decode it to feed its own encoder.
|
||||
_HW_DECODABLE = {"h264", "hevc", "h265", "vp9", "vc1", "mpeg2video"}
|
||||
# Color-transfer values that mark an HDR source (PQ / HLG). Even a HW-decodable HEVC that is HDR must
|
||||
# take the software path — see _probe_video_hwsafe.
|
||||
# Color-transfer values that mark an HDR source (PQ / HLG). A HW-decodable HDR/10-bit source can't take
|
||||
# the plain scale_vaapi=format=nv12 path (Gen 9.5 VPP refuses it) — it needs the OpenCL tonemap graph
|
||||
# (_venc_opts tonemap branch) when the OpenCL runtime is present, else software libx264. See _probe_video_hwsafe.
|
||||
_HDR_TRANSFERS = {"smpte2084", "arib-std-b67"}
|
||||
|
||||
|
||||
def _probe_video_hwsafe(src: Path) -> bool:
|
||||
"""Whether the source is safe for the VAAPI HW transcode pipeline — i.e. 8-bit SDR. A 10-bit (p010)
|
||||
or HDR source breaks `scale_vaapi=format=nv12` on Gen 9.5 VPP (`VAProfile not supported` → ffmpeg
|
||||
emits zero segments and the item never plays), so it must take the software libx264 path, which
|
||||
CPU-decodes and downconverts to 8-bit yuv420p cleanly. Ground truth from ffprobe (bit depth via
|
||||
pix_fmt / bits_per_raw_sample, HDR via color_transfer); the Plex mirror doesn't store these.
|
||||
"""Whether the source is safe for the PLAIN VAAPI transcode pipeline — i.e. 8-bit SDR. A 10-bit
|
||||
(p010) or HDR source breaks `scale_vaapi=format=nv12` on Gen 9.5 VPP (`VAProfile not supported` →
|
||||
ffmpeg emits zero segments and the item never plays), so it takes the OpenCL tonemap graph instead
|
||||
(when the OpenCL runtime is present) or software libx264 (which CPU-decodes and downconverts to 8-bit
|
||||
yuv420p cleanly). Ground truth from ffprobe (bit depth via pix_fmt / bits_per_raw_sample, HDR via
|
||||
color_transfer); the Plex mirror doesn't store these.
|
||||
|
||||
Returns True (assume HW-safe) on any probe failure: the runtime HW→SW fallback in start_session is
|
||||
the backstop for a HW session that still produces nothing, so a probe miss only costs the rare
|
||||
@@ -213,18 +220,48 @@ def _probe_video_hwsafe(src: Path) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _hwaccel_in_opts(hw: bool) -> list[str]:
|
||||
def _opencl_present() -> bool:
|
||||
"""Whether an OpenCL runtime (Intel NEO / intel-opencl-icd) is installed — the precondition for the
|
||||
HW HDR→SDR tonemap graph. Gen 9.5 VPP can't tonemap natively (tonemap_vaapi fails), so the working
|
||||
HW path for a 10-bit/HDR source is VAAPI-decode → OpenCL tonemap → VAAPI-encode, which needs an ICD
|
||||
registered here. Absent on the dev box and until the image bakes NEO in → those sources take
|
||||
software libx264. Its own function so it's monkeypatchable in tests, mirroring _render_node_present."""
|
||||
try:
|
||||
return any(_OPENCL_ICD_DIR.glob("*.icd"))
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _hwaccel_in_opts(hw: bool, tonemap: bool = False) -> list[str]:
|
||||
"""VAAPI decode options, placed before `-i` so the source is decoded (and kept) on the GPU.
|
||||
`-hwaccel_output_format vaapi` keeps frames as GPU surfaces all the way to `h264_vaapi` — no
|
||||
round-trip to system memory. Empty for the software path."""
|
||||
round-trip to system memory. Empty for the software path.
|
||||
|
||||
The tonemap variant additionally derives an OpenCL device FROM the VAAPI one (`opencl=ocl@va`) so
|
||||
tonemap_opencl can map the decoded surfaces zero-copy (see _venc_opts); it needs named devices, so it
|
||||
uses `-init_hw_device` + `-hwaccel_device` rather than the plain path's implicit `-vaapi_device`."""
|
||||
if not hw:
|
||||
return []
|
||||
if tonemap:
|
||||
return [
|
||||
"-init_hw_device",
|
||||
f"vaapi=va:{_VAAPI_DEVICE}",
|
||||
"-init_hw_device",
|
||||
"opencl=ocl@va",
|
||||
"-hwaccel",
|
||||
"vaapi",
|
||||
"-hwaccel_output_format",
|
||||
"vaapi",
|
||||
"-hwaccel_device",
|
||||
"va",
|
||||
]
|
||||
return ["-hwaccel", "vaapi", "-hwaccel_output_format", "vaapi", "-vaapi_device", _VAAPI_DEVICE]
|
||||
|
||||
|
||||
def _venc_opts(
|
||||
transcode: bool,
|
||||
hw: bool,
|
||||
tonemap: bool = False,
|
||||
preset: str = "veryfast",
|
||||
crf: int = 20,
|
||||
max_height: int = 0,
|
||||
@@ -237,6 +274,23 @@ def _venc_opts(
|
||||
if not transcode:
|
||||
return ["-c:v", "copy"]
|
||||
force_kf = ["-force_key_frames", f"expr:gte(t,n_forced*{_SEG_SECONDS})"]
|
||||
if hw and tonemap:
|
||||
# HDR/10-bit HW path: Gen 9.5 VPP can't tonemap (scale_vaapi=format=nv12 fails), so the decoded
|
||||
# VAAPI surfaces are mapped to OpenCL (hwmap=derive_device=opencl), tone-mapped HDR→SDR to 8-bit
|
||||
# nv12 there (tonemap_opencl, hable curve, bt709 out), then mapped back to VAAPI (reverse=1) for
|
||||
# h264_vaapi. `-filter_hw_device ocl` tells the filter graph which device to run the OpenCL
|
||||
# filters on. Full GPU pipeline — ~1 CPU core, several×realtime on the UHD 630 (measured).
|
||||
return [
|
||||
"-filter_hw_device",
|
||||
"ocl",
|
||||
"-vf",
|
||||
"hwmap=derive_device=opencl,"
|
||||
"tonemap_opencl=tonemap=hable:transfer=bt709:matrix=bt709:primaries=bt709:format=nv12,"
|
||||
"hwmap=derive_device=vaapi:reverse=1",
|
||||
"-c:v",
|
||||
"h264_vaapi",
|
||||
*force_kf,
|
||||
]
|
||||
if hw:
|
||||
# scale_vaapi with format=nv12 downconverts a 10-bit (p010) HEVC source to 8-bit nv12 so
|
||||
# h264_vaapi (8-bit High) can encode it; for an 8-bit source it's a cheap format pass-through.
|
||||
@@ -274,23 +328,44 @@ def _render_node_present() -> bool:
|
||||
return Path(_VAAPI_DEVICE).exists()
|
||||
|
||||
|
||||
def _want_hw(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> bool:
|
||||
"""Whether to use the VAAPI hardware encoder for this transcode session, honouring the admin
|
||||
`plex_transcode_hw` policy: "off" → always software; "on" → VAAPI wherever the render node exists
|
||||
(trusting the admin, skipping the source gates — the runtime HW→SW fallback still backstops a
|
||||
session that produces nothing); "auto" (default) → VAAPI only when the render node is present AND
|
||||
the iGPU can hardware-decode the source AND the source is 8-bit SDR (a 10-bit/HDR source breaks
|
||||
scale_vaapi=format=nv12 on Gen 9.5 → software libx264, which downconverts cleanly)."""
|
||||
def _select_encode(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> tuple[bool, bool]:
|
||||
"""Pick the transcode video pipeline for this session — returns `(hw, tonemap)`:
|
||||
* `(False, False)` → software libx264 (the fallback: no render node, non-decodable source, or a
|
||||
10-bit/HDR source with no OpenCL runtime to tonemap on the GPU).
|
||||
* `(True, False)` → full-GPU VAAPI (`scale_vaapi=format=nv12` + `h264_vaapi`) — an 8-bit SDR source.
|
||||
* `(True, True)` → VAAPI decode → OpenCL tonemap → VAAPI encode — a 10-bit/HDR source, when the
|
||||
OpenCL runtime is present (Gen 9.5 VPP can't tonemap natively).
|
||||
|
||||
Honours the admin `plex_transcode_hw` policy: "off" → always software; "on" → HW wherever the render
|
||||
node exists (trusting the admin, skipping the decodability gate — the runtime HW→SW fallback still
|
||||
backstops a session that produces nothing — but still selecting the tonemap graph for a 10-bit/HDR
|
||||
source so it doesn't hit the scale_vaapi wall); "auto" (default) → HW only when the render node is
|
||||
present AND the iGPU can hardware-decode the source, with tonemap chosen for a 10-bit/HDR source that
|
||||
has an OpenCL runtime (else that source drops to software)."""
|
||||
if not transcode:
|
||||
return False
|
||||
return (False, False)
|
||||
mode = (sysconfig.get_str(db, "plex_transcode_hw") or "auto").lower()
|
||||
if mode == "off" or not _render_node_present():
|
||||
return False
|
||||
return (False, False)
|
||||
if mode == "on":
|
||||
return True
|
||||
# A 10-bit/HDR source still needs the tonemap graph (scale_vaapi can't handle it); use it when
|
||||
# OpenCL is present, else plain HW and let the runtime backstop catch a miss. The probe is only
|
||||
# consulted when OpenCL exists (short-circuit), so "on" stays probe-free where it can't matter.
|
||||
tonemap = _opencl_present() and not _probe_video_hwsafe(src)
|
||||
return (True, tonemap)
|
||||
# auto
|
||||
if (item.codec_video or "").lower() not in _HW_DECODABLE:
|
||||
return False
|
||||
return _probe_video_hwsafe(src)
|
||||
return (False, False)
|
||||
if _probe_video_hwsafe(src):
|
||||
return (True, False) # 8-bit SDR → full-GPU scale path
|
||||
# 10-bit / HDR, HW-decodable: HW OpenCL tonemap when the runtime is present, else software.
|
||||
return (True, True) if _opencl_present() else (False, False)
|
||||
|
||||
|
||||
def _want_hw(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> bool:
|
||||
"""The HW gate on its own (the `hw` half of `_select_encode`). Kept as a thin wrapper for the
|
||||
callers/tests that only care whether a session runs on the GPU at all."""
|
||||
return _select_encode(db, item, transcode, src)[0]
|
||||
|
||||
|
||||
# CPU-nice increment for the SOFTWARE transcode child. The Plex ffmpeg is a Popen in the long-lived API
|
||||
@@ -345,15 +420,17 @@ def _ffmpeg_cmd(
|
||||
audio_langs: list[str] | None = None,
|
||||
transcode: bool = False,
|
||||
hw: bool = False,
|
||||
tonemap: bool = False,
|
||||
preset: str = "veryfast",
|
||||
crf: int = 20,
|
||||
max_height: int = 0,
|
||||
) -> list[str]:
|
||||
# For a transcode session the video is re-encoded (VAAPI or libx264) instead of stream-copied; the
|
||||
# hwaccel decode options go before -i, the encoder args replace `-c:v copy` below. Everything else is
|
||||
# identical to remux.
|
||||
venc = _venc_opts(transcode, hw, preset, crf, max_height)
|
||||
args = ["ffmpeg", "-nostdin", "-loglevel", "error", *_hwaccel_in_opts(hw)]
|
||||
# identical to remux. `tonemap` is the HDR/10-bit HW variant (OpenCL tonemap between VAAPI decode and
|
||||
# encode) — it needs matching decode-side (named devices) and encode-side (the OpenCL filter graph) opts.
|
||||
venc = _venc_opts(transcode, hw, tonemap, preset, crf, max_height)
|
||||
args = ["ffmpeg", "-nostdin", "-loglevel", "error", *_hwaccel_in_opts(hw, tonemap)]
|
||||
# Keep ORIGINAL timestamps on the output so the first segment carries the true absolute PTS of the
|
||||
# keyframe ffmpeg actually seeked to. hls.js still zero-bases playback to that PTS; we read it back
|
||||
# (see _probe_first_pts) to learn the real content offset. Without -copyts the segment PTS is
|
||||
@@ -491,7 +568,7 @@ def start_session(
|
||||
# multi-audio: the video rendition is variant v0 → seg_{i}_0.ts; single-audio → seg_{i}.ts
|
||||
return directory / (f"seg_{i}_0.ts" if is_multi else f"seg_{i}.ts")
|
||||
|
||||
def _spawn(hw: bool) -> HlsSession:
|
||||
def _spawn(hw: bool, tonemap: bool = False) -> HlsSession:
|
||||
# Software-fallback encode tuning — read only on the SW transcode path (the VAAPI path and
|
||||
# remux ignore preset/crf/max_height, and remux must not touch the DB for them).
|
||||
preset, crf, max_height = "veryfast", 20, 0
|
||||
@@ -519,6 +596,7 @@ def start_session(
|
||||
audio_langs if is_multi else None,
|
||||
transcode=transcode,
|
||||
hw=hw,
|
||||
tonemap=tonemap,
|
||||
preset=preset,
|
||||
crf=crf,
|
||||
max_height=max_height,
|
||||
@@ -535,6 +613,7 @@ def start_session(
|
||||
"master.m3u8" if is_multi else "index.m3u8",
|
||||
transcode=transcode,
|
||||
hw=hw,
|
||||
tonemap=tonemap,
|
||||
)
|
||||
_sessions[key] = s
|
||||
# Admin-configurable concurrency cap (Configuration → Plex → Max concurrent transcodes);
|
||||
@@ -542,20 +621,21 @@ def start_session(
|
||||
_enforce_cap(max(1, sysconfig.get_int(db, "plex_max_transcodes")))
|
||||
return s
|
||||
|
||||
s = _spawn(_want_hw(db, item, transcode, src))
|
||||
s = _spawn(*_select_encode(db, item, transcode, src))
|
||||
# Wait for the first video segment outside the lock (this blocks on ffmpeg, and we must not hold up
|
||||
# other sessions). Pass the proc so a HW session that dies at VAAPI init fails fast (not after 25s).
|
||||
ready = wait_for(_seg(0), timeout=25.0, proc=s.proc)
|
||||
if not ready and s.hw:
|
||||
# Runtime HW→SW fallback: the VAAPI session produced no segments (a VAProfile/init failure the
|
||||
# _want_hw gates didn't catch — e.g. an admin-forced "on" on a 10-bit source, or an unforeseen
|
||||
# driver refusal). Respawn on libx264 so a failed GPU session never strands playback; the
|
||||
# software path CPU-decodes and downconverts every source. s.hw is now False, so the route
|
||||
# honestly reports "CPU" and the player's Preparing-stream state stays correct.
|
||||
# Runtime HW→SW fallback: the VAAPI session produced no segments (a VAProfile/init/tonemap
|
||||
# failure the _select_encode gates didn't catch — e.g. an admin-forced "on" on a source the GPU
|
||||
# can't decode, or an unforeseen driver refusal). Respawn on libx264 (no HW, no tonemap) so a
|
||||
# failed GPU session never strands playback; the software path CPU-decodes and downconverts every
|
||||
# source. s.hw is now False, so the route honestly reports "CPU" and the player's Preparing-stream
|
||||
# state stays correct.
|
||||
log.warning(
|
||||
"plex transcode: HW session produced no segments — falling back to software key=%s", key
|
||||
)
|
||||
s = _spawn(False)
|
||||
s = _spawn(False, False)
|
||||
ready = wait_for(_seg(0), timeout=25.0, proc=s.proc)
|
||||
if ready:
|
||||
# For a seek (start_s>0) learn the REAL start offset K from the first segment's absolute PTS,
|
||||
@@ -572,7 +652,7 @@ def start_session(
|
||||
if not wait_for(_seg(i), timeout=_WARMUP_SEG_TIMEOUT_S, proc=s.proc):
|
||||
break
|
||||
log.info(
|
||||
"plex hls session start key=%s req=%.1f real=%.3f multi=%s audio=%s transcode=%s hw=%s",
|
||||
"plex hls session start key=%s req=%.1f real=%.3f multi=%s audio=%s transcode=%s hw=%s tonemap=%s",
|
||||
key,
|
||||
start_s,
|
||||
s.media_start_s,
|
||||
@@ -580,6 +660,7 @@ def start_session(
|
||||
audio_ord,
|
||||
transcode,
|
||||
s.hw,
|
||||
s.tonemap,
|
||||
)
|
||||
return s
|
||||
|
||||
|
||||
@@ -554,14 +554,16 @@ def stream_session(
|
||||
raise HTTPException(status_code=404, detail="Media file not found on disk")
|
||||
# Report the REAL media start (the keyframe ffmpeg landed on, which hls.js zero-bases to), not the
|
||||
# requested offset — the client's absolute clock and the subtitle cue-shift both key off this.
|
||||
# `transcode`/`hw` let the player show an honest cold-start ("Preparing stream…") and surface the
|
||||
# active mode + encoder in the stats overlay; both segment streams are otherwise identical.
|
||||
# `transcode`/`hw`/`tonemap` let the player show an honest cold-start ("Preparing stream…") and
|
||||
# surface the active mode + encoder in the stats overlay (`tonemap` = the HW HDR→SDR path); both
|
||||
# segment streams are otherwise identical.
|
||||
return {
|
||||
"mode": "hls",
|
||||
"start_s": s.media_start_s,
|
||||
"url": f"/api/plex/stream/{it.rating_key}/hls/{s.entry}",
|
||||
"transcode": s.transcode,
|
||||
"hw": s.hw,
|
||||
"tonemap": s.tonemap,
|
||||
}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user