3987cd42602fd7df56b17bcb3dfbac80f52d1b46
36
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 peter
|
dc4e8b9fcd |
fix(plex): playback sessions per viewer, bounded in total; a fair share of subtitle passes
HLS sessions were keyed by the item, so a second account opening the same film killed the first one's session; the displaced player reclaimed at its playhead and evicted the other in turn. The registry is now keyed by (viewer, rating_key), segment requests resolve by both, and the session directory carries the viewer (the orphan sweep still recognises the older shapes). Only transcodes counted toward a cap, so remuxes — each a whole segmented copy of the film on the scratch volume — were bounded by the library size. Three bounds now apply after every start, each dropping the least recently used: two sessions per account, the transcode cap, and a new admin knob, plex_max_sessions (default 8), over every session. A superseded session request returns at once, since it waits on the killed ffmpeg, so an account cannot park more request workers than it may hold sessions. The subtitle extractor's full-pass slots were global and minutes long each; one account opening subtitles on film after film could hold them all. An account may now hold three of the six, and still gets windows past that. |
||
|
peter
|
54d19cc386 |
fix(logs): deep-review round 1 — say each failure once, and scrub it everywhere
- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
`formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
file paths, and the job-failed, refused, edit-failed and backfill lines use
it. Done lines log the asset id, not the title-built path. The PII contract
now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
a process's last batch closes the repeat window, so a short-lived download
child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
`download.job.failed` with a cause; they left no line at all before. The
crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
carry reason `transport`. Subscribe recognises a duplicate by reason code
only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
|
||
|
peter
|
04b58e5c6f |
feat(logs): give the log the events that no request line already carries
The gap this sprint was scoped against turned out to be smaller than the survey said, and in a way worth writing down: `log_requests` already records every HTTP call with its route template, status, duration, request id and user id, and the audit log already records every admin act on someone else's account. Putting a line in each of routes/downloads.py's twenty mutating endpoints would have added a second copy of the first and nothing else. So this instruments what neither of them can see. yt-dlp now explains itself to our log. `quiet: True` was doing more than quieting: with no `logger` its whole account of a download went to the child's stdout, so "Requested format is not available", a signature-extraction failure or a POT refusal reached us only as whatever text ended up in the raised exception — the real reason a download failed was routinely one line the app never kept. Measured against the pinned yt-dlp 2026.07.04 before wiring it: with `logger` set it calls `debug` for extraction and progress lines and `warning`/`error` for the rest, and neither `quiet` nor `no_warnings` gates those calls. `debug` lands at TRACE, being several lines a second, so the noisy half is the half the user's own consent gates. The full source URL is trimmed to its host on the way in — it is a row of someone's download history, and `logs.set_job` already puts the job id on every one of these lines. The option is set AFTER the adapter merge, like `cookiefile` and the size cap: a source adapter must not be able to take away the one account of what went wrong that a failed job leaves behind. An HLS session now says which pipeline it got (transcode/remux, hw, tonemap, variants, offset) — everything downstream is about that choice, and it was only inferable from an ffmpeg command line nobody keeps. A session culled by the concurrent-transcode cap says so: somebody's playback stops there, the route turns it into a retryable 404, and "my film keeps rebuffering" had no other explanation anywhere. The HW→SW fallback keeps its line and gains an event code. A live search refused by a quota cap says WHICH cap: the 429 is in the request line either way, but the per-user daily cap is answered by tomorrow and the shared budget by the admin. The download and edit job lifecycle lines gain event codes, which is what E22.6's console filters on — the job id was already a structured field. tests/test_ytdlp_logging.py; three of its four cases mutation-checked. A fourth (a `%` in a progress line) survived its mutation — stdlib only applies `%` formatting to a record that has args — so it is gone rather than kept as a case nothing can break; the reasoning moved into the comment on `_emit`. |
||
|
peter
|
6b0401b922 |
fix(logs): rule on every swallowed exception in the backend
114 exception handlers neither logged nor re-raised. The defect was never that
they are silent — plenty of them should be — it is that their silence said
nothing, so nobody could tell a decision from an oversight. Each one has now been
read. 44 got a line; the other 70 carry `# silent-ok: <why>`.
What the silent ones had been hiding, in the cases that got a line:
- A failed ffprobe sent every HDR file down the 8-bit pipeline, so playback
degraded to a software transcode with nothing saying why; a failed audio probe
collapsed a multi-audio file to one track and the player showed no language
picker; a failed first-PTS probe left subtitles seconds out of sync.
- A subtitle sidecar that existed but would not parse made a PARTIAL track read as
complete, permanently — the viewer loses every cue past the cached window.
- An unreadable crop/trim/cut-list in an edit spec was dropped, so the clip came
out uncropped, untrimmed, or shorter than the one the user built.
- A dropped `max_height` downloaded at best quality against an explicit cap.
- A failed videos.list enrich made every candidate undroppable-from-judgement and
therefore dropped: a working search rendered as "no results".
- An unusable password hash answered as "wrong password", locking the owner out
and telling them they mistyped; a rotated TOKEN_ENCRYPTION_KEY turned every
stored token into "not connected".
- A stored admin setting that would not coerce fell back to the env default while
the settings page went on showing the saved value.
- A one-off InnerTube config failure pinned the whole process to the hard-coded
fallback key, and searches return nothing when that goes stale.
- The HLS sweep reported 0 swept — like a clean run — when it could not list its
own root, leaving every crashed session's segments on disk forever.
Four of them are counted rather than logged per item (dropped cues, dropped cut
list segments, unreclaimed GC files) and reported once per call, because the
alternative is one line per cue over a whole subtitle track.
ruff gains S110 and BLE001. Every blind `except Exception` that remains carries
`# noqa: BLE001 — <why>`, which is the point: catching everything is sometimes
right, and always worth a sentence. plex/sync.py's three best-effort backfill
catches now keep the traceback — a blind catch over a batch that fetches AND
writes could be Plex, the database or our own bug, and `%s` of the exception told
them apart badly. main.py's boot-time log-level read likewise.
logs.event's `code` is positional-only now: `event("x", code=...)` was a TypeError,
and `code` is exactly the word a caller reaches for when the thing that failed has
one of its own.
tests/test_swallowed_exceptions.py is the wider rule ruff cannot see: a handler
that neither logs nor re-raises must say why. Both cases mutation-checked — a
marker removed and a log line removed each fail it, naming the handler.
|
||
|
peter
|
0cff964ff2 |
refactor(logs): name every logger after its module, and hold it there
25 literal names covered 34 modules. plex/client.py, plex/stream.py, plex/subtitles.py, plex/sync.py and two Plex route modules all called themselves `siftlode.plex`; downloads/pipeline.py called itself `siftlode.worker`; routes/channels.py called itself `siftlode.api`. The `logger` column is in every stored row so E22.6's console can filter on it, and so an operator can pin a level to one part of the system — neither works when six modules share a name. `logs.get_logger(__name__)` derives it: app/plex/sync.py -> siftlode.plex.sync. One name per module, and it cannot drift from the file it sits in. tests/test_logger_names.py pins the derivation, that everything stays under the one `siftlode` tree the handler and the tests attach to, and that no module hand-rolls a name again unless the line says why (`# logger-name-ok: <why>`). Two do: get_logger's own call, and `siftlode.http`, which is named for the traffic it carries rather than the module it lives in — it replaced uvicorn's access log and is what an operator pins when they want request lines and nothing else. app/scheduler.py called its logger `logger`; every other module calls it `log`. ruff gains LOG, G, T20, TRY400, TRY401 and S112. Four findings cleared with them: worker.py logged `str(exc)[:120]` where the traceback was the whole question, and three prints/f-strings that are deliberate now say so on the line. S110 and BLE001 stay off until S3 — 23 handlers are outstanding and each is a judgement about one of them; enabling the rules first would just mean 23 blanket noqas. |
||
|
peter
|
07766c43e8 |
feat: closed-set admin knobs are pickers, not free text
log_level, search_source, download_layout and the transcode hw/preset were typed by hand, and every consumer carried its own guess at what a typo meant. The value set is now a Literal in config.py: pydantic rejects a bad env value at boot (any letter case accepted), sysconfig refuses a bad override and derives the admin picker's options from the same Literal, and the consumers read the value as-is. The Configuration page renders a choice key as a segmented bar, or a dropdown past six options, with translated option names. Its tabs now follow the registry's order instead of a hand-kept copy that had already dropped the explore group. A single-choice FilterSelect now opens on the option in use: in the browser it could land on the first row instead. |
||
|
peter
|
71d7646bf1 |
fix(plex): a name this session cannot write fails at once, and the layout has one spelling
Round 6 turned `artifact_may_arrive` into a positive rule, but keyed it to the HLS vocabulary in
general rather than to what THIS session produces: the playlist regex accepted master/index/stream_*
for every session and the `seq == 0` branch accepted any variant token unconditionally. The two argv
branches are disjoint — a single-audio session is given `seg_%d.ts` + `index.m3u8` and no `%v` at
all, so `master.m3u8`, `stream_*.m3u8` and every `seg_<variant>_<n>.ts` are names it can never write,
and vice versa — so each of those was read as "an HLS name, so wait and see" and held a threadpool
worker, plus its pooled DB connection, for the full timeout. That is verbatim the round-5 outage the
route's own comment claims it prevents.
Measured through the real route against a live single-audio session, before: master.m3u8 404 after
30.0s, stream_q7z.m3u8 404 after 30.0s, seg_a0_0.ts / seg_zzzz_0.ts / seg_0_0.ts 404 after 20.1s
each. After, on the running localdev API: 404 in 44-316ms for all of them, while index.m3u8,
seg_0.ts and seg_1.ts still answer 200 — nothing is over-refused.
The session now carries the rendition names its ffmpeg was told to write, and `entry` is derived from
them instead of passed. And "one place" now means the argv BUILDS its patterns from the same
functions the parser reads back (`segment_name("%d", "%v")`, `variant_playlist_name("%v")`): the
layout was still spelled out independently in six places, with `_VIDEO_VARIANT` read by no consumer,
which is the drift that caused round 6's HIGH. A test asserts the argv's own patterns are what those
functions produce, and that every name they expand to is one the guard accepts.
The load-bearing MEASURED comment cited ffmpeg 8.1.1 — the Windows host's. The binary that writes
these files is the API container's 7.1.5; re-measured there, the conclusion holds.
|
||
|
peter
|
7f17b17c13 |
fix(plex): an HLS session's file names are one vocabulary, and a name it cannot produce fails at once
Round 5 added a guard so a request for a segment beyond the produced region answers instead of polling out the full timeout — the route is a SYNC one, so every such wait parks a threadpool worker and, through `current_user`, a pooled DB connection. Measured against real ffmpeg, the guard could not fire where it mattered. `-hls_segment_filename seg_%v_%d.ts` substitutes the VARIANT first: a dual-audio session writes seg_0_0.ts (video) and seg_a0_0.ts / seg_a1_0.ts (the `name:aN` renditions). The regex read those as sequence-first, so every video segment parsed as segment 0 and every audio segment did not parse at all — the guard was inert on every multi-audio item, which is most of this library. `_seg()` carried the same inversion: the warm-up waited 8 s for `seg_1_0.ts`, a name ffmpeg never writes, on every multi-audio session start AND every seek-restart. The layout is now one constant plus one parser (`segment_name` / `_SEG_NAME_RE`), with a test pinning them to each other, instead of three comments that could drift — and they had drifted, all three ways. The guard is also a positive rule now. It was a blocklist, so anything that was not a single-audio segment name was waved through: `GET …/anything.m3u8` still parked a worker for 30 s, while the route's comment claimed a flood of hopeless names was what it prevented. A session produces its playlists and its segments and nothing else. `.vtt` leaves `_HLS_CT` with it — ffmpeg runs with `-sn` and subtitles are a separate route, so that extension could only ever name a file that will not arrive. Also here: - A 5000-digit segment index reached the client as a 500: `int()` on an unbounded digit run raises past 4300 digits and only OSError was caught. The name is bounded on the route and the index in the regex — the same bug class the subtitle routes were bounded for. - A session torn down between the wait and the read is a 404, not a 500. `_kill` rmtrees the directory and no lock spans those two steps; the player's recovery is keyed on the STATUS, so a 500 skipped the reclaim and ended playback with "couldn't play" instead of restarting at the playhead. The trigger is ordinary — `loadSession` awaits the new session before destroying the old hls instance. - Two comments claimed an open player checkpoints every 10 s "whether or not it is playing", which is the stated reason the 30 s finished-idle reap is safe. `flush` sends nothing once the finish latch is up, or before a position is known. The states it does cover — playing or paused mid-item — are the ones the rule needs; the comments now say that instead of more. - The route's tests leave the DB lane: the route reads nothing from the database since the auto-start was removed, and there those four regressions SKIP on a machine with no Postgres while pytest prints green. 14 tests, mutation-checked: the old parse, the old warm-up name, the blocklist, both bounds off the absurd name, the unguarded read, and the route dropping `proc` — each red on the test that names it. |
||
|
peter
|
1f81684b6e |
fix(ui,plex): a dialog keeps the focus its child asked for, and a hopeless segment fails at once
Both found by UAT in a real browser, after the tests for them were already green — which is the whole reason this repo settles focus and timing claims by running them. - **The autoFocus fix worked, and then the dialog took the focus back.** React 18's StrictMode runs every effect mount→cleanup→mount in development: the cleanup handed focus back to the trigger, and the second mount, finding the focus outside the dialog, moved it to the first focusable — off the `autoFocus` field a beat after the browser had put it there. Chrome measured it: INPUT (commitMount) → BODY (cleanup) → ✕ (re-mount). The effect is idempotent now — it remembers what the dialog was focused on and restores THAT on a re-mount — and the close-time restore is conditional on the dialog still holding the focus, which it tracks as it changes rather than computing at cleanup, where the panel is already out of the document. A dialog no longer yanks focus back from wherever the user has since moved it. - **Passing ffmpeg's handle to the segment wait only shortened the FINISHED case.** Measured against the running localdev API: a request for `seg_99999.ts` on a live session still took 20 195 ms, because that ffmpeg has not exited and the wait has no other reason to stop. ffmpeg writes segments in order, so a segment whose predecessor does not exist is not the next one and cannot arrive — and that is knowable at once. Same request now: 45 ms. Verified on localdev against the real library (Westworld S01E06, remux, multi-audio): Mod+K opens the command palette with the focus in its search field and typing reaches it; playback runs on the new session-scoped HLS URL; a request naming a stale session id is a 404; and a viewer paused for longer than the finished-session rule keeps their 2.4 GB session across two reaper passes — the progress checkpoint touches it — and resumes cleanly. vitest 1009, pytest 1127. |
||
|
peter
|
47af4d5c05 |
fix(plex): an HLS artifact names the session it belongs to, and a paused viewer says so
Deep-review round 5, over round 4's own fixes. - Sessions are keyed by rating_key alone while a session's DIRECTORY is per (key, offset, tag), and the artifact URL carried no session identity — so once a seek-restart or a second viewer of the same item took the registry slot, the old viewer's request for `seg_12.ts` was answered 200 with the new session's twelfth segment: bytes from a different `-ss` offset under `-copyts`, on a timeline the client is not on. Wrong content with a success status is the one failure a player cannot notice, and round 4's reclaim-on-404 could never fire on it. The id is in the path now (so the playlist's relative names resolve back into the same session), a mismatch is a 404, and the player reclaims at the playhead. The entry playlist's auto-start is gone with it: it could only ever begin at 0, silently discarding the viewer's position. - Round 4 exempted paused viewers from the reaper by keeping EVERY finished session for the full ten-minute idle window — one whole segmented copy of a film per item, on the volume the download center shares. The signal it needed already existed: an open player checkpoints its position every 10 s whether or not it is playing, so that route now touches the session and the periodic pass is back on the short rule. - POST /session's opportunistic reap was never scoped to the key being started: it walked the whole registry with the 30 s finished rule, so pressing play on ANY item — even a direct-play one, the call ran before that early return — killed another viewer's paused session. It is gone; the periodic pass owns reaping. - A segment that can never arrive no longer polls out the full 20 s: the producer's handle is passed to `wait_for`, so a finished ffmpeg answers at once. This is a sync route, so each such wait parked a threadpool worker AND a pooled DB connection, and an ordinary seek past the produced region was enough to do it. - The DataError handler's re-raise branch — the half that keeps a genuine DataError a 500 — is pinned, so simplifying it to an unconditional 422 cannot pass in silence. Six regression tests (two route-level, in the DB lane), each mutation-checked red on the assertion it is named after. pytest 1124, vitest 999. |
||
|
peter
|
31dc4184f5 |
fix(api): a paused viewer keeps its HLS session, a refused subtitle publish backs off, and a NUL is a 422 everywhere
Deep-review round 4, backend half. - The periodic HLS reaper (round 3) applied the "finished and unfetched for 30 s" rule that only a NEW session request may apply. A remux finishes long before the film, and a paused player with a full buffer fetches nothing, so a one-minute pause killed the session and playback died ~30 s after resuming. The periodic pass is `reap_abandoned` now (the long idle limit, finished or not). - A subtitle `publish` that could not keep its output (full / read-only cache volume) returned quietly, so every poll queued the window and the whole-container pass again. It reports failure and the pass is backed off like a failed read; a cached VTT that cannot be READ is answered `unavailable` (`cache_unreadable`) instead of scheduling reads that can never land. - The subtitle routes bound `rating_key` / `sub_ord` (they become a cache file name: a 250-digit one was ENAMETOOLONG, which pathlib's exists() re-raises -> a 500), and the cache helpers no longer throw on such a name. - One app-wide handler turns psycopg's "text fields cannot contain NUL" DataError into a 422: round 3 guarded only the by-rating_key lookup, and `contains=`, `contains_group=`, reorder/bulk key lists and the collection search still 500'd. - The hand-written `db.get(...) is None -> 404 "Unknown ..."` siblings use `get_or_404` with a `what` (channel / video / user), so the Hungarian UI no longer shows English for a gone row. |
||
|
peter
|
5a42d76e8a |
feat(plex): an off-by-default loudness normalisation toggle for playback
Bug hunt 2026-09, item 14 — "how does the multichannel downmix work, and is it
normalised?" It isn't: `plex/stream.py` has always encoded `-c:a aac -ac 2 -b:a
192k`, i.e. fixed stereo with ffmpeg's default downmix matrix and no loudness
handling at all, which is why a film mastered for a cinema arrives with whispered
dialogue and deafening action.
New admin knob `plex_audio_loudnorm` (Configuration → Plex), DEFAULT OFF: it makes
a film markedly louder and narrows its dynamic range, which is a taste call rather
than a fix. It sits at the end of the Plex group, after the software-transcode
knobs — it is an audio concern, not one of the libx264 ones.
-af aformat=channel_layouts=stereo,loudnorm=I=-16:TP=-1.5:LRA=11,aresample=48000
Three things about that chain are load-bearing and each has a test:
* the downmix is FORCED FIRST rather than left to `-ac 2` — measuring 5.1 and
folding it down afterwards would normalise something other than what is heard;
* the trailing resample is not decoration: `loudnorm` runs its own resampler at
192 kHz internally, which the AAC encoder will not take;
* one `-af`, no stream index, so EVERY audio rendition of a multi-audio file is
normalised — not just the first.
The knob is also part of the session directory name (`…_multi-ln_+0.00`), because
it changes the audio the segments contain, and a session left over from before the
admin flipped it must not be mistaken for one that honours it. The suffix is
appended with a DASH: `_SESSION_DIR_RE` treats the tag as one underscore-free
field, and that regex is the only thing keeping the orphan sweep off what may be a
shared scratch path. A test pins both halves — the name changes, and the sweep
still recognises it.
Measured end-to-end on the app's own HLS segments (a real 5.1 AC3 file, same item,
same 160 s offset, same ten segments): OFF -35.2 LUFS, ON -17.2 LUFS. On a 60 s
bench: I -27.6 → -16.6 LUFS with LRA 13.3 → 9.8 LU. A/V sync is untouched — under
this pipeline's own `-copyts` the audio start PTS is identical with and without the
filter, so the 3 s lookahead desyncs nothing.
Direct-played files are served untouched and stay so; a 5.1 mix is always `remux`,
so the reported case is fully covered.
|
||
|
peter
|
d7e9240e54 |
merge main (0.78.2) into playback-class before release
# Conflicts: # backend/app/plex/stream.py |
||
|
peter
|
b563155a37 |
docs(plex): correct the cull-yield fallback comment to name the real route 404s
Deep-review round 3: the only surviving finding — a comment-drift nit. The fallback's cull-yield comment named the route's 'Not available yet' path, but a culled-key segment request actually returns 'No active playback session' (the substantive invariant — never a None that 404s as file-not-found — is unchanged). Reworded to match the route. Concurrency + correctness came back clean. |
||
|
peter
|
915319fa35 |
fix(plex): make the HW->SW fallback supersede-check atomic; honest cull yield
Deep-review round 2 on the round-1 fix: - Race (medium): the round-1 identity guard did the supersede check and the respawn in TWO separate _lock sections. A newer same-key session installed in the gap was still torn down by _spawn's unconditional pop+kill (the round-1 comment even had the direction backwards). Fold the check into _spawn via a new replace_only= param: the identity test and the pop+replace now share ONE lock, so the respawn is a true no-op when superseded/culled. - Honest states (low): the superseded branch returned current_session(key), which is None when the key was cap-culled during our out-of-lock wait; the route turns None into a dishonest 'Media file not found on disk' 404 for a file that exists. Yield 'current_session(key) or s' so a culled session degrades to the route's retryable 'not available yet' path instead. - Hygiene: _want_hw docstring corrected (it is a test-only shim now; production decides via _select_encode) and the two stale _want_hw references refreshed (stream.py header, config.py plex_transcode_hw). - tests: the cap-cull no-respawn/no-None sub-case; the _opencl_present except OSError branch (glob raising, not just empty); tightened a tautological -vf assertion. Backend gate green (ruff + full pytest). |
||
|
peter
|
a2a8f5681d |
fix(plex): don't tonemap 10-bit SDR; identity-guard the HW→SW fallback
Deep-review round 1 on feature/plex-hdr-hw-tonemap: - F1 (correctness): _select_encode gated tonemap on bit-depth alone, so a 10-bit *SDR* (bt709) source got the HDR hable tone curve applied, visibly altering an SDR picture where only a 10->8-bit downconvert was needed. Split the ffprobe gate into a 3-way classifier (_probe_video_kind: hwsafe / hdr / sdr10); only true-HDR (PQ/HLG) takes the OpenCL tonemap graph, 10-bit SDR drops to software libx264 (an exact downconvert). Applies to both auto and on. - F2 (race): the runtime HW->SW fallback popped+killed _sessions[key] with no identity check. Because wait_for blocks OUTSIDE _lock, a concurrent same-key _spawn (rapid seek-restart) or _enforce_cap cull could kill this session; the fallback then respawned software over the NEWER session and breached the transcode cap. Guard the respawn: only fall back if _sessions[key] is still this session, else yield the currently-registered one. - tests: 3-way classifier (incl. HLG + empty-probe), real _opencl_present glob, real wait_for proc fast-fail branch, tonemap-variant fallback, the F1 regressions (auto + on), the supersede/no-respawn race, and the on/opencl matrix cell. Backend gate green (ruff + full pytest). |
||
|
peter
|
4827aebb7f |
feat(plex): HW HDR→SDR tonemap for 10-bit/HDR transcode (OpenCL on the iGPU)
Upgrade mode-3's 10-bit/HDR branch from "route to software libx264" (BUG 1's behaviour) to a full-GPU pipeline: VAAPI decode → OpenCL tonemap (hable, HDR10/HLG → bt709 SDR nv12) → h264_vaapi. Gen 9.5 (UHD 630) can't tonemap in the VAAPI VPP (tonemap_vaapi fails), so OpenCL is the only HW route — proven on the prod iGPU at ~4.2x realtime, ~1 CPU core. - stream.py: `_select_encode` returns (hw, tonemap); a 10-bit/HDR HW-decodable source picks the tonemap graph when the OpenCL runtime is present, else drops to software. `_opencl_present` gates on a registered ICD. `_venc_opts` / `_hwaccel_in_opts` gain the tonemap variant (named vaapi+opencl devices, `-filter_hw_device ocl`, hwmap→tonemap_opencl→hwmap reverse). `_want_hw` kept as the hw-half wrapper. The runtime HW→SW fallback still backstops a session that emits no segment. - Dockerfile: add the Intel NEO OpenCL runtime (intel-opencl-icd). It was removed from Debian trixie, so it's pulled from bookworm (22.43 still supports Gen 9.5; the .deb runs on the trixie base — verified live). Harmless without /dev/dri. - route + FE: thread `tonemap` through the session response; the stats overlay mode row reads "transcode · GPU · HDR→SDR" for the tonemap path. - tests: tonemap command branch, the (hw, tonemap) selection matrix (auto/on × 8-bit/10-bit × opencl), start_session tonemap wiring, and the fmtMode tag. |
||
|
peter
|
840526b1b8 |
feat(plex): filter the feed by playback class (Direct Play / Direct Stream / Transcode HW/SW)
Adds a movie-scoped "Playback" filter to the Plex feed so a user can isolate items by how they play — the groundwork for collecting HW-transcodable (or no-transcode) versions. Four classes derived from the stored `playable` + a new `transcode_hw` facet: - Direct Play (direct), Direct Stream (remux) - Transcode · HW — playable=transcode AND the iGPU can hardware-transcode it - Transcode · SW — playable=transcode, software libx264 only The HW/SW split is precise, not codec-only: a 10-bit/HDR HEVC is HW-decodable yet breaks the VAAPI nv12 pipeline (the same case plex/stream special-cases), so it must count as SW. Backend: - New PlexItem columns (migration 0077): transcode_hw, video_bit_depth, media_probed_at. Migration backfills transcode_hw codec-only so the filter works immediately. - `plex.sync.transcode_hw_capable` is the single classifier; the HW-decodable codec set moves to sync as `HW_DECODABLE_CODECS` and plex.stream imports it, so the catalog facet and the runtime encoder choice can't disagree by codec. - Bit-depth backfill (`_probe_media_batch`, wired into sync): probes only HW-decodable transcode items' video Stream for bit-depth + HDR and refines a 10-bit/HDR HEVC to software. Bounded per run, watermarked by media_probed_at. A codec change resets the watermark (stale probe). - Filter + self-excluded facet on a shared `playback_class_expr` CASE (movie-leg only — a show is an aggregate with no single class). transcode_hw threaded onto the feed/detail/card JSON. Frontend: - Multi-select "Playback" Combobox in the sidebar (movie scope, facet-empty drop-out), active chips, 4-way corner-dot badge (green/blue/amber/red) + tooltip, and an info-page playback chip with a codec · bit-depth subtitle. New `bg-info` (blue) token, i18n HU/EN parity. - Fix a regression where the card's playback dot lost its hover tooltip: the full-card reveal overlay swallowed the cursor; the dot now sits above it (z-20). Gate: backend full lane + frontend typecheck/lint/knip/prettier/vitest all green. Browser-UAT'd on localdev (filter narrows to the SW set, facet counts, badge colour + tooltip, info chip). |
||
|
peter
|
10d69cb316 |
fix(plex): route 10-bit/HDR HEVC to software + runtime VAAPI→libx264 fallback
A 10-bit/HDR HEVC source (e.g. an HDR10 x265 file) died at scale_vaapi=format=nv12 on the UHD 630 (Gen 9.5 VPP: "VAProfile not supported"); ffmpeg produced 0 segments and the item never played, with no fallback — a retry just re-picked the failing HW path. An 8-bit HEVC and the same file on libx264 both work. Two defenses: - `_want_hw` auto mode now also gates on an 8-bit-SDR source via a new `_probe_video_hwsafe` ffprobe (bit depth from pix_fmt/bits_per_raw_sample, HDR from color_transfer) — a 10-bit/HDR source takes the software libx264 path, which downconverts to yuv420p cleanly. The Plex mirror stores no bit-depth signal, so this is probed at session start. "on" still trusts the admin and skips the gate. - `start_session` adds a runtime HW→SW fallback: a HW session that emits no first segment respawns on libx264 and reports hw=False (the route/stats overlay honestly show CPU). `wait_for` gained an optional `proc` so a dead ffmpeg fails fast instead of burning the 25s budget before the fallback fires. Tests: TestProbeVideoHwsafe (parsing + probe-failure-assumes-safe), the 10-bit auto gate + "on" skip in TestWantHw, and the no-segment HW→SW respawn in TestTranscodeStartSession. Backend lane green. |
||
|
peter
|
00539a4d97 |
fix(plex): deep-review follow-ups on E6 transcode + player polish
Eight verified findings from the sprint deep-review: - Transcode cap: _enforce_cap now counts only transcode sessions, so a live remux (stream-copy) stream can no longer be LRU-evicted mid-play — matching the config comment + the HU/EN Config hint that promise remux is uncapped. - Player Back: the browser/mouse Back interceptor (BackClose) now also mounts for the stats overlay and up-next card, so Back closes them first instead of ejecting the whole player (keyboard/Esc already did). - Honest cold-start: loadSession clears sessionIsTranscode up front, so a transcode->direct/remux switch no longer flashes a stale "Preparing stream..." during the session request. - Stats overlay a11y: the close (X) buttons in the stats, help and info overlays now read as "Close"/"Bezaras" (new plex.player.close key), not "Back"; dropped aria-live from the per-second stat grid so a screen reader is not flooded with telemetry. Tests: route-level transcode routing (transcode item -> HLS re-encode session, remux stays copy, direct opens none); start_session threading the preset/crf/max_height knobs + hw/preexec into the ffmpeg argv; UpNextCard countdown auto-advance, Play-now, Cancel and the S?-E? heading fallback. Also brings UpNextCard.tsx / plexStats.ts / plexStats.test.ts to prettier. |
||
|
peter
|
ce96032edb |
feat(plex): transcode resource discipline + admin knobs (E6 S1c)
The software-fallback re-encode now runs deprioritized and admin-configurable. The SW transcode ffmpeg child is niced (preexec_fn → os.nice, applied per-child because the Popen lives in the long-lived API process, which must NOT itself be niced — unlike the download worker's whole-process _deprioritize) so a CPU-bound re-encode can't starve request handling or playback; the VAAPI path needs none of it. New sysconfig knobs (plex group, rendered on the admin Configuration page, i18n HU/EN): plex_transcode_hw (auto/on/off, layered onto _want_hw), plex_transcode_ preset, plex_transcode_crf, plex_transcode_max_height (software-only downscale-to cap; the GPU path encodes at source resolution). plex_max_transcodes stays the concurrency cap (comment corrected — the fallback is shipped, not "later"). pytest extends the command-builder + _want_hw coverage (preset/crf/max_height plumbing, the on/off/auto policy). Browser-UAT'd: max_height=240 drops the live resolution to 330x240; the SW ffmpeg child runs at nice 10 (ppid=api process). |
||
|
peter
|
0658f9550b |
feat(plex): transcode playback mode — VAAPI/libx264 re-encode (E6 S1b)
Non-h264 items (playable="transcode": HEVC/VP9/AV1/MPEG-2/XviD…) now play by re-encoding the video to H.264 inside the existing HLS session, instead of the old HTTP 501. VAAPI hardware encode (h264_vaapi, full GPU decode→scale→encode pipeline) where the render node is present and the source is HW-decodable; libx264 software fallback otherwise (yuv420p 8-bit, -threads reserve-one-core). The re-encode mints a keyframe at every segment boundary (-force_key_frames) so the HLS muxer still cuts independent segments and seeking works; everything else — -copyts, the HLS tail, multi-audio renditions, the AAC audio path — is the same machinery as remux, so hls.js/subtitles/seeking are unchanged. The session route returns transcode/hw so the player can surface the active mode. pytest pins both command branches (software + VAAPI shape) and the HW-selection gate (render node present AND source HW-decodable). Browser-UAT'd on the XviD fixture (Winning London): 501→plays, real 528×384 H.264 frames, 0 dropped, seek lands clean; stats overlay Mode row reads "transcode". |
||
|
peter
|
12d9561ba1 |
fix(plex): warm up HLS buffer on session start + player loading/buffering spinner
The Plex player stalled ~2s a few seconds into every auto-advanced next episode. Root cause: each auto-advance spins up a cold ffmpeg HLS remux session, and for a fresh start (start_s==0) start_session returned immediately with zero buffer — hls.js drained the first segment, then rebuffered waiting for the still-warming ffmpeg to write the next one. Once ffmpeg gets ahead (stream-copy is faster than realtime) it never stalls again. - stream.py: before reporting a session ready, wait for the first _WARMUP_SEGMENTS (3) video segments to exist — the seg0 wait is now unconditional (was start_s>0 only) and warm-up runs for all starts. Best-effort and bounded: stop at the first segment not ready in time. - PlexPlayer.tsx: a loading/buffering spinner. The existing overlay only covered the INITIAL load (!ready); add a `buffering` state driven by the video's waiting/stalled/seeking (set) and playing/canplay/seeked/pause (clear) events so a mid-playback stall or in-flight seek reads as "loading" instead of a frozen frame, and surface a fatal loadError even once playback has started. Spinner uses the shared Loader2. Merges roadmap E6·item1 (spinner while loading/seeking) into this fix, since the server-side warm-up deliberately holds ready a beat longer. Tests: 3 new in test_plex_stream.py (warm-up waits for the first few segments; stops at the first missing one; no warm-up when seg0 never arrives). Gate: backend pytest 460 (Docker lane), FE tsc/lint/knip/ prettier + vitest 483. |
||
|
peter
|
b1832afd18 |
style: apply ruff format across backend + scripts (line-length 100)
Mechanical, whole-tree `ruff format` pass — no behaviour change. Recorded in .git-blame-ignore-revs (next commit) so git blame skips it. |
||
|
peter
|
5fe6d32d38 |
fix(plex): give the sweep's boot-time cutoff cross-clock slack
Found by running the round-8 fix in the real container instead of trusting the unit test: a directory created 2ms AFTER `_PROCESS_START` was stamped with an mtime 2ms BEFORE it, so the sweep deleted it as a leftover. The two sides come from different clocks — a file's mtime from the kernel's coarse clock (one timer tick of lag), `time.time()` from the fine one — and the unit test could not see it because it supplies an artificial cutoff. The window is small and lands exactly where it hurts: the sweep runs at boot, so the session at risk is one started in the first instants after startup — the case the guard exists for. Keep-side slack of 2s; the cost is that a leftover from the last seconds before a restart survives until the next one. Verified in the container: a backdated leftover is swept, a directory created by this run is kept, and a registered live session with an old mtime is kept — 1 of 3. Test pins the skew (mutation-checked: with the grace at 0 it goes red). |
||
|
peter
|
bdf7c03b59 |
fix(r5): address the eighth /code-review high round — 7 findings
The worst one was, again, the previous round's own fix. - Round 7 stopped awaiting the HLS sweep so startup could serve immediately — which also made it run CONCURRENTLY with playback, and `sweep_orphans` snapshotted the live-session set BEFORE listing the root and rmtree'd outside the lock. Since `start_session` reuses a directory path for the same (key, offset, tag), a stale name can become a live session at any moment. Two guards now: nothing whose mtime is newer than this process's start is ever deleted, and the live check + rmtree happen together under the lock, one directory at a time. - The sweep task is awaited after cancel, so shutdown can't leave it pending. - `_rate_limiter` used a truthiness test on its timestamp, so a `now()` of exactly 0.0 read as "never ran" — unreachable in production, but `now` exists to be injected and a test clock starting at 0 is the obvious choice. - The RSS total-outage error now carries the first underlying error: "all N channels failed" is also what ONE dead feed id looks like on a small instance, and a red card the user can't act on is its own kind of dishonest. - The WebKit fullscreen handling moved into `lib/fullscreen` and PlexPlayer uses it too — it had the same unprefixed-only reads in three places, including an Escape branch that would have closed the whole player instead of leaving fullscreen. - The `--max` CSS comment claimed `inset: 0` did the sizing; Tailwind's `w-full` is still on the element and wins. Comment now states the real mechanism. Verified in real Chrome (the pane can't composite or do native fullscreen): - F fills the 1920x889 viewport; the exit pill renders top-right on the letterbox band and overlaps NO YouTube control — the collision this round suspected does not reproduce in this embed (its own buttons sit bottom-left / bottom-right). - Four real ArrowDown presses while maximised: persisted volume 100 -> 80, with the level flash visible on screen. - A real click on the exit pill un-maximises and returns focus to the card. - lib/fullscreen driven from a real click: enter (innerHeight 889 -> 1024), the change listener fires exactly once per transition, exit returns to 889. Backend suite 150 -> 158; the new sweep tests are mutation-checked, and they read their cutoff from the filesystem rather than a clock (the container VM and the bind-mounted host disagree by enough to make a clock-based assertion flaky). |
||
|
peter
|
c0db80d333 |
fix(r5): address the third /code-review high round — 7 findings
- the sweep guard anchors the rating_key to digits: the round-2 widening had degenerated to "anything ending in _<digits>" (backup_2024, release_10) - RSS counts apply-side failures too, so a read-only database no longer reports a wholly failed poll as "ok — new=0, failed=0" - the player reads/writes its volume directly (readAccount/writeAccount): the persisted-object hook rendered nothing and cost a second write per settle - new tests: the sweep guard's accept/reject table, and run_ffmpeg's threading (drain, cancel while silent, stall watchdog, stderr tail on a nonzero exit) - the breaker sentinel is checked one way (isinstance) at all four sites - run_rss_poll is typed dict[str, int] All three new test groups were mutation-checked: reverting each fix turns them red (or, for the cancel/stall pair, hangs — which is the bug itself). |
||
|
peter
|
53ff61653d |
fix(r5): address the second /code-review high round — 7 findings
- the volume keys no longer defer to a scrollable card while in fullscreen
(mirrored into a ref: the keydown handler is bound once and state would be stale)
- the debounced volume flush writes straight to storage; persisting inside a
setState updater is not guaranteed to run for an unmounting component
- the HLS sweep also matches the original {key}_{start} directory naming, which
is what the oldest leftovers on disk actually use
- run_rss_poll returns {new, failed} so an egress outage stops rendering as
"ok — 0" in the scheduler card and the audit log; /api/sync/rss reports it too
- drop the unused poll_rss_channel (it silently gained a new exception)
- _Breaker.run is generic, so a call site keeps its item-id type
- test helper instead of the generator-throw idiom
|
||
|
peter
|
2a7f49c981 |
fix(r5): address the /code-review high round — 10 findings
- HLS sweep only deletes directories matching the session-name shape; the root is admin-configurable and may be a shared path - playlist unwrap picks the first entry that actually downloaded, not entries[0] - concat quoting keeps backslashes literal (ffmpeg treats them so inside '') - ArrowUp/Down defer while the focused element can still scroll that way - the staging fallback skips yt-dlp working files (.part-Frag/.ytdl/.temp/.fNNN) - the boot HLS sweep runs off the event loop - volume persistence is debounced (was a setItem per wheel notch) and a spin no longer loses notches to the iframe's lagging getVolume - _Breaker.run owns the whole open/call/record protocol; four call sites shrink - should_prune extracted and unit-tested, plus RSS error-vs-empty tests - the worker exits non-zero when the schema never lands |
||
|
peter
|
2f46947dcf |
fix(downloads): R5 S1 — downloads robustness
- pick the produced file safely: Path("") is Path("."), which passes .exists(),
so a missing yt-dlp filepath skipped the staging fallback and crashed in
with_name ("PosixPath('.') has an empty name") on non-YouTube sources
- unwrap a playlist/multi_video extraction to the entry that carries the media
- run_ffmpeg: drain stderr while it runs (a full 64KB pipe blocked ffmpeg
forever and hung the worker thread), 1s cancel polling, stall watchdog and a
bounded wait on exit
- escape apostrophes/backslashes in the ffmpeg concat list (pre-0.51.0 names)
- sweep orphaned HLS segment dirs at startup (nothing else ever reaped them)
|
||
|
peter
|
9347d52a54 |
fix(plex): honor the admin "max concurrent transcodes" setting (config drift)
stream.py enforced a hardcoded _MAX_SESSIONS = 4 and never read the DB-overridable plex_max_transcodes ConfigSpec, so the Configuration → Plex → "Max concurrent transcodes" knob was dead (same env-vs-DB drift class as the Downloads/Admin fixes). _enforce_cap(cap) now takes the cap from sysconfig.get_int(db, "plex_max_transcodes") (>=1 so playback can't be capped to zero). Bumped the config default 1→4 so the effective default matches the old hardcoded cap (no behavior change for non-overriders; the knob now works). |
||
|
peter
|
ecd2b8eb97 |
feat(plex): multi-rendition audio (client-side switch) + reliable resume-on-F5
- Multi-audio items now ship every audio track as an HLS rendition in one session (stream.py var_stream_map -> master.m3u8), so hls.js switches audio CLIENT-SIDE with no ffmpeg restart, same timeline, no gap/drift. /session gains ?multi=1 (forces HLS even for direct-playable multi-audio files); K is probed from the video variant seg. - Restore the selected audio track on AUDIO_TRACKS_UPDATED, not MANIFEST_PARSED (the renditions aren't parsed yet on MANIFEST_PARSED, so the set was dropped -> after F5 the UI showed the restored track but playback stayed on the default). - Resume position now survives F5: a pagehide keepalive beacon (plexProgressBeacon) saves the current position on reload/close/navigate, since React effect cleanup does not run on a full reload; seekTo writes the target into absRef immediately so a save right after a seek is accurate. |
||
|
peter
|
836706ba38 |
feat(plex): rework player timing (copyts) + player settings & personalization
Streaming / subtitle sync (the core fix): - stream.py: add -copyts so HLS segments carry the true absolute PTS, and measure the real keyframe start K from seg_0 (ffprobe) -> return it as the session start. Fixes the seconds-long subtitle lead caused by using the requested seek offset instead of the keyframe ffmpeg actually lands on with video stream-copy. - Add -noaccurate_seek so the re-encoded audio starts at the same keyframe as the video (was starting (X-K)s later -> seconds of silence after each seek/audio switch). - Compensate the fixed ~1.0s lag hls.js introduces for non-zero-start copyts streams, folded into the session start so the clock, seeking and the subtitle shift are all content-accurate. - /subtitle gains an offset param; _shift_vtt shifts absolute cues onto the session's zero-based clock and DROPS fully-past cue blocks (collapsing them to 0->0 made every past cue active at currentTime 0 on resume -> a pile-up until playback advanced). Audio: - /session + stream.py gain an audio A/V-sync offset (-itsoffset, full +/-, second input only when non-zero). Player settings & personalization (per-account, persisted -> survive F5): - storage.ts: useAccountPersistedObject (per-account JSON prefs blob). - PlexPlayer: volume/mute, audio+subtitle language (index-based match, fixes the F5 audio-revert), sync offsets, seek steps, subtitle style, auto-hide, play intent. - Hotkeys A (cycle audio) / S (cycle subtitle), mouse-wheel volume, Ctrl+arrow fine seek, per-user plain/fine seek-step + auto-hide toggle. - Subtitle appearance: size / colour / vertical position / background via ::cue + line. - UI: split into a Tracks quick-menu + a tabbed gear panel (Sync | Playback | Subtitle); both dismiss on outside-click; edge-aware control tooltips. - i18n en/hu/de for all new strings. |
||
|
peter
|
8853b73dda |
fix(plex): play subtitles as WebVTT tracks (external sidecar subs no longer crash)
Selecting a subtitle restarted the HLS session with `-map 0:s:{ord}`, assuming an
EMBEDDED stream. Films whose subs are external sidecar .srt files (Plex reports
them, but they aren't in the mkv) matched no stream; the master-playlist's
declared subtitle group then made ffmpeg fail → "Playback couldn't start".
Subtitles now go through a new GET /api/plex/subtitle/{rk}/{ord} → text/vtt
(external subs fetched from Plex via the stream key + SRT→VTT; embedded text subs
extracted with ffmpeg; image subs → 415), served as native <video><track> that
the browser overlays. So choosing/switching a subtitle is instant with NO session
restart, and stream.py drops all subtitle muxing (`-sn`, no master playlist).
Image-based subs (PGS/VobSub) are marked text=false and hidden in the picker.
Verified on prod's Nymphomaniac Vol. II: HU sidecar → 1693 WebVTT cues, no crash.
|
||
|
peter
|
dfed471e70 |
feat(plex): P2 subtitle + audio track selector (closes P2)
Backend: item detail returns audio_streams + subtitle_streams (from Plex Part.Stream
ordinals). The HLS session accepts audio/subtitle stream ordinals — the selected audio
is mapped (transcoded to AAC), a selected subtitle is muxed in as a WebVTT rendition
via a MASTER playlist (-master_pl_name + -var_stream_map sgroup:subs) so ffmpeg keeps
it in sync per-session. Selecting a track forces the HLS path (even for direct files).
Generic /stream/{rk}/hls/{filename} endpoint serves the master/media/vtt/ts artifacts.
Frontend: gear menu lists real audio + subtitle tracks; changing one restarts the
session at the current position (seek-restart mechanism). hls.js subtitle enabled on
SUBTITLE_TRACKS_UPDATED (not just MANIFEST_PARSED, else the VTT is never fetched).
Cues auto-nudged to line 88% (the full-height <video> clips a default bottom-edge cue).
Controls no longer auto-hide while the menu is open. plex.player.* i18n en/hu/de.
Verified in a real browser (2 Broke Girls S1E1, 2 audio + 2 subs): menu shows the real
tracks, enabling English subtitles renders them correctly positioned; backend master
playlist + vtt segments validated over HTTP.
|
||
|
peter
|
a3198ebc94 |
feat(plex): P2 streaming backend — direct 206 + seek-restart HLS remux
Validated end-to-end on a real remux file (h264+ac3 mkv). The riskiest part of the
Plex epic — on-the-fly playback from the local file — now proven.
- app/plex/stream.py: seek-restart HLS session model (one ffmpeg per item, restarted
at a seek offset via -ss). Video stream-COPY (I/O-bound, CPU-light — fine on the
GPU-less prod host) + audio→aac only when not already aac. Maps first video+audio,
drops subtitles (no VTT rendition clutter). Session cap + idle reaper.
- routes: POST /stream/{rk}/session?start= (direct→raw url, remux→HLS session,
transcode→501 P3), GET /stream/{rk}/file (206 range, direct), /index.m3u8, /seg_n.ts.
- KEY FINDINGS (hard-won): (1) ffmpeg -hls_playlist_type VOD does NOT write the
playlist mid-run (only at finalize) → use EVENT (append-only, appears immediately),
which suits the seek-restart model; (2) naive per-segment copy + keyframe-indexed
extraction are both unreliable (non-uniform/imprecise segments) — ffmpeg's own HLS
muxer is the only correct segmenter; (3) dev slowness was the Windows /downloads
bind-mount write + SMB read → PLEX_HLS_DIR env points scratch at fast container-local
/var/tmp on dev (prod keeps download_root, fast local disk).
|