diff --git a/CHANGELOG.md b/CHANGELOG.md index bca46e62..5a533e01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,40 @@ edit the release notes and re-run the generator. Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading). +## 0.84.0 — 2026-09-12 + +A bug-fix release, from a list you wrote while using Siftlode. Two of these were destroying your own data: a YouTube video rewound itself when you reopened it, and clearing a resume point on a half-watched Plex item told Plex nothing. Subtitles now appear in seconds instead of minutes, the Plex player's bitrate is a real measurement rather than a guess, and a channel you turn full history on for fills in front of you instead of looking frozen. + +### Features + +- Subtitles arrive in seconds. Picking an embedded subtitle track used to take about two and a half minutes on a large film, and switching from Hungarian to English cost the same again — because the cost was never the conversion, it was reading the whole file. It now reads a window around where you are, shows that first, and finishes the rest behind it; every text track rides the SAME read, so switching tracks is instant. On a 17 GB film: 154 s before, 8 s for the window now. Nothing blocks while it works, there is a “Preparing subtitles” pill instead of a frozen player, and a track that can never be shown finally says so. +- The Plex player's info panel tells you the truth. Bitrate is measured from the bytes actually arriving, not copied off the manifest — and it shows a dash until there is something to measure rather than a confident wrong number. It also lists the audio and the subtitle track you are on, which it never did. +- A new admin switch: loudness normalisation for Plex playback (Configuration → Plex). Off by default, because it makes a film markedly louder and narrows its dynamic range — that is a taste call, not a fix. On a 5.1 film it moved the measured loudness from −35.2 to −17.2 LUFS, and it does not touch A/V sync. +- Collections can be sorted. The collection page had no sort control at all and always ordered by release year — it now carries the same Sort control as the library grid, with the same options, and your choice survives a reload without re-sorting the library behind it. A collection opened from a shared link also shows its real name instead of a raw id. +- Remove a video from a playlist without leaving the player, and the player steps to whatever followed instead of closing. Above the move-to submenu there is now a one-click “move to the list you last moved to”. +- Live YouTube search results can be sorted — by views, date or length, over the results already on screen, so sorting rearranges what you are looking at instead of running a different search. +- The channel name in the player is a real link again: middle-click and Ctrl-click open it in a new tab, the way the YouTube icon beside it always did, while a plain click still takes the in-app route. + +### Fixes + +- A YouTube video no longer rewinds when you reopen it. Closing at 13:50 and coming back put you at 12:31 — the checkpoint reached the server but not the page's own cache, so the stale position won. It now goes forward: 811 → 816 → 825 s across reopens. +- “Clear resume point” works on a half-watched Plex item. The push to Plex was skipped for exactly the case you would use it on, so Plex kept the old offset and handed it straight back on the next import. Verified against the real server, including a full 44,885-item re-import. +- Turning full history on for a channel fills the page in front of you. The grid and the item count stayed frozen for the best part of a minute while the header counted up, because the request that starts the backfill returns before anything is stored. Measured: the count now moves 54 → 139 within about twelve seconds, no reload. +- Starting or seeking a Plex film is about eight seconds faster. Every session start and every seek waited out a dead eight-second timer for a file the transcoder never writes — on a film with more than one audio track, which is most of them. A multi-audio session now starts in 0.9 s cold and 0.7 s on a seek. +- A Plex session the server no longer has now recovers or says so, instead of spinning forever. A stream that kept being lost could restart itself indefinitely behind a spinner that never became an error. +- A Plex stream request for something that cannot exist answers immediately rather than holding the server for 20–30 seconds. Enough of those at once could stall the whole app. +- The Plex player no longer counts one viewing twice, and a failed “watched” mark no longer half-happens: it used to leave the item looking watched in the app while the server disagreed, or quietly mark it watched anyway a few seconds after telling you it had failed. +- Dragging a Plex playlist into a new order sticks. Leaving and coming straight back showed the old order, and the next drag saved that. +- A refused action says WHY, in the language you are reading. Several places showed a generic “couldn't do that” — or nothing at all — where the server had explained itself, including the Plex “Add to playlist” dialog, the collection editor and the upload dialog. The demo account gets its reason back too. +- Dialogs behave for the keyboard. Opening one puts the cursor where you would expect (the jump-to box opens on its search field, not its close button), closing one hands the focus back to what you opened it from, and a dialog that closes underneath another one no longer steals the focus out of the one still on screen. +- The regular micro-stutter in the Plex player is NOT Siftlode's, and this release does not pretend to fix it. Measured per frame: the player delivers every source frame with nothing dropped; the hold comes from a 23.976 fps film on a 120.0148 Hz display, and it reproduces identically outside Siftlode. Setting the display to 119.88 Hz removes it for every application, not just this one. + +### Internal + +- The round started from sixteen items written from real use, not from an audit. Eleven were fixed, one was measured and closed without a code change (the display cadence above), two are parked for a reproducible case, and one — playlist multi-select and bulk actions — was scoped out to its own sprint. The measurements are the point: item 8's “35 s” turned out to be one file's number, and the real law is bytes-ffmpeg-must-read divided by ~140 MB/s, which is why the fix is a window rather than a faster conversion. +- Seven deep-review rounds followed, each reviewing the previous round's own fix commits: 32 → 24 → 43 → 31 → 22 → 26 → 18 raised. Round 7 was run TARGETED rather than full — four reviewers on the six surfaces round 6 had changed, and adversarial verification spent only where a wrong fix could re-break the previous round's — which is seven agents instead of about thirty-six for two HIGH findings. Every regression test added in a fix round was mutation-checked: reverted, run, and confirmed red for the defect it is named after. Thirteen of round 7's mutations were, and two were instructive — one guard no test could drive, and one round-6 test whose mock modelled a server that accepts a write and then serves the pre-write state. +- What the eight-lane gate structurally cannot catch, and what this round taught about it: every real defect of the last three rounds was a wrong belief about an external system, not broken logic. ffmpeg substitutes the VARIANT before the sequence in its segment filenames, so a guard written the other way round was inert on every multi-audio file and a warm-up waited eight seconds for a name that is never written. A reclaim budget's window was anchored at the first failure rather than the last, so any failure cadence between one and two minutes reset it forever — invisible to two tests that spaced their cases at zero and ten minutes. jsdom performs no default Tab, so a focus test can be green with no focus trap installed at all. The conclusion carried forward is to measure the boundary when writing the code, not when reviewing it: one ffmpeg run costs pennies, a review round costs a session. + ## 0.83.0 — 2026-09-09 Siftlode now opens on a page that answers one question: what should I watch right now. You say how much time you have — ten minutes, half an hour, a whole evening — and it gives you one suggestion, from YouTube or from Plex, with a button that picks up where you left off. Under it sits a shelf of the things you always come back to, which you fill yourself. diff --git a/VERSION b/VERSION index 83dcd12c..09c49413 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.83.0 +0.84.0 diff --git a/backend/app/auth.py b/backend/app/auth.py index 8a6859bb..95d027ec 100644 --- a/backend/app/auth.py +++ b/backend/app/auth.py @@ -341,7 +341,11 @@ def require_human(user: User = Depends(current_user)) -> User: no token, so has_read_scope/has_write_scope are False); this makes the boundary explicit where there's no scope check to lean on.""" if user.is_demo: - raise HTTPException(status_code=403, detail="Not available in the demo account.") + # Structured, because `req` leaves 403 to the caller and `notifyActionError` only shows a + # STRUCTURED detail — a plain string here reached the demo user as their caller's generic + # "couldn't do that", with no hint that the demo account is the reason (round 6). Affordances + # this gate sits behind are not all UI-gated on `is_demo` (the comfort shelf is not). + raise HTTPException(status_code=403, detail={"code": "demo"}) return user diff --git a/backend/app/config.py b/backend/app/config.py index 62acd26e..ad8d13ea 100644 --- a/backend/app/config.py +++ b/backend/app/config.py @@ -271,6 +271,11 @@ class Settings(BaseSettings): # trades quality for size (lower = better + bigger). `max_height` caps the SOFTWARE transcode's output # height (0 = keep source) so a CPU re-encode of a tall source stays affordable — the GPU path always # encodes at source resolution. + # Loudness normalisation of the downmixed stereo audio on the HLS path (EBU R128 `loudnorm`). + # OFF by default: it makes a film ~11 dB louder and narrows its dynamic range, which is a taste + # call, not a fix. Only affects files that go through remux/transcode — a direct-played file is + # served untouched, but those are browser-native stereo anyway, not the 5.1 mixes this is for. + plex_audio_loudnorm: bool = False plex_transcode_preset: str = "veryfast" plex_transcode_crf: int = 20 plex_transcode_max_height: int = 1080 diff --git a/backend/app/main.py b/backend/app/main.py index 1386116c..792d9335 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -1,10 +1,13 @@ import asyncio import logging +import math import sys from contextlib import asynccontextmanager, suppress from pathlib import Path -from fastapi import FastAPI, HTTPException +from fastapi import FastAPI, HTTPException, Request +from fastapi.encoders import jsonable_encoder +from fastapi.exceptions import RequestValidationError # When started via uvicorn --log-config the "siftlode" logger is already configured # (see log_config.json). This block is a timestamped fallback for other entrypoints @@ -21,12 +24,14 @@ log = logging.getLogger("siftlode.app") from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import FileResponse, HTMLResponse, JSONResponse from fastapi.staticfiles import StaticFiles +from sqlalchemy.exc import DataError from starlette.middleware.sessions import SessionMiddleware from app import state from app.config import settings from app.db import SessionLocal from app.plex import stream as plex_stream +from app.plex import subtitles as plex_subtitles from app.routes import ( admin, audit as audit_routes, @@ -77,6 +82,7 @@ async def lifespan(app: FastAPI): # alone doesn't buy that; not waiting does. The sweep only touches directories no live session # owns, so racing it against early playback is safe. sweep = asyncio.create_task(_sweep_hls_orphans()) + reaper = asyncio.create_task(_reap_hls_sessions()) start_scheduler() try: yield @@ -86,10 +92,36 @@ async def lifespan(app: FastAPI): # pending ("Task was destroyed but it is pending!" — noise that reads like a bug during # triage). Cancelling does NOT interrupt the rmtree thread itself; this only makes our own # bookkeeping honest. - sweep.cancel() - with suppress(asyncio.CancelledError): - await sweep + for task in (sweep, reaper): + task.cancel() + with suppress(asyncio.CancelledError): + await task shutdown_scheduler() + # Kills a running subtitle extraction: its worker would otherwise hold interpreter exit for + # the rest of the read (see plex_subtitles.shutdown). + plex_subtitles.shutdown() + + +# How often idle HLS sessions are reaped (see plex/stream.py `reap_idle`). +_HLS_REAP_EVERY_S = 60.0 + + +async def _reap_hls_sessions() -> None: + """The periodic reaper plex/stream.py's docstring promises. `reap_idle` used to run only inside + the NEXT session request, so after the last playback of the day an abandoned session — its + ffmpeg still working through the film, its directory a whole segmented copy of it — lived until + somebody pressed play again or the process restarted. + + A FINISHED session is reaped on the short rule here, not kept for the full idle window: an open + player checkpoints every 10 s and that route touches the session, so a paused viewer is no longer + invisible to this pass (see `reap_idle`). Keeping every finished session for ten minutes instead + held one whole segmented copy of a film per item on the download volume (round 5).""" + while True: + await asyncio.sleep(_HLS_REAP_EVERY_S) + try: + await asyncio.to_thread(plex_stream.reap_idle) + except Exception: + log.exception("HLS session reap failed") async def _sweep_hls_orphans() -> None: @@ -108,6 +140,34 @@ async def _sweep_hls_orphans() -> None: app = FastAPI(title=settings.app_name, lifespan=lifespan) + +@app.exception_handler(RequestValidationError) +async def _validation_error(request: Request, exc: RequestValidationError) -> JSONResponse: + """FastAPI's own 422, in its own shape — except that it can be ENCODED. The JSON parser accepts + `NaN`/`Infinity`, a float field refuses them, and the refusal echoes the value back: the default + handler then failed to serialise its own error (JSON has no NaN) and the request became a 500 + with a traceback. A non-finite value is echoed as its name instead.""" + finite = {float: lambda x: x if math.isfinite(x) else str(x)} + return JSONResponse( + status_code=422, content={"detail": jsonable_encoder(exc.errors(), custom_encoder=finite)} + ) + + +@app.exception_handler(DataError) +async def _nul_in_text(request: Request, exc: DataError) -> JSONResponse: + """A NUL byte in a string the request sent is a 422, on every route at once. Postgres cannot hold + NUL in text, so the driver refuses to even send the query — and a `%00` in a query parameter or + an escaped NUL in a JSON body (a playlist's `contains=`, a reorder's key list, a collection + search) was a 500 with a traceback wherever it reached a comparison. Guarding each site did not + hold: round 3 guarded the by-rating_key lookup and missed the rest. Any other DataError is left + alone — re-raised, it stays the 500 it always was.""" + if "NUL (0x00)" not in str(exc.orig): + raise exc + return JSONResponse( + status_code=422, content={"detail": {"code": "invalid_input", "reason": "nul"}} + ) + + app.add_middleware( SessionMiddleware, secret_key=settings.secret_key, diff --git a/backend/app/models.py b/backend/app/models.py index 02a2895a..6ea02966 100644 --- a/backend/app/models.py +++ b/backend/app/models.py @@ -24,6 +24,11 @@ from app.db import Base from app.utils import canonical_email +# The resume-position columns (VideoState / PlexState.position_seconds) are int4. The checkpoint +# routes bound their bodies by this, so an out-of-range position is a 422 rather than a 500. +POSITION_MAX_S = 2**31 - 1 + + class TimestampMixin: """Adds a server-defaulted `created_at`. Models that index created_at or order it differently (QuotaEvent, Notification, Message) keep their own column on purpose.""" @@ -423,6 +428,7 @@ class VideoState(Base, UpdatedAtMixin): # status: a partially-watched video keeps status "new" but a non-zero position, so # it can render a progress bar and match the "in progress" feed filter. 0 = not # started / finished (trivially-early and near-finished positions are not stored). + # int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500). position_seconds: Mapped[int] = mapped_column( Integer, default=0, server_default="0", nullable=False ) diff --git a/backend/app/plex/client.py b/backend/app/plex/client.py index a8a92dde..d3313323 100644 --- a/backend/app/plex/client.py +++ b/backend/app/plex/client.py @@ -197,7 +197,12 @@ class PlexClient: ) def unscrobble(self, rating_key: str) -> None: - """Mark the item *unwatched* on the token's Plex account (clears viewCount).""" + """Mark the item *unwatched* on the token's Plex account. + + Clears the view count AND the resume point (`viewOffset`) — measured against the live server + on 2026-09-09, from both a watched+offset state and an offset-only one. This one call is the + whole un-mark; `/:/timeline?time=0` and `/:/progress?time=0` were both tried there and do + NOT clear an offset, so do not reach for them.""" self._write( "GET", f"/:/unscrobble?identifier={self._LIBRARY_IDENTIFIER}" diff --git a/backend/app/plex/lookups.py b/backend/app/plex/lookups.py index a66b83d3..ff430253 100644 --- a/backend/app/plex/lookups.py +++ b/backend/app/plex/lookups.py @@ -11,5 +11,11 @@ _M = TypeVar("_M") def by_rating_key(db: Session, model: type[_M], rating_key: str) -> _M | None: """The mirrored row for a Plex rating_key, or None. `model` is any of the rating_key-keyed Plex tables (PlexItem / PlexShow / PlexSeason / PlexCollection); the return type is that model, so call - sites keep their `Model | None` static type.""" + sites keep their `Model | None` static type. + + A key with a NUL in it (`%00` in a path segment) names no row — a 404, like any unknown key. + Postgres refuses NUL in text, so asking would be a DataError; the app-wide handler turns the + places that DO ask (lists and filters compared to rating_key) into a 422 (see main.py).""" + if "\x00" in str(rating_key): + return None return db.execute(select(model).where(model.rating_key == str(rating_key))).scalars().first() diff --git a/backend/app/plex/query.py b/backend/app/plex/query.py index c3fd6f7e..20abd28b 100644 --- a/backend/app/plex/query.py +++ b/backend/app/plex/query.py @@ -14,6 +14,22 @@ from app.models import PlexItem, PlexLibrary TS_CONFIG = "public.unaccent_simple" +# The ONE Plex sort vocabulary, shared by every surface that offers the Sort control (the browse grid +# and the collection info page) and mirrored by the frontend's `plexSort.ts`. It lives here rather than +# inside a router because the two surfaces order DIFFERENT things — browse orders a union subquery in +# SQL, a collection orders its already-loaded member list in Python — and a key that one of them cannot +# honour would put an option in the UI that silently does nothing. Tests pin both implementations to +# this tuple, so adding a key here fails the gate until every surface can order by it. +PLEX_SORT_KEYS: tuple[str, ...] = ( + "added", + "lastwatched", + "release", + "year", + "rating", + "duration", + "title", +) + # --- Language-aware column pickers (E5 S6) ---------------------------------- # # The HU-overlay twin (E5 S5) stores Hungarian metadata in `*_hu` columns on the SAME EN row. When diff --git a/backend/app/plex/stream.py b/backend/app/plex/stream.py index e3053176..e65d65e8 100644 --- a/backend/app/plex/stream.py +++ b/backend/app/plex/stream.py @@ -21,10 +21,12 @@ CPU-only prod host; only P3's full transcode is CPU-heavy. import logging import os import re +import secrets import shutil import subprocess import threading import time +from collections.abc import Sequence from pathlib import Path from sqlalchemy.orm import Session as DbSession @@ -55,6 +57,10 @@ _SEG_SECONDS = 6 # worse than the old zero-buffer behaviour). _WARMUP_SEGMENTS = 3 _WARMUP_SEG_TIMEOUT_S = 8.0 +# The HLS variant ffmpeg gives the VIDEO rendition of a multi-audio session. `-var_stream_map` names +# each audio rendition (`name:a0`, `a1`, …) but leaves the video one unnamed, so ffmpeg falls back to +# its index for that one. See `segment_name` for why this is a constant and not a comment. +_VIDEO_VARIANT = "0" _SESSION_IDLE_S = 600 # reap a session with no access for this long # A session directory is named `{rating_key}_{int(start_s)}_{tag}_{aoff:+.2f}` (see start_session); # before multi-audio it was just `{rating_key}_{int(start_s)}`. `sweep_orphans` deletes ONLY names @@ -94,11 +100,19 @@ class HlsSession: directory: Path, proc: subprocess.Popen, start_s: float, - entry: str, + variants: "Sequence[str]", transcode: bool = False, hw: bool = False, tonemap: bool = False, ): + # Unguessable per-session id, carried in the URLs the client is handed. Sessions are keyed + # by rating_key alone, so a seek-restart or a SECOND viewer of the same item replaces the + # registered one — and the old viewer's next request for `seg_12.ts`, resolved by rating_key, + # was answered 200 with the NEW session's twelfth segment, produced from a different offset + # under `-copyts`. Wrong bytes, not an error, so the client's reclaim never fired on it + # (deep-review round 5). A request naming a session that is no longer registered is a 404, + # which the player recovers from by restarting at the playhead. + self.id = secrets.token_urlsafe(9) self.key = key self.dir = directory self.proc = proc @@ -113,7 +127,13 @@ class HlsSession: # subtitle cue-shift both key off the true content position — no drift. Provisional = X until # the first segment is probed. self.media_start_s = start_s - self.entry = entry # the playlist filename hls.js should load (master.m3u8 with subs, else index.m3u8) + # The rendition names this session's ffmpeg was told to write — () for a single-audio + # session. Held on the session because it is what `artifact_may_arrive` needs to tell a name + # this session CAN produce from one belonging to the other argv branch (see it). + self.variants = tuple(variants) + # The playlist filename hls.js should load: the master of a multi-audio session, else the + # single index. Derived, not passed, so it cannot disagree with the argv. + self.entry = _MASTER_PL if self.variants else _INDEX_PL self.last_access = time.time() @@ -271,6 +291,30 @@ def _hwaccel_in_opts(hw: bool, tonemap: bool = False) -> list[str]: return ["-hwaccel", "vaapi", "-hwaccel_output_format", "vaapi", "-vaapi_device", _VAAPI_DEVICE] +# Audio is ALWAYS re-encoded on the HLS path: the source is typically AC3/DTS 5.1, which no browser +# plays, so it is downmixed to stereo AAC. `-ac 2` uses ffmpeg's standard downmix matrix; nothing is +# normalised, which is why a film mastered for a cinema arrives with whispered dialogue and deafening +# action (bug hunt 2026-09, item 14). +_AUDIO_ENC = ["-c:a", "aac", "-ac", "2", "-b:a", "192k"] +# Optional loudness normalisation (admin knob `plex_audio_loudnorm`, default OFF). EBU R128 single-pass +# `loudnorm` — two-pass is impossible here, since a seek-restart session must start emitting segments +# immediately and cannot analyse the file first. +# +# The downmix is FORCED FIRST (`aformat=channel_layouts=stereo`) rather than left to `-ac 2`: measuring +# 5.1 and then folding it down would normalise something other than what the viewer hears. `loudnorm` +# runs its own resampler at 192 kHz internally, so the chain resamples back to 48 kHz for the AAC +# encoder. Measured on a real 5.1 AC3 film (Westworld S1E6, 60 s from 15:00): I -27.6 → -16.6 LUFS, +# LRA 13.3 → 9.8 LU — the quiet/loud gap narrows, and everything gets ~11 dB louder, so expect to turn +# the volume down after switching this on. The filter's 3 s lookahead does NOT desync A/V: measured +# under this pipeline's own `-copyts`, the audio start PTS is identical with and without it. +_LOUDNORM_AF = "aformat=channel_layouts=stereo,loudnorm=I=-16:TP=-1.5:LRA=11,aresample=48000" + + +def _aenc_opts(loudnorm: bool) -> list[str]: + """Audio encoder args for one output, with the normalisation filter when the admin enabled it.""" + return [*_AUDIO_ENC, *(["-af", _LOUDNORM_AF] if loudnorm else [])] + + def _venc_opts( transcode: bool, hw: bool, @@ -446,6 +490,7 @@ def _ffmpeg_cmd( preset: str = "veryfast", crf: int = 20, max_height: int = 0, + loudnorm: bool = False, ) -> list[str]: # For a transcode session the video is re-encoded (VAAPI or libx264) instead of stream-copied; the # hwaccel decode options go before -i, the encoder args replace `-c:v copy` below. Everything else is @@ -469,11 +514,15 @@ def _ffmpeg_cmd( if audio_langs: # MULTI-RENDITION: map EVERY audio track as an alternate HLS rendition in one group, so hls.js # switches audio CLIENT-SIDE on the same timeline (no session restart, no drift) — the premium - # audio-switch path. Produces master.m3u8 + stream_%v.m3u8 (v0=video, v1..=audio) + seg_%v_%d.ts. + # audio-switch path. Produces master.m3u8 + one stream_%v.m3u8 and seg_%v_*.ts per variant, + # where %v is the video rendition's index ("0") and each audio rendition's `name:` (a0, a1, + # …). Those names are spelled by `segment_name` / `variant_playlist_name` below, which the + # guard and the warm-up read back — see them for the measurement behind the ordering. args += ["-map", "0:v:0"] for i in range(len(audio_langs)): args += ["-map", f"{ai}:a:{i}?"] - args += ["-sn", *venc, "-c:a", "aac", "-ac", "2", "-b:a", "192k"] + # One `-af` covers EVERY audio output of this file, so each rendition is normalised. + args += ["-sn", *venc, *_aenc_opts(loudnorm)] var = ["v:0,agroup:aud"] for i, lang in enumerate(audio_langs): entry = f"a:{i},agroup:aud,name:a{i}" @@ -482,23 +531,25 @@ def _ffmpeg_cmd( if i == 0: entry += ",default:yes" var.append(entry) + # The name patterns come from the vocabulary functions, with ffmpeg's own placeholders passed + # through them — so the producer and the parser cannot spell the layout differently (round 7). args += _HLS_TAIL + [ "-master_pl_name", - "master.m3u8", + _MASTER_PL, "-var_stream_map", " ".join(var), "-hls_segment_filename", - str(out_dir / "seg_%v_%d.ts"), - str(out_dir / "stream_%v.m3u8"), + str(out_dir / segment_name("%d", "%v")), + str(out_dir / variant_playlist_name("%v")), ] else: ao = audio_ord if audio_ord is not None else 0 args += ["-map", "0:v:0", "-map", f"{ai}:a:{ao}?", "-sn"] - args += [*venc, "-c:a", "aac", "-ac", "2", "-b:a", "192k"] + args += [*venc, *_aenc_opts(loudnorm)] args += _HLS_TAIL + [ "-hls_segment_filename", - str(out_dir / "seg_%d.ts"), - str(out_dir / "index.m3u8"), + str(out_dir / segment_name("%d")), + str(out_dir / _INDEX_PL), ] return args @@ -579,16 +630,26 @@ def start_session( src = paths.local_media_path(db, item.file_path) if src is None: return None + # Read once, here: the knob has to be identical for the directory name and the ffmpeg argv, and a + # session respawn (the HW->SW fallback) must not pick up a mid-playback config change. + loudnorm = sysconfig.get_bool(db, "plex_audio_loudnorm") key = item.rating_key start_s = max(0.0, float(start_s)) audio_langs = _probe_audio_langs(src) if multi else [] is_multi = len(audio_langs) > 1 # only worth a master playlist when there really are ≥2 tracks - tag = "multi" if is_multi else str(audio_ord) + # The rendition names the argv below will produce — one source for the argv, the warm-up's + # segment names, the entry playlist and the artifact guard. + variants = session_variants(audio_langs if is_multi else None) + # Loudness normalisation changes the AUDIO THE SEGMENTS CONTAIN, so it belongs in the directory + # name beside the other parameters that do (offset, audio track, A/V offset): a session left over + # from before the admin flipped the knob must not be mistaken for one that honours it. Appended to + # the tag with a DASH — `_SESSION_DIR_RE` treats the tag as one underscore-free field, and that + # regex is the only thing keeping the orphan sweep off a shared scratch path (see its comment). + tag = ("multi" if is_multi else str(audio_ord)) + ("-ln" if loudnorm else "") directory = _HLS_ROOT / f"{key}_{int(start_s)}_{tag}_{aoff:+.2f}" def _seg(i: int) -> Path: - # multi-audio: the video rendition is variant v0 → seg_{i}_0.ts; single-audio → seg_{i}.ts - return directory / (f"seg_{i}_0.ts" if is_multi else f"seg_{i}.ts") + return directory / segment_name(i, _VIDEO_VARIANT if variants else None) def _spawn( hw: bool, tonemap: bool = False, *, replace_only: "HlsSession | None" = None @@ -632,6 +693,7 @@ def start_session( preset=preset, crf=crf, max_height=max_height, + loudnorm=loudnorm, ), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, @@ -642,7 +704,7 @@ def start_session( directory, proc, start_s, - "master.m3u8" if is_multi else "index.m3u8", + variants, transcode=transcode, hw=hw, tonemap=tonemap, @@ -714,6 +776,101 @@ def current_session(key: str) -> HlsSession | None: return s +# The names ffmpeg writes into a session directory, in ONE place — and "one place" means the argv +# BUILDS its patterns from these same functions (`segment_name("%d", "%v")`), not that a comment +# says so. Getting this wrong is invisible in both directions: a name we failed to anticipate used +# to fall back to "wait and see" (round 6's HIGH), and now that the rule is positive it falls back +# to a 404 instead — so the producer and the parser have to be literally the same spelling. +# +# MEASURED (ffmpeg 7.1.5 in the API container — the binary that actually writes these files — and +# re-checked on 8.1.1): `-hls_segment_filename seg_%v_%d.ts` substitutes `%v` with the VARIANT and +# `%d` with the SEQUENCE — variant FIRST. A dual-audio session writes seg_0_0.ts… (video, the +# unnamed variant) plus seg_a0_0.ts… / seg_a1_0.ts… (the `name:aN` audio renditions), and the +# playlists beside them are master.m3u8 + stream_0/a0/a1.m3u8. Single-audio uses `seg_%d.ts` + +# index.m3u8, with no variant field at all. +# +# The two argv branches are DISJOINT vocabularies, which is why the guard below is keyed to the +# SESSION and not to HLS in general: a single-audio session can never write `master.m3u8`, +# `stream_*.m3u8` or any `seg__.ts`, and a multi-audio one can never write `index.m3u8` +# or `seg_.ts`. Accepting the union meant a hopeless name of the other shape was waited out for +# the full playlist/segment timeout — the threadpool parking this guard exists to end (round 7). +_MASTER_PL = "master.m3u8" +_INDEX_PL = "index.m3u8" +_SEG_NAME_RE = re.compile(r"^seg_(?:([0-9a-z]{1,4})_)?(\d{1,9})\.ts$") + + +def session_variants(audio_langs: "Sequence[str | None] | None") -> tuple[str, ...]: + """The `-var_stream_map` rendition names a session will write, or () for a single-audio one. + + The video rendition is left unnamed in the map, so ffmpeg falls back to its index + (`_VIDEO_VARIANT`); each audio rendition is named `aN` in track order.""" + if not audio_langs: + return () + return (_VIDEO_VARIANT, *(f"a{i}" for i in range(len(audio_langs)))) + + +def segment_name(seq: "int | str", variant: "str | None" = None) -> str: + """The file ffmpeg writes for the `seq`-th segment of `variant`. + + `variant=None` is a single-audio session, whose argv carries no `%v` field at all. `seq` and + `variant` are typed loosely because `_ffmpeg_cmd` builds its own `-hls_segment_filename` pattern + through this function, passing ffmpeg's `"%d"` / `"%v"` placeholders — that is what keeps the + producer and the parser from drifting, rather than a comment asking the next editor not to.""" + return f"seg_{variant}_{seq}.ts" if variant else f"seg_{seq}.ts" + + +def variant_playlist_name(variant: str) -> str: + """The sub-playlist ffmpeg writes for one rendition of a multi-audio session.""" + return f"stream_{variant}.m3u8" + + +def playlist_names(variants: "Sequence[str]") -> frozenset[str]: + """Every playlist name this session's argv can write — the master plus one per rendition, or + just `index.m3u8` for a single-audio session.""" + if not variants: + return frozenset({_INDEX_PL}) + return frozenset({_MASTER_PL, *(variant_playlist_name(v) for v in variants)}) + + +def artifact_may_arrive(s: "HlsSession", filename: str) -> bool: + """Whether waiting for `filename` can possibly end in a file OF THIS SESSION. + + Positive rule, not a blocklist: a session produces segments and playlists and NOTHING else, so + any other name is refused at once. ffmpeg also produces segments strictly in order, so a segment + whose PREDECESSOR does not exist yet is not the next one — it is beyond the generated region, + and no amount of waiting will conjure it. Playlists are the one thing that must still be waited + for: they are rewritten continuously and have no ordering to read. + + Keyed to the SESSION's own vocabulary, not to HLS's: the two argv branches are disjoint, so a + name belonging to the other shape is as hopeless here as a random string. Reading those as "an + HLS name, so wait and see" left the whole class reachable — measured through the real route + against a live single-audio session, `master.m3u8` and `stream_q7z.m3u8` were 404s after 30.0 s + and `seg_a0_0.ts` / `seg_zzzz_0.ts` after 20.1 s each (deep-review round 7). + + Saying no at once matters because the route that waits is a SYNC one: it holds a thread off the + ~40-slot pool, and a pooled DB connection with it, for the whole timeout. An ordinary seek past + the produced region was enough to park one for 20 s, and forty concurrent requests for a name + that will never exist parked the whole API (deep-review round 5). Round 5's guard read only + single-audio segment names, so both halves of that were still fully reachable: every multi-audio + name fell through, and so did any `.m3u8`/`.vtt` name at all.""" + if filename in playlist_names(s.variants): + return True + m = _SEG_NAME_RE.match(filename) + if m is None: + return False + variant, seq = m.group(1), int(m.group(2)) + # The variant field must be one THIS session writes: present and known for a multi-audio + # session, absent for a single-audio one. + if variant not in (s.variants or (None,)): + return False + if seq == 0: + return True + try: + return (s.dir / segment_name(seq - 1, variant)).exists() + except OSError: + return False + + def wait_for(path: Path, timeout: float = 20.0, proc: subprocess.Popen | None = None) -> bool: """Wait until a session file (playlist / segment) exists and is non-empty. Segments are produced ~faster than realtime, so a segment just ahead of playback appears quickly; a segment @@ -785,14 +942,47 @@ def sweep_orphans(older_than: float | None = None) -> int: return dropped -def reap_idle() -> int: +# How long a session whose ffmpeg has FINISHED may go unfetched before a new session request reaps +# it (see reap_idle). +_FINISHED_IDLE_S = 30 + + +def reap_idle(finished_idle_s: float = _FINISHED_IDLE_S) -> int: + """Kill sessions nobody has said they still want for `_SESSION_IDLE_S` — or, once their ffmpeg + has finished, for `finished_idle_s`. + + A finished session is also what a PAUSED viewer has: a remux runs far faster than realtime, so + ffmpeg is done a minute or two in and writes ENDLIST; hls.js then stops asking for the playlist, + and with its forward buffer full a paused player fetches nothing at all. With the 30 s rule that + killed a one-minute pause, and playback died ~30 s after resuming (deep-review round 4). + + What makes the short rule safe again is that fetching is no longer the only signal: a player + parked MID-ITEM checkpoints its position every 10 s, and that route calls `touch_session`, so a + viewer who is merely paused says so far more often than this rule asks (round 5). The heartbeat + stops once the item is finished or before a position is known (`usePlexProgress.flush`) — both + states where reaping is correct, and where the player's 404 → reclaim recovers a viewer who + scrubs back in anyway. Do not restate this as "every 10 s whether or not it is playing": round 6 + found that claim in two comments and not in the code. Round 4's blanket + `reap_abandoned` bought the same safety by keeping every finished session for ten minutes — each + one a whole segmented copy of the film on a volume shared with the download center.""" now = time.time() dropped = 0 with _lock: for key, s in list(_sessions.items()): done = s.proc.poll() is not None - if now - s.last_access > _SESSION_IDLE_S or (done and now - s.last_access > 30): + idle = now - s.last_access + if idle > _SESSION_IDLE_S or (done and idle > finished_idle_s): _kill(s) _sessions.pop(key, None) dropped += 1 return dropped + + +def touch_session(key: str) -> None: + """Say a viewer is still there for `key`, without fetching anything. `last_access` used to move + only on an HLS file fetch, which a paused player does not do — so the only way to keep a paused + viewer's session alive was to keep EVERY abandoned one just as long.""" + with _lock: + s = _sessions.get(key) + if s is not None: + s.last_access = time.time() diff --git a/backend/app/plex/subtitles.py b/backend/app/plex/subtitles.py index 770c10b0..132f476e 100644 --- a/backend/app/plex/subtitles.py +++ b/backend/app/plex/subtitles.py @@ -1,15 +1,57 @@ -"""WebVTT conversion, HLS-clock time-shift, and on-disk cache for Plex subtitle streams. +"""WebVTT conversion, HLS-clock time-shift, and the on-disk cache + background extractor for Plex +subtitle streams. Text subtitles (embedded or external) are turned into WebVTT so the browser can render them; bitmap subtitles (Blu-ray/DVD) can't be (would need burn-in). Converted VTTs are cached on the shared Plex on-disk cache root (``cache.CACHE_DIR``). + +WHY THERE IS A BACKGROUND EXTRACTOR HERE — measured on prod, 2026-09-10, against a 17.06 GB 1080p +MKV with two embedded SubRip tracks on the 81 TB array: + +=========================================================== ======== + full pass, ONE track (what this module used to do) 154 s + full pass, BOTH text tracks in a SINGLE ffmpeg run 111 s + `-c:s copy` instead of converting to WebVTT 143 s + from 1:00:00 to the end of the file 66 s + a 630 s WINDOW at 1:00:00, both tracks 8 s +=========================================================== ======== + +The conversion is not the cost (dropping it saved 7%): ffmpeg has to stream every byte of the +container between its seek point and its stop point, and the array delivers ~140 MB/s — so the time +is (bytes read) / 140 MB/s, and the ONLY lever is reading less. Hence: + +* extract a WINDOW around the playhead first — the viewer sees subtitles in seconds — then complete + the whole track in the background so every later seek is instant and the next session is free; +* extract EVERY embedded text track in ONE pass, because the read is shared: picking Hungarian used + to cost a full read and then switching to English cost another one (the "hektikus" report); +* never do any of it inside a request (it froze the with no feedback AND held an API worker + thread for the whole read). + +`-copyts` CANNOT be used for a window: it makes ffmpeg ignore `-t`/`-to` (measured — a [0, 600 s] +window still took the full 145 s and emitted the whole track), so windows are extracted on a +zero-based clock and shifted back to absolute time here with `shift_vtt`. """ +import json +import logging import re import subprocess +import tempfile +import threading +import time +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path +from sqlalchemy.orm import Session + +from app.db import SessionLocal +from app.models import PlexItem +from app.plex import paths as plex_paths from app.plex.cache import CACHE_DIR -from app.plex.client import PlexError +from app.plex.client import PlexClient, PlexError +from app.plex.lookups import by_rating_key + +log = logging.getLogger("siftlode.plex") # Image-based subtitle codecs (Blu-ray/DVD) — can't be turned into text WebVTT (would need burn-in). BITMAP_SUBS = { @@ -31,6 +73,9 @@ def srt_to_vtt(text: str) -> str: def _vtt_ts_to_s(ts: str) -> float: + # Also reads ffmpeg's spelling of a NEGATIVE time (a windowed pass, see `_ffmpeg_extract`): its + # webvtt muxer splits the milliseconds with C's truncating division, so every field carries the + # sign — -5.005 s is `00:-5.-05`, -65.3 s is `-1:-5.-300` — and summing the signed fields is exact. parts = ts.split(":") h = "0" if len(parts) == 2 else parts[0] m, rest = parts[-2], parts[-1] @@ -39,16 +84,12 @@ def _vtt_ts_to_s(ts: str) -> float: def _vtt_s_to_ts(x: float) -> str: - x = max(0.0, x) - h = int(x // 3600) - x -= h * 3600 - m = int(x // 60) - x -= m * 60 - s = int(x) - ms = int(round((x - s) * 1000)) - if ms >= 1000: - s += 1 - ms -= 1000 + # Rounded to whole milliseconds FIRST, then split: rounding the fraction last carried a rounded-up + # millisecond into the seconds and no further, so 59.9996 came out as `00:00:60.000` — a timestamp + # the WebVTT parser rejects, silently dropping that cue. + h, rest = divmod(int(round(max(0.0, x) * 1000)), 3_600_000) + m, rest = divmod(rest, 60_000) + s, ms = divmod(rest, 1000) return f"{h:02d}:{m:02d}:{s:02d}.{ms:03d}" @@ -109,7 +150,7 @@ def ffmpeg_to_vtt(src: str, sub_ord: int | None) -> str: "webvtt", "pipe:1", ] - r = subprocess.run(cmd, capture_output=True) + r = subprocess.run(cmd, capture_output=True, timeout=EXTRACT_TIMEOUT_S) if r.returncode != 0 or not r.stdout: raise PlexError( (r.stderr or b"").decode("utf-8", "replace")[:200] or "ffmpeg subtitle convert failed" @@ -125,11 +166,686 @@ def cache_read(key: str) -> str | None: return None -def cache_write(key: str, vtt: str) -> None: +def cache_write(key: str, vtt: str) -> bool: + """Atomically replace the cached VTT; False when it could not be written (`publish` must then + not claim the region it was carrying).""" try: CACHE_DIR.mkdir(parents=True, exist_ok=True) tmp = CACHE_DIR / f"{key}.vtt.tmp" tmp.write_text(vtt, encoding="utf-8") tmp.replace(CACHE_DIR / f"{key}.vtt") # atomic publish + return True + except OSError: + return False + + +# --- cache identity ----------------------------------------------------------------------------- + +# The cached VTT holds ABSOLUTE cue times (per rating_key + subtitle ordinal); the per-session shift +# is applied on the way out, so a resume/seek to any offset reuses the same file. Beside it, a small +# JSON sidecar records WHICH absolute-time ranges the file actually covers — a windowed extraction +# publishes a usable but incomplete track, and "incomplete" must never be mistaken for "all there is". +_FULL_END_S = 1_000_000_000.0 # stand-in for "to the end of the file" (JSON can't hold infinity) + + +def cache_key(rating_key: str, sub_ord: int) -> str: + return f"sub_{rating_key}_{sub_ord}" + + +def _vtt_path(key: str) -> Path: + return CACHE_DIR / f"{key}.vtt" + + +def _meta_path(key: str) -> Path: + return CACHE_DIR / f"{key}.json" + + +def read_meta(key: str) -> tuple[list[list[float]], bool]: + """Covered absolute-time ranges + whether the track is fully extracted. A VTT with no sidecar + (written by an older build, which only ever cached complete tracks) counts as complete.""" + try: + raw = json.loads(_meta_path(key).read_text(encoding="utf-8")) + return [[float(a), float(b)] for a, b in raw.get("ranges", [])], bool(raw.get("complete")) + except (OSError, ValueError, TypeError): + return ([[0.0, _FULL_END_S]], True) if _exists(_vtt_path(key)) else ([], False) + + +def _exists(p: Path) -> bool: + """`Path.exists` that answers False for a path the OS refuses outright: pathlib swallows only + ENOENT-like errors, so an over-long name (ENAMETOOLONG, from an absurd rating_key or ordinal) + escaped as a 500 from the status route.""" + try: + return p.exists() + except (OSError, ValueError): + return False + + +def _write_meta(key: str, ranges: list[list[float]], complete: bool) -> None: + try: + CACHE_DIR.mkdir(parents=True, exist_ok=True) + tmp = CACHE_DIR / f"{key}.json.tmp" + tmp.write_text(json.dumps({"ranges": ranges, "complete": complete}), encoding="utf-8") + tmp.replace(_meta_path(key)) except OSError: pass + + +def generation(key: str) -> int: + """Monotonic id of the cached file's current content. It goes into the URL so that when a + window is superseded by a wider one the browser refetches instead of keeping the short VTT.""" + try: + return _vtt_path(key).stat().st_mtime_ns + except (OSError, ValueError): # ValueError: a NUL in the key — `stat` refuses the path outright + return 0 + + +def merge_ranges(ranges: list[list[float]]) -> list[list[float]]: + """Sort + coalesce touching/overlapping covered ranges.""" + out: list[list[float]] = [] + for a, b in sorted((r for r in ranges if r[1] > r[0]), key=lambda r: r[0]): + if out and a <= out[-1][1]: + out[-1][1] = max(out[-1][1], b) + else: + out.append([a, b]) + return out + + +def covers(ranges: list[list[float]], at: float) -> bool: + return any(a <= at <= b for a, b in ranges) + + +def merge_vtt(*sources: str) -> str: + """Union of the cue blocks of several absolute-time VTTs, ordered by start time. + + Two extraction passes over the same file emit identical cues for the region they share, so they + are de-duplicated — but NOT on the raw timing text. The two passes do not spell a timestamp the + same way: a window comes back through `shift_vtt`, which always writes `HH:MM:SS.mmm`, while a + full pass is ffmpeg's own output, and its webvtt muxer drops the hour below one hour + (`10:00.123`). Keyed on the text, every cue of a window in the first hour survived twice and the + viewer saw each line stacked on itself. So each timing line is parsed and re-emitted in one + canonical form, and the key is that canonical timing PLUS the cue text: two different cues that + happen to share a time (two speakers, a sign over dialogue) are both kept.""" + seen: set[str] = set() + cues: list[tuple[float, str]] = [] + for src in sources: + for block in re.split(r"\n[ \t]*\n", src or ""): + canon = _canonical_cue(block.strip()) + if canon is None: + continue # header / NOTE / STYLE — the merged file gets its own single header + start, key, text = canon + if key in seen: + continue + seen.add(key) + cues.append((start, text)) + cues.sort(key=lambda c: c[0]) + return "WEBVTT\n\n" + "\n\n".join(b for _, b in cues) + "\n" + + +def _canonical_cue(block: str) -> tuple[float, str, str] | None: + """(start seconds, identity, the block with its timing line canonicalised) — or None for a block + that is not a cue. The identity leaves out anything before the timing line (an SRT-style cue + number), which is a label, not content.""" + lines = block.split("\n") + for i, ln in enumerate(lines): + if "-->" not in ln: + continue + left, _, right = ln.partition("-->") + rparts = right.strip().split(None, 1) + settings = (" " + rparts[1]) if len(rparts) > 1 else "" + try: + st = _vtt_ts_to_s(left.strip()) + en = _vtt_ts_to_s(rparts[0]) + except (ValueError, IndexError): + return None + lines[i] = f"{_vtt_s_to_ts(st)} --> {_vtt_s_to_ts(en)}{settings}" + return st, "\n".join(lines[i:]), "\n".join(lines) + return None + + +def _last_cue_end(vtt: str) -> float: + """Where the last cue of a VTT ends — how far a pass that died early provably got.""" + end = 0.0 + for ln in vtt.split("\n"): + if "-->" not in ln: + continue + try: + end = max(end, _vtt_ts_to_s(ln.partition("-->")[2].strip().split(None, 1)[0])) + except (ValueError, IndexError): + continue + return end + + +def publish(key: str, vtt: str, covered: tuple[float, float], complete: bool) -> bool: + """Merge a freshly extracted region into the cached track and widen its covered ranges. False + when the region did not land (the cache is unwritable, or the VTT there cannot be read) — the + caller must then record a FAILED pass: returning quietly let the next poll schedule the same read + again, a whole-container pass every time, while the player said "preparing" for good. + + A read-merge-write, so it runs under `_PUBLISH_LOCK`: two jobs publish into the same keys (a + window and the full pass, and every embedded track of the file rides each of them), and without + the lock the later writer drops whatever the earlier one merged — permanently, once `complete` is + set. It is also the only writer of the cache files, which is what keeps their fixed `.tmp` + staging names safe.""" + with _PUBLISH_LOCK: + old = cache_read(key) + if old is None and _vtt_path(key).exists(): + # There, but unreadable (`cache_read` swallows the OSError). Merging into "" would replace + # a whole track with this one region while its sidecar still claimed all of it. + log.warning("subtitle cache %s unreadable; not publishing into it", key) + return False + old_ranges, was_complete = read_meta(key) + if old is None: + # A sidecar with no VTT behind it claims cues that are not there, so it is not merged + # into. And a VTT with no sidecar reads as a COMPLETE track (see read_meta — how older + # builds cached), while a publish is two renames: so before a key's first VTT lands, its + # sidecar is made to claim nothing. Dying between the two then leaves a window that is + # extracted again, never one taken for the whole track for good. + old_ranges, was_complete = [], False + _write_meta(key, [], False) + if not cache_write(key, merge_vtt(old or "", vtt)): + log.warning("subtitle cache %s could not be written", key) + return False # the region never landed, so the sidecar must not claim it + _write_meta(key, merge_ranges(old_ranges + [list(covered)]), complete or was_complete) + return True + + +# --- background extraction ---------------------------------------------------------------------- + +# How much of the track to extract before the viewer sees anything. WINDOW_PRE_S starts a little +# before the playhead so a cue that is already on screen is not missed; WINDOW_LEN_S is the span +# published as the first, provisional VTT (~4 s of work on the measured file). It only has to hold +# until the full pass lands behind it — 111 s for that file — so it is deliberately much longer than +# any realistic full pass. +WINDOW_PRE_S = 30.0 +WINDOW_LEN_S = 300.0 +# …but it does not always land: a full pass can be backed off (FULL_RETRY_S) or simply slower than +# the window lasts (a file past ~40 GB, or several passes sharing the array). A window was only asked +# for once the playhead had LEFT coverage, so every edge unmounted the track for a poll plus an +# extraction. Within this distance of a covered stretch's end, the next window is read ahead. +LOOKAHEAD_S = 60.0 +# Ceiling on one ffmpeg run so a wedged extraction can't hold a pool thread forever (the old +# in-request extraction had no timeout at all). +EXTRACT_TIMEOUT_S = 900.0 +# How long a timed-out ffmpeg gets, after a TERM, to close its outputs before it is killed. +_TERM_GRACE_S = 10.0 +# A failed track (or window) is remembered briefly so a 1 s status poll doesn't respawn ffmpeg every +# second, and forgotten again soon after so fixing the file (or Plex coming back) doesn't need a +# restart. +FAIL_TTL_S = 60.0 +# …except a FULL pass that failed. That is a read of the whole container (minutes), a failure that +# reproduces does so every time, and the viewer is typically inside what it did publish — so the +# short memo re-armed a whole-file read every minute for as long as they watched. Windows around the +# playhead still run meanwhile; only the full pass waits this out. +FULL_RETRY_S = 3600.0 +# The furthest position the status poll accepts. No film is this long, and an absurd one used to +# reach ffmpeg's `-ss` (which rejects anything past its int64 microseconds). +MAX_POSITION_S = 100_000.0 +# Windows and full passes run on SEPARATE pools, so a window never waits behind a full pass. They +# shared one FIFO pool of three whose comment promised "room for a window" that nothing enforced: +# stepping through three episodes with subtitles on left three full passes (minutes each) holding every +# worker, and a seek window on the third queued behind them. Two of each: bounded, so a burst can't +# thrash the array. +_WIN_POOL = ThreadPoolExecutor(max_workers=2, thread_name_prefix="plex-subs-win") +_FULL_POOL = ThreadPoolExecutor(max_workers=2, thread_name_prefix="plex-subs-full") +# Bounds on what may wait for those pools — separate too, so held full-pass slots (minutes each) +# cannot refuse a window (seconds). The queues are unbounded and every job is a read of the library: +# a viewer scrubbing through a film queued a window per stop (the one where they stopped ran LAST), +# and a script could queue thousands. Past a bound nothing is queued — the next poll asks again, by +# which time something has finished. +_MAX_WINDOWS_INFLIGHT = 8 +_MAX_FULL_INFLIGHT = 6 +_MAX_TRACK_WINDOWS = 2 + +_LOCK = threading.Lock() +# Queued or running jobs → the absolute span a window job reads (None for a full pass). +_INFLIGHT: dict[tuple, tuple[float, float] | None] = {} +# Track-level failures: a track that cannot be served at all (no item, a bitmap track, Plex down…). +_FAILURES: dict[str, tuple[str, float]] = {} +# rating_key → when its full pass last failed (see FULL_RETRY_S). +_FULL_FAILED: dict[str, float] = {} +# cache key → the spans windows failed to read, with when. A failed window says nothing about the rest +# of the track — the full pass may be about to deliver it — so it is remembered as a SPAN: it must be +# neither respawned every poll nor taken as a failure of positions it never tried. (It used to set the +# track-level memo, which told every viewer at any uncovered position "failed" — a final answer that +# ends the player's poll — including one planted by a request for an absurd position.) +_WIN_FAILED: dict[str, list[tuple[float, float, float]]] = {} +_PUBLISH_LOCK = threading.Lock() +# The ffmpeg children currently running, so a shutdown can kill them (see `shutdown`), and whether it +# has: a worker that reaches its Popen after the kill sweep must not start a read. +_PROCS: set[subprocess.Popen] = set() +_STOPPING = False + + +class SubtitleUnavailable(Exception): + """A track that cannot be served as text, carrying the stable code the route maps to a status.""" + + def __init__(self, code: str): + super().__init__(code) + self.code = code + + +# code -> HTTP status. An extraction/upstream failure is 422 with a structured detail (502 is +# reserved for the raw byte proxies); the rest are genuine "no such thing" / "not text". +UNAVAILABLE_STATUS = { + "no_item": 404, + "no_track": 404, + "no_file": 404, + "not_fetchable": 404, + "bitmap": 415, + "extract_failed": 422, + "plex_error": 422, +} +# The codes that are a FINAL answer about the track: it can never be served as text, so the player +# stops asking. The others (`plex_error`, `extract_failed`) are about now or about one stretch — a +# memo that lapses, a window that failed where the rest is fine — and answered as `unavailable`: said, +# but asked again. Given as `failed`, one Plex hiccup ended the subtitles for the rest of the session, +# at cached positions too, and after the full pass had completed behind it. +FINAL_CODES = frozenset({"no_item", "no_track", "no_file", "not_fetchable", "bitmap"}) + + +def reset_state() -> None: + """Drop the in-flight/failure registries (tests; nothing in the app needs it).""" + global _STOPPING + with _LOCK: + _INFLIGHT.clear() + _FAILURES.clear() + _FULL_FAILED.clear() + _WIN_FAILED.clear() + _STOPPING = False + + +def shutdown() -> None: + """Stop extracting, for the app's shutdown. The pool's workers are non-daemon threads and Python + joins them at interpreter exit — so a worker still inside a full pass (minutes on a big file, up + to EXTRACT_TIMEOUT_S) held the process open after uvicorn had stopped, until the container runtime + SIGKILLed it. Queued jobs are dropped and running ffmpeg children are killed, which returns their + workers at once — and a worker still BEFORE its ffmpeg (the Plex lookup, the DB) finds the flag + and never starts one. Nothing that matters is lost: an interrupted publish cannot pass for a whole + track (see `publish`), and an interrupted track is simply extracted again on the next poll.""" + global _STOPPING + with _LOCK: + _STOPPING = True + procs = list(_PROCS) + for pool in (_WIN_POOL, _FULL_POOL): + pool.shutdown(wait=False, cancel_futures=True) + for p in procs: + p.kill() + + +def _sweep_memos() -> None: + """(Caller holds _LOCK.) Drop every lapsed failure memo. They were only ever pruned when the SAME + key was read again, and the status route takes any rating_key/ordinal — so a script polling fresh + keys grew the API process without bound. Swept on every write, they hold at most a TTL's worth.""" + now = time.monotonic() + for k in [k for k, (_, t) in _FAILURES.items() if now - t >= FAIL_TTL_S]: + del _FAILURES[k] + for k in [k for k, t in _FULL_FAILED.items() if now - t >= FULL_RETRY_S]: + del _FULL_FAILED[k] + for k in list(_WIN_FAILED): + _live_window_failures(k) + + +def _fail(key: str, code: str) -> None: + with _LOCK: + _sweep_memos() + _FAILURES[key] = (code, time.monotonic()) + + +def _recent_failure(key: str) -> str | None: + with _LOCK: + hit = _FAILURES.get(key) + if hit and time.monotonic() - hit[1] < FAIL_TTL_S: + return hit[0] + _FAILURES.pop(key, None) + return None + + +def _full_failed(rating_key: str) -> bool: + with _LOCK: + when = _FULL_FAILED.get(rating_key) + if when is not None and time.monotonic() - when < FULL_RETRY_S: + return True + _FULL_FAILED.pop(rating_key, None) + return False + + +def _live_window_failures(key: str) -> list[tuple[float, float, float]]: + """(Caller holds _LOCK.) The key's failed-window spans that are still remembered.""" + now = time.monotonic() + live = [f for f in _WIN_FAILED.get(key, []) if now - f[2] < FAIL_TTL_S] + if live: + _WIN_FAILED[key] = live + else: + _WIN_FAILED.pop(key, None) + return live + + +def _window_fail(key: str, span: tuple[float, float]) -> None: + with _LOCK: + _sweep_memos() + _WIN_FAILED[key] = [*_live_window_failures(key), (span[0], span[1], time.monotonic())] + + +def _window_failed(key: str, at: float) -> bool: + with _LOCK: + return any(a <= at <= b for a, b, _ in _live_window_failures(key)) + + +def ensure(rating_key: str, sub_ord: int, at: float = 0.0) -> dict: + """State of one subtitle track, scheduling whatever extraction is still missing. NEVER blocks. + + ``ready`` = fully extracted · ``partial`` = usable, covers ``at``, still growing · ``preparing`` + = nothing to show for ``at`` yet · ``unavailable`` = nothing for ``at`` and nothing being read + for it right now (``code`` says why), ask again · ``failed`` = the track can never be shown + (``code`` says why) — the only final answer. ``gen`` changes whenever the cached file grows, so + the client can refetch a window that has been superseded; 0 = nothing is cached at all.""" + key = cache_key(rating_key, sub_ord) + ranges, complete = read_meta(key) + have = cache_read(key) is not None + covered = have and covers(ranges, at) + gen = generation(key) + if complete and have: + return {"state": "ready", "code": None, "gen": gen} + if not have and _exists(_vtt_path(key)): + # On disk but unreadable (permissions, a bad volume): nothing can serve it, and no extraction + # can help — `publish` refuses to merge into it, and a job whose sidecar covers the position + # skips as already done — so scheduling only kept the player on "preparing" for good. `gen` 0: + # there is nothing a could fetch. + return {"state": "unavailable", "code": "cache_unreadable", "gen": 0} + code = _recent_failure(key) + if code and not covered: + return { + "state": "failed" if code in FINAL_CODES else "unavailable", + "code": code, + "gen": gen, + } + # Nothing left that could cover the position for now: the full pass failed, AND so did a window + # here. Still not final — the memos lapse, and the rest of the track may be right there. + full_failed = _full_failed(rating_key) + window_failed = not covered and _window_failed(key, at) + if full_failed and window_failed: + return {"state": "unavailable", "code": "extract_failed", "gen": gen} + if not code: + probe, want_window = at, not covered and not window_failed + if covered: + # Read ahead of an edge instead of waiting to fall off it (see LOOKAHEAD_S). The probe is + # just PAST the edge: `covers` is inclusive, so the edge itself reads as already done. + edge = max(b for a, b in ranges if a <= at <= b) + if edge - at <= LOOKAHEAD_S and edge < _FULL_END_S: + probe = edge + 1.0 + want_window = not _window_failed(key, probe) + _schedule(rating_key, sub_ord, probe, want_window=want_window, want_full=not full_failed) + return {"state": "partial" if covered else "preparing", "code": None, "gen": gen} + + +def cached_track(rating_key: str, sub_ord: int) -> tuple[str | None, str | None]: + """What the fetch can serve RIGHT NOW, scheduling nothing: the cached VTT (whatever it + covers), else the recent failure that explains why there is none. + + Scheduling belongs to `ensure` — the status poll, which knows where the viewer is. The fetch does + not: when it scheduled too, it judged readiness at position 0, so a viewer who resumed an hour in + got an unrequested window over the head of the file (plus a remount when it landed), and with a + failure memo alive it was refused a window that WAS on disk while the poll called it showable.""" + key = cache_key(rating_key, sub_ord) + hit = cache_read(key) + return (hit, None) if hit is not None else (None, _recent_failure(key)) + + +def _schedule( + rating_key: str, sub_ord: int, at: float, want_window: bool, want_full: bool = True +) -> None: + """Kick off the jobs this track still needs: the window that makes it watchable now, and the one + full pass that makes it complete. A 1 s status poll (or a second viewer) joins a job already + queued or running instead of starting another ffmpeg. + + A window is joined when one of this track's in-flight windows will COVER the position asked about + — decided by what it reads, not by an identity derived from the position. The poll sends the live + playhead, so a position-keyed window was a new job every second; bucketing the position instead + still queued a second read when the playhead crossed a bucket edge, and joined a backward seek to + a window that could not cover it. Nor may it be one window per track: a window cut around the + wrong spot (the first poll after a resume can still report 0) made the right one wait for it. The + full pass is per ITEM, because one read fans out to every embedded text track of the file.""" + jobs: list[tuple[tuple, float | None]] = [] + with _LOCK: + if _STOPPING: + return + n_windows = sum(1 for span in _INFLIGHT.values() if span is not None) + n_full = len(_INFLIGHT) - n_windows + if want_window and n_windows < _MAX_WINDOWS_INFLIGHT: + spans = [ + span + for ident, span in _INFLIGHT.items() + if span is not None and ident[1:3] == (rating_key, sub_ord) + ] + if not any(a <= at <= b for a, b in spans) and len(spans) < _MAX_TRACK_WINDOWS: + # Rounded to what ffmpeg is told (`-ss %.3f`): the cues come back on ffmpeg's zero, + # and shifting them by an unrounded start put them up to half a millisecond off — + # enough to spell a timestamp differently from the full pass and double the cue. + w_start = float(f"{max(0.0, at - WINDOW_PRE_S):.3f}") + ident = ("win", rating_key, sub_ord, w_start) + _INFLIGHT[ident] = (w_start, w_start + WINDOW_PRE_S + WINDOW_LEN_S) + jobs.append((ident, w_start)) + if want_full and n_full < _MAX_FULL_INFLIGHT and ("full", rating_key) not in _INFLIGHT: + _INFLIGHT[("full", rating_key)] = None + jobs.append((("full", rating_key), None)) + for ident, window_start in jobs: + pool = _FULL_POOL if window_start is None else _WIN_POOL + pool.submit(_run, ident, rating_key, sub_ord, window_start, at) + + +def _already_done(key: str, window_start: float | None, at: float) -> bool: + """A queued job looks again before it reads anything. While it waited, another job may have + covered it: every embedded track rides each pass over its file, so a window for one language also + publishes the others.""" + if not _exists(_vtt_path(key)): + return False + ranges, complete = read_meta(key) + return complete or (window_start is not None and covers(ranges, at)) + + +def _run( + ident: tuple, rating_key: str, sub_ord: int, window_start: float | None, at: float +) -> None: + key = cache_key(rating_key, sub_ord) + try: + if _STOPPING or _already_done(key, window_start, at): + return + _extract_and_publish(rating_key, sub_ord, window_start) + except SubtitleUnavailable as e: + if e.code == "extract_failed": + _pass_failed(rating_key, [key], window_start) + else: + _fail(key, e.code) + except PlexError as e: + log.warning("subtitle extraction failed for %s/%s: %s", rating_key, sub_ord, e) + _fail(key, "plex_error") + except ( + Exception + ): # a worker thread has nobody to raise to — record it, let the next poll see it + log.exception("subtitle extraction crashed for %s/%s", rating_key, sub_ord) + _pass_failed(rating_key, [key], window_start) + finally: + with _LOCK: + _INFLIGHT.pop(ident, None) + + +def _pass_failed( + rating_key: str, + keys: list[str], + window_start: float | None, + span: tuple[float, float] | None = None, +) -> None: + """Record an ffmpeg pass that failed: a full pass backs off the whole item (FULL_RETRY_S); a + window marks only the stretch it failed to read, for the tracks it was reading.""" + if window_start is None: + with _LOCK: + _sweep_memos() + _FULL_FAILED[rating_key] = time.monotonic() + return + for key in keys: + _window_fail(key, span or (window_start, window_start + WINDOW_PRE_S + WINDOW_LEN_S)) + + +def _text_streams(db: Session, rating_key: str) -> tuple[PlexItem, list[dict]]: + """The item plus its subtitle streams, in the ordinal order the API and the UI both use.""" + it = by_rating_key(db, PlexItem, rating_key) + if it is None: + raise SubtitleUnavailable("no_item") + with PlexClient(db) as plex: + part = ((plex.metadata(it.rating_key) or {}).get("Media") or [{}])[0].get("Part") or [{}] + return it, [s for s in (part[0].get("Stream") or []) if s.get("streamType") == 3] + + +def _is_bitmap(stream: dict) -> bool: + return (stream.get("codec") or stream.get("format") or "").lower() in BITMAP_SUBS + + +def _extract_and_publish(rating_key: str, sub_ord: int, window_start: float | None) -> None: + """One extraction job. A sidecar track is fetched whole from Plex (cheap, no ffmpeg); an embedded + one is extracted together with EVERY other embedded text track in the file, because they all ride + on the same single read — the whole point of the measurement at the top of this module.""" + with SessionLocal() as db: + it, subs = _text_streams(db, rating_key) + if sub_ord < 0 or sub_ord >= len(subs): + raise SubtitleUnavailable("no_track") + want = subs[sub_ord] + if _is_bitmap(want): + raise SubtitleUnavailable("bitmap") + if want.get("key"): + # An external sidecar is one small fetch from Plex, never a read of the media file, so a + # window job fetches the whole thing too. It must not be left to the full job: that + # identity is per ITEM (the embedded fan-out shares it), so a sidecar picked while an + # embedded track's full pass was running sat behind that pass for minutes. + vtt = _sidecar_vtt(db, want) + if not publish(cache_key(rating_key, sub_ord), vtt, (0.0, _FULL_END_S), True): + raise SubtitleUnavailable("extract_failed") + return + if want.get("index") is None: + raise SubtitleUnavailable("not_fetchable") + src = plex_paths.local_media_path(db, it.file_path) + if src is None: + raise SubtitleUnavailable("no_file") + # Ordinal among the file's EMBEDDED subtitle streams — what `-map 0:s:N` counts. Sidecars sit + # in the same list but not in the file, so they must not shift the mapping. + targets = { + sum(1 for x in subs[:i] if x.get("index") is not None): i + for i, s in enumerate(subs) + if s.get("index") is not None and not _is_bitmap(s) + } + start = window_start or 0.0 + length = (WINDOW_LEN_S + WINDOW_PRE_S) if window_start is not None else None + end = _FULL_END_S if length is None else start + length + texts, clean = _ffmpeg_extract(str(src), sorted(targets), window_start, length) + for emb_ord, vtt in texts.items(): + key = cache_key(rating_key, targets[emb_ord]) + # A pass that exited non-zero holds only what ffmpeg reached before it died. Claim no more + # than its last cue, never call it complete, and remember the failure: published as complete, + # the track read "ready" for good while its subtitles stopped wherever ffmpeg did (the cache + # has no eviction, so nothing would ever have extracted it again). + covered_end = end if clean else start + _last_cue_end(vtt) + landed = publish( + key, + shift_vtt(vtt, -start), # zero-based window clock → absolute cue times + (start, covered_end), + clean and length is None, + ) + if not landed: + # A read that could not be kept is a failed read (see publish) — backed off like one. + _pass_failed(rating_key, [key], window_start, (start, end)) + elif not clean: + _pass_failed(rating_key, [key], window_start, (covered_end, end)) + + +def _sidecar_vtt(db: Session, stream: dict) -> str: + """An external subtitle file, fetched through Plex and converted to WebVTT.""" + with PlexClient(db) as plex: + raw = plex.raw_get(stream["key"]) + codec = (stream.get("codec") or stream.get("format") or "").lower() + if codec in ("srt", "subrip", ""): + return srt_to_vtt(raw.decode("utf-8", "replace")) + with tempfile.NamedTemporaryFile(suffix="." + (codec or "sub"), delete=False) as fh: + fh.write(raw) + tmp = fh.name + try: + return ffmpeg_to_vtt(tmp, None) + finally: + Path(tmp).unlink(missing_ok=True) + + +def _ffmpeg_extract( + src: str, emb_ords: list[int], start: float | None, length: float | None +) -> tuple[dict[int, str], bool]: + """Extract several embedded subtitle streams to WebVTT in ONE ffmpeg run (= one read of the file). + Returns the texts and whether ffmpeg exited cleanly — a non-zero exit can still leave partial + output, which is worth publishing but must never be taken for the whole track. + + `start`/`length` bound the read to a window. Without `-copyts` the cues come out on a zero-based + clock (the caller shifts them back) — which is exactly why `-copyts` is absent: with it ffmpeg + ignores `-t` and reads the whole container anyway (measured).""" + if not emb_ords: + return {}, True + with tempfile.TemporaryDirectory(prefix="plexsub-") as tmpdir: + cmd = ["ffmpeg", "-nostdin", "-loglevel", "error"] + if start: + cmd += ["-ss", f"{start:.3f}"] + cmd += ["-i", src] + outs = {o: Path(tmpdir) / f"{o}.vtt" for o in emb_ords} + for emb_ord, out in outs.items(): + cmd += ["-map", f"0:s:{emb_ord}"] + if length is not None: + cmd += ["-t", f"{length:.3f}"] + # The input seek lands on the keyframe at or BEFORE `start`, so a cue already running + # at `start` comes out at a negative time — and by default the muxer then shifts the + # WHOLE output so that cue sits at 0: every cue of the window late by the distance to + # that keyframe (1–5 s measured on a real library file, a whole GOP on a synthetic + # one), and doubled for good once the full pass merged in beside it. Kept negative, the + # times are exact on the zero-based clock (`_vtt_ts_to_s` reads the sign). + cmd += ["-avoid_negative_ts", "disabled"] + cmd += ["-f", "webvtt", "-y", str(out)] + # Popen rather than run(): the child is registered so `shutdown` can kill it — and one that + # would start after the shutdown's kill sweep never does, or is killed right here. + with _LOCK: + if _STOPPING: + raise SubtitleUnavailable("extract_failed") + proc = subprocess.Popen( + cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE + ) + with _LOCK: + _PROCS.add(proc) + if _STOPPING: + proc.kill() + timed_out = False + try: + try: + _, err = proc.communicate(timeout=EXTRACT_TIMEOUT_S) + except subprocess.TimeoutExpired: + # A pass that ran out of time is a pass that died partway: what it reached is kept and + # published as partial (the caller's unclean path), like any other non-zero exit. It + # used to be killed and its outputs deleted unread — and with the full-pass backoff, the + # same doomed read came back every hour. TERM first: ffmpeg closes its outputs on it, + # whereas a KILL leaves only what was already flushed in whole blocks (measured: none). + timed_out = True + proc.terminate() + try: + _, err = proc.communicate(timeout=_TERM_GRACE_S) + except subprocess.TimeoutExpired: + proc.kill() + _, err = proc.communicate() + finally: + with _LOCK: + _PROCS.discard(proc) + got = { + o: p.read_text(encoding="utf-8", errors="replace") + for o, p in outs.items() + if p.exists() + } + if not got: + raise SubtitleUnavailable("extract_failed") + clean = proc.returncode == 0 and not timed_out + if not clean: + log.warning( + "ffmpeg subtitle pass %s: %s", + "timed out" if timed_out else f"exited {proc.returncode}", + (err or b"")[:200], + ) + return got, clean diff --git a/backend/app/routes/_common.py b/backend/app/routes/_common.py index 32590e7b..7223ce93 100644 --- a/backend/app/routes/_common.py +++ b/backend/app/routes/_common.py @@ -17,12 +17,20 @@ from app.models import User M = TypeVar("M", bound=Base) -def get_or_404(db: Session, model: type[M], id_: Any, *, detail: str = "Not found") -> M: +def not_found(what: str = "generic") -> HTTPException: + """The 404 for a row that is gone — structured, so the bilingual UI says it in the reader's + language (`errors.notFound.`, generic fallback). This is the 404 a user actually meets: the + thing was deleted in another tab or on another device. It used to be a bare English string + ("Unknown view", "Not found"), which the Hungarian UI toasted verbatim.""" + return HTTPException(status_code=404, detail={"code": "not_found", "reason": what}) + + +def get_or_404(db: Session, model: type[M], id_: Any, *, what: str = "generic") -> M: """Fetch a row by primary key or raise 404. Replaces the `db.get(Model, id); if None: 404` idiom repeated across routes.""" obj = db.get(model, id_) if obj is None: - raise HTTPException(status_code=404, detail=detail) + raise not_found(what) return obj @@ -33,13 +41,13 @@ def owned_or_404( user: User, *, owner_attr: str = "user_id", - detail: str = "Not found", + what: str = "generic", ) -> M: """Fetch a row by primary key that `user` owns, or raise 404. A row owned by someone else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger. Generalizes the near-identical per-route ownership helpers (e.g. plex's `_own_playlist_or_404`, the saved-views owner check).""" - obj = get_or_404(db, model, id_, detail=detail) + obj = get_or_404(db, model, id_, what=what) if getattr(obj, owner_attr) != user.id: - raise HTTPException(status_code=404, detail=detail) + raise not_found(what) return obj diff --git a/backend/app/routes/admin.py b/backend/app/routes/admin.py index b9b3a04f..86ad5ade 100644 --- a/backend/app/routes/admin.py +++ b/backend/app/routes/admin.py @@ -173,7 +173,7 @@ def _activate_user_by_email(db: Session, email: str) -> None: def _decide(db: Session, invite_id: int, admin: User, approved: bool) -> Invite: - inv = get_or_404(db, Invite, invite_id, detail="No such invite") + inv = get_or_404(db, Invite, invite_id, what="invite") # Idempotency guard: only a still-pending invite can be decided. Without this a repeat/stale # approve or deny re-runs its side effects — re-activating a user, or (now that deny emails) # sending a duplicate decision mail, even flipping an already-approved invite to "denied" and @@ -291,7 +291,7 @@ def send_invitation( Only an APPROVED row can be invited: there is nothing to invite someone to while their request is undecided, and inviting a denied address would contradict the decision.""" - inv = get_or_404(db, Invite, invite_id, detail="No such invite") + inv = get_or_404(db, Invite, invite_id, what="invite") if inv.status != "approved": raise HTTPException( status_code=409, @@ -364,7 +364,7 @@ def delete_invite( falls back to the env ALLOWED_EMAILS / ADMIN_EMAILS bootstrap lists once no invite exists, so for an address on those the deletion leaves access intact (and an ADMIN_EMAILS address is still promoted to admin on its next Google sign-in). The admin has to hear that, not guess it.""" - inv = get_or_404(db, Invite, invite_id, detail="No such invite") + inv = get_or_404(db, Invite, invite_id, what="invite") if _accounts_for(db, [inv.email]): raise HTTPException( status_code=409, @@ -434,9 +434,7 @@ def set_user_role( role = body.role if role not in ("user", "admin"): raise HTTPException(status_code=400, detail="role must be 'user' or 'admin'") - target = db.get(User, user_id) - if target is None: - raise HTTPException(status_code=404, detail="No such user") + target = get_or_404(db, User, user_id, what="user") if target.is_demo: raise HTTPException(status_code=400, detail="The demo account's role can't be changed.") if target.id == admin.id: @@ -491,9 +489,7 @@ def set_user_suspended( un-suspend the user is emailed they can return (suspend is announced reactively, on a blocked sign-in).""" suspended = body.suspended - target = db.get(User, user_id) - if target is None: - raise HTTPException(status_code=404, detail="No such user") + target = get_or_404(db, User, user_id, what="user") if target.is_demo: raise HTTPException(status_code=400, detail="The demo account can't be suspended.") if target.id == admin.id: @@ -537,9 +533,7 @@ def admin_delete_user( """Admin-triggered account deletion — the same full erasure a user can perform on themselves (cascade delete of all personal data + access-request cleanup + Google-grant revoke). Guards: not the demo account, not yourself (use Settings → Account), and not the last admin.""" - target = db.get(User, user_id) - if target is None: - raise HTTPException(status_code=404, detail="No such user") + target = get_or_404(db, User, user_id, what="user") if target.is_demo: raise HTTPException(status_code=400, detail="The demo account can't be deleted.") if target.id == admin.id: @@ -632,7 +626,7 @@ def user_access( Its own endpoint rather than extra columns on /users: `is_allowed` and `can_activate` each run a query, and the panel opens for ONE user at a time — folding them into the list would pay that cost per row on every load to answer a question nobody asked yet.""" - target = get_or_404(db, User, user_id, detail="No such user") + target = get_or_404(db, User, user_id, what="user") return _serialize_user(target) | { "is_allowed": is_allowed(db, target.email), "can_activate": can_activate(db, target), @@ -657,7 +651,7 @@ def create_set_password_link( an instance with no mail — narrowing it to one of those would only make the admin's real case the one it refuses. The demo account is the single exception: it is shared, so handing anyone a way to change its password is handing them everyone else's session.""" - target = get_or_404(db, User, user_id, detail="No such user") + target = get_or_404(db, User, user_id, what="user") if target.is_demo: raise HTTPException( status_code=400, detail={"code": "setpw", "reason": "demo", "email": target.email} @@ -693,7 +687,7 @@ def activate_user( This endpoint changes nothing about what an unauthenticated caller can cause — the admin could already do exactly this by re-entering the address in the whitelist box (`add_invite` → `_activate_user_by_email`). What was missing was any way to know they had to.""" - target = get_or_404(db, User, user_id, detail="No such user") + target = get_or_404(db, User, user_id, what="user") if target.is_demo: raise HTTPException(status_code=400, detail="The demo account can't be activated.") if target.is_active: @@ -732,7 +726,7 @@ def resend_user_verification( Not rate-limited: an admin is not a threat model to enumerate against, and the deliberate silence of the public endpoint is exactly what this exists to escape.""" - target = get_or_404(db, User, user_id, detail="No such user") + target = get_or_404(db, User, user_id, what="user") if target.is_demo or target.email_verified: raise HTTPException( status_code=409, diff --git a/backend/app/routes/auth.py b/backend/app/routes/auth.py index c76bd49f..b96d4304 100644 --- a/backend/app/routes/auth.py +++ b/backend/app/routes/auth.py @@ -1246,7 +1246,7 @@ def logout_others( are rejected, then re-stamp THIS cookie so the caller stays signed in. For "I left myself logged in somewhere" / a suspected session compromise. The demo account (shared) is excluded.""" if user.is_demo: - raise HTTPException(status_code=403, detail="Not available in the demo account.") + raise HTTPException(status_code=403, detail={"code": "demo"}) # Commit the bump BEFORE re-stamping this cookie, so a failed commit can't strand the current # session at a newer epoch than the DB (see set_password). bump_session_epoch(user) @@ -1293,7 +1293,7 @@ def set_password( account) needs no current password — the session already proves identity. Changing an existing one requires the current password, so a hijacked session can't silently rotate it.""" if user.is_demo: - raise HTTPException(status_code=403, detail="Not available in the demo account.") + raise HTTPException(status_code=403, detail={"code": "demo"}) new_password = body.password or "" if len(new_password) < PASSWORD_MIN_LEN: raise HTTPException( diff --git a/backend/app/routes/channels.py b/backend/app/routes/channels.py index d848d6cd..4c868976 100644 --- a/backend/app/routes/channels.py +++ b/backend/app/routes/channels.py @@ -26,7 +26,7 @@ from app.models import ( User, Video, ) -from app.routes._common import owned_or_404 +from app.routes._common import get_or_404, not_found, owned_or_404 from app.sync.explore import explore_ingest_page from app.sync.runner import backfill_channel_now, run_recent_backfill from app.sync.subscriptions import apply_channel_details @@ -287,9 +287,7 @@ def channel_detail( needed — a stub, or a channel last enriched before those fields existed (published_at is a reliable sentinel: every real channel has a creation date). The enrich uses the API key (1 unit, cached after), so it's skipped for the quota-less demo account.""" - channel = db.get(Channel, channel_id) - if channel is None: - raise HTTPException(status_code=404, detail="Unknown channel") + channel = get_or_404(db, Channel, channel_id, what="channel") if channel.published_at is None and not user.is_demo: try: with ( @@ -366,9 +364,7 @@ def explore_channel( it, then ingests one page of its uploads (newest-first) marked via_explore + enriched. Returns the next page token so the page can "load more". Guarded against the shared backfill quota reserve; blocked for the demo account (it spends shared quota).""" - channel = db.get(Channel, channel_id) - if channel is None: - raise HTTPException(status_code=404, detail="Unknown channel") + channel = get_or_404(db, Channel, channel_id, what="channel") if _is_blocked(db, user, channel_id): raise HTTPException(status_code=403, detail="You've blocked this channel.") if quota.backfill_reserve_reached(): @@ -426,7 +422,7 @@ def _user_subscription(db: Session, user: User, channel_id: str) -> Subscription ) ).scalar_one_or_none() if sub is None: - raise HTTPException(status_code=404, detail="Not subscribed to this channel") + raise not_found("subscription") return sub @@ -512,9 +508,7 @@ def reset_backfill( state (a "reset" trigger). Clears the channel's backfill markers, opts it back into deep backfill, and re-runs its recent pull immediately; the deep scheduler re-pages the full back-catalog on its next run. Idempotent — videos upsert by id, so nothing duplicates.""" - channel = db.get(Channel, channel_id) - if channel is None: - raise HTTPException(status_code=404, detail="Unknown channel") + channel = get_or_404(db, Channel, channel_id, what="channel") channel.backfill_done = False channel.backfill_cursor = None channel.recent_synced_at = None @@ -544,9 +538,7 @@ def subscribe( local subscription with YouTube's returned resource id and enriches the channel's metadata if it's still a stub, so the new row renders properly in the manager. It does NOT pull the channel's videos — the scheduler picks those up on its next run, like any subscribed channel.""" - channel = db.get(Channel, channel_id) - if channel is None: - raise HTTPException(status_code=404, detail="Unknown channel") + channel = get_or_404(db, Channel, channel_id, what="channel") existing = db.execute( select(Subscription).where( Subscription.user_id == user.id, Subscription.channel_id == channel_id @@ -663,8 +655,7 @@ def block_channel( """Block a channel for this user: its videos won't appear in their live search (and won't be ingested for them) and are hidden from their feed / explore. Idempotent. The now-hidden un-kept search/explore videos are reclaimed by the discovery-cleanup job in due course.""" - if db.get(Channel, channel_id) is None: - raise HTTPException(status_code=404, detail="Unknown channel") + get_or_404(db, Channel, channel_id, what="channel") if not _is_blocked(db, user, channel_id): db.add(BlockedChannel(user_id=user.id, channel_id=channel_id)) # Stop any active exploration of it so it can be cleaned up. @@ -706,7 +697,7 @@ def attach_tag( _user_subscription(db, user, channel_id) # Only the user's own tags may be attached as personal links (owned_or_404 → 404 if the tag # is missing OR belongs to someone else, without leaking which). - owned_or_404(db, Tag, body.tag_id, user, detail="Unknown tag") + owned_or_404(db, Tag, body.tag_id, user, what="tag") exists = db.execute( select(ChannelTag).where( ChannelTag.channel_id == channel_id, diff --git a/backend/app/routes/discover.py b/backend/app/routes/discover.py index f029f0e2..e7447c09 100644 --- a/backend/app/routes/discover.py +++ b/backend/app/routes/discover.py @@ -168,7 +168,7 @@ def remove_pin( user: User = Depends(require_human), db: Session = Depends(get_db), ) -> dict: - pin = owned_or_404(db, ComfortPin, pin_id, user) + pin = owned_or_404(db, ComfortPin, pin_id, user, what="pin") db.delete(pin) db.commit() return {"deleted": pin_id} diff --git a/backend/app/routes/downloads.py b/backend/app/routes/downloads.py index a6dbb662..01a56f56 100644 --- a/backend/app/routes/downloads.py +++ b/backend/app/routes/downloads.py @@ -38,6 +38,7 @@ from app.models import ( MediaAsset, User, ) +from app.routes._common import get_or_404, not_found from app.security import hash_password from app.userscope import is_messageable_user, messageable_clauses from app.utils import iso @@ -801,7 +802,7 @@ def share_download( # Uniform "no user" for a missing OR non-messageable (demo/suspended/deactivated) target, so # sharing can't be used to probe which addresses exist or to reach a suspended account. if not is_messageable_user(recipient): - raise HTTPException(status_code=404, detail="No user with that email.") + raise not_found("recipient") if recipient.id == user.id: raise HTTPException(status_code=400, detail="That's already your download.") exists = db.execute( @@ -1091,8 +1092,7 @@ def admin_set_quota( admin: User = Depends(admin_user), db: Session = Depends(get_db), ) -> dict: - if db.get(User, user_id) is None: - raise HTTPException(status_code=404, detail="Unknown user") + get_or_404(db, User, user_id, what="user") cur = quota.resolve(db, user_id) row = db.get(DownloadQuota, user_id) if row is None: diff --git a/backend/app/routes/feed.py b/backend/app/routes/feed.py index 2d6f3635..e037295b 100644 --- a/backend/app/routes/feed.py +++ b/backend/app/routes/feed.py @@ -1,7 +1,7 @@ from datetime import date, datetime, timezone from fastapi import APIRouter, Depends, HTTPException, Query -from pydantic import BaseModel +from pydantic import BaseModel, ConfigDict, Field from sqlalchemy import and_, func, or_, select from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.orm import Session @@ -22,12 +22,14 @@ from app.feedquery import ( sort_keys, ) from app.models import ( + POSITION_MAX_S, ChannelTag, Tag, User, Video, VideoState, ) +from app.routes._common import get_or_404 from app.sync.videos import parse_iso8601_duration from app.utils import iso from app.youtube.client import YouTubeClient, YouTubeError @@ -234,8 +236,7 @@ def set_video_state( status = body.status if status not in VALID_STATES: raise HTTPException(status_code=400, detail=f"status must be one of {VALID_STATES}") - if db.get(Video, video_id) is None: - raise HTTPException(status_code=404, detail="Unknown video") + get_or_404(db, Video, video_id, what="video") now = datetime.now(timezone.utc) @@ -303,11 +304,16 @@ def clear_video_state( class VideoProgressIn(BaseModel): + # Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned + # one into a 500. + model_config = ConfigDict(allow_inf_nan=False) + # Accept a float and truncate (below) — the player may post a raw currentTime (a float), which # the old `int(payload.get(...))` silently truncated. A plain `int` field would 422 that on the # 10s progress hot path, losing the resume position; `float` keeps the old tolerance. - position_seconds: float = 0 - duration_seconds: float = 0 + # Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500. + position_seconds: float = Field(0, le=POSITION_MAX_S) + duration_seconds: float = Field(0, le=POSITION_MAX_S) @router.post("/videos/{video_id}/progress") @@ -326,8 +332,7 @@ def set_video_progress( if position < 0: position = 0 - if db.get(Video, video_id) is None: - raise HTTPException(status_code=404, detail="Unknown video") + get_or_404(db, Video, video_id, what="video") # Decide whether this position is worth keeping (else clear it). near_end = duration > 0 and position > duration - FINISH_MARGIN_SECONDS diff --git a/backend/app/routes/me.py b/backend/app/routes/me.py index 6de8d72d..ec625c5f 100644 --- a/backend/app/routes/me.py +++ b/backend/app/routes/me.py @@ -159,8 +159,13 @@ def delete_my_account( which all cascade on the users row. Quota-audit events are kept but anonymised (FK SET NULL). The shared demo account can't be deleted, and the last remaining admin can't delete itself (that would lock everyone out of the admin surfaces).""" + # Structured, like the other three demo refusals: `req` leaves 403 to the caller and + # `notifyActionError` discards a plain-string detail in favour of the caller's translated + # fallback, so a hand-written English sentence here reaches a Hungarian UI as a generic + # "couldn't do that" — or, on this affordance, as nothing at all. `reason` distinguishes it + # from the other sites' bare `{code: "demo"}` so the UI can say what specifically is refused. if user.is_demo: - raise HTTPException(status_code=403, detail="The shared demo account can't be deleted.") + raise HTTPException(status_code=403, detail={"code": "demo", "reason": "delete_account"}) if user.role == "admin" and count_admins(db) <= 1: raise HTTPException( status_code=400, diff --git a/backend/app/routes/playlists.py b/backend/app/routes/playlists.py index 3a1bcd12..c7617c50 100644 --- a/backend/app/routes/playlists.py +++ b/backend/app/routes/playlists.py @@ -23,6 +23,7 @@ from app.sync.playlists import ( sync_user_playlists, ) from app.ratelimit import RateLimiter +from app.routes._common import get_or_404, owned_or_404 from app.youtube.client import YouTubeClient, YouTubeError router = APIRouter(prefix="/api/playlists", tags=["playlists"]) @@ -48,10 +49,13 @@ def _check_pull_cooldown(user: User) -> None: def _own_playlist(db: Session, user: User, playlist_id: int) -> Playlist: - pl = db.get(Playlist, playlist_id) - if pl is None or pl.user_id != user.id: - raise HTTPException(status_code=404, detail="Unknown playlist") - return pl + return owned_or_404(db, Playlist, playlist_id, user, what="playlist") + + +def _refused(status: int, reason: str) -> HTTPException: + """A playlist action the server refuses, as a reason the UI translates + (`playlists.errors.`) — a bare English detail reached the Hungarian UI verbatim.""" + return HTTPException(status_code=status, detail={"code": "playlist", "reason": reason}) def _touch(pl: Playlist) -> None: @@ -256,7 +260,7 @@ def create_playlist( ) -> dict: name = body.name.strip() if not name: - raise HTTPException(status_code=400, detail="name is required") + raise _refused(400, "name_required") pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id)) db.add(pl) db.commit() @@ -288,8 +292,7 @@ def add_watch_later( ) -> dict: """Bookmark shortcut: add a video to the built-in Watch later playlist.""" video_id = body.video_id - if db.get(Video, video_id) is None: - raise HTTPException(status_code=404, detail="Unknown video") + get_or_404(db, Video, video_id, what="video") pl = _watch_later(db, user) _add_item(db, pl, video_id, mark_dirty=False) return {"saved": True, "playlist_id": pl.id} @@ -317,10 +320,7 @@ def sync_youtube( """Pull the user's YouTube playlists into local mirrors now (read-only direction). Also runs automatically on the scheduler; this is the manual 'sync now'.""" if not has_read_scope(user): - raise HTTPException( - status_code=403, - detail="Connect YouTube (read access) to sync your playlists.", - ) + raise _refused(403, "youtube_read_scope") _check_pull_cooldown(user) with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PULL): return sync_user_playlists(db, user) @@ -337,18 +337,16 @@ def revert_youtube( once the in-session undo history is gone.""" pl = _own_playlist(db, user, playlist_id) if not pl.yt_playlist_id: - raise HTTPException(status_code=400, detail="This playlist isn't linked to YouTube.") + raise _refused(400, "not_on_youtube") if not has_read_scope(user): - raise HTTPException( - status_code=403, detail="Connect YouTube (read access) to sync your playlists." - ) + raise _refused(403, "youtube_read_scope") _check_pull_cooldown(user) try: with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PULL): return repull_playlist(db, user, pl) except YouTubeError: db.rollback() - raise HTTPException(status_code=422, detail="Couldn't refresh from YouTube.") + raise _refused(422, "youtube_failed") def _pushable(db: Session, user: User, playlist_id: int) -> Playlist: @@ -356,9 +354,7 @@ def _pushable(db: Session, user: User, playlist_id: int) -> Playlist: Watch later and read-only YouTube mirrors are never pushable.""" pl = _own_playlist(db, user, playlist_id) if pl.kind == "watch_later": - raise HTTPException( - status_code=400, detail="The Watch later list can't be sent to YouTube." - ) + raise _refused(400, "watch_later_not_pushable") return pl @@ -376,7 +372,7 @@ def push_plan( with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PUSH): plan = plan_merge(db, user, pl) except YouTubeError: - raise HTTPException(status_code=422, detail="Couldn't read the playlist on YouTube.") + raise _refused(422, "youtube_failed") plan["remaining_today"] = quota.remaining_today(db) plan["affordable"] = plan["units_estimate"] <= plan["remaining_today"] return plan @@ -406,14 +402,11 @@ def push( ) plan = plan_merge(db, user, pl, live=live) if plan["units_estimate"] > quota.remaining_today(db): - raise HTTPException( - status_code=429, - detail="Not enough YouTube quota left today for this sync.", - ) + raise _refused(429, "quota_exceeded") return merge_playlist(db, user, pl, live=live) except YouTubeError: db.rollback() - raise HTTPException(status_code=422, detail="YouTube playlist sync failed.") + raise _refused(422, "youtube_failed") @router.get("/{playlist_id}") @@ -458,7 +451,7 @@ def rename_playlist( if "name" in body.model_fields_set: name = (body.name or "").strip() if not name: - raise HTTPException(status_code=400, detail="name cannot be empty") + raise _refused(400, "name_required") if name != pl.name: pl.name = name recompute_dirty(db, pl) @@ -493,18 +486,18 @@ def delete_playlist( ) -> dict: pl = _own_playlist(db, user, playlist_id) if pl.kind == "watch_later": - raise HTTPException(status_code=400, detail="The Watch later list can't be deleted.") + raise _refused(400, "watch_later_undeletable") if on_youtube: if not pl.yt_playlist_id: - raise HTTPException(status_code=400, detail="This playlist isn't on YouTube.") + raise _refused(400, "not_on_youtube") if not has_write_scope(user): - raise HTTPException(status_code=403, detail="Enable YouTube editing in Settings first.") + raise _refused(403, "youtube_editing_off") try: with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_DELETE): with YouTubeClient(db, user) as yt: yt.delete_playlist(pl.yt_playlist_id) except YouTubeError: - raise HTTPException(status_code=422, detail="YouTube delete failed.") + raise _refused(422, "youtube_failed") db.delete(pl) # items cascade db.commit() return {"deleted": playlist_id} @@ -519,8 +512,7 @@ def add_item( ) -> dict: pl = _own_playlist(db, user, playlist_id) video_id = body.video_id - if db.get(Video, video_id) is None: - raise HTTPException(status_code=404, detail="Unknown video") + get_or_404(db, Video, video_id, what="video") _add_item(db, pl, video_id, mark_dirty=True) return {"playlist_id": pl.id, "video_id": video_id} @@ -557,7 +549,9 @@ def move_item( case. Idempotent: if the video is already in the target, the source row is simply dropped (moving into a list that already has it is a removal, not a duplicate).""" src = _own_playlist(db, user, playlist_id) - dst = _own_playlist(db, user, body.target_playlist_id) + # Its own reason: "this playlist doesn't exist any more" about a TARGET that is gone was read as + # the list on screen — which is still there. + dst = owned_or_404(db, Playlist, body.target_playlist_id, user, what="move_target") if src.id == dst.id: raise HTTPException( status_code=400, detail={"code": "playlist_move", "reason": "same_playlist"} diff --git a/backend/app/routes/plex/browse.py b/backend/app/routes/plex/browse.py index 5cdda59a..42d04506 100644 --- a/backend/app/routes/plex/browse.py +++ b/backend/app/routes/plex/browse.py @@ -54,6 +54,22 @@ from app.utils import to_tsquery_str router = APIRouter(prefix="/api/plex", tags=["plex"]) +def sort_column(u, sort: str): + """The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline + in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control + offers but this cannot order would quietly fall back to `added_at` and look like a broken control. + An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback.""" + return { + "added": u.c.added_at, + "title": func.lower(u.c.title), + "year": u.c.year, + "rating": u.c.rating, + "duration": u.c.duration_s, + "release": u.c.rel, + "lastwatched": u.c.last_watched, + }.get(sort, u.c.added_at) + + @router.get("/libraries") def list_libraries(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict: """The mirrored libraries the user can browse (drives the Plex scope selector). HU overlay @@ -221,14 +237,7 @@ def unified_library( if tsq is not None: order = [u.c.rank.desc()] else: - col = { - "title": func.lower(u.c.title), - "year": u.c.year, - "rating": u.c.rating, - "duration": u.c.duration_s, - "release": u.c.rel, - "lastwatched": u.c.last_watched, - }.get(sort, u.c.added_at) + col = sort_column(u, sort) order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()] order.append(u.c.rk.desc()) rows = db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all() diff --git a/backend/app/routes/plex/collections.py b/backend/app/routes/plex/collections.py index 862f5f9a..5719cf3a 100644 --- a/backend/app/routes/plex/collections.py +++ b/backend/app/routes/plex/collections.py @@ -1,5 +1,9 @@ """Plex collections: browse mirrored collections + the admin manual-collection editor.""" +from collections.abc import Callable +from datetime import date, datetime +from typing import Any, NamedTuple + from fastapi import APIRouter, Depends, HTTPException from pydantic import BaseModel from sqlalchemy import func, select @@ -12,6 +16,7 @@ from app.models import ( PlexItem, PlexLibrary, PlexShow, + PlexState, User, ) from app.plex import rollup @@ -83,13 +88,20 @@ def _collection_card(c: PlexCollection) -> dict: def collections( library: str | None = None, q: str | None = None, + keys: str | None = None, user: User = Depends(current_user), db: Session = Depends(get_db), ) -> dict: """Mirrored collections as cards (most-populated first), optionally name-filtered by `q`. With a `library` plex_key → that one library's collections (used by the collection editor). WITHOUT one → the UNION across all enabled libraries (the unified-scope sidebar picker, which has no single - library selected).""" + library selected). + + `keys` (CSV of rating_keys) narrows to just those. It exists because a Plex link carries the + collection FILTER as bare rating_keys — nothing in the URL says what "17045" is called — so the + sidebar token and the applied-filter chip had nothing to render but the raw id. Resolving them + through this endpoint costs one small row each; pulling the whole list to find two titles is + ~200 KB.""" if library is not None: lib = ( db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(library))) @@ -113,6 +125,13 @@ def collections( term = (q or "").strip() if term: query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%")) + if keys is not None: + wanted = [k for k in (keys or "").split(",") if k] + # An explicit but empty `keys=` asks for nothing — answer nothing, rather than falling through + # to "every collection" (which is what an unguarded empty IN () would have to become). + if not wanted: + return {"collections": []} + query = query.where(PlexCollection.rating_key.in_(wanted)) rows = ( db.execute( query.order_by( @@ -149,14 +168,69 @@ def collections( } +class Member(NamedTuple): + """One collection member flattened to everything the Sort control can order by. The page's two + legs are different tables (movies and shows), so there is no single query to ORDER BY — the + ordering is a pure list sort over these, which is also what makes it unit-testable.""" + + kind: str # "movie" | "show" + obj: Any # PlexItem | PlexShow, carried through to the card serializer + title: str # lower-cased; never NULL (the column is not nullable) + year: int | None + rating: float | None + added: datetime | None + release: date | None + duration: int | None # movies only — a show has no single runtime + last_watched: datetime | None + + +# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page +# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option +# that silently does nothing here. +MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = { + "added": lambda m: m.added, + "lastwatched": lambda m: m.last_watched, + "release": lambda m: m.release, + "year": lambda m: m.year, + "rating": lambda m: m.rating, + "duration": lambda m: m.duration, + "title": lambda m: m.title, +} + +# What the page ordered by before it had a control, and still its default: a film series wants to be +# read in release order, not by whenever Plex happened to ingest it. +DEFAULT_MEMBER_SORT = "year" + + +def sort_members(members: list[Member], sort: str, sort_dir: str) -> list[Member]: + """Order a collection's members by one of `PLEX_SORT_KEYS`. + + Two properties worth stating because they are easy to get wrong with a plain `reverse=True`: + members with no value for the key sink to the END in BOTH directions (matching the browse grid's + `nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve + title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside + it. An unknown key falls back to the page's default instead of returning an arbitrary order.""" + key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT]) + ordered = sorted(members, key=lambda m: m.title) # stable secondary, applied first + have = [m for m in ordered if key(m) is not None] + missing = [m for m in ordered if key(m) is None] + have.sort(key=key, reverse=sort_dir != "asc") + return have + missing + + @router.get("/collections/{rating_key}") def collection_detail( rating_key: str, + sort: str = DEFAULT_MEMBER_SORT, + sort_dir: str = "asc", user: User = Depends(current_user), db: Session = Depends(get_db), ) -> dict: - """One collection's hero data + its members (movies + shows), ordered by release year then title — - the Collection info page. Read-only here; edits go through the admin mutation endpoints below.""" + """One collection's hero data + its members (movies + shows) in the requested order — the + Collection info page. The sort vocabulary is the browse grid's (`PLEX_SORT_KEYS`) so the same Sort + control means the same thing on both surfaces; the default is release year ascending, which is + what this page did before it had a control. Read-only here; edits go through the admin mutation + endpoints below.""" col = by_rating_key(db, PlexCollection, rating_key) if col is None: raise HTTPException(status_code=404, detail="Unknown collection") @@ -179,19 +253,63 @@ def collection_detail( .all() ) mstate = rollup.item_state_map(db, user.id, [m.id for m in movies]) - sstate = rollup.state_map(db, user.id, [s.id for s in shows]) - # Merge the two type-homogeneous lists into ONE member list ordered by (year, title). - tagged = [(m.year, (m.title or "").lower(), "movie", m) for m in movies] + [ - (s.year, (s.title or "").lower(), "show", s) for s in shows + # ONE rollup pass over the show members, for both the aggregate badge status and `resumed_at` — + # the browse grid's "last watched" for a show (the newest resume/watched across its episodes). The + # cheaper `rollup.state_map` gives only the status, which would leave `lastwatched` unorderable on + # a show collection while the control still offered it. + show_rows = ( + db.execute( + rollup.agg_subquery(db, user.id, include_resumed=True).where( + PlexItem.show_id.in_([s.id for s in shows]) + ) + ).all() + if shows + else [] + ) + sstate = {r.show_id: rollup.show_status(r.total, r.watched, r.inprog) for r in show_rows} + sresumed = {r.show_id: r.resumed_at for r in show_rows} + + def _movie_last_watched(st: PlexState | None) -> datetime | None: + # Same definition as the browse grid's: when it was last resumed, else when it was marked + # watched (coalesce(progress_updated_at, watched_at)). + return (st.progress_updated_at or st.watched_at) if st else None + + # Merge the two type-homogeneous lists into ONE member list, then order it. + members = [ + Member( + kind="movie", + obj=m, + title=(m.title or "").lower(), + year=m.year, + rating=m.rating, + added=m.added_at, + release=m.originally_available_at, + duration=m.duration_s, + last_watched=_movie_last_watched(mstate.get(m.id)), + ) + for m in movies + ] + [ + Member( + kind="show", + obj=sh, + title=(sh.title or "").lower(), + year=sh.year, + rating=sh.rating, + added=sh.added_at, + release=sh.originally_available_at, + duration=None, + last_watched=sresumed.get(sh.id), + ) + for sh in shows ] - tagged.sort(key=lambda t: (t[0] is None, t[0] or 0, t[1])) + members = sort_members(members, sort, sort_dir) items = [ - movie_card(o, mstate.get(o.id)) - if kind == "movie" - else show_card(o, sstate.get(o.id, "new")) - for _, _, kind, o in tagged + movie_card(m.obj, mstate.get(m.obj.id)) + if m.kind == "movie" + else show_card(m.obj, sstate.get(m.obj.id, "new")) + for m in members ] - years = [y for y, _, _, _ in tagged if y is not None] + years = [m.year for m in members if m.year is not None] certs = [m.content_rating for m in movies] + [s.content_rating for s in shows] return { **_collection_card(col), @@ -229,7 +347,9 @@ def _reject_duplicate_name( .first() ) if dup is not None and dup.rating_key != exclude_rk: - raise HTTPException(status_code=409, detail="A collection with this name already exists") + # Structured, so the bilingual UI says it in the reader's language (it was toasted in + # Hungarian beside an English error dialog saying the same thing). + raise HTTPException(status_code=409, detail={"code": "collection", "reason": "name_taken"}) def _write_or_pending_error(wrote: bool, e: Exception) -> HTTPException: diff --git a/backend/app/routes/plex/playback.py b/backend/app/routes/plex/playback.py index d77eb462..d08b81ca 100644 --- a/backend/app/routes/plex/playback.py +++ b/backend/app/routes/plex/playback.py @@ -1,8 +1,6 @@ """Plex playback: image/subtitle proxies, HLS streaming, and watch-state writes.""" import hashlib -import os -import tempfile import threading from datetime import datetime, timedelta, timezone @@ -12,17 +10,20 @@ from fastapi import ( BackgroundTasks, Depends, HTTPException, + Path, + Query, Request, Response, ) from fastapi.responses import FileResponse -from pydantic import BaseModel +from pydantic import BaseModel, ConfigDict, Field from sqlalchemy import select from sqlalchemy.orm import Session from app.auth import current_user, resolved_user_id from app.db import SessionLocal, get_db from app.models import ( + POSITION_MAX_S, PlexItem, PlexSeason, PlexShow, @@ -89,62 +90,70 @@ def image(rating_key: str, request: Request, variant: str = "thumb") -> Response return imgcache.response(data, ct) +# Both subtitle routes authenticate with the full `current_user`, not the cookie-only +# `_require_session` the image proxies use. That lighter gate exists for bursts of low-sensitivity +# poster fetches and skips the is_active / is_suspended / session-epoch checks — acceptable for a +# poster, not for a route that starts reads of the media library: a suspended or "log out +# everywhere"-revoked cookie kept scheduling them. They are not bursty (one poll a second, one fetch +# per track), so the DB user-load costs nothing that matters. +# +# Both path parameters are bounded, the same way on both routes: they become a cache FILE name +# (`sub__`), and an absurd one — a 250-digit key or ordinal — outgrew the OS's name +# limit and turned the status poll into a 500. A Plex rating_key is a short number; the bound only +# has to be generous. +_RatingKey = Path(max_length=32) +_SubOrd = Path(ge=0, le=9999) + + +@router.get("/subtitle/{rating_key}/{sub_ord}/status") +def subtitle_status( + rating_key: str = _RatingKey, + sub_ord: int = _SubOrd, + user: User = Depends(current_user), + # Bounded both ways: a non-finite position used to reach the scheduler's arithmetic and 500, and + # an absurd one reached ffmpeg's `-ss`, which refused it. + at: float = Query(0.0, ge=0.0, le=subtitles.MAX_POSITION_S, allow_inf_nan=False), +) -> dict: + """Is this subtitle track showable yet — and if not, get it moving. Extraction runs in the + background (see plex/subtitles.py for the measurements that forced it there), so the player polls + this while it waits instead of hanging on a that will not load for a minute or more. + + `at` is the viewer's absolute position: the extractor windows around it, so what gets read first + is the part actually about to be watched.""" + return subtitles.ensure(rating_key, sub_ord, at) + + @router.get("/subtitle/{rating_key}/{sub_ord}") -def subtitle(rating_key: str, sub_ord: int, request: Request, offset: float = 0.0) -> Response: +def subtitle( + rating_key: str = _RatingKey, + sub_ord: int = _SubOrd, + user: User = Depends(current_user), + # Finite, like every float this router takes: `-inf` crashed the shift (a 500), `nan` stacked + # every cue at 0. It can legitimately be negative (a subtitle nudged later than the session start). + offset: float = Query(0.0, allow_inf_nan=False), +) -> Response: """Serve the ord-th subtitle track as WebVTT for the browser's — so choosing a subtitle overlays it directly (no HLS re-mux / session restart). Handles EXTERNAL sidecar subs (fetched from Plex via the stream `key` — the old `-map 0:s` broke on these since they're not in the file) - AND embedded text subs (extracted from the local file). Image subs (PGS/VobSub) → 415. Disk-cached.""" - _require_session(request) - # Cache the ABSOLUTE-time VTT once (per rk+ord); the per-session cue-shift is applied on the way - # out, so a resume/seek to any offset reuses the same cached track (no cache-per-offset explosion). - cache_key = f"sub_{rating_key}_{sub_ord}" - hit = subtitles.cache_read(cache_key) - if hit is not None: - return Response(subtitles.shift_vtt(hit, offset), media_type="text/vtt") - try: - with SessionLocal() as db: - it = item_or_404(db, rating_key) - with PlexClient(db) as plex: - part = ((plex.metadata(it.rating_key) or {}).get("Media") or [{}])[0].get( - "Part" - ) or [{}] - subs = [s for s in (part[0].get("Stream") or []) if s.get("streamType") == 3] - if sub_ord < 0 or sub_ord >= len(subs): - raise HTTPException(status_code=404, detail="No such subtitle") - s = subs[sub_ord] - codec = (s.get("codec") or s.get("format") or "").lower() - if codec in subtitles.BITMAP_SUBS: - raise HTTPException( - status_code=415, - detail="Image-based subtitle can't be shown as text", - ) - if s.get("key"): # external sidecar → fetch from Plex, convert to VTT - raw = plex.raw_get(s["key"]) - if codec in ("srt", "subrip", ""): - vtt = subtitles.srt_to_vtt(raw.decode("utf-8", "replace")) - else: - fd, tmp = tempfile.mkstemp(suffix="." + (codec or "sub")) - try: - os.write(fd, raw) - os.close(fd) - vtt = subtitles.ffmpeg_to_vtt(tmp, None) - finally: - os.unlink(tmp) - elif s.get("index") is not None: # embedded → extract from the local file - src = plex_paths.local_media_path(db, it.file_path) - if src is None: - raise HTTPException(status_code=404, detail="Media file not found on disk") - emb_ord = sum(1 for x in subs[:sub_ord] if x.get("index") is not None) - vtt = subtitles.ffmpeg_to_vtt(str(src), emb_ord) - else: - raise HTTPException(status_code=404, detail="Subtitle not fetchable") - except PlexNotConfigured as e: - raise HTTPException(status_code=400, detail=str(e)) - except PlexError as e: - raise HTTPException(status_code=502, detail=f"Subtitle fetch failed: {e}") - subtitles.cache_write(cache_key, vtt) - return Response(subtitles.shift_vtt(vtt, offset), media_type="text/vtt") + AND embedded text subs (extracted from the local file). Image subs (PGS/VobSub) → 415. Disk-cached. + + A pure READ: it serves whatever is cached and schedules nothing — the status poll owns that, + because only it knows where the viewer is (see `subtitles.cached_track`). The player mounts the + only once /status says there is something to show, so in practice this is only ever hit + warm; on a cache eviction race the 202 sends the player back to polling.""" + # The cached VTT holds ABSOLUTE cue times (per rk+ord); the per-session cue-shift is applied on + # the way out, so a resume/seek to any offset reuses the same track (no cache-per-offset explosion). + hit, code = subtitles.cached_track(rating_key, sub_ord) + if hit is None and code: + raise HTTPException( + status_code=subtitles.UNAVAILABLE_STATUS.get(code, 422), + detail={"code": "subtitle", "reason": code}, + ) + if hit is None: + return Response( + status_code=202, media_type="application/json", content='{"state":"preparing"}' + ) + return Response(subtitles.shift_vtt(hit, offset), media_type="text/vtt") # Resume/watched thresholds. _PROGRESS_MIN_S matches the feed's PROGRESS_MIN_SECONDS (5), but @@ -287,10 +296,15 @@ def _push_watch( class ItemProgressIn(BaseModel): + # Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned + # one into a 500. + model_config = ConfigDict(allow_inf_nan=False) + # float, truncated in the handler — the player may post a raw currentTime (a float); a plain # `int` field would 422 it on the periodic progress checkpoint (the old `int(...)` truncated). - position_seconds: float = 0 - duration_seconds: float = 0 + # Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500. + position_seconds: float = Field(0, le=POSITION_MAX_S) + duration_seconds: float = Field(0, le=POSITION_MAX_S) final: bool = False @@ -309,6 +323,14 @@ def item_progress( `_should_push_resume`). `final` (sent on pause / seek / pagehide / unmount) pushes immediately and resets the throttle window, so Plex sees a settled position the instant the user pauses or seeks.""" it = item_or_404(db, rating_key) + # A player parked mid-item — playing or PAUSED — checkpoints every 10 s, so this is the signal + # that such a viewer still wants their HLS session: the one thing `last_access` could not see, + # since it moved only on a file fetch and a paused player fetches nothing (see + # stream.reap_idle). It is NOT every open player: `usePlexProgress.flush` sends nothing once the + # finish latch is up, or before the first position is known. Those are the states in which + # reaping is the right answer anyway, and a viewer who scrubs back into a reaped session gets + # the same 404 → reclaim the player already handles. + plex_stream.touch_session(str(rating_key)) position = max(0, int(body.position_seconds)) duration = int(body.duration_seconds) final = body.final @@ -406,10 +428,20 @@ def item_state( ) prev_status = st.status if st is not None else "new" if status == "new": + # Does Plex hold anything for this item that the un-mark has to clear? Two things qualify: a + # view count, and a resume point. An IN-PROGRESS item carries status "new" WITH a position — + # that is how the importer maps a bare `viewOffset` (`_plex_watch_to_state`) — so keying the + # push on the status alone silently skipped the single case "Clear resume point" exists for: + # the un-mark never reached Plex, the offset stayed, and the next pull imported it straight + # back (Protector, cleared twice by the user and back at 92s both times). `hidden` is + # Siftlode-only and still pushes nothing, exactly as this endpoint's docstring says. + mirrored_on_plex = st is not None and ( + st.status == "watched" or (st.position_seconds or 0) > 0 + ) if st is not None: db.delete(st) db.commit() - if prev_status != "new": # something → unwatched: mirror the un-mark to Plex + if mirrored_on_plex: _push_watch(background, db, user.id, it.id, rating_key, "unwatched") return {"status": "new"} if st is None: @@ -524,9 +556,10 @@ def season_state( @router.post("/stream/{rating_key}/session") def stream_session( rating_key: str, - start: float = 0.0, + # Finite: `start=inf` reached `int()` in the session builder and came back as a 500. + start: float = Query(0.0, allow_inf_nan=False), audio: int | None = None, - aoff: float = 0.0, + aoff: float = Query(0.0, allow_inf_nan=False), multi: bool = False, user: User = Depends(current_user), db: Session = Depends(get_db), @@ -536,7 +569,6 @@ def stream_session( HEVC/VP9/AV1/…) take the same HLS path but with the video re-encoded to H.264 (VAAPI or libx264). Selecting a non-default audio track (`audio` = stream ordinal) forces the HLS path so the chosen audio can be muxed in. Subtitles are independent WebVTT tracks (GET /subtitle) — no session restart.""" - plex_stream.reap_idle() # opportunistic cleanup of idle sessions on each new playback it = item_or_404(db, rating_key) transcode = it.playable == "transcode" # `multi` (item has ≥2 audio tracks) forces the HLS path so ALL audio tracks ship as renditions and @@ -560,7 +592,9 @@ def stream_session( return { "mode": "hls", "start_s": s.media_start_s, - "url": f"/api/plex/stream/{it.rating_key}/hls/{s.entry}", + # The session id is part of the path (not a query param) so the playlist's RELATIVE segment + # names resolve back into the same session — see `stream_hls_file` for why it has to. + "url": f"/api/plex/stream/{it.rating_key}/hls/{s.id}/{s.entry}", "transcode": s.transcode, "hw": s.hw, "tonemap": s.tonemap, @@ -592,45 +626,82 @@ def stream_file( return FileResponse(str(src)) # Starlette honours the Range header (206) +# What a session directory can contain, and nothing else. No `.vtt`: ffmpeg runs with `-sn` and +# subtitles are independent tracks served by GET /subtitle, so a WebVTT name here could only ever be +# a name that will not arrive — and accepting it cost a 20 s parked worker per request. _HLS_CT = { ".m3u8": "application/vnd.apple.mpegurl", ".ts": "video/mp2t", - ".vtt": "text/vtt", } +# A real artifact name is ~20 characters. The bound is what keeps an absurd one out of the parser: +# `int()` on an unbounded digit run raises ValueError past 4300 digits (CPython's +# `sys.get_int_max_str_digits`), which reached the client as a 500 — the same shape of bug the +# subtitle routes were bounded for (`_RatingKey`, above). +_HLS_NAME_MAX = 128 -@router.get("/stream/{rating_key}/hls/{filename}") +@router.get("/stream/{rating_key}/hls/{session_id}/{filename}") def stream_hls_file( rating_key: str, + session_id: str, filename: str, user: User = Depends(current_user), - db: Session = Depends(get_db), ) -> Response: - """Serve a file (playlist / segment / subtitle) from the current HLS session directory. The - playlist references its segments/subtitles by relative name, all resolving back here.""" + """Serve a file (playlist / segment / subtitle) from ONE HLS session's directory. The playlist + references its segments/subtitles by relative name, all resolving back here. + + The session id in the path is what makes "back here" mean the same session. Sessions are keyed by + rating_key alone and a seek-restart or a second viewer of the same item replaces the registered + one, while the DIRECTORY is per (key, offset, tag) — so resolving by rating_key handed the old + viewer the new session's `seg_12.ts`, produced from a different offset under `-copyts`: wrong + bytes with a 200, which the player has no way to notice (deep-review round 5). A request for a + session that is no longer the registered one is a 404, and the player reclaims at the playhead. + + That reclaim is also why a request with no live session is no longer auto-started here: the + auto-start could only ever begin at 0, silently throwing away the viewer's position.""" # Filename must be a plain hls artifact — no path separators, only the expected extensions. ext = "." + filename.rsplit(".", 1)[-1] if "." in filename else "" - if ext not in _HLS_CT or "/" in filename or "\\" in filename or ".." in filename: + if ( + len(filename) > _HLS_NAME_MAX + or ext not in _HLS_CT + or "/" in filename + or "\\" in filename + or ".." in filename + ): raise HTTPException(status_code=404, detail="Not found") s = plex_stream.current_session(str(rating_key)) - if s is None: - # No session yet (e.g. reaped): the ENTRY playlist may auto-start one at 0 — master.m3u8 for a - # multi-audio item, index.m3u8 for a single-audio remux item. - it = item_or_404(db, rating_key) - if filename == "master.m3u8": - s = plex_stream.start_session(db, it, 0.0, multi=True) - elif filename == "index.m3u8" and it.playable == "remux": - s = plex_stream.start_session(db, it, 0.0) - else: - raise HTTPException(status_code=404, detail="No active playback session") - if s is None: - raise HTTPException(status_code=404, detail="Media file not found on disk") + if s is None or s.id != session_id: + raise HTTPException(status_code=404, detail="No active playback session") path = s.dir / filename - timeout = 30 if ext == ".m3u8" else 20 - if not plex_stream.wait_for(path, timeout): + # Only a name THIS session can actually produce is worth waiting for: its own playlists, and a + # segment of one of its own renditions whose predecessor is already on disk (ffmpeg writes them + # in order, so if it is not there this one is not next). The session is passed, not just its + # directory, because the single- and multi-audio argv branches spell their names differently and + # neither can produce the other's. Answering now instead of waiting it out is what keeps a seek + # past the produced region — or a deliberate flood of hopeless names — from parking threadpool + # workers and their DB connections (see `artifact_may_arrive`). + if not plex_stream.artifact_may_arrive(s, filename): raise HTTPException(status_code=404, detail="Not available yet") + timeout = 30 if ext == ".m3u8" else 20 + # `proc` so a session whose ffmpeg has already exited answers at once instead of polling out the + # whole timeout: this is a SYNC route on the ~40-slot threadpool, holding a pooled DB connection + # with it, so every such wait is a worker (and a connection) parked for 20 s — an ordinary seek + # past the produced region was enough to do it (round 5). + if not plex_stream.wait_for(path, timeout, proc=s.proc): + raise HTTPException(status_code=404, detail="Not available yet") + try: + content = path.read_bytes() + except OSError: + # The session was torn down between the wait and the read — `_kill` rmtrees the whole + # directory, and nothing holds `_lock` across those two steps. It has to present as the same + # 404 a gone session does: the player's recovery is keyed on the STATUS (`code === 404`), so + # a 500 here skipped the reclaim and ended playback with "couldn't play" instead of + # restarting at the playhead. Ordinary trigger: `loadSession` awaits the new session — which + # kills and rmtrees the old one — before destroying the old hls instance, so the outgoing + # instance's in-flight segment request races the rmtree on every seek-restart. + raise HTTPException(status_code=404, detail="No active playback session") from None return Response( - content=path.read_bytes(), + content=content, media_type=_HLS_CT[ext], headers={"Cache-Control": "no-store"}, ) diff --git a/backend/app/routes/plex/playlists.py b/backend/app/routes/plex/playlists.py index 81739a34..022af3ae 100644 --- a/backend/app/routes/plex/playlists.py +++ b/backend/app/routes/plex/playlists.py @@ -25,7 +25,7 @@ router = APIRouter(prefix="/api/plex", tags=["plex"]) def _own_playlist_or_404(db: Session, pid: int, user: User) -> PlexPlaylist: - return owned_or_404(db, PlexPlaylist, pid, user, detail="Playlist not found") + return owned_or_404(db, PlexPlaylist, pid, user, what="playlist") def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict: diff --git a/backend/app/routes/saved_views.py b/backend/app/routes/saved_views.py index 4ac5b835..e2d6f68e 100644 --- a/backend/app/routes/saved_views.py +++ b/backend/app/routes/saved_views.py @@ -91,7 +91,7 @@ def create_view( def _own_view(db: Session, user: User, view_id: int) -> SavedView: - return owned_or_404(db, SavedView, view_id, user, detail="Unknown view") + return owned_or_404(db, SavedView, view_id, user, what="view") class UpdateViewIn(BaseModel): diff --git a/backend/app/routes/scheduler.py b/backend/app/routes/scheduler.py index b605fea6..c5a39258 100644 --- a/backend/app/routes/scheduler.py +++ b/backend/app/routes/scheduler.py @@ -23,6 +23,7 @@ from app.scheduler import ( trigger_job, ) from app.sync.runner import estimate_deep_backfill +from app.routes._common import not_found router = APIRouter(prefix="/api/admin/scheduler", tags=["admin"]) @@ -41,7 +42,7 @@ def set_job_interval( """Change how often a scheduler job runs (minutes). Persisted as an override and applied to the live scheduler immediately.""" if job_id not in JOB_INTERVALS: - raise HTTPException(status_code=404, detail="Unknown job") + raise not_found("job") minutes = body.interval_minutes if not (MIN_INTERVAL <= minutes <= MAX_INTERVAL): raise HTTPException( @@ -73,7 +74,7 @@ def run_job_now( try: result = trigger_job(job_id, actor_id=user.id) except KeyError: - raise HTTPException(status_code=404, detail="Unknown job") + raise not_found("job") if result == "already_running": raise HTTPException(status_code=409, detail="Job is already running") return {"id": job_id, "started": True} diff --git a/backend/app/routes/tags.py b/backend/app/routes/tags.py index 0000cb7c..d5b8c8d8 100644 --- a/backend/app/routes/tags.py +++ b/backend/app/routes/tags.py @@ -80,7 +80,7 @@ def create_tag( def _own_tag(db: Session, user: User, tag_id: int) -> Tag: # System tags (user_id NULL) are not user-editable → owned_or_404 404s them too. - return owned_or_404(db, Tag, tag_id, user, detail="Unknown tag") + return owned_or_404(db, Tag, tag_id, user, what="tag") class TagUpdateIn(BaseModel): diff --git a/backend/app/sysconfig.py b/backend/app/sysconfig.py index 1fa7802f..5ade8ac2 100644 --- a/backend/app/sysconfig.py +++ b/backend/app/sysconfig.py @@ -167,6 +167,8 @@ SPECS: tuple[ConfigSpec, ...] = ( ConfigSpec( "plex_transcode_max_height", "int", "plex", "plex_transcode_max_height", min=0, max=4320 ), + # Audio loudness normalisation on the HLS path (remux + transcode alike, not direct play). + ConfigSpec("plex_audio_loudnorm", "bool", "plex", "plex_audio_loudnorm"), # NOTE: plex_watch_sync_interval_min is a config.py default read by the watch-sync scheduler job # (scheduler.py); it is intentionally NOT a runtime admin knob, so it stays out of SPECS. # --- TMDB enrichment (E5 S2; optional) — the API key is the only secret --- diff --git a/backend/tests/test_demo_refusal_detail.py b/backend/tests/test_demo_refusal_detail.py new file mode 100644 index 00000000..f629f9cf --- /dev/null +++ b/backend/tests/test_demo_refusal_detail.py @@ -0,0 +1,90 @@ +"""The shared demo account's refusal says WHY, in the reader's language (deep-review round 6). + +`require_human` answered `403 "Not available in the demo account."` — a plain string. `req` leaves +403 to the caller, and round 5 taught `notifyActionError` to discard a plain-string detail in favour +of the caller's already-translated fallback, so from that commit on the demo user got a generic +"couldn't do that" and no hint that the demo account was the reason. Not every affordance behind +this gate is UI-gated on `is_demo` either — the comfort shelf is not, so the pin button is live. + +A structured `{code: "demo"}` is what `localizeDetail` can turn into `errors.demo` in both locales. +Pinned at all FOUR sites: the shared dependency, the two auth routes that hand-roll the same check, +and the account-delete route in `routes/me.py` — which round 6 missed while its docstring claimed +all the sites were covered, so the next reader had no reason to look (round 7). An asymmetric fix is +the smell this repo bans, and a docstring asserting symmetry it does not have is worse than none. + +The list is derived from the source, not typed out: `test_every_is_demo_refusal_is_on_the_list` +below fails when a fifth `is_demo` 403 appears anywhere in `app/`, so this file cannot fall behind +the code the way round 6's did. Needs the DB. +""" + +import re +from pathlib import Path + +from app.models import User + + +def test_every_demo_refusal_carries_the_same_structured_reason(db, make_user, as_user): + demo = make_user(email="demo@example.com", is_demo=True) + client = as_user(demo) + for method, url, body in [ + # `require_human` itself, through an affordance the UI does NOT gate on is_demo. + ("post", "/api/discover/pins", {"kind": "video", "ref": "v1"}), + # The three routes that repeat the check inline rather than depending on the gate. + ("post", "/auth/logout-others", None), + ("post", "/auth/set-password", {"password": "a long enough password"}), + ("delete", "/api/me/account", None), + ]: + r = getattr(client, method)(url, **({"json": body} if body is not None else {})) + assert r.status_code == 403, (url, r.text) + detail = r.json()["detail"] + # Every one of them is a dict whose `code` the frontend can localize; the delete route adds + # a `reason` so its refusal can be worded specifically. + assert isinstance(detail, dict), (url, detail) + assert detail["code"] == "demo", (url, detail) + + +def test_every_is_demo_refusal_is_on_the_list(): + """A fifth site must not be able to appear unnoticed. + + Round 6 converted three of four `is_demo` 403s and wrote a docstring saying it had converted + them all; the fourth kept answering a bare English string for another round because nothing + counted them. So count them here: every `is_demo`-guarded 403 in `app/` has to raise a + structured detail, and the number of them has to match what this file exercises.""" + app_dir = Path(__file__).resolve().parents[1] / "app" + sites = [] + for path in sorted(app_dir.rglob("*.py")): + lines = path.read_text(encoding="utf-8").splitlines() + for i, line in enumerate(lines): + # `if user.is_demo:` / `if u.is_demo:` — the hand-rolled gate, not the ORM filters + # (`User.is_demo.is_(False)`) and not the column definition. + if not re.search(r"^\s*if\s+\w+\.is_demo\s*:", line): + continue + block = "\n".join(lines[i : i + 6]) + if "403" not in block: + continue + sites.append(f"{path.relative_to(app_dir)}:{i + 1}") + assert 'detail={"code": "demo"' in block, ( + f"{path.relative_to(app_dir)}:{i + 1} refuses the demo account with an " + f"unstructured detail — the frontend cannot localize it:\n{block}" + ) + # auth.py's shared gate + routes/auth.py's two + routes/me.py's delete. A new one is welcome, + # but it has to be added to the loop above in the same commit. + assert len(sites) == 4, sites + + +def test_a_real_account_is_not_refused_by_the_gate(db, make_user, as_user): + """The other half: the structured detail must not come from having broken the gate open the + other way. A non-demo account gets past `require_human` and fails (or succeeds) on its own + merits — anything but a 403 demo refusal.""" + client = as_user(make_user(email="real@example.com")) + r = client.post("/auth/logout-others") + assert r.status_code == 200, r.text + assert not isinstance(r.json().get("detail"), dict) + + +def test_the_gate_reads_the_flag_and_not_the_email(db): + """`is_demo` is a column, not a convention about the address — a real account that happens to + be called demo@ must not be refused.""" + from app.auth import require_human + + assert require_human(User(email="demo@example.com", is_demo=False)) is not None diff --git a/backend/tests/test_hls_reaper.py b/backend/tests/test_hls_reaper.py new file mode 100644 index 00000000..667474cc --- /dev/null +++ b/backend/tests/test_hls_reaper.py @@ -0,0 +1,204 @@ +"""Idle HLS sessions are reaped on a timer, and a viewer who is merely PAUSED keeps theirs. + +plex/stream.py promises "a periodic reaper kills idle ones", but `reap_idle` ran only inside +POST /session: after the last playback, an abandoned session (its ffmpeg still working through the +film, its directory a whole segmented copy of it) lived until somebody pressed play again +(deep-review round 4). + +And it must not take a PAUSED viewer's session: a remux finishes long before the film does, and a +paused player with a full buffer fetches nothing, so "finished and unfetched for 30 s" does not mean +"abandoned". Round 4 bought that safety by keeping EVERY finished session for the full ten-minute +idle window — one segmented copy of a film per item, on the volume the download center shares. Round +5 replaced the blanket retention with the signal that already existed: an open player checkpoints its +position every 10 s whether or not it is playing, and that route says "still here". + +Both call sites are pinned here, because the bug each time was that only one of them moved: the +periodic pass, and the fact that starting a session no longer sweeps OTHER viewers' sessions at all. +""" + +import asyncio +import time +from types import SimpleNamespace + +import pytest +from fastapi.testclient import TestClient + +import app.main as app_main +import app.state as state +from app.auth import current_user +from app.db import get_db +from app.main import app +from app.models import PlexItem, PlexLibrary, User +from app.plex import paths as plex_paths +from app.plex import stream + + +class _Configured: + def __enter__(self): + return None + + def __exit__(self, *exc): + return False + + +def test_the_periodic_pass_is_the_one_that_runs_and_it_uses_the_short_rule(monkeypatch): + async def _no_sweep(): + return None + + reaped: list[int] = [] + monkeypatch.setattr(app_main, "SessionLocal", _Configured) + monkeypatch.setattr(app_main.state, "is_configured", lambda db: True) + monkeypatch.setattr(app_main, "_sweep_hls_orphans", _no_sweep) + monkeypatch.setattr(app_main, "start_scheduler", lambda: None) + monkeypatch.setattr(app_main, "shutdown_scheduler", lambda: None) + monkeypatch.setattr(app_main.plex_subtitles, "shutdown", lambda: None) + monkeypatch.setattr(app_main, "_HLS_REAP_EVERY_S", 0.01) + # `reap_idle`, i.e. the SHORT finished rule: a finished session nobody has said they still want + # is a whole copy of a film sitting on the download volume. + monkeypatch.setattr(app_main.plex_stream, "reap_idle", lambda: reaped.append(1) or 0) + + async def _cycle(): + async with app_main.lifespan(app): + await asyncio.sleep(0.2) + + asyncio.run(_cycle()) + assert len(reaped) >= 2, "no periodic reap ran while the app was up" + + +class _Proc: + def __init__(self, finished: bool): + self.finished = finished + self.killed = False + + def poll(self): + return 0 if self.finished else None + + def terminate(self): + self.killed = True + + def wait(self, timeout=None): + return 0 + + +class _Session: + def __init__(self, tmp_path, name: str, finished: bool, idle_s: float): + self.dir = tmp_path / name + self.dir.mkdir() + self.proc = _Proc(finished) + self.last_access = time.time() - idle_s + + +@pytest.fixture +def sessions(monkeypatch): + live: dict = {} + monkeypatch.setattr(stream, "_sessions", live) + return live + + +def test_the_periodic_pass_reaps_a_finished_session_nobody_has_claimed(tmp_path, sessions): + sessions["12345"] = _Session(tmp_path, "done", finished=True, idle_s=45) + assert stream.reap_idle() == 1 + assert sessions == {} + + +def test_the_periodic_pass_still_reaps_what_nobody_has_touched_for_the_idle_limit( + tmp_path, sessions +): + sessions["1"] = _Session( + tmp_path, "gone_done", finished=True, idle_s=stream._SESSION_IDLE_S + 5 + ) + sessions["2"] = _Session( + tmp_path, "gone_live", finished=False, idle_s=stream._SESSION_IDLE_S + 5 + ) + assert stream.reap_idle() == 2 + assert sessions == {} + + +def test_touching_a_session_keeps_it_through_the_short_rule(tmp_path, sessions): + s = _Session(tmp_path, "paused", finished=True, idle_s=90) + sessions["12345"] = s + stream.touch_session("12345") + assert stream.reap_idle() == 0 + assert "12345" in sessions and s.dir.exists() and not s.proc.killed + + +# --- the two call sites, driven through the real routes (DB lane) ------------------------------- + + +def _library_item(db, rating_key: str, *, playable: str = "remux"): + lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True) + db.add(lib) + db.flush() + it = PlexItem( + rating_key=rating_key, + library_id=lib.id, + kind="movie", + title="A Movie", + file_path="/media/a.mkv", + playable=playable, + ) + db.add(it) + db.commit() + return it + + +def _client(db, user): + app.dependency_overrides[get_db] = lambda: db + app.dependency_overrides[current_user] = lambda: user + return TestClient(app) + + +def test_a_paused_viewers_checkpoint_keeps_their_session_alive(db, monkeypatch, tmp_path, sessions): + """The whole reason the short rule is safe again: a paused player fetches nothing, so + `last_access` never moved on its own — but it does keep checkpointing its position.""" + monkeypatch.setattr(state, "_configured_cache", True) + it = _library_item(db, "paused1") + user = User(email="paused@example.com") + db.add(user) + db.commit() + s = _Session(tmp_path, "paused", finished=True, idle_s=90) + sessions["paused1"] = s + try: + resp = _client(db, user).post( + f"/api/plex/item/{it.rating_key}/progress", + json={"position_seconds": 600, "duration_seconds": 3000, "final": False}, + ) + finally: + app.dependency_overrides.clear() + assert resp.status_code == 200, resp.text + assert stream.reap_idle() == 0, "the checkpoint did not say the viewer is still there" + assert "paused1" in sessions and not s.proc.killed + + +def test_starting_a_session_leaves_another_viewers_paused_one_alone( + db, monkeypatch, tmp_path, sessions +): + """The opportunistic sweep this route used to run was not scoped to the key being started: it + walked the WHOLE registry with the 30 s finished rule, so pressing play on any item killed a + paused viewer's session — the very case round 4 set out to protect (found in round 5).""" + monkeypatch.setattr(state, "_configured_cache", True) + monkeypatch.setattr(plex_paths, "local_media_path", lambda db_, fp: "/local/a.mkv") + started = _library_item(db, "starting") + user = User(email="starter@example.com") + db.add(user) + db.commit() + paused = _Session(tmp_path, "paused", finished=True, idle_s=90) + sessions["someone-else"] = paused + + def fake_start(db_, item, start, audio=None, aoff=0.0, multi=False, transcode=False): + return SimpleNamespace( + id="sid", + media_start_s=start, + entry="index.m3u8", + transcode=transcode, + hw=False, + tonemap=False, + ) + + monkeypatch.setattr(stream, "start_session", fake_start) + try: + resp = _client(db, user).post(f"/api/plex/stream/{started.rating_key}/session") + finally: + app.dependency_overrides.clear() + assert resp.status_code == 200, resp.text + assert "someone-else" in sessions and not paused.proc.killed diff --git a/backend/tests/test_not_found_detail.py b/backend/tests/test_not_found_detail.py new file mode 100644 index 00000000..b7e9645e --- /dev/null +++ b/backend/tests/test_not_found_detail.py @@ -0,0 +1,148 @@ +"""A row that is gone answers with a reason the bilingual UI can translate (deep-review round 3). + +The 404 a user actually meets is the one for something deleted in another tab or on another device. +`owned_or_404` / `get_or_404` sent it as a bare English string ("Unknown view", "Not found"), and the +Hungarian UI toasted it verbatim. The structured-404 fix had gone into one ad-hoc helper instead of +the shared one, so every other owner check kept the English. Needs the DB. +""" + +from app.auth import require_write_scope +from app.models import DownloadJob, MediaAsset + + +def test_a_gone_view_tag_or_pin_says_what_is_gone(db, make_user, as_user): + client = as_user(make_user()) + for method, url, what in [ + ("patch", "/api/saved-views/999999", "view"), + ("delete", "/api/saved-views/999999", "view"), + ("delete", "/api/tags/999999", "tag"), + ("delete", "/api/discover/pins/999999", "pin"), + ]: + r = getattr(client, method)(url, **({"json": {"name": "x"}} if method == "patch" else {})) + assert r.status_code == 404, (url, r.text) + assert r.json()["detail"] == {"code": "not_found", "reason": what}, url + + +def test_a_gone_channel_video_or_user_says_what_is_gone(db, make_user, as_user): + """The siblings that still hand-wrote `db.get(...) is None → 404 "Unknown …"` (deep-review + round 4): the same gone row, but the Hungarian UI showed the English.""" + admin = make_user(role="admin") + client = as_user(admin) + for method, url, body, what in [ + ("post", "/api/channels/UCgone000000000000000000/block", None, "channel"), + ("post", "/api/videos/gone0000000/state", {"status": "watched"}, "video"), + ( + "post", + "/api/videos/gone0000000/progress", + {"position_seconds": 60, "duration_seconds": 600}, + "video", + ), + ("put", "/api/admin/downloads/quota/999999", {"max_bytes": 1}, "user"), + ]: + r = getattr(client, method)(url, **({"json": body} if body is not None else {})) + assert r.status_code == 404, (url, r.text) + assert r.json()["detail"] == {"code": "not_found", "reason": what}, url + + +def test_every_gone_channel_route_says_it_the_same_way(db, make_user, as_user, override_dep): + """The round-4 sweep converted ONE of the five `db.get(Channel, …) is None -> 404 "Unknown + channel"` sites (block), and shipped the HU/EN string with it — so the Hungarian UI said it in + Hungarian on the Block button and in English on the four beside it (deep-review round 5). The + asymmetric fix is the smell this repo bans, so all five are pinned together. + + Literally all five: round 6 found this loop issuing three requests under a docstring claiming + five, with explore silently absent and subscribe excused as "covered by the tags route" — which + reaches `_user_subscription`, a different site answering `reason: subscription`. Reverting + either of those two to its bare English string left the whole lane green.""" + admin = make_user(role="admin") + client = as_user(admin) + # Subscribe's own gate (the optional YouTube write scope) sits in FRONT of the lookup, so a + # plain account 403s before reaching it. The gate is not what is under test here. + override_dep(require_write_scope, lambda: admin) + gone = "UCgone000000000000000000" + for method, url in [ + ("get", f"/api/channels/{gone}"), + ("post", f"/api/channels/{gone}/explore"), + ("post", f"/api/channels/{gone}/reset-backfill"), + ("post", f"/api/channels/{gone}/subscribe"), + ("post", f"/api/channels/{gone}/block"), + ]: + r = getattr(client, method)(url) + assert r.status_code == 404, (url, r.text) + assert r.json()["detail"] == {"code": "not_found", "reason": "channel"}, url + + +def test_a_gone_subscription_recipient_or_job_says_what_is_gone(db, make_user, as_user): + """The three bare-English 404s the same sweep left behind, which `notifyActionError` printed + verbatim into an otherwise Hungarian UI: "Not subscribed to this channel", "No user with that + email.", "Unknown job". + + All three, with a request each. Round 6 found this test naming three and issuing two: the + recipient site had no test in the whole suite (nor did the `errors.notFound.recipient` keys + added beside it), and `run_job_now` rode on its PATCH sibling's coverage.""" + admin = make_user(role="admin") + client = as_user(admin) + r = client.post("/api/channels/UCgone000000000000000000/tags", json={"tag_id": 1}) + assert r.status_code == 404, r.text + assert r.json()["detail"] == {"code": "not_found", "reason": "subscription"} + for method, url in [ + ("patch", "/api/admin/scheduler/jobs/no_such_job"), + ("post", "/api/admin/scheduler/jobs/no_such_job/run"), + ]: + kw = {"json": {"interval_minutes": 30}} if method == "patch" else {} + r = getattr(client, method)(url, **kw) + assert r.status_code == 404, (url, r.text) + assert r.json()["detail"] == {"code": "not_found", "reason": "job"}, url + # Sharing needs a finished download of one's own to share; the recipient lookup is the next check. + asset = MediaAsset( + source_kind="url", source_ref="share-404", format_sig="sig", status="ready", ref_count=1 + ) + db.add(asset) + db.flush() + job = DownloadJob( + user_id=admin.id, + asset_id=asset.id, + source_kind="url", + source_ref="share-404", + status="done", + queue_pos=1, + profile_snapshot={}, + ) + db.add(job) + db.commit() + r = client.post(f"/api/downloads/{job.id}/share", json={"email": "nobody@example.com"}) + assert r.status_code == 404, r.text + assert r.json()["detail"] == {"code": "not_found", "reason": "recipient"} + + +def test_a_nul_in_a_request_string_is_a_422_not_a_500(db, make_user, as_user): + """Postgres cannot hold NUL in text, so the driver refuses the query outright: a DataError, i.e. + a 500 with a traceback, wherever a `%00` or an escaped NUL reached a comparison (deep-review + round 4 — round 3 had guarded only the by-rating_key lookup). One app-wide handler now answers + all of them.""" + client = as_user(make_user()) + r = client.get("/api/plex/playlists?contains=%00") + assert r.status_code == 422, r.text + assert r.json()["detail"] == {"code": "invalid_input", "reason": "nul"} + assert client.get("/api/plex/playlists?contains_group=1,%00").status_code == 422 + + +def test_any_other_data_error_is_still_a_500(): + """The other half of the app-wide handler, and the half a refactor can delete in silence: it is + registered for EVERY DataError on every route and decides on a substring of the driver's message, + so only a NUL may become a 422. Without this, simplifying it to an unconditional 422 — it reads + like dead defensive code — would answer a numeric overflow or an over-long ingested title with + "your input was bad", losing the traceback that names the real fault (deep-review round 5).""" + import asyncio + + import pytest + from sqlalchemy.exc import DataError + + from app.main import _nul_in_text + + overflow = DataError("SELECT 1", {}, Exception("numeric field overflow")) + with pytest.raises(DataError): + asyncio.run(_nul_in_text(None, overflow)) # re-raised: it stays the 500 it always was + + nul = DataError("SELECT 1", {}, Exception("invalid byte sequence: NUL (0x00)")) + assert asyncio.run(_nul_in_text(None, nul)).status_code == 422 diff --git a/backend/tests/test_playlist_move.py b/backend/tests/test_playlist_move.py index 8590c2a8..9a936d32 100644 --- a/backend/tests/test_playlist_move.py +++ b/backend/tests/test_playlist_move.py @@ -128,6 +128,60 @@ def test_move_into_someone_elses_list_404s_and_changes_nothing(db, make_user, as assert _ids(db, theirs.id) == [] +def test_a_list_that_is_not_yours_answers_with_a_reason_the_ui_can_translate( + db, make_user, as_user +): + # The one 404 a user meets in practice (the list went away in another tab). It used to be a bare + # English "Unknown playlist", which the Hungarian UI then showed verbatim. + user, other = make_user(), make_user() + _seed_videos(db, ["v1"]) + theirs = _playlist(db, other, "Theirs", ["v1"]) + + r = as_user(user).delete(f"/api/playlists/{theirs.id}/items/v1") + assert r.status_code == 404 + assert r.json()["detail"] == {"code": "not_found", "reason": "playlist"} + db.expire_all() + assert _ids(db, theirs.id) == ["v1"] + + +def test_a_move_into_a_list_that_is_gone_names_the_target_not_the_list_on_screen( + db, make_user, as_user +): + # Both lists 404'd with the same reason, which the UI reads as "THIS playlist doesn't exist any + # more" — said about the source the reader is looking at, which is still there. + user = make_user() + _seed_videos(db, ["v1"]) + src = _playlist(db, user, "Mine", ["v1"]) + gone = _playlist(db, user, "Gone", []) + gone_id = gone.id + db.delete(gone) + db.commit() + + r = _move(as_user(user), src.id, "v1", gone_id) + assert r.status_code == 404 + assert r.json()["detail"] == {"code": "not_found", "reason": "move_target"} + db.expire_all() + assert _ids(db, src.id) == ["v1"] + + +def test_a_refused_playlist_action_answers_with_a_reason_the_ui_can_translate( + db, make_user, as_user +): + # The rest of the router's refusals were bare English too ("The Watch later list can't be + # deleted.", "YouTube delete failed.", ...), shown verbatim under the Hungarian UI. + user = make_user() + client = as_user(user) + wl = client.post("/api/playlists/watch-later", json={"video_id": "nope"}) + assert wl.status_code == 404 + assert wl.json()["detail"] == {"code": "not_found", "reason": "video"} + _seed_videos(db, ["v1"]) + saved = client.post("/api/playlists/watch-later", json={"video_id": "v1"}) + wl_id = saved.json()["playlist_id"] + r = client.delete(f"/api/playlists/{wl_id}") + assert r.status_code == 400 + assert r.json()["detail"] == {"code": "playlist", "reason": "watch_later_undeletable"} + + def test_move_marks_both_linked_sides_dirty(db, make_user, as_user): """A move edits two YouTube mirrors at once — the source lost an item, the target gained one. Marking only the source is the asymmetric fix this repo keeps re-learning.""" diff --git a/backend/tests/test_plex_clear_resume_push.py b/backend/tests/test_plex_clear_resume_push.py new file mode 100644 index 00000000..b088a84d --- /dev/null +++ b/backend/tests/test_plex_clear_resume_push.py @@ -0,0 +1,83 @@ +""" "Clear resume point" has to reach Plex — the case the status-only guard skipped. + +`POST /api/plex/item/{rk}/state {"status": "new"}` backs the *Clear resume point* / *Mark unwatched* +buttons. It deletes the local row and mirrors the un-mark to Plex — but it used to decide whether to +push from the PREVIOUS STATUS alone (`prev_status != "new"`), and an in-progress item's row is +`status="new"` carrying a position: that is how the importer maps a bare `viewOffset` +(`_plex_watch_to_state`). So for the one case the button exists for, nothing was ever sent to Plex, +Plex kept the offset, and the next pull imported it straight back — the user cleared *Protector* +twice and it returned to Continue watching at 92s both times (2026-09-09). + +One `unscrobble` clears the view count AND the offset (measured against the live server), so the +push itself needed no change; what needed fixing is WHEN it is scheduled. + +DB lane (`db` fixture) — skipped when no Postgres is reachable. +""" + +from types import SimpleNamespace + +from fastapi.testclient import TestClient + +import app.state as state +from app.auth import current_user +from app.db import get_db +from app.main import app +from app.models import PlexItem, PlexLibrary, PlexState, User +from app.plex import watch_sync as plex_watch + + +def _seed(db, *, status: str, position: int): + lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True) + db.add(lib) + db.flush() + it = PlexItem( + rating_key="98614", library_id=lib.id, kind="movie", title="Protector", duration_s=5495 + ) + db.add(it) + user = User(email="clear@example.com") + db.add(user) + db.flush() + db.add(PlexState(user_id=user.id, item_id=it.id, status=status, position_seconds=position)) + db.commit() + return it, user + + +def _clear(db, user, it, monkeypatch) -> list[tuple]: + """Drive the endpoint with the Plex link satisfied and the background push recorded.""" + pushes: list[tuple] = [] + monkeypatch.setattr(state, "_configured_cache", True) # pass the instance-setup request gate + monkeypatch.setattr(plex_watch, "link_for_push", lambda db_, uid: SimpleNamespace(id=1)) + monkeypatch.setattr( + plex_watch, + "push_state_to_plex", + lambda user_id, item_id, rating_key, action, *rest: pushes.append((rating_key, action)), + ) + app.dependency_overrides[get_db] = lambda: db + app.dependency_overrides[current_user] = lambda: user + try: + resp = TestClient(app).post(f"/api/plex/item/{it.rating_key}/state", json={"status": "new"}) + assert resp.status_code == 200, resp.text + finally: + app.dependency_overrides.clear() + return pushes + + +def test_clearing_an_in_progress_item_tells_plex(db, monkeypatch): + # The regression: status "new" + a position IS the in-progress shape, and it must push. + it, user = _seed(db, status="new", position=92) + assert _clear(db, user, it, monkeypatch) == [("98614", "unwatched")] + assert db.query(PlexState).count() == 0 + + +def test_clearing_a_watched_item_still_tells_plex(db, monkeypatch): + # The case the old guard did cover — kept, so the fix cannot narrow into a new hole. + it, user = _seed(db, status="watched", position=0) + assert _clear(db, user, it, monkeypatch) == [("98614", "unwatched")] + + +def test_unhiding_touches_plex_not_at_all(db, monkeypatch): + # `hidden` is a Siftlode-only concept "with no Plex analogue, so it never touches Plex" — the + # endpoint's own docstring. The old status-only guard pushed here anyway, un-marking an item on + # Plex that the user had merely hidden. + it, user = _seed(db, status="hidden", position=0) + assert _clear(db, user, it, monkeypatch) == [] diff --git a/backend/tests/test_plex_collection_sort.py b/backend/tests/test_plex_collection_sort.py new file mode 100644 index 00000000..431e69f0 --- /dev/null +++ b/backend/tests/test_plex_collection_sort.py @@ -0,0 +1,291 @@ +"""The Collection info page's member ordering (`GET /api/plex/collections/{rk}?sort=&sort_dir=`) and +the `keys=` lookup that gives the sidebar token / filter chip a title instead of a raw rating_key. + +Reported from daily use: "Collectionben nem működik a rendezés". The browse grid sorts fine — the +collection page had no control at all and a hardcoded year-ascending order. Adding the control means +the SAME vocabulary must actually work here, hence the pin against `PLEX_SORT_KEYS`. + +Pure-logic tests run everywhere; the endpoint tests use the `db` fixture (DB lane). +""" + +from datetime import date, datetime, timedelta, timezone + +from fastapi.testclient import TestClient +from sqlalchemy import literal_column + +import app.state as state +from app.auth import current_user +from app.db import get_db +from app.main import app +from app.models import PlexCollection, PlexItem, PlexLibrary, PlexShow, PlexState, User +from app.plex.query import PLEX_SORT_KEYS +from app.routes.plex.browse import sort_column +from app.routes.plex.collections import ( + DEFAULT_MEMBER_SORT, + MEMBER_SORT_KEYS, + Member, + sort_members, +) + +NOW = datetime(2024, 6, 1, tzinfo=timezone.utc) + + +def _m(title: str, **over) -> Member: + base = dict( + kind="movie", + obj=None, + title=title, + year=None, + rating=None, + added=None, + release=None, + duration=None, + last_watched=None, + ) + base.update(over) + return Member(**base) # type: ignore[arg-type] + + +def _titles(members: list[Member]) -> list[str]: + return [m.title for m in members] + + +# --- the vocabulary is ONE vocabulary --------------------------------------------------------- + + +def test_every_sort_key_is_orderable_on_a_collection(): + """The collection page shows the same Sort control as the browse grid, so a key it cannot order + would be an option that silently does nothing.""" + assert set(MEMBER_SORT_KEYS) == set(PLEX_SORT_KEYS) + + +def test_every_sort_key_is_orderable_on_the_browse_grid(): + """The other half of the same pin: browse falls back to `added_at` for an unknown key, so a key + missing from its map would order by date while the control claimed otherwise.""" + + class _Cols: + title = literal_column("title") + year = literal_column("year") + rating = literal_column("rating") + duration_s = literal_column("duration_s") + rel = literal_column("rel") + last_watched = literal_column("last_watched") + added_at = literal_column("added_at") + + class _U: + c = _Cols() + + u = _U() + fallback = u.c.added_at + unhandled = [k for k in PLEX_SORT_KEYS if k != "added" and sort_column(u, k) is fallback] + assert unhandled == [] + # …and the fallback itself is still what an unknown key gets. + assert sort_column(u, "no-such-key") is fallback + + +# --- ordering behaviour ----------------------------------------------------------------------- + + +def test_orders_by_the_requested_key_in_both_directions(): + members = [_m("b", year=2006), _m("c", year=2017), _m("a", year=2003)] + assert _titles(sort_members(members, "year", "asc")) == ["a", "b", "c"] + assert _titles(sort_members(members, "year", "desc")) == ["c", "b", "a"] + + +def test_members_without_a_value_sink_to_the_end_in_both_directions(): + """A movie with no rating is not "the worst rated" — nullslast, like the browse grid.""" + members = [_m("rated-low", rating=6.5), _m("unrated"), _m("rated-high", rating=8.1)] + assert _titles(sort_members(members, "rating", "desc")) == [ + "rated-high", + "rated-low", + "unrated", + ] + assert _titles(sort_members(members, "rating", "asc")) == ["rated-low", "rated-high", "unrated"] + + +def test_ties_stay_title_ascending_when_the_direction_flips(): + """Flipping the direction must reverse the RANKING, not the alphabet inside a tie.""" + members = [_m("zulu", year=2003), _m("alpha", year=2003), _m("mike", year=2017)] + assert _titles(sort_members(members, "year", "asc")) == ["alpha", "zulu", "mike"] + assert _titles(sort_members(members, "year", "desc")) == ["mike", "alpha", "zulu"] + + +def test_an_unknown_key_falls_back_to_the_pages_default(): + members = [_m("b", year=2006), _m("a", year=2017)] + assert _titles(sort_members(members, "no-such-key", "asc")) == _titles( + sort_members(members, DEFAULT_MEMBER_SORT, "asc") + ) + + +def test_a_show_has_no_duration_and_therefore_sinks_on_that_key(): + members = [ + _m("short movie", duration=3600), + _m("a show", kind="show", duration=None), + _m("long movie", duration=9000), + ] + assert _titles(sort_members(members, "duration", "desc")) == [ + "long movie", + "short movie", + "a show", + ] + + +# --- the endpoint (DB lane) ------------------------------------------------------------------- + + +def _seed(db): + lib_m = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True) + lib_s = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True) + db.add_all([lib_m, lib_s]) + db.flush() + + col = PlexCollection(rating_key="col1", library_id=lib_m.id, title="Franchise", child_count=3) + other = PlexCollection(rating_key="col2", library_id=lib_m.id, title="Elsewhere", child_count=1) + db.add_all([col, other]) + db.flush() + + # Titles deliberately alphabetical-opposite to the years, so a year sort can't accidentally pass + # by falling through to the title tiebreak. + old = PlexItem( + rating_key="mo_old", + library_id=lib_m.id, + kind="movie", + title="Zulu", + year=2003, + rating=8.1, + duration_s=8000, + added_at=NOW - timedelta(days=1), + originally_available_at=date(2003, 7, 9), + collection_keys=["col1"], + ) + mid = PlexItem( + rating_key="mo_mid", + library_id=lib_m.id, + kind="movie", + title="Mike", + year=2006, + rating=7.4, + duration_s=9000, + added_at=NOW - timedelta(days=3), + originally_available_at=date(2006, 7, 7), + collection_keys=["col1"], + ) + new = PlexItem( + rating_key="mo_new", + library_id=lib_m.id, + kind="movie", + title="Alpha", + year=2017, + rating=6.5, + duration_s=7000, + added_at=NOW, + originally_available_at=date(2017, 5, 26), + collection_keys=["col1"], + ) + # A show member with a resumed episode: `lastwatched` must work on this leg too, not just movies. + show = PlexShow( + rating_key="sh1", + library_id=lib_s.id, + title="Series", + year=2010, + child_count=1, + collection_keys=["col1"], + ) + db.add_all([old, mid, new, show]) + db.flush() + ep = PlexItem( + rating_key="sh1_e1", + library_id=lib_s.id, + kind="episode", + title="E1", + show_id=show.id, + season_number=1, + episode_number=1, + ) + db.add(ep) + db.flush() + + user = User(email="colsort@example.com") + db.add(user) + db.flush() + db.add_all( + [ + # The show's episode is the most recently touched thing in the collection… + PlexState( + user_id=user.id, + item_id=ep.id, + status="new", + position_seconds=60, + progress_updated_at=NOW, + ), + # …and one movie was watched earlier. + PlexState( + user_id=user.id, + item_id=old.id, + status="watched", + watched_at=NOW - timedelta(days=2), + ), + ] + ) + db.commit() + return user + + +def _get(db, user, monkeypatch, path): + monkeypatch.setattr(state, "_configured_cache", True) + app.dependency_overrides[get_db] = lambda: db + app.dependency_overrides[current_user] = lambda: user + try: + return TestClient(app).get(path) + finally: + app.dependency_overrides.clear() + + +def _order(r) -> list[str]: + return [i["title"] for i in r.json()["items"]] + + +def test_collection_detail_defaults_to_release_year_ascending(db, monkeypatch): + """The order the page had before it grew a control — a franchise reads in release order.""" + user = _seed(db) + r = _get(db, user, monkeypatch, "/api/plex/collections/col1") + assert r.status_code == 200 + assert _order(r) == ["Zulu", "Mike", "Series", "Alpha"] + + +def test_collection_detail_honours_sort_and_direction(db, monkeypatch): + user = _seed(db) + r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=year&sort_dir=desc") + assert _order(r) == ["Alpha", "Series", "Mike", "Zulu"] + r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=title&sort_dir=asc") + assert _order(r) == ["Alpha", "Mike", "Series", "Zulu"] + r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=rating&sort_dir=desc") + # The show has no rating, so it sinks — it is not "rated 0". + assert _order(r) == ["Zulu", "Mike", "Alpha", "Series"] + + +def test_lastwatched_ranks_a_shows_episodes_alongside_the_movies(db, monkeypatch): + """The show leg needs the rollup's `resumed_at`; with only a status map this key would order + every show as never-watched.""" + user = _seed(db) + r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=lastwatched&sort_dir=desc") + # Series (episode resumed NOW) > Zulu (watched 2 days ago) > the two never-touched, title-asc. + assert _order(r) == ["Series", "Zulu", "Alpha", "Mike"] + + +def test_collections_list_resolves_titles_by_key(db, monkeypatch): + """What a shared Plex link needs: the URL carries bare rating_keys, so the sidebar token and the + filter chip have nothing to show but `17045` until these are resolved.""" + user = _seed(db) + r = _get(db, user, monkeypatch, "/api/plex/collections?keys=col1") + assert r.status_code == 200 + assert [c["title"] for c in r.json()["collections"]] == ["Franchise"] + + +def test_an_empty_keys_param_asks_for_nothing(db, monkeypatch): + """`keys=` with nothing in it must not fall through to "every collection" — that would be a + 200 KB answer to a question about zero ids.""" + user = _seed(db) + r = _get(db, user, monkeypatch, "/api/plex/collections?keys=") + assert r.status_code == 200 + assert r.json()["collections"] == [] diff --git a/backend/tests/test_plex_collections_filter.py b/backend/tests/test_plex_collections_filter.py index 67707360..36668044 100644 --- a/backend/tests/test_plex_collections_filter.py +++ b/backend/tests/test_plex_collections_filter.py @@ -155,6 +155,7 @@ def test_create_collection_rejects_duplicate_name(db, monkeypatch): json={"library": "1", "title": "marvel", "item_rating_key": "m1"}, ) assert r.status_code == 409, r.text + assert r.json()["detail"] == {"code": "collection", "reason": "name_taken"} finally: app.dependency_overrides.clear() diff --git a/backend/tests/test_plex_hls_file_route.py b/backend/tests/test_plex_hls_file_route.py new file mode 100644 index 00000000..81fb645c --- /dev/null +++ b/backend/tests/test_plex_hls_file_route.py @@ -0,0 +1,322 @@ +"""An HLS artifact request names the session it belongs to, and gets nothing else. + +Sessions are keyed by rating_key ALONE, while a session's directory is per (key, offset, tag). So a +seek-restart — or a second viewer of the same item — replaces the registered session, and the old +viewer's next request for `seg_12.ts`, resolved by rating_key, was answered 200 with the NEW +session's twelfth segment: bytes produced from a different `-ss` offset under `-copyts`, i.e. a +timeline the client is not on. Wrong content with a success status is the one failure a player cannot +notice, and the reclaim-on-404 added in round 4 could never fire on it (deep-review round 5). + +No `db` fixture: the route reads nothing from the database (its own `Depends(get_db)` went with the +removed auto-start, and `current_user` is overridden wholesale), so these must not sit in the DB +lane — there they SKIP on a machine with no Postgres while pytest prints green (round 6). +""" + +import time + +import pytest +from fastapi.testclient import TestClient + +import app.state as state +from app.auth import current_user +from app.main import app +from app.models import User +from app.plex import stream as plex_stream + + +class _Proc: + def __init__(self, running: bool = False): + self.running = running + + def poll(self): + return None if self.running else 0 + + def terminate(self): + pass + + def wait(self, timeout=None): + return 0 + + +class _Session: + def __init__(self, session_id: str, directory, running: bool = False, audio_langs=None): + self.id = session_id + self.dir = directory + self.proc = _Proc(running) + self.last_access = time.time() + # The real session derives both of these from the argv it spawned; the guard reads + # `variants` to tell a name this session can produce from one only the OTHER argv branch + # writes, so a stand-in without it would test a shape that cannot exist. + self.variants = plex_stream.session_variants(audio_langs) + self.entry = "master.m3u8" if self.variants else "index.m3u8" + + +@pytest.fixture +def sessions(monkeypatch): + live: dict = {} + monkeypatch.setattr(plex_stream, "_sessions", live) + return live + + +@pytest.fixture +def client(monkeypatch): + monkeypatch.setattr(state, "_configured_cache", True) + user = User(email="hls@example.com") + app.dependency_overrides[current_user] = lambda: user + try: + yield TestClient(app) + finally: + app.dependency_overrides.pop(current_user, None) + + +def test_a_segment_is_served_from_the_session_the_url_names(client, tmp_path, sessions): + d = tmp_path / "m1_0_0_+0.00" + d.mkdir() + (d / "seg_11.ts").write_bytes(b"") # its predecessor, so seg_12 is the next one + (d / "seg_12.ts").write_bytes(b"mine") + sessions["m1"] = _Session("sid-a", d) + r = client.get("/api/plex/stream/m1/hls/sid-a/seg_12.ts") + assert r.status_code == 200, r.text + assert r.content == b"mine" + + +def test_a_segment_of_a_replaced_session_is_a_404_not_the_new_sessions_bytes( + client, tmp_path, sessions +): + """Viewer A is at 0:00; viewer B seeks to 25:00 and their session takes the registry slot. A's + next segment must fail honestly — the player reclaims at the playhead — not arrive as B's.""" + theirs = tmp_path / "m1_1500_0_+0.00" + theirs.mkdir() + (theirs / "seg_11.ts").write_bytes(b"") + (theirs / "seg_12.ts").write_bytes(b"theirs") + sessions["m1"] = _Session("sid-b", theirs) + r = client.get("/api/plex/stream/m1/hls/sid-a/seg_12.ts") + assert r.status_code == 404, r.text + assert b"theirs" not in r.content + + +def test_a_playlist_with_no_live_session_is_a_404_not_a_silent_restart_at_zero( + client, monkeypatch, sessions +): + """The entry playlist used to auto-start a session when none was registered — always at 0.0, + silently throwing away the viewer's position. A 404 sends the player through its own reclaim, + which restarts at the playhead.""" + started: list[float] = [] + monkeypatch.setattr( + plex_stream, + "start_session", + lambda *a, **k: started.append(a[2] if len(a) > 2 else 0.0), + ) + r = client.get("/api/plex/stream/m1/hls/sid-gone/index.m3u8") + assert r.status_code == 404, r.text + assert started == [] + + +def test_a_segment_that_will_never_arrive_fails_at_once(client, tmp_path, sessions): + """`stream_hls_file` is a SYNC route: it runs on the ~40-slot threadpool and holds a pooled DB + connection while it waits. It used to poll out the whole 20 s for a name that could never appear + — an ordinary seek past the produced region was enough to park a worker and a connection, and + forty hopeless names parked the API (deep-review round 5). ffmpeg writes segments in order, so a + segment whose predecessor is absent is not the next one, and that is knowable at once — even + while ffmpeg is still running, which is the case `proc` alone cannot shorten.""" + d = tmp_path / "m1_0_0_+0.00" + d.mkdir() + (d / "seg_0.ts").write_bytes(b"x") # the session is producing; seg_99999 is simply not next + sessions["m1"] = _Session("sid-a", d, running=True) + started = time.monotonic() + r = client.get("/api/plex/stream/m1/hls/sid-a/seg_99999.ts") + elapsed = time.monotonic() - started + assert r.status_code == 404, r.text + # The wait is 20 s; anything in that neighbourhood means the producer's exit was not consulted. + assert elapsed < 2.0, f"waited {elapsed:.1f}s for a segment that can never arrive" + + +@pytest.mark.parametrize( + "name", + [ + "seg_0_9000.ts", # the VIDEO rendition of a multi-audio session: variant 0, segment 9000 + "seg_a0_9000.ts", # the first audio rendition: `name:a0` from -var_stream_map + ], +) +def test_a_multi_audio_segment_beyond_the_region_fails_at_once_too( + client, tmp_path, sessions, name +): + """The round-5 guard read `seg__.ts`, but ffmpeg substitutes `%v` FIRST — so the + real names are `seg__.ts` and every multi-audio request fell through to the full + 20 s wait: the video rendition because its variant "0" parsed as segment 0, the audio renditions + (`a0`, `a1`) because they did not parse at all. Dual-audio is the common shape in this library, + so the guard was inert exactly where it was needed (round 6).""" + d = tmp_path / "m1_0_multi_+0.00" + d.mkdir() + (d / "seg_0_0.ts").write_bytes(b"x") + (d / "seg_a0_0.ts").write_bytes(b"x") + sessions["m1"] = _Session("sid-a", d, running=True, audio_langs=["hu", "en"]) + started = time.monotonic() + r = client.get(f"/api/plex/stream/m1/hls/sid-a/{name}") + elapsed = time.monotonic() - started + assert r.status_code == 404, r.text + assert elapsed < 2.0, f"waited {elapsed:.1f}s for {name}" + + +def test_a_multi_audio_segment_whose_predecessor_exists_is_served(client, tmp_path, sessions): + """The other half of the same rule: reading the names correctly must not start refusing real + ones. `seg_a0_1.ts` follows `seg_a0_0.ts` — the previous SEGMENT of that rendition, not the + previous variant.""" + d = tmp_path / "m1_0_multi_+0.00" + d.mkdir() + (d / "seg_a0_0.ts").write_bytes(b"x") + (d / "seg_a0_1.ts").write_bytes(b"audio") + sessions["m1"] = _Session("sid-a", d, running=True, audio_langs=["hu", "en"]) + r = client.get("/api/plex/stream/m1/hls/sid-a/seg_a0_1.ts") + assert r.status_code == 200, r.text + assert r.content == b"audio" + + +def test_a_name_the_session_cannot_produce_fails_at_once(client, tmp_path, sessions): + """The guard was a blocklist: anything that was not a single-audio segment name was waved + through to the timeout, so `GET …/anything.m3u8` still parked a worker for the full 30 s — and + the route comment claimed a flood of hopeless names was what it prevented. A session produces + its playlists and its segments and nothing else, so any other name is answerable now.""" + d = tmp_path / "m1_0_0_+0.00" + d.mkdir() + sessions["m1"] = _Session("sid-a", d, running=True) + started = time.monotonic() + r = client.get("/api/plex/stream/m1/hls/sid-a/nope.m3u8") + elapsed = time.monotonic() - started + assert r.status_code == 404, r.text + assert elapsed < 2.0, f"waited {elapsed:.1f}s for a playlist this session never writes" + + +@pytest.mark.parametrize("digits", [100, 5000]) +def test_an_absurd_segment_index_is_a_404_not_a_500(client, tmp_path, sessions, digits): + """`int()` on an unbounded digit run raises ValueError past 4300 digits, and only OSError was + caught — so a long enough name reached the client as a 500 with a traceback. Same bug class the + subtitle routes were bounded for; the answer is the same 404 any other unproducible name gets.""" + d = tmp_path / "m1_0_0_+0.00" + d.mkdir() + sessions["m1"] = _Session("sid-a", d, running=True) + r = client.get(f"/api/plex/stream/m1/hls/sid-a/seg_{'9' * digits}.ts") + assert r.status_code == 404, r.text + + +def test_a_session_torn_down_during_the_read_is_a_404_not_a_500( + client, monkeypatch, tmp_path, sessions +): + """`_kill` rmtrees the whole directory and nothing holds `_lock` across the wait and the read, + so the file can vanish in between — on every seek-restart, since `loadSession` awaits the new + session before destroying the old hls instance. The player's recovery is keyed on the 404 + STATUS, so a 500 here skipped the reclaim and ended playback with "couldn't play".""" + d = tmp_path / "m1_0_0_+0.00" + d.mkdir() + (d / "seg_11.ts").write_bytes(b"") + sessions["m1"] = _Session("sid-a", d) + # The file existed when the wait checked, and is gone by the time the route reads it. + monkeypatch.setattr(plex_stream, "wait_for", lambda *a, **k: True) + r = client.get("/api/plex/stream/m1/hls/sid-a/seg_12.ts") + assert r.status_code == 404, r.text + + +def test_a_dead_producer_is_not_waited_out(client, tmp_path, sessions): + """The route passes `proc` to `wait_for`. Without it a session whose ffmpeg has already exited + — a finished remux, a VAAPI init refusal — polls the whole 20 s for a segment that will never + be written, holding a threadpool worker and its pooled DB connection. The predecessor IS on + disk here, so `artifact_may_arrive` says yes and only `proc` can end the wait.""" + d = tmp_path / "m1_0_0_+0.00" + d.mkdir() + (d / "seg_5.ts").write_bytes(b"x") + sessions["m1"] = _Session("sid-a", d, running=False) + started = time.monotonic() + r = client.get("/api/plex/stream/m1/hls/sid-a/seg_6.ts") + elapsed = time.monotonic() - started + assert r.status_code == 404, r.text + assert elapsed < 2.0, f"waited {elapsed:.1f}s on a producer that had already exited" + + +@pytest.mark.parametrize("audio_langs", [None, ["hu", "en"]]) +def test_the_name_ffmpeg_is_asked_for_is_a_name_the_guard_understands(tmp_path, audio_langs): + """One vocabulary, enforced. `segment_name` is what the warm-up and the PTS probe wait for, what + the argv's own `-hls_segment_filename` pattern is built from, and what `artifact_may_arrive` + parses back — they were three separate statements of the layout and two of them were wrong. Pin + them to each other instead.""" + sess = _Session("sid", tmp_path, audio_langs=audio_langs) + variant = sess.variants[0] if sess.variants else None + name = plex_stream.segment_name(3, variant) + prev = plex_stream.segment_name(2, variant) + assert not plex_stream.artifact_may_arrive(sess, name) + (tmp_path / prev).write_bytes(b"x") + assert plex_stream.artifact_may_arrive(sess, name) + + +def test_the_argv_asks_for_exactly_the_names_the_guard_accepts(tmp_path): + """The producer and the parser, pinned to each other through the argv itself. + + Round 6's HIGH was a guard that read `seg_%v_%d.ts` sequence-first, so it was inert on every + multi-audio item, and a warm-up that waited 8 s for a name ffmpeg never writes. The fix was to + give the layout one spelling — but a comment saying so is what had just failed, so assert that + the `-hls_segment_filename` pattern in the real argv is literally what `segment_name` produces, + and that every playlist the argv names is one `playlist_names` accepts.""" + for audio_langs in (None, ["hu", "en", "de"]): + args = plex_stream._ffmpeg_cmd( + tmp_path / "in.mkv", 0.0, tmp_path, 0, 0.0, audio_langs, loudnorm=False + ) + variants = plex_stream.session_variants(audio_langs) + sess = _Session("sid", tmp_path, audio_langs=audio_langs) + # The two patterns in the argv are the vocabulary's own spelling, `%d`/`%v` and all. + seg_pattern = args[args.index("-hls_segment_filename") + 1] + pl_pattern = args[-1].rsplit("/", 1)[-1].rsplit("\\", 1)[-1] + if variants: + assert seg_pattern.endswith(plex_stream.segment_name("%d", "%v")), seg_pattern + assert pl_pattern == plex_stream.variant_playlist_name("%v"), pl_pattern + assert args[args.index("-master_pl_name") + 1] == plex_stream._MASTER_PL + else: + assert seg_pattern.endswith(plex_stream.segment_name("%d")), seg_pattern + assert pl_pattern == plex_stream._INDEX_PL, pl_pattern + assert "-master_pl_name" not in args + + # And every name those patterns can actually expand to is one the guard lets through — the + # entry playlist hls.js is handed included, since a session whose own manifest is refused + # cannot start at all. + for v in variants or (None,): + assert plex_stream.artifact_may_arrive(sess, plex_stream.segment_name(0, v)) + if v is not None: + assert plex_stream.artifact_may_arrive(sess, plex_stream.variant_playlist_name(v)) + assert plex_stream.artifact_may_arrive(sess, sess.entry) + + +def test_a_name_only_the_other_argv_branch_writes_is_refused_at_once(tmp_path, client, sessions): + """The guard is keyed to THIS session's vocabulary, not to HLS in general. + + The two argv branches are disjoint: a single-audio session is given `seg_%d.ts` + `index.m3u8` + and no `%v` at all, so `master.m3u8`, `stream_*.m3u8` and every `seg__.ts` are names + it can never write — and vice versa. Accepting the union read them as "an HLS name, so wait and + see", and each one then held a threadpool worker and its pooled DB connection for the full + timeout: measured through this route against a live single-audio session, `master.m3u8` took + 30.0 s and `seg_a0_0.ts` 20.1 s to answer 404 (deep-review round 7).""" + single = tmp_path / "m1_0_0_+0.00" + single.mkdir() + sessions["m1"] = _Session("sid-a", single, running=True) + multi = tmp_path / "m2_0_multi_+0.00" + multi.mkdir() + sessions["m2"] = _Session("sid-b", multi, running=True, audio_langs=["hu", "en"]) + + # Each of these belongs to the other shape (or to no shape at all) and must be refused without + # a wait — the assertion is the ELAPSED time as much as the status. + for key, sid, names in ( + ("m1", "sid-a", ["master.m3u8", "stream_0.m3u8", "seg_0_0.ts", "seg_zzzz_0.ts"]), + ("m2", "sid-b", ["index.m3u8", "seg_0.ts", "seg_a9_0.ts", "stream_zz.m3u8"]), + ): + for name in names: + started = time.monotonic() + r = client.get(f"/api/plex/stream/{key}/hls/{sid}/{name}") + elapsed = time.monotonic() - started + assert r.status_code == 404, f"{name}: {r.text}" + assert elapsed < 2.0, ( + f"{name}: waited {elapsed:.1f}s for a name this session cannot write" + ) + + # And the control: the names each session CAN write are still waited for, not refused. + assert plex_stream.artifact_may_arrive(sessions["m1"], "index.m3u8") + assert plex_stream.artifact_may_arrive(sessions["m1"], "seg_0.ts") + assert plex_stream.artifact_may_arrive(sessions["m2"], "master.m3u8") + assert plex_stream.artifact_may_arrive(sessions["m2"], "stream_a1.m3u8") + assert plex_stream.artifact_may_arrive(sessions["m2"], "seg_a1_0.ts") diff --git a/backend/tests/test_plex_playback_session.py b/backend/tests/test_plex_playback_session.py index 51a952ac..ec5d7331 100644 --- a/backend/tests/test_plex_playback_session.py +++ b/backend/tests/test_plex_playback_session.py @@ -58,7 +58,12 @@ def _post(db, user, monkeypatch, rating_key, *, captured): def fake_start(db_, it, start, audio, aoff, multi, transcode=False): captured["transcode"] = transcode return SimpleNamespace( - media_start_s=start, entry="index.m3u8", transcode=transcode, hw=False, tonemap=False + id="sid", + media_start_s=start, + entry="index.m3u8", + transcode=transcode, + hw=False, + tonemap=False, ) monkeypatch.setattr(plex_stream, "start_session", fake_start) diff --git a/backend/tests/test_plex_stream.py b/backend/tests/test_plex_stream.py index 39be017f..5c354d9b 100644 --- a/backend/tests/test_plex_stream.py +++ b/backend/tests/test_plex_stream.py @@ -483,6 +483,7 @@ class TestColdStartWarmup: src.write_bytes(b"x") monkeypatch.setattr(stream.paths, "local_media_path", lambda db, fp: src) monkeypatch.setattr(stream.sysconfig, "get_int", lambda db, k: 2) + monkeypatch.setattr(stream.sysconfig, "get_bool", lambda db, k: False) monkeypatch.setattr(stream.subprocess, "Popen", lambda *a, **k: _FakeProc()) return tmp_path @@ -554,6 +555,12 @@ class TestTranscodeStartSession: "get_str", lambda db, k: {"plex_transcode_preset": "slow"}.get(k, "auto"), ) + # Loudness normalisation (item 14) is off unless a test turns it on via `stream._loudnorm_on`. + monkeypatch.setattr( + stream.sysconfig, + "get_bool", + lambda db, k: {"plex_audio_loudnorm": getattr(stream, "_loudnorm_on", False)}[k], + ) monkeypatch.setattr(stream, "wait_for", lambda path, timeout=20.0, proc=None: True) _CapturingProc.calls = [] monkeypatch.setattr(stream.subprocess, "Popen", _CapturingProc) @@ -868,3 +875,73 @@ class TestWaitForProcFastFail: seg = tmp_path / "seg_0.ts" seg.write_bytes(b"") assert stream.wait_for(seg, timeout=30.0, proc=self._DeadProc()) is False + + +class TestAudioLoudnorm: + """`plex_audio_loudnorm` (bug hunt 2026-09, item 14): the HLS path always downmixes 5.1 to stereo + AAC and never normalised it, so a cinema mix arrives with whispered dialogue and deafening action. + The knob is OFF by default — it makes a film ~11 dB louder and narrows its dynamic range, which is + a taste call. Measured on a real 5.1 AC3 film: I -27.6 -> -16.6 LUFS, LRA 13.3 -> 9.8 LU. + + Reuses TestTranscodeStartSession's capturing fixture: what matters is the argv ffmpeg is actually + spawned with, not `_ffmpeg_cmd` called with literals.""" + + capturing_env = TestTranscodeStartSession.capturing_env + + @staticmethod + def _spawn(monkeypatch, *, on: bool, multi: bool = False): + monkeypatch.setattr(stream, "_loudnorm_on", on, raising=False) + monkeypatch.setattr( + stream, "_select_encode", lambda db, item, transcode, src: (False, False) + ) + if multi: + monkeypatch.setattr(stream, "_probe_audio_langs", lambda src: ["hun", "eng"]) + s = stream.start_session(db=None, item=_FakeItem(), start_s=0.0, multi=multi) + assert s is not None + (cmd, *_), _ = _CapturingProc.calls[0] + return s, cmd + + def test_the_knob_off_leaves_the_audio_exactly_as_it_was(self, capturing_env, monkeypatch): + _, cmd = self._spawn(monkeypatch, on=False) + assert "-af" not in cmd + assert cmd[cmd.index("-c:a") + 1] == "aac" and cmd[cmd.index("-ac") + 1] == "2" + + def test_the_knob_on_normalises_AFTER_the_downmix(self, capturing_env, monkeypatch): + # Order is the whole point: measuring 5.1 and folding it down afterwards would normalise + # something other than what the viewer hears. + _, cmd = self._spawn(monkeypatch, on=True) + af = cmd[cmd.index("-af") + 1] + stages = af.split(",") + assert stages[0] == "aformat=channel_layouts=stereo" + assert stages[1].startswith("loudnorm=") + # loudnorm runs its own 192 kHz resampler internally; the AAC encoder needs 48 kHz back. + assert stages[-1] == "aresample=48000" + + def test_every_audio_rendition_of_a_multi_audio_file_is_normalised( + self, capturing_env, monkeypatch + ): + # A multi-audio session emits one AAC output per language. `-af` without a stream index + # covers all of them — a per-stream filter would have normalised only the first. + _, cmd = self._spawn(monkeypatch, on=True, multi=True) + assert cmd.count("-map") == 3 # video + two audio tracks + assert cmd.count("-af") == 1 + assert "loudnorm=" in cmd[cmd.index("-af") + 1] + + def test_the_session_directory_records_the_setting(self, capturing_env, monkeypatch): + # The knob changes the AUDIO THE SEGMENTS CONTAIN, so it belongs in the directory name with + # the other parameters that do. Otherwise a session left from before the flip is reused. + off, _ = self._spawn(monkeypatch, on=False) + _CapturingProc.calls = [] + on, _ = self._spawn(monkeypatch, on=True) + assert off.dir.name != on.dir.name + assert on.dir.name.endswith("-ln_+0.00") + + @pytest.mark.parametrize("on", [False, True]) + def test_the_orphan_sweep_still_recognises_its_own_directory( + self, capturing_env, monkeypatch, on + ): + # The tag is appended with a DASH, not an underscore, precisely so `_SESSION_DIR_RE` (the only + # thing keeping the sweep off a shared scratch path) still matches. A session directory the + # sweep no longer recognises is one that never gets cleaned up. + s, _ = self._spawn(monkeypatch, on=on) + assert _SESSION_DIR_RE.match(s.dir.name) diff --git a/backend/tests/test_plex_subtitle_auth.py b/backend/tests/test_plex_subtitle_auth.py new file mode 100644 index 00000000..a4f6b2fe --- /dev/null +++ b/backend/tests/test_plex_subtitle_auth.py @@ -0,0 +1,45 @@ +"""The subtitle routes must refuse a session the rest of the app refuses. + +They used to sit behind `_require_session`, the cookie-only gate the poster proxies use — a signed +cookie was enough, with none of `current_user`'s checks (is_active, is_suspended, the session epoch +that "log out everywhere" bumps). Fine for a poster; not for a route that starts reads of the media +library. Found by the 2026-09 bug-hunt deep-review; the gate itself pre-dates that round. +""" + +from fastapi.testclient import TestClient + +import app.auth as auth +import app.state as app_state +from app.db import get_db +from app.main import app +from app.plex import subtitles +from app.routes.plex import playback + + +class _NoPool: + def __init__(self) -> None: + self.jobs: list = [] + + def submit(self, fn, *args) -> None: + self.jobs.append((fn, args)) + + +def test_a_suspended_account_cannot_drive_the_subtitle_extractor( + db, make_user, override_dep, monkeypatch, tmp_path +): + user = make_user(is_suspended=True) + override_dep(get_db, lambda: db) + monkeypatch.setattr(app_state, "_configured_cache", True) + # A validly SIGNED cookie for that account — which is all the old gate looked at. + for mod in (auth, playback): + monkeypatch.setattr(mod, "resolved_user_id", lambda request: (user.id, True)) + monkeypatch.setattr(subtitles, "CACHE_DIR", tmp_path) + pool = _NoPool() + monkeypatch.setattr(subtitles, "_WIN_POOL", pool) + monkeypatch.setattr(subtitles, "_FULL_POOL", pool) + subtitles.reset_state() + + client = TestClient(app) + assert client.get("/api/plex/subtitle/98614/0/status?at=600").status_code == 401 + assert client.get("/api/plex/subtitle/98614/0").status_code == 401 + assert pool.jobs == [], "a refused session still scheduled an extraction" diff --git a/backend/tests/test_plex_subtitle_prepare.py b/backend/tests/test_plex_subtitle_prepare.py new file mode 100644 index 00000000..fcaa4aa0 --- /dev/null +++ b/backend/tests/test_plex_subtitle_prepare.py @@ -0,0 +1,1087 @@ +"""Subtitle extraction must not happen inside the request, and must read as little as possible. + +Bug hunt 2026-09, item 8 — "switching the subtitle track shows nothing; after a few stop/starts it +suddenly works". Measured on prod against a 17.06 GB MKV with two embedded SubRip tracks: + + full pass, one track 154 s | both tracks in ONE run 111 s | a 630 s window at 1:00:00 8 s + +The cost is not the WebVTT conversion, it is that ffmpeg streams every byte of the container between +its seek point and its stop point. The old endpoint did that pass INSIDE the 's request, once +per track, with no timeout — so the browser sat on an empty track for minutes (and an API worker +thread sat with it), and picking the second language cost the whole read again. + +These tests pin the three things that fixes it: the request never waits for ffmpeg, the extraction +publishes a window around the playhead before the whole track, and one ffmpeg run covers every +embedded text track — plus what the deep-review of that fix found (cues doubled at the window/full +seam, a truncated pass cached as complete, a poll that respawned ffmpeg every second, a sidecar stuck +behind an unrelated pass). No Postgres (the auth half is in test_plex_subtitle_auth.py, which needs +the DB). Almost every test stubs ffmpeg out, but not all: a few run a real child process, and +`test_a_window_lines_up_with_the_full_pass_on_a_real_file` needs the real ffmpeg — it is the only +guard of `-avoid_negative_ts disabled`. It skips without one, except in the gate's DB lane, where +conftest refuses to run without ffmpeg rather than let it skip. +""" + +import asyncio +import errno +import shutil +import subprocess +import sys +import threading +import time +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path +from types import SimpleNamespace + +import pytest +from fastapi.testclient import TestClient + +import app.state as app_state +import app.main as app_main +from app.auth import current_user +from app.main import app +from app.plex import subtitles + +needs_ffmpeg = pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available") + + +@pytest.fixture(autouse=True) +def _isolated_cache(tmp_path, monkeypatch): + """A throwaway cache root + empty registries: this module's state is process-global.""" + monkeypatch.setattr(subtitles, "CACHE_DIR", tmp_path) + subtitles.reset_state() + yield + subtitles.reset_state() + + +@pytest.fixture +def client(monkeypatch, override_dep) -> TestClient: + # Neither route reads the user; the real `current_user` (and what it rejects) is pinned in + # test_plex_subtitle_auth.py against the DB. + override_dep(current_user, lambda: SimpleNamespace(id=1)) + monkeypatch.setattr(app_state, "_configured_cache", True) # pass the instance-setup gate, no DB + return TestClient(app) + + +class _ManualPool: + """Deterministic stand-in for the extraction thread pool: jobs run when the test says so.""" + + def __init__(self) -> None: + self.jobs: list[tuple] = [] + + def submit(self, fn, *args) -> None: + self.jobs.append((fn, args)) + + def run_next(self) -> None: + fn, args = self.jobs.pop(0) + fn(*args) + + def run_all(self) -> int: + jobs, self.jobs = self.jobs, [] + for fn, args in jobs: + fn(*args) + return len(jobs) + + def shutdown(self, wait=True, cancel_futures=False) -> None: + pass # jobs still run only when the test says so — which is what the shutdown tests probe + + +def _manual_pool(monkeypatch) -> _ManualPool: + """One manual pool standing in for BOTH extraction pools, so a test sees every job in the order + it was submitted (window or full pass — `window_start` tells them apart).""" + pool = _ManualPool() + monkeypatch.setattr(subtitles, "_WIN_POOL", pool) + monkeypatch.setattr(subtitles, "_FULL_POOL", pool) + return pool + + +def _real_pools(monkeypatch) -> None: + """Fresh single-worker pools (the module's own may have been shut down by an earlier test).""" + monkeypatch.setattr(subtitles, "_WIN_POOL", ThreadPoolExecutor(max_workers=1)) + monkeypatch.setattr(subtitles, "_FULL_POOL", ThreadPoolExecutor(max_workers=1)) + + +def _cue(start: str, end: str, text: str) -> str: + return f"{start} --> {end}\n{text}" + + +def _vtt(*cues: str) -> str: + return "WEBVTT\n\n" + "\n\n".join(cues) + "\n" + + +# --- the request must never wait for the extraction --------------------------------------------- + + +def test_a_cold_track_answers_at_once_instead_of_waiting_for_ffmpeg(client, monkeypatch): + """THE regression. A cold track used to block the request for the length of a full file read; + now it answers 202 and the extraction runs behind it. Timed, because "does it block?" is the + whole question — the fake extraction holds a worker for 5 s, which the request must not feel.""" + started = threading.Event() + release = threading.Event() + + def _slow(rating_key, sub_ord, window_start): + started.set() + release.wait(timeout=5) + + monkeypatch.setattr(subtitles, "_extract_and_publish", _slow) + try: + t0 = time.monotonic() + st = client.get("/api/plex/subtitle/98614/0/status") + elapsed = time.monotonic() - t0 + assert st.status_code == 200 and st.json()["state"] == "preparing", st.text + assert elapsed < 2.0, f"the request waited {elapsed:.1f}s on the extraction" + assert started.wait(timeout=5), "the extraction never started in the background" + # …and the fetch answers just as fast while that extraction is still running. + t0 = time.monotonic() + resp = client.get("/api/plex/subtitle/98614/0") + assert resp.status_code == 202, resp.text + assert time.monotonic() - t0 < 2.0 + finally: + release.set() + + +def test_a_burst_of_polls_starts_one_window_and_one_full_pass(monkeypatch): + """The player polls once a second while it waits. Each poll must JOIN the running jobs, not + start another ffmpeg over the same 17 GB.""" + pool = _manual_pool(monkeypatch) + for _ in range(5): + subtitles.ensure("98614", 0, at=600.0) + assert len(pool.jobs) == 2 # one window (at 600 s) + the one full pass + # submit(_run, ident, rating_key, sub_ord, window_start, at) — window_start tells them apart + assert [a[3] for _, a in pool.jobs] == [570.0, None] # the window first, then the full pass + + +# --- window first, whole track after ------------------------------------------------------------ + + +def test_status_walks_preparing_to_partial_to_ready(client, monkeypatch): + """A window around the playhead makes the track showable long before it is complete, and the + generation changes when it grows so the browser refetches instead of keeping the short VTT.""" + pool = _manual_pool(monkeypatch) + key = subtitles.cache_key("98614", 0) + + def _publish(rating_key, sub_ord, window_start): + if window_start is not None: + subtitles.publish( + key, _vtt(_cue("59:54.187", "59:58.650", "window")), (3570.0, 3900.0), False + ) + else: + subtitles.publish(key, _vtt(_cue("01:12.000", "01:15.000", "whole")), (0.0, 1e9), True) + + monkeypatch.setattr(subtitles, "_extract_and_publish", _publish) + + first = client.get("/api/plex/subtitle/98614/0/status?at=3600").json() + assert first["state"] == "preparing" and first["gen"] == 0 + + pool.run_next() # the window job, submitted first — the full pass stays queued behind it + windowed = client.get("/api/plex/subtitle/98614/0/status?at=3600").json() + assert windowed["state"] == "partial", "a window around the playhead is showable" + assert windowed["gen"] > 0 + # …and the request now gets real WebVTT rather than another 202. + served = client.get("/api/plex/subtitle/98614/0?at=3600") + assert served.status_code == 200 and "window" in served.text + + pool.run_next() # the full pass, scheduled by that same first poll + done = client.get("/api/plex/subtitle/98614/0/status?at=3600").json() + assert done["state"] == "ready" + assert done["gen"] != windowed["gen"], "a widened track must not reuse the window's generation" + assert "whole" in client.get("/api/plex/subtitle/98614/0").text + + +def test_a_position_outside_the_extracted_window_is_not_called_showable(client, monkeypatch): + """Honest states: cues exist, but none for where the viewer just seeked to. Saying "ready" there + would put an empty track on screen with no explanation — the original complaint.""" + _manual_pool(monkeypatch) + subtitles.publish( + subtitles.cache_key("98614", 0), + _vtt(_cue("00:10.000", "00:12.000", "x")), + (0.0, 330.0), + False, + ) + assert client.get("/api/plex/subtitle/98614/0/status?at=100").json()["state"] == "partial" + assert client.get("/api/plex/subtitle/98614/0/status?at=5000").json()["state"] == "preparing" + + +def test_an_unshowable_track_says_why_and_maps_to_a_status(client, monkeypatch): + pool = _manual_pool(monkeypatch) + + def _bitmap(rating_key, sub_ord, window_start): + raise subtitles.SubtitleUnavailable("bitmap") + + monkeypatch.setattr(subtitles, "_extract_and_publish", _bitmap) + client.get("/api/plex/subtitle/98614/0/status") + pool.run_all() + st = client.get("/api/plex/subtitle/98614/0/status").json() + assert st["state"] == "failed" and st["code"] == "bitmap" + resp = client.get("/api/plex/subtitle/98614/0") + assert resp.status_code == 415 # a structured detail, so the UI can translate it + assert resp.json()["detail"] == {"code": "subtitle", "reason": "bitmap"} + + +# --- the ffmpeg pass itself --------------------------------------------------------------------- + + +class _FakeFfmpeg: + """Stands in for the ffmpeg child: writes the outputs its argv asked for, exits with `returncode`.""" + + def __init__(self, cmd: list[str], returncode: int) -> None: + self.cmd, self.returncode = cmd, returncode + + def communicate(self, timeout=None): + for i, arg in enumerate(self.cmd): + if arg == "-y": + Path(self.cmd[i + 1]).write_text( + _vtt(_cue("00:01.000", "00:02.000", "hi")), "utf-8" + ) + return b"", b"" + + def kill(self) -> None: + pass + + +def _capture_ffmpeg(monkeypatch, returncode: int = 0) -> list[list[str]]: + """Record the argv of each ffmpeg run and satisfy it by writing the outputs it asked for.""" + runs: list[list[str]] = [] + + def _fake_popen(cmd, **kwargs): + runs.append(cmd) + return _FakeFfmpeg(cmd, returncode) + + monkeypatch.setattr(subtitles.subprocess, "Popen", _fake_popen) + return runs + + +def test_one_ffmpeg_run_extracts_every_embedded_text_track(monkeypatch): + """The read is the cost and it is SHARED. Extracting one track at a time is what made picking + Hungarian cost a full pass and then switching to English cost another one.""" + runs = _capture_ffmpeg(monkeypatch) + out, clean = subtitles._ffmpeg_extract("/plex-media/f.mkv", [0, 1], None, None) + assert len(runs) == 1, "each extra ffmpeg run is another full read of the file" + assert set(out) == {0, 1} and clean + assert runs[0].count("-map") == 2 + + +def test_a_non_zero_exit_is_reported_even_when_it_left_output(monkeypatch): + """ffmpeg can die partway and still leave a VTT behind. That text is worth keeping — but the + caller has to be TOLD, or it publishes a truncated track as the whole thing.""" + _capture_ffmpeg(monkeypatch, returncode=1) + out, clean = subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], None, None) + assert out and not clean + + +def test_a_windowed_pass_bounds_the_read_and_never_uses_copyts(monkeypatch): + """`-copyts` is the trap: with it ffmpeg IGNORES -t and reads the whole container anyway + (measured — a [0, 600 s] window still took the full 145 s). So the window is extracted on a + zero-based clock and shifted back to absolute time by the caller.""" + runs = _capture_ffmpeg(monkeypatch) + subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], 3570.0, 330.0) + cmd = runs[0] + assert "-copyts" not in cmd + assert cmd[cmd.index("-ss") + 1].startswith( + "3570" + ) # input seek — before -i, so it is a real seek + assert cmd.index("-ss") < cmd.index("-i") + assert cmd[cmd.index("-t") + 1].startswith("330") + + +def test_the_full_pass_is_not_bounded(monkeypatch): + runs = _capture_ffmpeg(monkeypatch) + subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], None, None) + assert "-t" not in runs[0] and "-ss" not in runs[0] + + +# --- the pure pieces the above rests on --------------------------------------------------------- + + +def test_merge_vtt_unions_windows_in_time_order(): + late = _vtt(_cue("59:54.187", "59:58.650", "late")) + early = _vtt(_cue("01:12.000", "01:15.000", "early"), _cue("59:54.187", "59:58.650", "late")) + merged = subtitles.merge_vtt(late, early) + assert merged.startswith("WEBVTT\n\n") + assert merged.count("-->") == 2, "the shared region must not be duplicated" + assert merged.index("early") < merged.index("late") + + +def test_covered_ranges_coalesce_and_answer_membership(): + ranges = subtitles.merge_ranges([[300.0, 600.0], [0.0, 330.0], [900.0, 1200.0]]) + assert ranges == [[0.0, 600.0], [900.0, 1200.0]] + assert subtitles.covers(ranges, 450.0) + assert not subtitles.covers(ranges, 700.0) + + +def test_a_cached_vtt_from_an_older_build_counts_as_complete(tmp_path): + """Before this change the cache only ever held whole tracks, and none of them has a sidecar. + Reading one as "covers nothing" would re-extract every already-cached track after the upgrade.""" + (tmp_path / "sub_98614_0.vtt").write_text(_vtt(_cue("00:01.000", "00:02.000", "x")), "utf-8") + assert subtitles.read_meta("sub_98614_0") == ([[0.0, 1_000_000_000.0]], True) + + +# --- the deep-review round: the seams between the window and the full pass ---------------------- + + +def test_a_window_and_the_full_pass_do_not_double_a_cue(): + """THE doubled-subtitle bug. The two passes spell the same timestamp differently — a window comes + back through shift_vtt (always HH:MM:SS.mmm), a full pass is ffmpeg's own output, whose webvtt + muxer drops the hour below one hour — so de-duplicating on the text kept both, and every cue of a + window in the first hour was on screen twice.""" + window = subtitles.shift_vtt(_vtt(_cue("00:30.123", "00:32.000", "hello")), -570.0) + full = _vtt(_cue("10:00.123", "10:02.000", "hello")) # ffmpeg's spelling of the same cue + merged = subtitles.merge_vtt(window, full) + assert merged.count("-->") == 1, merged + assert "00:10:00.123 --> 00:10:02.000" in merged # one canonical spelling in the served file + + +def test_two_different_cues_at_the_same_time_are_both_kept(): + """De-duplication is by timing AND text: a sign and a line of dialogue can share a timestamp.""" + both = _vtt(_cue("00:10.000", "00:12.000", "sign"), _cue("00:10.000", "00:12.000", "dialogue")) + merged = subtitles.merge_vtt(both, both) + assert merged.count("-->") == 2 and "sign" in merged and "dialogue" in merged + + +class _NullSession: + def __enter__(self): + return None + + def __exit__(self, *exc): + return False + + +# A file's subtitle streams as Plex lists them: a downloaded sidecar first (no `index` — it is not in +# the file), then the embedded ones, one of them a bitmap track that can never become text. +_STREAMS = [ + {"key": "/library/streams/1", "codec": "srt"}, # sub_ord 0 — sidecar + {"index": 3, "codec": "subrip"}, # sub_ord 1 — embedded ordinal 0 (Hungarian) + {"index": 4, "codec": "pgs"}, # sub_ord 2 — embedded ordinal 1, bitmap + {"index": 5, "codec": "subrip"}, # sub_ord 3 — embedded ordinal 2 (English) +] + + +def _stub_item(monkeypatch, *, clean: bool = True, cues=("00:05.000", "00:07.000")): + """Everything around `_extract_and_publish` stubbed, so the function itself is what runs.""" + monkeypatch.setattr(subtitles, "SessionLocal", _NullSession) + monkeypatch.setattr( + subtitles, "_text_streams", lambda db, rk: (SimpleNamespace(file_path="/f.mkv"), _STREAMS) + ) + monkeypatch.setattr( + subtitles.plex_paths, "local_media_path", lambda db, fp: "/plex-media/f.mkv" + ) + monkeypatch.setattr( + subtitles, "_sidecar_vtt", lambda db, s: _vtt(_cue("00:01.000", "00:03.000", "side")) + ) + calls: list[tuple] = [] + + def _extract(src, emb_ords, start, length): + calls.append((list(emb_ords), start, length)) + return {o: _vtt(_cue(*cues, f"track {o}")) for o in emb_ords}, clean + + monkeypatch.setattr(subtitles, "_ffmpeg_extract", _extract) + return calls + + +def test_one_window_publishes_every_embedded_text_track_at_absolute_time(monkeypatch): + """What `_extract_and_publish` itself is for — every other test stubs it out. One read over the + file's EMBEDDED text streams (the sidecar is not in the file and must not shift the `-map` + ordinals; the bitmap track is skipped), each published under its own sub_ord, and a window's + zero-based cues shifted back to where they sit in the film.""" + calls = _stub_item(monkeypatch) + subtitles._extract_and_publish("98614", 3, 3570.0) + assert calls == [([0, 2], 3570.0, 330.0)], "one ffmpeg run over embedded ordinals 0 and 2" + english = subtitles.cache_read(subtitles.cache_key("98614", 3)) or "" + assert "00:59:35.000 --> 00:59:37.000" in english, english # 3570 + 5 s + assert "track 2" in english + assert "track 0" in (subtitles.cache_read(subtitles.cache_key("98614", 1)) or "") + for never in (0, 2): # the sidecar and the bitmap track come from nowhere in this pass + assert subtitles.cache_read(subtitles.cache_key("98614", never)) is None + + +def test_a_full_pass_that_died_is_not_cached_as_the_whole_track(monkeypatch): + """A truncated full pass used to be published as complete: `ensure` answered "ready" forever, the + player stopped polling, and — the cache has no eviction — the subtitles simply ended wherever + ffmpeg had died, in every later session too. Now it covers what it provably reached; past that + a window is tried, and only when that fails too is the position answered as unavailable.""" + _stub_item(monkeypatch, clean=False, cues=("39:58.000", "40:00.000")) + pool = _manual_pool(monkeypatch) + subtitles._extract_and_publish("98614", 3, None) + key = subtitles.cache_key("98614", 3) + assert subtitles.read_meta(key) == ([[0.0, 2400.0]], False) + assert subtitles.ensure("98614", 3, at=100.0)["state"] == "partial" + assert subtitles.ensure("98614", 3, at=3000.0)["state"] == "preparing" + assert [a[3] for _, a in pool.jobs] == [2970.0], "a window past the break — not the full pass" + _window_raises(monkeypatch) + pool.run_all() + after = subtitles.ensure("98614", 3, at=3000.0) + assert (after["state"], after["code"]) == ("unavailable", "extract_failed") + + +def test_polls_that_follow_the_playhead_join_one_window(monkeypatch): + """The poll sends the LIVE position, so it moves a second per poll. A window keyed on that + position was a new job every second — another read of the file per poll until the first landed.""" + pool = _manual_pool(monkeypatch) + for at in (600.0, 601.0, 602.0, 603.0, 604.0): + subtitles.ensure("98614", 1, at=at) + assert len(pool.jobs) == 2, "one window + the one full pass" + + +def test_a_window_for_a_different_stretch_does_not_wait_behind_the_first(monkeypatch): + """Measured on localdev: after a resume the first poll can still report 0 (the position is only + known once playback ticks), so the first window is cut around the wrong spot. The poll that + follows, at the real position, must get its own window at once — not queue behind that one.""" + pool = _manual_pool(monkeypatch) + subtitles.ensure("98614", 1, at=0.0) + subtitles.ensure("98614", 1, at=1198.0) + assert sorted(a[3] for _, a in pool.jobs if a[3] is not None) == [0.0, 1168.0] + + +def test_a_queued_window_that_was_covered_meanwhile_reads_nothing(monkeypatch): + """Every embedded track rides each pass over its file, so by the time a queued window for one + language runs, another language's window may already have published it.""" + pool = _manual_pool(monkeypatch) + calls = _stub_item(monkeypatch) + subtitles.ensure("98614", 3, at=600.0) + subtitles.publish( + subtitles.cache_key("98614", 3), + _vtt(_cue("10:00.000", "10:02.000", "x")), + (570.0, 900.0), + False, + ) + pool.run_next() # the window job + assert calls == [] + + +def test_a_sidecar_does_not_wait_behind_an_embedded_full_pass(monkeypatch): + """The full pass is one job per ITEM (the embedded fan-out shares it). A sidecar picked while an + embedded track's pass was running used to find that identity taken and wait out the whole read — + minutes — for a track that is one small fetch.""" + pool = _manual_pool(monkeypatch) + _stub_item(monkeypatch) + subtitles._INFLIGHT[("full", "98614")] = None # an embedded track's full pass is running + subtitles.ensure("98614", 0, at=600.0) + pool.run_all() + assert subtitles.ensure("98614", 0, at=600.0)["state"] == "ready" + + +def test_publishes_into_one_track_do_not_lose_each_other(monkeypatch): + """`publish` is a read-merge-write. Two jobs publishing into the same track at once — the window + and the full pass both fan out to every embedded track — used to let the later writer drop the + earlier one's cues. Forced interleaving: the first publisher pauses right after its read.""" + key = subtitles.cache_key("98614", 0) + real_read = subtitles.cache_read + paused, resume = threading.Event(), threading.Event() + reads = {"n": 0} + + def _read(k): + reads["n"] += 1 + out = real_read(k) + if reads["n"] == 1: + paused.set() + resume.wait(timeout=5) + return out + + monkeypatch.setattr(subtitles, "cache_read", _read) + first = threading.Thread( + target=subtitles.publish, + args=(key, _vtt(_cue("00:10.000", "00:12.000", "alpha")), (0.0, 330.0), False), + ) + first.start() + assert paused.wait(timeout=5) + second = threading.Thread( + target=subtitles.publish, + args=(key, _vtt(_cue("10:00.000", "10:02.000", "bravo")), (570.0, 900.0), False), + ) + second.start() + time.sleep(0.3) # without the lock, the second publisher finishes in here + resume.set() + first.join(timeout=5) + second.join(timeout=5) + merged = real_read(key) or "" + assert "alpha" in merged and "bravo" in merged, merged + assert subtitles.read_meta(key)[0] == [[0.0, 330.0], [570.0, 900.0]] + + +def test_shutdown_kills_a_running_extraction(monkeypatch): + """The pool's workers are non-daemon threads that Python joins at exit, so one still inside a + full pass held a restarting process open for the rest of the read. Stand-in child: a real + process that would sleep for a minute.""" + real_popen = subprocess.Popen + monkeypatch.setattr( + subtitles.subprocess, + "Popen", + lambda cmd, **kw: real_popen([sys.executable, "-c", "import time; time.sleep(60)"], **kw), + ) + _real_pools(monkeypatch) + finished = threading.Event() + + def _extract(): + try: + subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], None, None) + except subtitles.SubtitleUnavailable: + pass # killed before it wrote anything — exactly what a shutdown does + finally: + finished.set() + + threading.Thread(target=_extract, daemon=True).start() + deadline = time.monotonic() + 10 + while not subtitles._PROCS and time.monotonic() < deadline: + time.sleep(0.02) + assert subtitles._PROCS, "the extraction never registered its child" + subtitles.shutdown() + assert finished.wait(timeout=10), "the extraction was still running after shutdown" + + +def test_the_track_fetch_serves_what_is_cached_and_schedules_nothing(client, monkeypatch): + """The URL carries no position, and it used to run the scheduler anyway — judging + readiness at position 0. A viewer resumed an hour in got an unrequested extraction over the head + of the file, and with a failure memo alive the fetch refused a window that WAS on disk while the + status poll (which knows the position) called the track showable: silently blank subtitles.""" + pool = _manual_pool(monkeypatch) + assert client.get("/api/plex/subtitle/98614/0").status_code == 202 + assert pool.jobs == [], "the fetch scheduled an extraction" + key = subtitles.cache_key("98614", 0) + subtitles.publish(key, _vtt(_cue("59:54.187", "59:58.650", "window")), (3570.0, 3900.0), False) + subtitles._fail(key, "extract_failed") # the full pass behind that window failed + resp = client.get("/api/plex/subtitle/98614/0") + assert resp.status_code == 200 and "window" in resp.text + assert pool.jobs == [] + + +@pytest.mark.parametrize("at", ["inf", "-1", "nan"]) +def test_an_impossible_position_is_rejected(client, monkeypatch, at): + """`?at=inf` reached the scheduler's arithmetic and came back as a 500 with a traceback.""" + pool = _manual_pool(monkeypatch) + assert client.get(f"/api/plex/subtitle/98614/0/status?at={at}").status_code == 422 + assert pool.jobs == [] + + +@pytest.mark.parametrize("q", ["offset=-inf", "offset=nan", "offset=inf"]) +def test_an_impossible_offset_is_rejected(client, q): + """The sibling of `at`: `-inf` crashed the shift with a 500, `nan` stacked every cue at 0.""" + assert client.get(f"/api/plex/subtitle/98614/0?{q}").status_code == 422 + + +def test_an_absurd_position_is_rejected_before_it_reaches_ffmpeg(client, monkeypatch): + """ffmpeg refuses an `-ss` past its int64 microseconds, and that failure was recorded against the + TRACK — every other viewer at an uncovered position was told "failed" for a minute.""" + pool = _manual_pool(monkeypatch) + assert client.get("/api/plex/subtitle/98614/0/status?at=1e13").status_code == 422 + assert pool.jobs == [] + + +# --- the second deep-review round --------------------------------------------------------------- + + +def _window_raises(monkeypatch): + """From here on every window read fails outright; a full pass still works.""" + real = subtitles._ffmpeg_extract + + def _extract(src, emb_ords, start, length): + if length is not None: + raise subtitles.SubtitleUnavailable("extract_failed") + return real(src, emb_ords, start, length) + + monkeypatch.setattr(subtitles, "_ffmpeg_extract", _extract) + + +def _later(monkeypatch, seconds: float) -> None: + """Move the module's clock forward (the failure memos are monotonic-time TTLs).""" + now = time.monotonic() + seconds + monkeypatch.setattr(subtitles.time, "monotonic", lambda: now) + + +def test_a_full_pass_that_failed_is_not_read_again_while_its_output_is_watched(monkeypatch): + """Published as partial, a failed full pass left the viewer inside what it DID publish — so the + status said `partial`, the poll kept going, and each time the one-minute failure memo lapsed it + re-armed a read of the whole container. A failure that reproduces did so every time: a full + read of the file every few minutes for as long as they watched.""" + _stub_item(monkeypatch, clean=False, cues=("39:58.000", "40:00.000")) + pool = _manual_pool(monkeypatch) + subtitles._extract_and_publish("98614", 3, None) + _later(monkeypatch, subtitles.FAIL_TTL_S + 1) + for _ in range(3): + assert subtitles.ensure("98614", 3, at=100.0)["state"] == "partial" + assert pool.jobs == [], "a failed full pass was queued again" + + +def test_a_failed_window_does_not_end_the_poll_while_the_full_pass_may_cover_it(monkeypatch): + """ "failed" is final — the player stops asking. A window that failed used to set the TRACK's + failure memo, so a viewer anywhere outside the covered part got "failed" and never heard about + the full pass that landed a minute later.""" + pool = _manual_pool(monkeypatch) + _stub_item(monkeypatch) + _window_raises(monkeypatch) + subtitles.ensure("98614", 3, at=3000.0) + pool.run_next() # the window fails; the full pass is still queued behind it + assert subtitles.ensure("98614", 3, at=3000.0)["state"] == "preparing" + assert subtitles.ensure("98614", 3, at=500.0)["state"] == "preparing" + assert [a[3] for _, a in pool.jobs] == [None, 470.0], "the failed stretch was not tried again" + + +def test_a_backward_seek_gets_its_own_window(monkeypatch): + """A window bucketed by position joined a seek back to 6:40 onto a running window at 9:10 that + could not cover it — the right one waited for the wrong one.""" + pool = _manual_pool(monkeypatch) + subtitles.ensure("98614", 1, at=580.0) + subtitles.ensure("98614", 1, at=400.0) + assert sorted(a[3] for _, a in pool.jobs if a[3] is not None) == [370.0, 550.0] + + +def test_a_playhead_crossing_a_bucket_edge_joins_the_window_that_covers_it(monkeypatch): + """…and the same bucketing queued a second read when a playhead crossed an edge while the first + window — which already covered it — was still running.""" + pool = _manual_pool(monkeypatch) + for at in (628.0, 629.0, 630.0, 631.0, 632.0): + subtitles.ensure("98614", 1, at=at) + assert len(pool.jobs) == 2, "one window + the one full pass" + + +def test_a_scrubbing_viewer_cannot_queue_a_window_per_stop(monkeypatch): + """The pool's queue is unbounded and every job is a read of the library.""" + pool = _manual_pool(monkeypatch) + for at in (0.0, 1000.0, 2000.0, 3000.0, 4000.0): + subtitles.ensure("98614", 1, at=at) + assert len([1 for _, a in pool.jobs if a[3] is not None]) == subtitles._MAX_TRACK_WINDOWS + + +def test_nothing_more_is_queued_while_the_extractor_is_saturated(monkeypatch): + pool = _manual_pool(monkeypatch) + for rk in range(20): + subtitles.ensure(str(rk), 0, at=0.0) + windows = [a for _, a in pool.jobs if a[3] is not None] + assert len(windows) == subtitles._MAX_WINDOWS_INFLIGHT + assert len(pool.jobs) - len(windows) == subtitles._MAX_FULL_INFLIGHT + + +def test_a_window_starts_where_ffmpeg_is_told_to_seek(monkeypatch): + """ffmpeg gets `-ss` to the millisecond, and its cues come back on THAT zero. Shifted back by an + unrounded start they landed up to half a millisecond off the full pass's — spelled differently, + and so kept twice by the merge.""" + pool = _manual_pool(monkeypatch) + subtitles.ensure("98614", 1, at=1198.4375) + (window_start,) = [a[3] for _, a in pool.jobs if a[3] is not None] + assert window_start == 1168.438 + + +def test_a_millisecond_rounded_up_carries_into_the_minute(): + """`00:00:60.000` is not a WebVTT timestamp — the browser drops the cue.""" + assert subtitles._vtt_s_to_ts(59.9996) == "00:01:00.000" + assert subtitles._vtt_s_to_ts(3599.9996) == "01:00:00.000" + shifted = subtitles.shift_vtt(_vtt(_cue("00:01:00.000", "00:01:02.000", "x")), 0.0004) + assert "00:01:00.000 --> 00:01:02.000" in shifted, shifted + + +def test_ffmpegs_spelling_of_a_negative_time_reads_exactly(): + """With `-avoid_negative_ts disabled` a cue already running at the window's start comes out + before zero, and ffmpeg's muxer puts the sign on every field.""" + assert subtitles._vtt_ts_to_s("00:-5.-05") == pytest.approx(-5.005) + assert subtitles._vtt_ts_to_s("-1:-5.-300") == pytest.approx(-65.3) + + +def test_dying_between_the_two_renames_cannot_pass_a_window_off_as_the_whole_track(monkeypatch): + """A VTT without a sidecar is how an older build cached a COMPLETE track. A publish is two + renames, the VTT first — so dying between them on a key's first publish left a window that + read as complete, "ready" for good.""" + key = subtitles.cache_key("98614", 0) + real_write = subtitles.cache_write + + def _die_after(k, vtt): + real_write(k, vtt) + raise KeyboardInterrupt # the process goes away right after the VTT landed + + monkeypatch.setattr(subtitles, "cache_write", _die_after) + with pytest.raises(KeyboardInterrupt): + subtitles.publish(key, _vtt(_cue("10:00.000", "10:02.000", "x")), (570.0, 900.0), False) + assert subtitles.read_meta(key) == ([], False) + + +def test_shutdown_drops_queued_extractions(monkeypatch): + """Only the kill half used to be pinned. A job still QUEUED at shutdown would otherwise start + after the kill sweep — a fresh read of the file holding the process open.""" + pool = ThreadPoolExecutor(max_workers=1) + monkeypatch.setattr(subtitles, "_WIN_POOL", pool) + monkeypatch.setattr(subtitles, "_FULL_POOL", pool) + busy, release, ran = threading.Event(), threading.Event(), threading.Event() + pool.submit(lambda: (busy.set(), release.wait(timeout=5))) + pool.submit(ran.set) + assert busy.wait(timeout=5) + subtitles.shutdown() + release.set() + pool.shutdown(wait=True) + assert not ran.is_set(), "a queued extraction ran after shutdown" + + +def test_a_worker_that_reaches_ffmpeg_after_shutdown_never_starts_it(monkeypatch): + """A worker still in its Plex lookup when the kill sweep ran used to start its ffmpeg anyway.""" + _real_pools(monkeypatch) + started: list = [] + monkeypatch.setattr(subtitles.subprocess, "Popen", lambda cmd, **kw: started.append(cmd)) + subtitles.shutdown() + with pytest.raises(subtitles.SubtitleUnavailable): + subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], None, None) + assert started == [] + + +def test_the_app_stops_the_extractor_when_it_shuts_down(monkeypatch): + """No test ran the lifespan, so deleting this one call switched all of the above off with every + lane green.""" + + class _Configured: + def __enter__(self): + return None + + def __exit__(self, *exc): + return False + + async def _no_sweep(): + return None + + calls: list[str] = [] + monkeypatch.setattr(app_main, "SessionLocal", _Configured) + monkeypatch.setattr(app_main.state, "is_configured", lambda db: True) + monkeypatch.setattr(app_main, "_sweep_hls_orphans", _no_sweep) + monkeypatch.setattr(app_main, "start_scheduler", lambda: None) + monkeypatch.setattr(app_main, "shutdown_scheduler", lambda: None) + monkeypatch.setattr(app_main.plex_subtitles, "shutdown", lambda: calls.append("stopped")) + + async def _cycle(): + async with app_main.lifespan(app): + pass + + asyncio.run(_cycle()) + assert calls == ["stopped"] + + +# --- a window against a REAL ffmpeg ------------------------------------------------------------- + + +def _make_mkv(tmp_path: Path) -> Path: + """A minute of video with a keyframe every 10 s and a subtitle every 2 s — so a seek to 21.3 s + lands on the keyframe at 20 s, with the 20.0–21.5 s cue already running at the window's start.""" + srt = tmp_path / "s.srt" + srt.write_text( + "".join( + f"{i + 1}\n00:00:{2 * i:02d},000 --> 00:00:{2 * i + 1:02d},500\nline {i}\n\n" + for i in range(30) + ), + "utf-8", + ) + out = tmp_path / "f.mkv" + subprocess.run( + [ + "ffmpeg", "-nostdin", "-loglevel", "error", + "-f", "lavfi", "-i", "color=c=black:s=32x32:r=10:d=60", + "-i", str(srt), + "-map", "0:v", "-map", "1:s", "-c:v", "mpeg4", "-g", "100", "-c:s", "srt", + "-y", str(out), + ], + check=True, + ) # fmt: skip + return out + + +@needs_ffmpeg +def test_a_window_lines_up_with_the_full_pass_on_a_real_file(tmp_path, monkeypatch): + """THE late-and-doubled window. The input seek lands on the keyframe before the window's start, + a cue already running there comes out at a negative time, and by default ffmpeg's muxer shifted + the whole window so that cue sat at 0 — every cue late by the distance to the keyframe (1–5 s on + a real library file), and kept twice for good once the full pass merged in beside it.""" + src = _make_mkv(tmp_path) + monkeypatch.setattr(subtitles, "CACHE_DIR", tmp_path / "cache") + full, clean = subtitles._ffmpeg_extract(str(src), [0], None, None) + assert clean + win, clean = subtitles._ffmpeg_extract(str(src), [0], 21.3, 20.0) + assert clean + key = subtitles.cache_key("1", 0) + subtitles.publish(key, subtitles.shift_vtt(win[0], -21.3), (21.3, 41.3), False) + served = subtitles.cache_read(key) or "" + assert "00:00:22.000 --> 00:00:23.500\nline 11" in served, served # on time, not 1.3 s late + subtitles.publish(key, full[0], (0.0, subtitles._FULL_END_S), True) + merged = subtitles.cache_read(key) or "" + assert merged.count("-->") == 30, "a window cue survived beside its full-pass twin" + + +# --- the third deep-review round ---------------------------------------------------------------- + + +def test_a_transient_failure_is_said_but_is_not_a_final_answer(monkeypatch): + """`failed` ends the player's poll and unmounts the track. A Plex hiccup memo (a minute) was given + as `failed`, so one blip ended the subtitles for the rest of the session — at cached positions + too, and after the full pass had completed behind it. It is `unavailable` now: said, asked again.""" + pool = _manual_pool(monkeypatch) + key = subtitles.cache_key("98614", 0) + subtitles.publish(key, _vtt(_cue("01:40.000", "01:42.000", "x")), (70.0, 400.0), False) + subtitles._fail(key, "plex_error") + here = subtitles.ensure("98614", 0, at=3000.0) + assert (here["state"], here["code"]) == ("unavailable", "plex_error") + assert here["gen"] > 0, "the client needs to know there are cues elsewhere to keep showing" + assert subtitles.ensure("98614", 0, at=100.0)["state"] == "partial" + assert pool.jobs == [], "nothing is re-read while the memo is alive" + _later(monkeypatch, subtitles.FAIL_TTL_S + 1) + assert subtitles.ensure("98614", 0, at=3000.0)["state"] == "preparing" + assert [a[3] for _, a in pool.jobs] == [2970.0, None], "asked again once the memo lapsed" + + +def test_the_next_window_is_read_ahead_of_its_edge(monkeypatch): + """With the full pass backed off (or slower than a window lasts) a window was only asked for once + the playhead had LEFT coverage — the track unmounted at every edge for a poll plus a read.""" + pool = _manual_pool(monkeypatch) + subtitles._FULL_FAILED["98614"] = time.monotonic() + key = subtitles.cache_key("98614", 0) + subtitles.publish(key, _vtt(_cue("16:20.000", "16:22.000", "x")), (970.0, 1300.0), False) + assert subtitles.ensure("98614", 0, at=1200.0)["state"] == "partial" + assert pool.jobs == [], "far from the edge: nothing to read yet" + assert subtitles.ensure("98614", 0, at=1250.0)["state"] == "partial" + assert [a[3] for _, a in pool.jobs] == [1271.0], "the next window, before the edge is reached" + + +def test_a_window_that_died_partway_marks_the_stretch_it_did_not_reach(monkeypatch): + """Without the span memo, a window that died before the playhead left it uncovered AND not + failed — so every poll queued the same read of the library again.""" + _stub_item(monkeypatch, clean=False, cues=("00:10.000", "00:12.000")) + pool = _manual_pool(monkeypatch) + subtitles._extract_and_publish("98614", 3, 3570.0) + key = subtitles.cache_key("98614", 3) + assert subtitles.read_meta(key) == ([[3570.0, 3582.0]], False) + assert subtitles.ensure("98614", 3, at=3700.0)["state"] == "preparing" + assert [a[3] for _, a in pool.jobs] == [None], "the dead stretch was queued again" + + +def test_a_crashed_pass_is_recorded_like_a_failed_one(monkeypatch): + """A timeout or a crash lands in `_run`'s catch-all. Recorded against the TRACK, a crashed window + told every viewer at any uncovered position "failed", and a crashed full pass re-armed a + whole-file read every minute instead of backing off.""" + + def _crash(rating_key, sub_ord, window_start): + raise RuntimeError("boom") + + monkeypatch.setattr(subtitles, "_extract_and_publish", _crash) + key = subtitles.cache_key("98614", 3) + subtitles._run(("win", "98614", 3, 570.0), "98614", 3, 570.0, 600.0) + assert subtitles._window_failed(key, 600.0) and not subtitles._window_failed(key, 3000.0) + subtitles._run(("full", "98614"), "98614", 3, None, 600.0) + assert subtitles._full_failed("98614") + assert subtitles._recent_failure(key) is None, "a crash is not a verdict on the whole track" + + +def test_the_failure_memos_lapse(monkeypatch): + """Both memos exist to stop a re-read, not to forbid one: permanent, a transient failure would + have held until the process restarted.""" + pool = _manual_pool(monkeypatch) + key = subtitles.cache_key("98614", 3) + subtitles._pass_failed("98614", [key], None) + subtitles._pass_failed("98614", [key], 570.0) + assert subtitles.ensure("98614", 3, at=600.0)["state"] == "unavailable" + _later(monkeypatch, subtitles.FAIL_TTL_S + 1) + subtitles.ensure("98614", 3, at=600.0) + assert [a[3] for _, a in pool.jobs] == [570.0], "the window memo lapsed; the full one has not" + pool.jobs.clear() + subtitles._INFLIGHT.clear() + _later(monkeypatch, subtitles.FULL_RETRY_S + 1) + subtitles.ensure("98614", 3, at=600.0) + assert None in [a[3] for _, a in pool.jobs], "the full pass was never tried again" + + +def test_lapsed_memos_are_swept_not_kept_forever(monkeypatch): + """They were pruned only when the SAME key was read again, and the status route takes any key: + polling fresh ones grew the API process without bound.""" + for i in range(50): + subtitles._fail(f"sub_x{i}_0", "no_item") + subtitles._pass_failed(f"x{i}", [f"sub_x{i}_0"], None) + subtitles._pass_failed(f"x{i}", [f"sub_x{i}_0"], 0.0) + _later(monkeypatch, subtitles.FULL_RETRY_S + 1) + subtitles._fail("sub_y_0", "no_item") + assert list(subtitles._FAILURES) == ["sub_y_0"] + assert subtitles._FULL_FAILED == {} and subtitles._WIN_FAILED == {} + + +def test_a_window_is_not_refused_while_full_passes_hold_their_slots(monkeypatch): + """One pool and one cap for both kinds of job: full passes (minutes each) held every worker and + every slot, and a seek window — seconds of work — waited behind them or was not queued at all.""" + win, full = _ManualPool(), _ManualPool() + monkeypatch.setattr(subtitles, "_WIN_POOL", win) + monkeypatch.setattr(subtitles, "_FULL_POOL", full) + for rk in range(subtitles._MAX_FULL_INFLIGHT): + subtitles._INFLIGHT[("full", str(rk))] = None + subtitles.ensure("98614", 1, at=600.0) + assert [a[3] for _, a in win.jobs] == [570.0] + assert full.jobs == [], "past its bound no full pass is queued" + + +def test_a_poll_after_shutdown_queues_nothing(monkeypatch): + pool = _manual_pool(monkeypatch) + subtitles.shutdown() + subtitles.ensure("98614", 1, at=600.0) + assert pool.jobs == [] + + +def test_a_job_dequeued_after_shutdown_reads_nothing(monkeypatch): + pool = _manual_pool(monkeypatch) + ran: list = [] + monkeypatch.setattr(subtitles, "_extract_and_publish", lambda *a: ran.append(a)) + subtitles.ensure("98614", 1, at=600.0) + subtitles.shutdown() + pool.run_all() + assert ran == [] + + +def test_a_child_started_while_the_shutdown_sweep_runs_is_killed(monkeypatch): + """The sweep can run between the pre-Popen check and the child's registration.""" + killed: list = [] + + class _Child: + returncode = -9 + + def kill(self): + killed.append(True) + + def communicate(self, timeout=None): + return b"", b"" + + def _popen(cmd, **kw): + subtitles._STOPPING = True # the shutdown sweep ran while this child was being started + return _Child() + + monkeypatch.setattr(subtitles.subprocess, "Popen", _popen) + with pytest.raises(subtitles.SubtitleUnavailable): + subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], None, None) + assert killed == [True] + + +def test_a_pass_that_times_out_keeps_what_it_reached(monkeypatch): + """It was killed and its outputs deleted unread — so a file too big to read inside the timeout + never got past the head, and with the full-pass backoff the same doomed read came back hourly.""" + real_popen = subprocess.Popen + cue = _vtt(_cue("00:05.000", "00:07.000", "reached")) + + def _popen(cmd, **kw): + out = cmd[cmd.index("-y") + 1] + script = f"import time\nopen({out!r}, 'w').write({cue!r})\ntime.sleep(60)" + return real_popen([sys.executable, "-c", script], **kw) + + monkeypatch.setattr(subtitles.subprocess, "Popen", _popen) + monkeypatch.setattr(subtitles, "EXTRACT_TIMEOUT_S", 1.0) + out, clean = subtitles._ffmpeg_extract("/plex-media/f.mkv", [0], None, None) + assert not clean, "a pass that ran out of time must not pass for a whole one" + assert "reached" in out[0] + + +def test_an_orphan_sidecar_is_not_merged_into(monkeypatch): + """A sidecar with no VTT behind it claims cues that are not there. Merged into, it made one + window read as the whole track — `ready`, and the full pass skipped as already done.""" + key = subtitles.cache_key("98614", 0) + subtitles._write_meta(key, [[0.0, subtitles._FULL_END_S]], True) + subtitles.publish(key, _vtt(_cue("16:20.000", "16:22.000", "x")), (970.0, 1300.0), False) + assert subtitles.read_meta(key) == ([[970.0, 1300.0]], False) + assert subtitles.ensure("98614", 0, at=5000.0)["state"] != "ready" + + +def test_an_unreadable_track_is_not_overwritten_by_one_window(monkeypatch): + """`cache_read` answers None for a file it could not READ as well as for one that is not there. + Taken for absent, a whole track was replaced by the one window being published.""" + key = subtitles.cache_key("98614", 0) + whole = _vtt(_cue("00:01.000", "00:02.000", "whole")) + subtitles.publish(key, whole, (0.0, subtitles._FULL_END_S), True) + monkeypatch.setattr(subtitles, "cache_read", lambda k: None) # an I/O error on the read + subtitles.publish(key, _vtt(_cue("16:20.000", "16:22.000", "x")), (970.0, 1300.0), False) + assert "whole" in subtitles._vtt_path(key).read_text("utf-8") + + +def test_a_region_that_could_not_be_written_is_not_claimed(monkeypatch): + monkeypatch.setattr(subtitles, "cache_write", lambda key, vtt: False) + key = subtitles.cache_key("98614", 0) + subtitles.publish(key, _vtt(_cue("16:20.000", "16:22.000", "x")), (0.0, 1e9), True) + assert subtitles.read_meta(key) == ([], False) + + +def test_a_nul_in_the_rating_key_is_not_a_500(client, monkeypatch): + """`%00` reached `Path.stat`, which raises ValueError rather than OSError — a 500 on every poll, + after the scheduler had already queued two jobs for a key that names nothing.""" + _manual_pool(monkeypatch) + assert client.get("/api/plex/subtitle/%00/0/status").status_code == 200 + + +def test_a_nul_rating_key_names_no_row(): + """Postgres refuses NUL in text, so looking it up was a DataError (a 500) on every Plex route.""" + from app.models import PlexItem + from app.plex.lookups import by_rating_key + + assert by_rating_key(None, PlexItem, "12\x0034") is None # type: ignore[arg-type] + + +# --- the fourth deep-review round --------------------------------------------------------------- + + +def test_a_pass_whose_output_could_not_be_kept_is_backed_off_like_a_failed_one(monkeypatch): + """`publish` refusing (a full or read-only cache volume) returned quietly, so `_run` recorded + nothing and every 1 s poll queued the window AND the whole-container pass again — the file read + end to end over and over, the player on "preparing" for good.""" + _stub_item(monkeypatch) + pool = _manual_pool(monkeypatch) + monkeypatch.setattr(subtitles, "cache_write", lambda key, vtt: False) + assert subtitles.ensure("98614", 3, at=600.0)["state"] == "preparing" + assert [a[3] for _, a in pool.jobs] == [570.0, None] + pool.run_all() + after = subtitles.ensure("98614", 3, at=600.0) + assert (after["state"], after["code"]) == ("unavailable", "extract_failed") + for _ in range(3): + subtitles.ensure("98614", 3, at=600.0) + assert pool.jobs == [], "the same reads were queued again" + + +def test_a_sidecar_that_could_not_be_kept_is_recorded_as_failed(monkeypatch): + _stub_item(monkeypatch) + pool = _manual_pool(monkeypatch) + monkeypatch.setattr(subtitles, "cache_write", lambda key, vtt: False) + subtitles.ensure("98614", 0, at=600.0) + pool.run_all() + subtitles.ensure("98614", 0, at=600.0) + assert pool.jobs == [], "a refused sidecar publish was queued again" + + +def test_an_unreadable_cached_track_is_said_once_and_not_rescheduled(monkeypatch): + """A VTT on disk that cannot be read: `publish` will not merge into it and a job whose sidecar + covers the position skips as done, so scheduling could only ever leave the player on + "preparing".""" + pool = _manual_pool(monkeypatch) + key = subtitles.cache_key("98614", 0) + subtitles.publish(key, _vtt(_cue("16:20.000", "16:22.000", "x")), (970.0, 1300.0), False) + monkeypatch.setattr(subtitles, "cache_read", lambda k: None) # an I/O error on the read + st = subtitles.ensure("98614", 0, at=1000.0) + assert (st["state"], st["code"], st["gen"]) == ("unavailable", "cache_unreadable", 0) + assert pool.jobs == [] + + +def test_an_absurd_rating_key_or_ordinal_is_refused_not_a_500(client, monkeypatch): + """They become a cache FILE name: a 250-digit key outgrew the OS name limit, and pathlib's + `exists()` re-raises that (ENAMETOOLONG) instead of answering False — a 500 on every poll.""" + pool = _manual_pool(monkeypatch) + long_key = "9" * 250 + for url in ( + f"/api/plex/subtitle/{long_key}/0/status", + f"/api/plex/subtitle/98614/{long_key}/status", + f"/api/plex/subtitle/{long_key}/0", + ): + assert client.get(url).status_code == 422, url + assert pool.jobs == [] + + # …and the cache helpers themselves no longer throw on such a name. Asserted against a forced + # ENAMETOOLONG rather than the real filesystem: on Linux pathlib re-raises it (the defect), but + # Windows reports ERROR_INVALID_NAME, which pathlib's `exists()` already swallows — so on a dev + # machine this assertion passed with `_exists` reverted and only the container lane was honest + # about it (deep-review round 5). + def _too_long(self): + raise OSError(errno.ENAMETOOLONG, "File name too long") + + monkeypatch.setattr(Path, "exists", _too_long) + assert subtitles._exists(Path("whatever")) is False + assert subtitles.read_meta(subtitles.cache_key(long_key, 0)) == ([], False) + + +def test_a_read_ahead_window_that_failed_is_not_queued_again(monkeypatch): + """Near a covered edge the probe moves just past it, and `_schedule` has no failed-span check of + its own: without the guard, every poll re-queued the same failing window read.""" + pool = _manual_pool(monkeypatch) + subtitles._FULL_FAILED["98614"] = time.monotonic() + key = subtitles.cache_key("98614", 0) + subtitles.publish(key, _vtt(_cue("16:20.000", "16:22.000", "x")), (970.0, 1300.0), False) + subtitles._window_fail(key, (1271.0, 1601.0)) + for at in (1250.0, 1252.0, 1254.0): + assert subtitles.ensure("98614", 0, at=at)["state"] == "partial" + assert pool.jobs == [], "the failed read-ahead window was queued again" diff --git a/backend/tests/test_progress_bounds.py b/backend/tests/test_progress_bounds.py new file mode 100644 index 00000000..7736cb7c --- /dev/null +++ b/backend/tests/test_progress_bounds.py @@ -0,0 +1,63 @@ +"""A progress checkpoint takes a FINITE position. + +The JSON parser under FastAPI accepts `Infinity` and `NaN`, and both progress handlers truncate the +position with `int()` — so one hand-written checkpoint came back as a 500 with a traceback. Found in +the 2026-09 bug hunt's second review round, as the sibling of the subtitle routes' non-finite +`at`/`offset`. Pure: the body is refused before the handler touches the database. +""" + +from types import SimpleNamespace + +import pytest +from fastapi.testclient import TestClient + +import app.state as app_state +from app.auth import current_user +from app.main import app + + +@pytest.fixture +def client(monkeypatch, override_dep) -> TestClient: + override_dep(current_user, lambda: SimpleNamespace(id=1)) + monkeypatch.setattr(app_state, "_configured_cache", True) # pass the instance-setup gate, no DB + return TestClient(app) + + +@pytest.mark.parametrize("path", ["/api/videos/v1/progress", "/api/plex/item/98614/progress"]) +@pytest.mark.parametrize("raw", ["Infinity", "-Infinity", "NaN"]) +def test_a_non_finite_position_is_refused(client, path, raw): + r = client.post( + path, + content=f'{{"position_seconds": {raw}, "duration_seconds": 3000}}', + headers={"content-type": "application/json"}, + ) + assert r.status_code == 422, r.text + + +@pytest.mark.parametrize("path", ["/api/videos/v1/progress", "/api/plex/item/98614/progress"]) +@pytest.mark.parametrize("field", ["position_seconds", "duration_seconds"]) +def test_a_position_past_the_int4_column_is_refused_not_a_500(client, path, field): + """Finite but past the int4 column: the flush was a DataError, i.e. a 500 with a traceback.""" + body = {"position_seconds": 10, "duration_seconds": 3000, field: 2**31} + assert client.post(path, json=body).status_code == 422 + + +def test_the_refusal_keeps_the_shape_the_ui_reads(client): + """The handler replaces FastAPI's own for every validation error in the app, and the UI turns + exactly this shape — a LIST of errors — into its localized "check your input" sentence. The + refused value is echoed by its name, because JSON cannot carry NaN.""" + r = client.post( + "/api/videos/v1/progress", + content='{"position_seconds": NaN, "duration_seconds": 3000}', + headers={"content-type": "application/json"}, + ) + detail = r.json()["detail"] + assert isinstance(detail, list) and detail[0]["loc"] == ["body", "position_seconds"] + assert detail[0]["input"] == "nan" + + +@pytest.mark.parametrize("q", ["start=inf", "start=nan", "aoff=inf", "aoff=-inf", "aoff=nan"]) +def test_a_session_at_an_impossible_offset_is_refused(client, q): + """`start=inf` reached `int()` in the session builder (a 500); `aoff=inf` spawned an ffmpeg + with a non-finite offset.""" + assert client.post(f"/api/plex/stream/98614/session?{q}").status_code == 422 diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 8d9de728..707e07c6 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -279,7 +279,8 @@ export default function App() { label: t("common.accessRequestsReview"), onClick: () => { focusAccessRequestsTab(); - setPage("users"); + // Over whatever overlay the toast was clicked in — as navigateTo does (see lib/nav). + navigateOverOverlays(() => setPage("users")); }, }, }); diff --git a/frontend/src/components/AddToPlaylist.tsx b/frontend/src/components/AddToPlaylist.tsx index 716279ef..e795b38a 100644 --- a/frontend/src/components/AddToPlaylist.tsx +++ b/frontend/src/components/AddToPlaylist.tsx @@ -4,6 +4,7 @@ import { useQuery, useQueryClient } from "@tanstack/react-query"; import { qk } from "../lib/queryKeys"; import { Check, ListPlus, Plus, Youtube } from "lucide-react"; import { api, type Playlist } from "../lib/api"; +import { notifyActionError } from "../lib/actionErrors"; import { useScrollFade } from "../lib/useScrollFade"; import { playlistName } from "../lib/playlistName"; import { PopoverItem, PopoverSurface, useAnchoredPopover } from "./ui/AnchoredPopover"; @@ -49,14 +50,19 @@ export default function AddToPlaylist({ qc.invalidateQueries({ queryKey: qk.playlist() }); } + // A failure is said, and the list refreshed either way. With no catch, a playlist deleted on + // another device (a 404 `not_found/playlist` — still offered, from a cache up to 30 s old) was an + // unhandled rejection: the tick did not change, nothing was said, the gone row stayed. async function toggle(pl: Playlist) { if (busy) return; setBusy(true); try { if (pl.has_video) await api.removeFromPlaylist(pl.id, videoId); else await api.addToPlaylist(pl.id, videoId); - refresh(); + } catch (e) { + notifyActionError(e, t(pl.has_video ? "playlists.removeFailed" : "playlists.addFailed")); } finally { + refresh(); setBusy(false); } } @@ -69,8 +75,10 @@ export default function AddToPlaylist({ const pl = await api.createPlaylist(name); await api.addToPlaylist(pl.id, videoId); setNewName(""); - refresh(); + } catch (e) { + notifyActionError(e, t("playlists.addFailed")); } finally { + refresh(); setBusy(false); } } diff --git a/frontend/src/components/AdminUsers.dom.test.tsx b/frontend/src/components/AdminUsers.dom.test.tsx new file mode 100644 index 00000000..d94ad5e8 --- /dev/null +++ b/frontend/src/components/AdminUsers.dom.test.tsx @@ -0,0 +1,103 @@ +// @vitest-environment jsdom +// +// A failed user action is said once, and a user who is gone is not left listed (deep-review round 4). +// setRole / suspend / delete had no onError at all — "the global error dialog says it" — but a 404 +// (the user was deleted in another tab) gets no dialog, so the click did nothing and the row stayed. +// And the invite approve/deny toasted unconditionally on top of the dialog `req` opens for a 409. +import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; +import { render, cleanup, screen, fireEvent, waitFor } from "@testing-library/react"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; + +vi.mock("react-i18next", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }), + }; +}); +vi.mock("../lib/useMe", () => ({ + useMe: () => ({ + id: 1, + email: "admin@x", + role: "admin", + pending_invites: 0, + pending_activations: 0, + }), +})); +vi.mock("./ConfirmProvider", () => ({ useConfirm: () => async () => true })); +const { notify } = vi.hoisted(() => ({ + notify: vi.fn((_n: { level: string; message: string }) => 0), +})); +vi.mock("../lib/notifications", async (importOriginal) => ({ + ...(await importOriginal()), + notify, +})); +vi.mock("../lib/errorDialog", () => ({ reportError: vi.fn() })); + +import AdminUsers from "./AdminUsers"; +import { api, type AdminUserRow } from "../lib/api"; +import i18n from "../i18n"; + +const row = { + id: 2, + email: "gone@x", + role: "user", + is_suspended: false, + is_demo: false, + email_verified: true, + is_active: true, +} as unknown as AdminUserRow; + +function answer(status: number, detail: unknown) { + return new Response(JSON.stringify({ detail }), { + status, + headers: { "content-type": "application/json" }, + }); +} + +beforeEach(() => { + notify.mockClear(); + localStorage.clear(); +}); +afterEach(() => { + cleanup(); + vi.restoreAllMocks(); +}); + +function mount() { + const users = vi.spyOn(api, "adminUsers").mockResolvedValue([row] as never); + render( + + + , + ); + return users; +} + +describe("a user action that fails", () => { + it("says a user who is gone, and refetches the list", async () => { + const users = mount(); + vi.spyOn(globalThis, "fetch").mockResolvedValue( + answer(404, { code: "not_found", reason: "user" }), + ); + fireEvent.click(await screen.findByText("users.roles.makeAdmin")); + await waitFor(() => + expect(notify).toHaveBeenCalledWith({ + level: "error", + message: i18n.t("errors.notFound.user"), // the request layer localizes a structured detail + }), + ); + await waitFor(() => expect(users).toHaveBeenCalledTimes(2)); + }); + + it("is not toasted on top of the error dialog the request already opened", async () => { + mount(); + vi.spyOn(globalThis, "fetch").mockResolvedValue(answer(409, "Cannot demote the last admin")); + fireEvent.click(await screen.findByText("users.roles.makeAdmin")); + const { reportError } = await import("../lib/errorDialog"); + await waitFor(() => expect(reportError).toHaveBeenCalled()); + expect(notify.mock.calls.filter(([n]) => n.level === "error")).toEqual([]); + }); +}); diff --git a/frontend/src/components/AdminUsers.tsx b/frontend/src/components/AdminUsers.tsx index 55333a27..90ab5f27 100644 --- a/frontend/src/components/AdminUsers.tsx +++ b/frontend/src/components/AdminUsers.tsx @@ -38,6 +38,7 @@ import { useConfirm } from "./ConfirmProvider"; import Tabs, { usePersistedTab } from "./ui/Tabs"; import { PageToolbar } from "./ui/PageShell"; import { useMe } from "../lib/useMe"; +import { notifyActionError } from "../lib/actionErrors"; // Admin-only user & access management: roles, access requests (the Invite whitelist), and the // demo whitelist + reset. Each section is its own tab (persisted across reloads); the Access tab @@ -120,6 +121,13 @@ function UsersRoles({ me }: { me: Me }) { n.delete(id); return n; }); + // A failed user action: said once (notifyActionError skips what `req` already put in its dialog), + // and the list refetched — a user deleted in another tab would otherwise stay listed, every click + // on the row doing nothing. + const onGoneOrFailed = (e: unknown, fallback: string) => { + notifyActionError(e, fallback); + qc.invalidateQueries({ queryKey: qk.adminUsers() }); + }; const setRole = useMutation({ mutationFn: ({ id, role }: { id: number; role: "user" | "admin"; email: string }) => api.setUserRole(id, role), @@ -129,7 +137,9 @@ function UsersRoles({ me }: { me: Me }) { notify({ level: "success", message: t("users.roles.updated", { email: vars.email }) }); }, onSettled: (_d, _e, vars) => endPending(vars.id), - // Errors (e.g. last-admin / self) surface via the global error dialog with the server's detail. + // Guards (last admin / self) surface via the global error dialog with the server's detail; what + // it does not show (a user deleted meanwhile, a 404) is said here — see onGoneOrFailed. + onError: (e) => onGoneOrFailed(e, t("users.roles.failed")), }); const suspend = useMutation({ @@ -146,7 +156,8 @@ function UsersRoles({ me }: { me: Me }) { }); }, onSettled: (_d, _e, vars) => endPending(vars.id), - // Guards (demo / self / last admin) surface via the global error dialog. + // Guards (demo / self / last admin) surface via the global error dialog; the rest, as above. + onError: (e) => onGoneOrFailed(e, t("users.suspend.failed")), }); const del = useMutation({ mutationFn: ({ id }: { id: number; email: string }) => api.adminDeleteUser(id), @@ -156,6 +167,7 @@ function UsersRoles({ me }: { me: Me }) { notify({ level: "success", message: t("users.delete.done", { email: vars.email }) }); }, onSettled: (_d, _e, vars) => endPending(vars.id), + onError: (e) => onGoneOrFailed(e, t("users.delete.failed")), }); const onToggle = async (u: AdminUserRow) => { @@ -518,7 +530,7 @@ function AdminInvites() { refresh(); notify({ level: "success", message: t("settings.invites.approved", { email: vars.email }) }); }, - onError: () => notify({ level: "error", message: t("settings.invites.approveFailed") }), + onError: (e) => notifyActionError(e, t("settings.invites.approveFailed")), }); const deny = useMutation({ mutationFn: (id: number) => api.denyInvite(id), @@ -526,7 +538,7 @@ function AdminInvites() { refresh(); notify({ level: "success", message: t("settings.invites.denied") }); }, - onError: () => notify({ level: "error", message: t("settings.invites.denyFailed") }), + onError: (e) => notifyActionError(e, t("settings.invites.denyFailed")), }); const onDeny = async (i: { id: number; email: string }) => { const ok = await confirm({ @@ -630,7 +642,7 @@ function AdminInvites() { refresh(); notify({ level: "success", message: t("settings.invites.addedToWhitelist", { email }) }); }, - onError: () => notify({ level: "error", message: t("settings.invites.addFailed") }), + onError: (e) => notifyActionError(e, t("settings.invites.addFailed")), }); const list = invites.data ?? []; @@ -780,7 +792,7 @@ function AdminDemo() { qc.invalidateQueries({ queryKey: qk.demoWhitelist() }); notify({ level: "success", message: t("settings.demo.added", { email }) }); }, - onError: () => notify({ level: "error", message: t("settings.demo.addFailed") }), + onError: (e) => notifyActionError(e, t("settings.demo.addFailed")), }); const remove = useMutation({ mutationFn: (id: number) => api.removeDemoWhitelist(id), @@ -788,7 +800,7 @@ function AdminDemo() { qc.invalidateQueries({ queryKey: qk.demoWhitelist() }); notify({ level: "success", message: t("settings.demo.removed") }); }, - onError: () => notify({ level: "error", message: t("settings.demo.removeFailed") }), + onError: (e) => notifyActionError(e, t("settings.demo.removeFailed")), }); const onRemove = async (r: { id: number; email: string }) => { const ok = await confirm({ @@ -806,7 +818,7 @@ function AdminDemo() { level: "success", message: t("settings.demo.resetDone", { count: r.playlists_seeded }), }), - onError: () => notify({ level: "error", message: t("settings.demo.resetFailed") }), + onError: (e) => notifyActionError(e, t("settings.demo.resetFailed")), }); const rows = list.data ?? []; diff --git a/frontend/src/components/AuditLog.tsx b/frontend/src/components/AuditLog.tsx index e382a24a..0fa2618e 100644 --- a/frontend/src/components/AuditLog.tsx +++ b/frontend/src/components/AuditLog.tsx @@ -12,6 +12,7 @@ import { useConfirm } from "./ConfirmProvider"; import DataTable from "./ui/DataTable"; import { LoadingState, StateMessage } from "./ui/QueryState"; import { auditColumns } from "./auditColumns"; +import { notifyActionError } from "../lib/actionErrors"; // Admin-only audit trail: an append-only who/what/when log of admin actions + scheduler run // summaries (never per-row). Read-only apart from Clear. Client-side sort/filter/paginate via @@ -36,7 +37,7 @@ export default function AuditLog() { qc.invalidateQueries({ queryKey: qk.adminAudit() }); notify({ level: "success", message: t("audit.cleared", { count: r.cleared }) }); }, - onError: () => notify({ level: "error", message: t("audit.clearFailed") }), + onError: (e) => notifyActionError(e, t("audit.clearFailed")), }); const onClear = async () => { if ( diff --git a/frontend/src/components/ChannelNameLink.dom.test.tsx b/frontend/src/components/ChannelNameLink.dom.test.tsx new file mode 100644 index 00000000..481d7a48 --- /dev/null +++ b/frontend/src/components/ChannelNameLink.dom.test.tsx @@ -0,0 +1,109 @@ +// @vitest-environment jsdom +// +// Reported from daily use: middle-clicking the channel name in the YT player did nothing. It looked +// like a link — the YouTube icon beside it IS one — but it was a ` + /> )} + {!navigated && onRemoveFromList && ( + + )} {!navigated && ( )} + {/* Toasts raised while the player is open (a failed Remove, a lost connection) — the app's own + Toaster is underneath this scrim. See ui/Toaster. In true fullscreen it moves into the + stage, the element holding the lock: out here it was painted under it, i.e. not at all. */} + {!isFullscreen && } , document.body, ); diff --git a/frontend/src/components/PlaylistEditFailures.dom.test.tsx b/frontend/src/components/PlaylistEditFailures.dom.test.tsx new file mode 100644 index 00000000..625af9a0 --- /dev/null +++ b/frontend/src/components/PlaylistEditFailures.dom.test.tsx @@ -0,0 +1,169 @@ +// @vitest-environment jsdom +// +// A playlist or tag edit that fails is said, and what the screen shows goes back to what is true +// (deep-review round 4). The Plex playlist page, the add-to-playlist popover and the tag manager had +// no failure handling at all: a 404 for a list or tag deleted elsewhere — now a structured +// `not_found`, which the request layer leaves to the caller — failed in silence, and an optimistic +// removal stayed removed. +import { describe, it, expect, vi, beforeAll, beforeEach, afterEach } from "vitest"; +import { render, cleanup, screen, fireEvent, waitFor, act } from "@testing-library/react"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; + +vi.mock("react-i18next", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }), + }; +}); +vi.mock("./ConfirmProvider", () => ({ useConfirm: () => async () => true })); +const { notify } = vi.hoisted(() => ({ + notify: vi.fn((_n: { level: string; message: string }) => 0), +})); +vi.mock("../lib/notifications", async (importOriginal) => ({ + ...(await importOriginal()), + notify, +})); + +import PlexPlaylistView from "./PlexPlaylistView"; +import AddToPlaylist from "./AddToPlaylist"; +import PlexPlaylistAdd from "./PlexPlaylistAdd"; +import TagManager from "./TagManager"; +import { api, HttpError, type PlexCard } from "../lib/api"; + +beforeAll(() => { + class Stub { + observe() {} + unobserve() {} + disconnect() {} + takeRecords() { + return []; + } + } + globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver; + globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver; +}); +beforeEach(() => notify.mockClear()); +afterEach(() => { + cleanup(); + vi.restoreAllMocks(); +}); + +const gone = (what: string) => + new HttpError(404, `gone ${what}`, { code: "not_found", reason: what }); +const errors = () => notify.mock.calls.filter(([n]) => n.level === "error").map(([n]) => n.message); +const client = () => new QueryClient({ defaultOptions: { queries: { retry: false } } }); + +const movie = (id: string) => + ({ id, type: "movie", title: `Movie ${id}`, year: 2020, thumb: "" }) as unknown as PlexCard; + +describe("the Plex playlist page", () => { + it("puts back a row whose removal failed, and says why", async () => { + vi.spyOn(api, "plexPlaylist").mockResolvedValue({ + id: 5, + title: "Films", + items: [movie("1"), movie("2")], + }); + vi.spyOn(api, "plexPlaylistRemoveItem").mockRejectedValue(gone("playlist")); + render( + + {}} onPlay={() => {}} /> + , + ); + await screen.findByText("Movie 1"); + fireEvent.click(screen.getAllByLabelText("remove")[0]!); + await waitFor(() => expect(errors()).toEqual(["gone playlist"])); + await act(async () => void (await new Promise((r) => setTimeout(r, 20)))); + expect(screen.getByText("Movie 1")).toBeTruthy(); + }); + + it("says so when a rename is refused, and keeps the field open", async () => { + vi.spyOn(api, "plexPlaylist").mockResolvedValue({ id: 5, title: "Films", items: [] }); + vi.spyOn(api, "plexRenamePlaylist").mockRejectedValue(gone("playlist")); + render( + + {}} onPlay={() => {}} /> + , + ); + await screen.findByText("Films"); + fireEvent.click(screen.getByTitle("plex.playlist.rename")); + const field = screen.getByDisplayValue("Films"); + fireEvent.change(field, { target: { value: "Movies" } }); + fireEvent.keyDown(field, { key: "Enter" }); + await waitFor(() => expect(errors()).toEqual(["gone playlist"])); + expect(screen.getByDisplayValue("Movies")).toBeTruthy(); // still open to try again + }); + + it("leaves a playlist that is already gone when asked to delete it", async () => { + vi.spyOn(api, "plexPlaylist").mockResolvedValue({ id: 5, title: "Films", items: [] }); + vi.spyOn(api, "plexDeletePlaylist").mockRejectedValue(gone("playlist")); + const onBack = vi.fn(); + render( + + {}} /> + , + ); + fireEvent.click(await screen.findByTitle("plex.playlist.delete")); + await waitFor(() => expect(onBack).toHaveBeenCalled()); + }); +}); + +describe("the add-to-playlist popover", () => { + it("says so when the playlist ticked is gone, and refreshes the list", async () => { + const lists = vi + .spyOn(api, "playlists") + .mockResolvedValue([{ id: 3, name: "Gone", kind: "user", has_video: false }] as never); + vi.spyOn(api, "addToPlaylist").mockRejectedValue(gone("playlist")); + render( + + + , + ); + fireEvent.click(screen.getByLabelText("playlists.addToPlaylist")); + fireEvent.click(await screen.findByRole("menuitemcheckbox", { hidden: true })); + await waitFor(() => expect(errors()).toEqual(["gone playlist"])); + await waitFor(() => expect(lists.mock.calls.length).toBeGreaterThan(1)); + }); + + it("says so on the PLEX side too, where the same sweep had missed the dialog", async () => { + // Same shape, same 404, same silence — `req` leaves 403/404/429 to the caller, and this one had + // no catch: the tick did not flip, nothing was said, the gone row stayed, and the floating + // promise became an unhandled rejection. Round 5 fixed the collection editor and the YouTube + // twin above; this dialog was left behind (round 6). + const lists = vi + .spyOn(api, "plexPlaylists") + .mockResolvedValue({ playlists: [{ id: 9, title: "Gone", has_item: false }] } as never); + vi.spyOn(api, "plexPlaylistAddItem").mockRejectedValue(gone("playlist")); + render( + + {}} + /> + , + ); + fireEvent.click(await screen.findByText("Gone")); + // The server's own localized reason wins over the caller's fallback when the detail is + // STRUCTURED — same as the twin above. What is under test is that anything is said at all. + await waitFor(() => expect(errors()).toEqual(["gone playlist"])); + await waitFor(() => expect(lists.mock.calls.length).toBeGreaterThan(1)); + }); +}); + +describe("the tag manager", () => { + it("says so when the tag deleted is gone, and refreshes the list", async () => { + const tags = vi + .spyOn(api, "tags") + .mockResolvedValue([{ id: 4, name: "Gone", system: false, channel_count: 0 }] as never); + vi.spyOn(api, "channels").mockResolvedValue([] as never); + vi.spyOn(api, "deleteTag").mockRejectedValue(gone("tag")); + render( + + {}} onFocusChannel={() => {}} /> + , + ); + fireEvent.click(await screen.findByLabelText("tagManager.delete")); + await waitFor(() => expect(errors()).toEqual(["gone tag"])); + await waitFor(() => expect(tags.mock.calls.length).toBeGreaterThan(1)); + }); +}); diff --git a/frontend/src/components/Playlists.dom.test.tsx b/frontend/src/components/Playlists.dom.test.tsx new file mode 100644 index 00000000..38ec9004 --- /dev/null +++ b/frontend/src/components/Playlists.dom.test.tsx @@ -0,0 +1,386 @@ +// @vitest-environment jsdom +// +// The Playlists page as the player sees it: which row it is launched with, and what it is told to +// do when the viewer removes that row from inside it. PlayerModal itself is replaced by a stand-in +// that records its props — its own behaviour is pinned in PlayerModal.dom.test.tsx. +import { describe, it, expect, vi, afterEach, beforeAll, beforeEach } from "vitest"; +import { render, screen, fireEvent, waitFor, cleanup, act } from "@testing-library/react"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; + +const h = vi.hoisted(() => ({ + notify: vi.fn(), + selectedId: 1 as number | null, + mounted: false, + mounts: 0, + player: null as null | { + video: { id: string; position_seconds: number }; + onClose: () => void; + onState: (id: string, status: string) => void; + onRemoveFromList?: (v: unknown) => Promise; + }, +})); + +vi.mock("react-i18next", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + useTranslation: () => ({ + t: (key: string, o?: Record) => (o?.title ? `${key}:${o.title}` : key), + i18n: { language: "en" }, + }), + }; +}); +vi.mock("../lib/notifications", async (importOriginal) => ({ + ...(await importOriginal()), + notify: (...a: unknown[]) => h.notify(...a), +})); +vi.mock("./PlaylistsProvider", () => ({ usePlaylists: () => ({ selectedId: h.selectedId }) })); +vi.mock("./ConfirmProvider", () => ({ useConfirm: () => async () => true })); +vi.mock("./usePlaylistActions", () => ({ usePlaylistActions: () => ({ remove: vi.fn() }) })); +vi.mock("./PlayerModal", async () => { + const { useEffect } = await import("react"); + return { + default: function PlayerStandIn(props: NonNullable) { + h.player = props; + useEffect(() => { + h.mounted = true; + h.mounts++; + return () => { + h.mounted = false; + }; + }, []); + return null; + }, + }; +}); + +import Playlists from "./Playlists"; +import { api, HttpError, type Video } from "../lib/api"; +import { qk } from "../lib/queryKeys"; +import { patchCachedProgress } from "../lib/progressCache"; + +beforeAll(() => { + class Stub { + observe() {} + unobserve() {} + disconnect() {} + takeRecords() { + return []; + } + } + globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver; + globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver; +}); + +const vid = (id: string, position_seconds: number): Video => + ({ + id, + title: `Video ${id}`, + channel_id: "UCx", + channel_title: "Channel", + status: "new", + position_seconds, + duration_seconds: 3000, + published_at: "2026-01-01T00:00:00Z", + watch_url: `https://www.youtube.com/watch?v=${id}`, + }) as unknown as Video; + +const WATCH_LATER = 9; +const list = (id: number, items: Video[]) => ({ + id, + name: `List ${id}`, + kind: id === WATCH_LATER ? "watch_later" : "user", + yt_playlist_id: null, + item_count: items.length, + items, +}); + +let qc: QueryClient; + +function mount(lists: Record, prefs: Record = {}) { + vi.spyOn(api, "playlists").mockResolvedValue( + Object.entries(lists).map(([id, items]) => list(Number(id), items)) as never, + ); + vi.spyOn(api, "playlist").mockImplementation( + async (id: number) => list(id, lists[id] ?? []) as never, + ); + qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + qc.setQueryData(qk.me(), { id: 1, can_write: true, prefs: {}, preferences: prefs }); + const tree = () => ( + + + + ); + const view = render(tree()); + return { rerender: () => view.rerender(tree()) }; +} + +const play = async (title: string) => + fireEvent.click(await screen.findByRole("button", { name: `card.play — ${title}` })); + +beforeEach(() => { + h.selectedId = 1; + h.player = null; + h.mounted = false; + h.mounts = 0; + h.notify.mockClear(); +}); +afterEach(() => { + cleanup(); + vi.restoreAllMocks(); +}); + +describe("Playlists — the row the player is launched with", () => { + it("carries the position the player just stored, not the one the list first loaded with", async () => { + // The page renders from its own undoable copy of the list, re-seeded only when the SET of ids + // changes — so the position the player wrote into the cache never reached it: reopen a video + // closed at 830 s and it resumed at the old spot, and the next checkpoint wrote that over 830. + mount({ 1: [vid("V", 748)] }); + await play("Video V"); + await waitFor(() => expect(h.player?.video.position_seconds).toBe(748)); + act(() => h.player!.onClose()); + h.player = null; + // What the player's cleanup does with the position the server confirmed. react-query hands a + // cache write to its observers on a setTimeout(0); outside act, the re-render it triggers is + // scheduled by React on its own and raced the click below under the gate's parallel load (the + // reopen read 748). The write is SYNCHRONOUS, so its notify timer is registered before ours and + // fires first — and inside act, the render it causes is flushed before act returns. + await act(async () => { + patchCachedProgress(qc, new Map([["V", 830]])); + await new Promise((r) => setTimeout(r, 0)); + }); + await play("Video V"); + await waitFor(() => expect(h.player?.video.position_seconds).toBe(830)); + }); +}); + +describe("Playlists — removing a video from inside the player", () => { + it("removes it from the list the player was launched from, even after the selection moved", async () => { + // The selection can change under an open player (the command palette opens over it); the + // removal read the LIVE selection, so it took the video out of a list the viewer never touched. + const remove = vi.spyOn(api, "removeFromPlaylist").mockResolvedValue(undefined as never); + const { rerender } = mount({ 1: [vid("V", 0)], 2: [vid("V", 0), vid("W", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.player).not.toBeNull()); + h.selectedId = 2; + rerender(); + await screen.findByRole("button", { name: "card.play — Video W" }); // list 2 is on screen now + await act(async () => { + await h.player!.onRemoveFromList!(vid("V", 0)); + }); + expect(remove).toHaveBeenCalledWith(1, "V"); + // …and the list on screen keeps its own copy of V: nothing was taken out of it. + expect(screen.getByRole("button", { name: "card.play — Video V" })).toBeTruthy(); + }); + + it("says so when the removal fails because the list is gone", async () => { + // `req` leaves 403/404 to the caller; the catch assumed a global dialog had already reported + // it, so a confirmed removal failed in total silence behind the full-screen player. + vi.spyOn(api, "removeFromPlaylist").mockRejectedValue(new HttpError(404, "Unknown playlist")); + mount({ 1: [vid("V", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.player).not.toBeNull()); + let done: boolean | undefined; + await act(async () => { + done = await h.player!.onRemoveFromList!(vid("V", 0)); + }); + expect(done).toBe(false); + expect(h.notify).toHaveBeenCalledWith(expect.objectContaining({ level: "error" })); + }); + + it("offers Remove by the list it was launched from, not by whichever list is selected now", async () => { + // Watch later deliberately has no Remove. Gated on the SELECTED list, it appeared on a player + // launched from Watch later as soon as another list was selected under it — and deleted from + // Watch later, since the removal itself targets the launch list. + h.selectedId = WATCH_LATER; + const { rerender } = mount({ [WATCH_LATER]: [vid("V", 0)], 2: [vid("W", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.player).not.toBeNull()); + expect(h.player!.onRemoveFromList).toBeUndefined(); + h.selectedId = 2; + rerender(); + await screen.findByRole("button", { name: "card.play — Video W" }); + expect(h.player!.onRemoveFromList).toBeUndefined(); + }); + + it("takes the removed video out of the cached list, so a progress checkpoint cannot bring it back", async () => { + // The player steps to the next item after a removal and stores the last one's position into + // every cached list. Landing before the refetch, that write handed the page a list still holding + // the video, and the page — seeing a different set of ids — reset: the row came back. + vi.spyOn(api, "removeFromPlaylist").mockResolvedValue(undefined as never); + mount({ 1: [vid("V", 0), vid("W", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.player).not.toBeNull()); + vi.mocked(api.playlist).mockReturnValue(new Promise(() => {})); // the refetch is still out + await act(async () => { + await h.player!.onRemoveFromList!(vid("V", 0)); + }); + await act(async () => { + patchCachedProgress(qc, new Map([["V", 99]])); + await new Promise((r) => setTimeout(r, 0)); + }); + expect(screen.queryByRole("button", { name: "card.play — Video V" })).toBeNull(); + expect(screen.getByRole("button", { name: "card.play — Video W" })).toBeTruthy(); + }); +}); + +describe("Playlists — a failed refetch of a list that is on screen", () => { + it("keeps the list, instead of replacing it with the error", async () => { + // A background refetch — on focus, after a move or a removal — failing during an API restart + // replaced a loaded list with "couldn't load" (the `isError && ` convention, ignored). + mount({ 1: [vid("V", 0)] }); + await screen.findByRole("button", { name: "card.play — Video V" }); + vi.mocked(api.playlist).mockRejectedValue(new HttpError(502, "gateway")); + await act(async () => { + await qc.invalidateQueries({ queryKey: qk.playlist(1) }); + await new Promise((r) => setTimeout(r, 20)); // react-query notifies its observers on a timer + }); + expect(qc.getQueryState(qk.playlist(1))?.status).toBe("error"); + expect(screen.queryByText("playlists.loadFailed")).toBeNull(); + expect(screen.getByRole("button", { name: "card.play — Video V" })).toBeTruthy(); + }); +}); + +describe("Playlists — the player outlives a change of selection", () => { + it("stays mounted while a newly selected list is still loading", async () => { + // The page swaps itself for "Loading…" until the new list arrives; with the player rendered only + // in the loaded branch, that took it down mid-playback and put it back at its launch video. + const { rerender } = mount({ 1: [vid("V", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.mounted).toBe(true)); + vi.mocked(api.playlist).mockReturnValue(new Promise(() => {})); + h.selectedId = 2; + rerender(); + // The shared loading state, which a screen reader hears (the old bare line had no live region). + expect( + (await screen.findByText("playlists.loading")).closest('[role="status"]'), + ).not.toBeNull(); + expect(h.mounted).toBe(true); + expect(h.mounts).toBe(1); // never went down — a remount would also end `mounted` + }); + + it("stays mounted when the newly selected list fails to load — a list deleted elsewhere", async () => { + const { rerender } = mount({ 1: [vid("V", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.mounted).toBe(true)); + vi.mocked(api.playlist).mockRejectedValue(new HttpError(404, "gone")); + h.selectedId = 2; + rerender(); + await screen.findByText("playlists.loadFailed"); + expect(h.mounted).toBe(true); + expect(h.mounts).toBe(1); + }); + + it("stays mounted when nothing is selected any more", async () => { + const { rerender } = mount({ 1: [vid("V", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.mounted).toBe(true)); + h.selectedId = null; + rerender(); + await screen.findByText("playlists.pickOne"); + expect(h.mounted).toBe(true); + expect(h.mounts).toBe(1); + }); + + it("refreshes the list it was launched from when the viewer marks the video in it", async () => { + // Invalidating the SELECTED list left the launch list's row showing the old status. + vi.spyOn(api, "setState").mockResolvedValue(undefined as never); + const { rerender } = mount({ 1: [vid("V", 0)], 2: [vid("W", 0)] }); + await play("Video V"); + await waitFor(() => expect(h.player).not.toBeNull()); + h.selectedId = 2; + rerender(); + await screen.findByRole("button", { name: "card.play — Video W" }); + await act(async () => { + h.player!.onState("V", "watched"); + await new Promise((r) => setTimeout(r, 0)); + }); + expect(qc.getQueryState(qk.playlist(1))?.isInvalidated).toBe(true); + }); +}); + +const rowsOnScreen = () => + screen + .getAllByRole("button", { name: /^card\.play — / }) + .map((b) => b.getAttribute("aria-label")?.replace("card.play — ", "")); + +async function rowMenu(title: string, item: string) { + const row = (await screen.findByRole("button", { name: `card.play — ${title}` })).closest("tr")!; + fireEvent.contextMenu(row); + await waitFor(() => + expect( + screen.getAllByRole("menuitem", { hidden: true }).some((b) => b.textContent === item), + ).toBe(true), + ); + fireEvent.click( + screen.getAllByRole("menuitem", { hidden: true }).find((b) => b.textContent === item)!, + ); +} + +describe("Playlists — a change the server refused is taken back on screen", () => { + // The resync refetch returns the SAME list the server had, react-query keeps the same `data` + // object for a deep-equal answer, and the page's re-seed effect is keyed on that object — so the + // optimistic change used to stay on screen for good. + it("puts back a row whose move failed", async () => { + vi.spyOn(api, "moveToPlaylist").mockRejectedValue( + new HttpError(404, "gone", { code: "not_found", reason: "move_target" }), + ); + mount({ 1: [vid("V", 0), vid("W", 0)], 2: [] }, { lastMovePlaylistId: 2 }); + await rowMenu("Video V", "playlists.moveToRecent"); + await waitFor(() => + expect(h.notify).toHaveBeenCalledWith(expect.objectContaining({ level: "error" })), + ); + await act(async () => void (await new Promise((r) => setTimeout(r, 20)))); + expect(rowsOnScreen()).toEqual(["Video V", "Video W"]); + }); + + it("puts back the order when saving a reorder failed", async () => { + vi.spyOn(api, "reorderPlaylist").mockRejectedValue(new HttpError(500, "boom")); + mount({ 1: [vid("V", 0), vid("W", 0)] }); + await rowMenu("Video V", "playlists.orderDown"); + await waitFor(() => expect(api.reorderPlaylist).toHaveBeenCalled()); + await act(async () => void (await new Promise((r) => setTimeout(r, 20)))); + expect(rowsOnScreen()).toEqual(["Video V", "Video W"]); + }); + + it("puts back the order the server ACCEPTED, not the one it had two reorders ago", async () => { + // The rollback reads the cached detail, and nothing wrote the accepted order into it — only the + // refetch that follows a successful reorder did. So a second reorder rejected while that refetch + // was still in flight rolled the rows back past the FIRST one, which the server had accepted. + // The membership guard cannot correct an order (its key is the sorted id set), so the wrong + // order stayed on screen for good and the next drag persisted it (deep-review round 5). + const reorder = vi + .spyOn(api, "reorderPlaylist") + .mockResolvedValueOnce(undefined as never) + .mockRejectedValue(new HttpError(500, "boom")); + mount({ 1: [vid("V", 0), vid("W", 0), vid("X", 0)] }); + await screen.findByRole("button", { name: "card.play — Video V" }); + // From here no refetch ever lands, so the cache is whatever the page itself put there. + vi.mocked(api.playlist).mockReturnValue(new Promise(() => {}) as never); + + await rowMenu("Video V", "playlists.orderDown"); + await waitFor(() => expect(reorder).toHaveBeenCalledTimes(1)); + expect(rowsOnScreen()).toEqual(["Video W", "Video V", "Video X"]); + + await rowMenu("Video X", "playlists.orderUp"); + await waitFor(() => expect(reorder).toHaveBeenCalledTimes(2)); + await act(async () => void (await new Promise((r) => setTimeout(r, 20)))); + expect(rowsOnScreen()).toEqual(["Video W", "Video V", "Video X"]); + }); + + it("puts back a row whose removal failed, even when the resync fails too", async () => { + // A removal during an API restart: the DELETE fails, and so does the refetch meant to put the row + // back — the list kept showing it as removed while it was still in the playlist. + vi.spyOn(api, "removeFromPlaylist").mockRejectedValue(new TypeError("Failed to fetch")); + mount({ 1: [vid("V", 0), vid("W", 0)] }); + await screen.findByRole("button", { name: "card.play — Video V" }); + vi.mocked(api.playlist).mockRejectedValue(new TypeError("Failed to fetch")); + await rowMenu("Video V", "playlists.removeItem"); + await waitFor(() => expect(api.removeFromPlaylist).toHaveBeenCalled()); + await act(async () => void (await new Promise((r) => setTimeout(r, 20)))); + expect(rowsOnScreen()).toEqual(["Video V", "Video W"]); + expect(h.notify).toHaveBeenCalledWith( + expect.objectContaining({ level: "error", message: "playlists.removeFailed" }), + ); + }); +}); diff --git a/frontend/src/components/Playlists.tsx b/frontend/src/components/Playlists.tsx index 093f34d7..f30f7d98 100644 --- a/frontend/src/components/Playlists.tsx +++ b/frontend/src/components/Playlists.tsx @@ -20,7 +20,6 @@ import { } from "@dnd-kit/sortable"; import { CSS } from "@dnd-kit/utilities"; import { - AlertTriangle, ArrowDown, ArrowDownToLine, ArrowUp, @@ -28,6 +27,7 @@ import { Check, ExternalLink, FolderInput, + History, GripVertical, ListPlus, Pencil, @@ -39,8 +39,9 @@ import { X, Youtube, } from "lucide-react"; -import { api, HttpError, type Playlist, type Video } from "../lib/api"; +import { api, type Playlist, type PlaylistDetail, type Video } from "../lib/api"; import { notify } from "../lib/notifications"; +import { notifyActionError } from "../lib/actionErrors"; import { notifyYouTubeActionError } from "../lib/youtubeErrors"; import { playlistName } from "../lib/playlistName"; import { useUndoable } from "../lib/useUndoable"; @@ -49,6 +50,7 @@ import { formatRuntime } from "../lib/format"; import { applyView, sameOrder } from "../lib/playlistView"; const PlayerModal = lazy(() => import("./PlayerModal")); import { Button } from "./ui/Button"; +import { LoadingState, StateMessage } from "./ui/QueryState"; import { type ActionMenuItem } from "./ui/ActionMenu"; import { useComfortPin, pinVideo } from "./comfortPin"; import { ContextMenu } from "./ui/ContextMenu"; @@ -59,6 +61,7 @@ import { useConfirm } from "./ConfirmProvider"; import { usePlaylistActions } from "./usePlaylistActions"; import { usePlaylists } from "./PlaylistsProvider"; import { useMe } from "../lib/useMe"; +import { usePref } from "../lib/usePref"; import { ColumnHead, alignClass, @@ -181,27 +184,34 @@ export default function Playlists() { const order = useUndoable([], { onApply: (next) => { if (selectedId == null) return; + const listId = selectedId; api .reorderPlaylist( - selectedId, + listId, next.map((v) => v.id), ) .then(() => { + // Write the accepted order into the cache BEFORE the refetch. The cache is what + // `restoreFromCache` rolls back to, and until this refetch landed it still held the order + // from before this reorder — so a SECOND reorder failing in that window rolled the rows + // back to an order the server does not have, permanently (the membership guard cannot + // correct an order). One accepted write, one cache entry that matches it (round 5). + qc.setQueryData(qk.playlist(listId), (old) => + old ? { ...old, items: next } : old, + ); // Refresh the sidebar (cover may change) and the detail (so the now-dirty state, // and the Reset/Unsynced indicators, show without an F5). The membership guard // keeps the refetch from wiping the undo history on a pure reorder. qc.invalidateQueries({ queryKey: qk.playlists() }); - qc.invalidateQueries({ queryKey: qk.playlist(selectedId) }); + qc.invalidateQueries({ queryKey: qk.playlist(listId) }); }) .catch((e) => { // Was a bare .then(): a failed reorder left the UI showing an order the server had - // rejected, and produced an unhandled rejection. Say so and resync from the server. + // rejected, and produced an unhandled rejection. Say so, and put back the order the server + // still has — or the rejected one becomes the base for the next drag. notifyYouTubeActionError(e, t("playlists.reorderFailed")); - // Clear the membership guard first: a reorder never changes the id SET, so without this - // the arriving detail is judged "same list" and the rejected order stays on screen — and - // becomes the base for the next drag. - lastSetRef.current = ""; - qc.invalidateQueries({ queryKey: qk.playlist(selectedId) }); + restoreFromCache(listId); + qc.invalidateQueries({ queryKey: qk.playlist(listId) }); }); }, }); @@ -214,10 +224,46 @@ export default function Playlists() { // re-confirms our own reorder doesn't wipe the undo history — we reset history only when // the actual set of items changes (different playlist, or an add/remove). const lastSetRef = useRef(""); - // The open player is tracked by video ID, not by a position: `shown` is derived, so a refetch or - // a change of view would otherwise slide a different video under the index — or unmount the player - // mid-playback. PlayerModal already tracks its own queue by id; this is the parent half of that. - const [playingId, setPlayingId] = useState(null); + // The selection as of NOW, for code that runs after an await or on behalf of the open player. + const selectedRef = useRef(selectedId); + selectedRef.current = selectedId; + // Put the rows back to what the CACHED detail says — the server's list as last fetched. A failed + // reorder or move must not wait for the resync to do it: the server's list is unchanged, so the + // refetch comes back deep-equal, react-query keeps the same `data` object, and the effect below + // (keyed on it) never runs. The moved row stayed gone and the rejected order stayed on screen + // (deep-review round 4). Only for the list on screen: the selection may have moved meanwhile. + function restoreFromCache(listId: number) { + if (listId !== selectedRef.current) return; + const cur = qc.getQueryData(qk.playlist(listId)); + if (!cur) return; + lastSetRef.current = `${listId}:${idsKey(cur.items)}`; + order.reset(cur.items); + } + // What the player was LAUNCHED with, snapshotted when it opens: the video, the queue and the index + // inside it. `shown` is derived (sort / filter / view), and the player freezes its queue and owns + // its own cursor from mount — so deriving the mount from `shown` every render meant anything that + // dropped the launch row yanked the player off the screen mid-playback. That was a latent edge + // (a view change) until "remove this from the playlist" moved INSIDE the player, where it is the + // normal case. Only onClose takes the player down now. `listId` is part of that snapshot for the + // same reason: the selection can change under an open player (the command palette opens over it), + // and "remove this from the playlist" means the list it was LAUNCHED from, not whichever is + // selected by the time the button is pressed. So does whether that button exists at all + // (`editable`): gating it on the current selection offered Remove on a player launched from + // Watch later — which deliberately has none — as soon as another list was selected under it. + const [playing, setPlaying] = useState<{ + video: Video; + queue: Video[]; + index: number; + listId: number | null; + editable: boolean; + } | null>(null); + // Where the last move went. `LS.playlist` is the SELECTED (source) list and `updated_at` moves on + // any edit, so neither answers "where do I keep sending things"; this does, and it is a per-account + // preference like every other remembered choice. Null until the first move of this account's life. + const [lastMoveTargetId, setLastMoveTargetId] = usePref( + "lastMovePlaylistId", + null, + ); // ONE context menu for the whole table, not one popover per (see ui/ContextMenu): the row it // belongs to is `rowMenu.target`, and the panel anchors to the pointer. const rowMenu = useContextMenu
- {/* Header (title, actions, view controls) is fixed chrome — it portals into the shell's + <> +
+ {/* Header (title, actions, view controls) is fixed chrome — it portals into the shell's fixed band, so only the video list scrolls. */} - - {/* Two tight rows beside a small cover: title + actions on the first, the meta line and the + + {/* Two tight rows beside a small cover: title + actions on the first, the meta line and the view controls sharing the second. This band is FIXED chrome (it never scrolls), so every row it sheds is list height won back — the old layout stacked three rows next to the cover plus a fourth block under it, for roughly twice this height. */} -
-
-
- {items[0]?.thumbnail_url && ( - - )} -
-
-
- {renaming ? ( - setRenameValue(e.target.value)} - onBlur={saveRename} - onKeyDown={(e) => { - if (e.key === "Enter") saveRename(); - if (e.key === "Escape") setRenaming(false); - }} - aria-label={t("playlists.rename")} - className="min-w-0 flex-1 text-lg font-semibold bg-surface border border-border rounded-md px-2 py-0.5 focus:outline-hidden focus:ring-2 focus:ring-accent/40" - /> - ) : ( - <> -

{plName(detail)}

- {editable && ( - - )} - {detail.yt_playlist_id && ( - - - - )} - +
+
+
+ {items[0]?.thumbnail_url && ( + )} -
- {linked && detail.dirty && ( - - - {t("playlists.unsynced")} - - )} - - {editable && canWrite && ( - - )} - {linked && detail.dirty && ( - - )} - {editable && ( -
- -
- - {t("playlists.itemCount", { count: items.length })} - {totalSeconds > 0 && ` · ${formatRuntime(totalSeconds)}`} - {filterActive && ` · ${t("playlists.filteredCount", { count: shown.length })}`} - - {(items.length > 1 || viewActive) && ( - <> - - - {viewActive && ( - + )} + {detail.yt_playlist_id && ( + + + + )} + + )} +
+ {linked && detail.dirty && ( + - {t("playlists.clearView")} + + {t("playlists.unsynced")} + + )} + + {editable && canWrite && ( + )} - {canSaveOrder && ( + {linked && detail.dirty && ( )} - {/* ONE sentence that names the state. Three loose texts used to pile up here + {editable && ( +
+
+ +
+ + {t("playlists.itemCount", { count: items.length })} + {totalSeconds > 0 && ` · ${formatRuntime(totalSeconds)}`} + {filterActive && ` · ${t("playlists.filteredCount", { count: shown.length })}`} + + {(items.length > 1 || viewActive) && ( + <> + + + {viewActive && ( + + )} + {canSaveOrder && ( + + )} + {/* ONE sentence that names the state. Three loose texts used to pile up here (reset link + save hint + "reordering is off"), and none of them answered the question a user actually has — "did I just change my playlist?". Worse, sorting a list whose stored order ALREADY matches the sort produced a warning, no Save button and no visible change, which reads as a malfunction. Now the line says which of the three situations you are in; the disabled grip still explains itself on hover/focus. */} - {viewActive && {viewStatus}} - {canEdit && ( -
- -
- )} - - )} + {viewActive && {viewStatus}} + {canEdit && ( +
+ +
+ )} + + )} +
-
- + - {items.length === 0 ? ( -
-
- - {t("playlists.empty")} + {items.length === 0 ? ( +
+
+ + {t("playlists.empty")} +
-
- ) : ( - - {/* Not clipped, so the header's filter popovers can overflow the table box. */} -
- {/* table-fixed, not auto: every column but the title has a declared width, so the + ) : ( + + {/* Not clipped, so the header's filter popovers can overflow the table box. */} +
+ {/* table-fixed, not auto: every column but the title has a declared width, so the title absorbs the rest and its `truncate` works. Auto layout sizes columns from min-content, and a nowrap title cell's min-content is the entire title. */} - - - v.id)} - strategy={verticalListSortingStrategy} - > - - {shown.map((v) => ( - - ))} - - -
- {shown.length === 0 && ( -
- {t("playlists.noneMatchFilter")} -
- )} -
- - )} + + + v.id)} + strategy={verticalListSortingStrategy} + > + + {shown.map((v) => ( + + ))} + + +
+ {shown.length === 0 && ( +
+ {t("playlists.noneMatchFilter")} +
+ )} +
+ + )} - {/* One surface for the whole table, outside the rows — it anchors to the pointer, not the row. */} - - - {playingIndex >= 0 && ( - - - - )} -
+ {/* One surface for the whole table, outside the rows — it anchors to the pointer, not the row. */} + +
+ {player} + ); } diff --git a/frontend/src/components/PlexCollectionEditor.tsx b/frontend/src/components/PlexCollectionEditor.tsx index 6182d8b5..f6091804 100644 --- a/frontend/src/components/PlexCollectionEditor.tsx +++ b/frontend/src/components/PlexCollectionEditor.tsx @@ -4,7 +4,7 @@ import { useTranslation } from "react-i18next"; import { useQuery, useQueryClient } from "@tanstack/react-query"; import { Check, Plus } from "lucide-react"; import { api } from "../lib/api"; -import { notifyCollectionEditError } from "../lib/actionErrors"; +import { notifyActionError } from "../lib/actionErrors"; import { useScrollFade } from "../lib/useScrollFade"; import { Button } from "./ui/Button"; import Modal from "./ui/Modal"; @@ -77,6 +77,11 @@ export default function PlexCollectionEditor({ return next; }); refresh(); + } catch (e) { + // 403/404 are left to the caller by `req` (see reportFailure), so without this the tick + // simply never appeared and the user clicked again — a collection deleted on the Plex server + // or flipped read-only failed in silence (deep-review round 5). + notifyActionError(e, t("plex.collection.editFailed")); } finally { setBusy(null); } @@ -88,6 +93,8 @@ export default function PlexCollectionEditor({ try { await api.plexSetCollectionEditable(id, true); refresh(); + } catch (e) { + notifyActionError(e, t("plex.collection.editFailed")); } finally { setBusy(null); } @@ -103,9 +110,9 @@ export default function PlexCollectionEditor({ setMembers((prev) => new Set(prev).add(col.id)); refresh(); } catch (e) { - // The backend rejects a duplicate name with 409 (Plex conflates same names) — localize it to - // nameTaken rather than surfacing the server's raw English detail. - notifyCollectionEditError(e, t("plex.collection.editFailed"), t("plex.collection.nameTaken")); + // The backend rejects a duplicate name with 409 (Plex conflates same names), structured — so + // it reads in the reader's language (see localizeDetail). + notifyActionError(e, t("plex.collection.editFailed")); } finally { setBusy(null); } @@ -123,6 +130,10 @@ export default function PlexCollectionEditor({ return next; }); refresh(); + } catch (e) { + // The worst of the three to lose: the user has just confirmed a destructive dialog, and the + // row stayed where it was with nothing said. + notifyActionError(e, t("plex.collection.deleteFailed")); } finally { setBusy(null); } diff --git a/frontend/src/components/PlexCollectionView.dom.test.tsx b/frontend/src/components/PlexCollectionView.dom.test.tsx index 5255e399..b57763ec 100644 --- a/frontend/src/components/PlexCollectionView.dom.test.tsx +++ b/frontend/src/components/PlexCollectionView.dom.test.tsx @@ -20,6 +20,8 @@ beforeAll(() => { } } globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver; + // …and the Sort control's popover watches its anchor with an IntersectionObserver. + globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver; }); vi.mock("react-i18next", async (importOriginal) => { @@ -36,7 +38,13 @@ vi.mock("../lib/notifications", async (importOriginal) => ({ })); import PlexCollectionView from "./PlexCollectionView"; -import { api, type PlexCard, type PlexCollectionDetail, type PlexShowDetail } from "../lib/api"; +import { + api, + HttpError, + type PlexCard, + type PlexCollectionDetail, + type PlexShowDetail, +} from "../lib/api"; afterEach(() => { cleanup(); @@ -126,3 +134,72 @@ describe("PlexCollectionView — Play-all", () => { expect(onPlay).not.toHaveBeenCalled(); }); }); + +// Bug-hunt deep-review: the Sort control put the sort into the detail query's key, and two things +// on this page did not follow it. +describe("PlexCollectionView — after the sort joined the query key", () => { + it("flips a poster's watched state on the page it was clicked on", async () => { + // The optimistic write kept the key's OLD spelling — an entry nothing reads — and the resync + // only touches the library root, so the click did nothing visible until a reload. + // The request never answers, so the resync never runs: only the optimistic write can flip it. + vi.spyOn(api, "plexSetState").mockReturnValue(new Promise(() => {})); + mount([card("movie1", "movie")]); + fireEvent.click(await screen.findByRole("button", { name: "plex.markWatched" })); + expect(await screen.findByRole("button", { name: "plex.markUnwatched" })).toBeTruthy(); + }); + + it("puts a toggle that failed back the way it was", async () => { + // With the flip landing on the entry the page reads, the resync is what corrects it — and it only + // touched the library root, so a failed toggle stayed "watched" beside its own error toast. + vi.spyOn(api, "plexSetState").mockRejectedValue(new HttpError(422, "nope")); + mount([card("movie1", "movie")]); + fireEvent.click(await screen.findByRole("button", { name: "plex.markWatched" })); + await waitFor(() => expect(api.plexCollection).toHaveBeenCalledTimes(2)); + expect(await screen.findByRole("button", { name: "plex.markWatched" })).toBeTruthy(); + expect(screen.queryByRole("button", { name: "plex.markUnwatched" })).toBeNull(); + }); + + it("keeps the page, and the open Sort panel, on screen while the new order loads", async () => { + // Every sort change was a key with no data yet: the whole page — the control and its open panel + // included — was swapped for a spinner, so key and direction could not be set in one visit. + vi.spyOn(api, "savePrefs").mockResolvedValue(undefined as never); + vi.spyOn(api, "plexCollection") + .mockResolvedValueOnce(detail([card("movie1", "movie"), card("movie2", "movie")])) + .mockReturnValueOnce(new Promise(() => {})); // the re-sorted page is still on its way + mount([card("movie1", "movie"), card("movie2", "movie")]); + fireEvent.click(await screen.findByRole("button", { name: /^plex\.filter\.sort:/ })); + // Popover rows are portalled and only exist while it is open, so `hidden: true`. + const other = (await screen.findAllByRole("menuitemradio", { hidden: true })).find( + (o) => o.getAttribute("aria-checked") !== "true", + ); + fireEvent.click(other!); + await waitFor(() => expect(api.plexCollection).toHaveBeenCalledTimes(2)); + expect(screen.queryByText("plex.loading")).toBeNull(); + expect(screen.getAllByRole("menuitemradio", { hidden: true }).length).toBeGreaterThan(1); + expect(screen.getByText("plex.collection.playAll")).toBeTruthy(); + }); + + it("does not show the previous collection while stepping to another one", async () => { + // Keeping the page on screen is for a re-sort of THIS collection only. + vi.spyOn(api, "plexCollection") + .mockResolvedValueOnce(detail([card("movie1", "movie")])) + .mockReturnValueOnce(new Promise(() => {})); + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + const view = (id: string) => ( + + {}} + onPlay={() => {}} + onOpenCard={() => {}} + onInfoCard={() => {}} + /> + + ); + const { rerender } = render(view("col1")); + await screen.findByText("plex.collection.playAll"); + rerender(view("col2")); + expect(await screen.findByText("plex.loading")).toBeTruthy(); + expect(screen.queryByText("plex.collection.playAll")).toBeNull(); + }); +}); diff --git a/frontend/src/components/PlexCollectionView.tsx b/frontend/src/components/PlexCollectionView.tsx index 5feef76d..8b9d88d4 100644 --- a/frontend/src/components/PlexCollectionView.tsx +++ b/frontend/src/components/PlexCollectionView.tsx @@ -5,9 +5,13 @@ import { Play, Shuffle, Pencil, Trash2, Plus, X, Lock, Check, Search } from "luc import { qk } from "../lib/queryKeys"; import { api, isIntrinsicReadOnly, type PlexCard, type PlexCollectionDetail } from "../lib/api"; import { formatRuntime } from "../lib/format"; -import { notifyActionError, notifyCollectionEditError } from "../lib/actionErrors"; +import { notifyActionError } from "../lib/actionErrors"; import { notify } from "../lib/notifications"; import { showEpisodeKeys } from "../lib/plexQueue"; +import { healPlexSort, plexSortOptions } from "../lib/plexSort"; +import { usePref } from "../lib/usePref"; +import SortControl from "./ui/SortControl"; +import { type SortState } from "../lib/columnSort"; import { useConfirm } from "./ConfirmProvider"; import { BackBtn, ActionBtn, DetailHero } from "../lib/plexDetailUi"; import { PlexPosterCard } from "./plexCards"; @@ -22,6 +26,11 @@ import { LoadingState, StateMessage } from "./ui/QueryState"; // admin-enabled collection — rename / delete / add / remove, all write-through to Plex (the mutation // endpoints round-trip to Plex, so each edit awaits and re-fetches; no optimism). Read-only ones (smart // / imdb / tmdb / … auto-lists) are marked with a lock. +// The order this page had before it grew a Sort control, and still its default: a franchise reads in +// release order. MUST match the endpoint's `DEFAULT_MEMBER_SORT` — the two only diverge if a request +// goes out without the params, which is why the control always sends them. +const COLLECTION_SORT_DEFAULT: NonNullable = { key: "year", dir: "asc" }; + function shuffled(a: string[]): string[] { const r = [...a]; for (let i = r.length - 1; i > 0; i--) { @@ -52,9 +61,27 @@ export default function PlexCollectionView({ // One comfort-pin menu for the member grid (see ui/ContextMenu). const pinMenu = useContextMenu(); const isAdmin = qc.getQueryData<{ role?: string }>(qk.me())?.role === "admin"; + // The member order is a PREFERENCE (server-synced, adopted on login), not an ephemeral filter — the + // same treatment the feed's view and the player's auto-advance get. It is deliberately its OWN key + // rather than the browse grid's sort: opening a collection must not silently re-sort the library + // behind it. + const [sortPref, setSortPref] = usePref>( + "plexCollectionSort", + COLLECTION_SORT_DEFAULT, + ); + // ONE key for the read and every write into it: when the sort joined the key, the watched toggle's + // optimistic write kept the old spelling and landed on an entry nothing reads (setQueryData matches + // exactly), so the click did nothing visible until a reload. + const detailKey = qk.plexCollections("detail", collectionId, sortPref.key, sortPref.dir); const q = useQuery({ - queryKey: qk.plexCollections("detail", collectionId), - queryFn: () => api.plexCollection(collectionId), + queryKey: detailKey, + queryFn: () => api.plexCollection(collectionId, sortPref), + // A re-sort keeps the current page on screen while the new order loads. Without it every sort + // change was a key with no data yet, and the whole page — the Sort control and its open panel + // with it — was swapped for a spinner, so key and direction could not be set in one visit. Only + // within the SAME collection: stepping to another one must not show this one's hero meanwhile. + placeholderData: (prev, prevQuery) => + prevQuery?.queryKey[2] === collectionId ? prev : undefined, }); const d = q.data; @@ -73,6 +100,11 @@ export default function PlexCollectionView({ }; const items = d?.items ?? []; + // A collection lives in ONE library, so its members are all movies or all shows. That decides both + // what the add-panel searches and which sort keys are on offer (only a movie has a runtime). + const scope = items.some((c) => c.type === "show") ? "show" : "movie"; + const sortState = healPlexSort(sortPref, scope); + const sortOptions = plexSortOptions(scope, (k) => t(`plex.sort.${k}`)); // Movies/episodes are playable leaves that queue as themselves; a show can't queue directly, so // Play-all expands it into its episodes (buildQueue). Either kind means there's something to play — // this is what a show-collection needs: before, `playable` held only leaves, so a show-collection's @@ -124,7 +156,7 @@ export default function PlexCollectionView({ const toggleWatched = (card: PlexCard) => toggleWatchState(card, t, { optimistic: (next) => { - qc.setQueryData(qk.plexCollections("detail", collectionId), (old) => + qc.setQueryData(detailKey, (old) => old ? { ...old, @@ -133,7 +165,14 @@ export default function PlexCollectionView({ : old, ); }, - resync: () => qc.invalidateQueries({ queryKey: qk.plexLibrary() }), + // This page's OWN key too, not just the library's: the flip above lands on the entry the page + // reads, and resync is what corrects it to the truth when the toggle fails — invalidating only + // `plex-library` left a failed toggle flipped on screen. The prefix (no sort in it) also + // refreshes the other sort orders of this collection cached behind this one. + resync: () => { + qc.invalidateQueries({ queryKey: qk.plexLibrary() }); + qc.invalidateQueries({ queryKey: qk.plexCollections("detail", collectionId) }); + }, }); async function doRename() { @@ -148,8 +187,8 @@ export default function PlexCollectionView({ invalidate(); setRenaming(false); } catch (e) { - // The backend enforces per-library name uniqueness — localize its 409 to nameTaken. - notifyCollectionEditError(e, t("plex.collection.editFailed"), t("plex.collection.nameTaken")); + // The backend enforces per-library name uniqueness — a taken name comes back localized. + notifyActionError(e, t("plex.collection.editFailed")); } finally { setBusy(false); } @@ -334,7 +373,7 @@ export default function PlexCollectionView({ {adding && canEdit && ( c.type === "show") ? "show" : "movie"} + scope={scope} existing={new Set(items.map((c) => c.id))} onAdded={invalidate} onClose={() => setAdding(false)} @@ -344,30 +383,46 @@ export default function PlexCollectionView({ {items.length === 0 ? (

{t("plex.collection.empty")}

) : ( -
- {items.map((c) => ( -
- onOpenCard(c)} - onInfo={() => onInfoCard(c)} - onToggleWatched={toggleWatched} - menuProps={pinMenu.triggerProps} + <> + {/* Sort sits directly above the grid, pinned right — the same placement (and the same + control, options and labels) as the browse grid, so the two read as one idea. */} +
+
+ setSortPref(next ?? COLLECTION_SORT_DEFAULT)} + label={t("plex.filter.sort")} + ascLabel={t("plex.filter.dir.asc")} + descLabel={t("plex.filter.dir.desc")} /> - {canEdit && ( - - )}
- ))} -
+
+
+ {items.map((c) => ( +
+ onOpenCard(c)} + onInfo={() => onInfoCard(c)} + onToggleWatched={toggleWatched} + menuProps={pinMenu.triggerProps} + /> + {canEdit && ( + + )} +
+ ))} +
+ )} )} diff --git a/frontend/src/components/PlexDiscover.dom.test.tsx b/frontend/src/components/PlexDiscover.dom.test.tsx index 548f5314..df49dee5 100644 --- a/frontend/src/components/PlexDiscover.dom.test.tsx +++ b/frontend/src/components/PlexDiscover.dom.test.tsx @@ -235,4 +235,31 @@ describe("PlexDiscover", () => { ); await screen.findByText("plex.discover.empty"); }); + + it("puts a Surprise card back when its watched toggle fails", async () => { + // The resync refetches the rails but not the Surprise draw (that would reshuffle it), so a + // refused toggle stayed flipped there — the sibling of the collection page's fix. + plexDiscover.mockResolvedValue({ + ...fullPayload, + continue_watching: [], + recently_watched: [], + recently_added: [], + because_you_watched: null, + }); + plexSurprise.mockResolvedValue({ scope: "both", items: [card("sp1", "Random Pick")] }); + plexSetState.mockRejectedValueOnce(new Error("refused")); + wrap( + , + ); + fireEvent.click(await screen.findByLabelText("plex.markWatched")); + await waitFor(() => expect(plexSetState).toHaveBeenCalledWith("sp1", "watched")); + await waitFor(() => expect(screen.getByLabelText("plex.markWatched")).toBeTruthy()); + expect(screen.queryByLabelText("plex.markUnwatched")).toBeNull(); + }); }); diff --git a/frontend/src/components/PlexDiscover.tsx b/frontend/src/components/PlexDiscover.tsx index c7414ef5..435b06c9 100644 --- a/frontend/src/components/PlexDiscover.tsx +++ b/frontend/src/components/PlexDiscover.tsx @@ -105,6 +105,14 @@ export default function PlexDiscover({ // Watched toggle over BOTH this surface's cache (so a rail card flips instantly) and the surprise // list, then persist + resync via the shared helper — invalidating the grid too so it agrees. + const setSurpriseStatus = (card: PlexCard, status: PlexCard["status"]) => + qc.setQueryData<{ scope: string; items: PlexCard[] }>( + qk.plexSurprise(scope, lang, filters), + (old) => + old + ? { ...old, items: old.items.map((c) => (c.id === card.id ? { ...c, status } : c)) } + : old, + ); const toggleWatched = (card: PlexCard) => toggleWatchState(card, t, { optimistic: (next) => { @@ -122,11 +130,11 @@ export default function PlexDiscover({ } : old, ); - qc.setQueryData<{ scope: string; items: PlexCard[] }>( - qk.plexSurprise(scope, lang, filters), - (old) => (old ? { ...old, items: flip(old.items) } : old), - ); + setSurpriseStatus(card, next); }, + // The resync below leaves the Surprise rail alone (a refetch would reshuffle it), so a failed + // toggle was left flipped there — put back in place instead. + revert: () => setSurpriseStatus(card, card.status), resync: () => { qc.invalidateQueries({ queryKey: qk.plexDiscover() }); qc.invalidateQueries({ queryKey: qk.plexLibrary() }); diff --git a/frontend/src/components/PlexPlayer.hls.dom.test.tsx b/frontend/src/components/PlexPlayer.hls.dom.test.tsx new file mode 100644 index 00000000..1e83fb9e --- /dev/null +++ b/frontend/src/components/PlexPlayer.hls.dom.test.tsx @@ -0,0 +1,259 @@ +// @vitest-environment jsdom +// +// The Plex player's HLS path, with hls.js replaced by a stand-in the test drives (jsdom has no +// MediaSource, so the real one never runs here). Its own file because the module mock is file-wide. +import { describe, it, expect, vi, beforeAll, beforeEach, afterEach } from "vitest"; +import { render, cleanup, waitFor, screen, act } from "@testing-library/react"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; + +vi.mock("react-i18next", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }), + }; +}); + +// Hoisted with the mock below (vi.mock runs before the module body). +const { FakeHls, hlsInstances } = vi.hoisted(() => { + type Handler = (event: string, data: unknown) => void; + const hlsInstances: InstanceType[] = []; + class FakeHls { + handlers = new Map(); + audioTrack = 0; + constructor() { + hlsInstances.push(this); + } + on(event: string, fn: Handler) { + this.handlers.set(event, [...(this.handlers.get(event) ?? []), fn]); + } + emit(event: string, data: unknown = {}) { + for (const fn of this.handlers.get(event) ?? []) fn(event, data); + } + loadSource() {} + attachMedia() {} + destroy() {} + } + return { FakeHls, hlsInstances }; +}); +vi.mock("hls.js", async (importOriginal) => { + const actual = (await importOriginal<{ default: typeof import("hls.js").default }>()).default; + return { + default: Object.assign(FakeHls, { + isSupported: () => true, + Events: actual.Events, + ErrorTypes: actual.ErrorTypes, + }), + }; +}); + +import Hls from "hls.js"; +import PlexPlayer from "./PlexPlayer"; +import { api, type PlexItemDetail } from "../lib/api"; +import { ACTIVE_ACCOUNT_KEY } from "../lib/activeAccount"; + +type Media = HTMLMediaElement & { _t?: number }; + +beforeAll(() => { + class Stub { + observe() {} + unobserve() {} + disconnect() {} + takeRecords() { + return []; + } + } + globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver; + globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver; + Object.defineProperty(HTMLMediaElement.prototype, "currentTime", { + configurable: true, + get(this: Media) { + return this._t ?? 0; + }, + set(this: Media, v: number) { + this._t = v; + }, + }); + Object.defineProperty(HTMLMediaElement.prototype, "play", { + configurable: true, + value: () => Promise.resolve(), + }); + Object.defineProperty(HTMLMediaElement.prototype, "pause", { configurable: true, value() {} }); + Object.defineProperty(HTMLMediaElement.prototype, "load", { configurable: true, value() {} }); + Object.defineProperty(HTMLMediaElement.prototype, "seekable", { + configurable: true, + get: () => ({ length: 0, end: () => 0 }), + }); + Object.defineProperty(HTMLMediaElement.prototype, "textTracks", { + configurable: true, + get: () => Object.assign([], { addEventListener() {}, removeEventListener() {} }), + }); +}); + +beforeEach(() => { + sessionStorage.setItem(ACTIVE_ACCOUNT_KEY, "7"); + hlsInstances.length = 0; +}); + +afterEach(() => { + cleanup(); + vi.restoreAllMocks(); + sessionStorage.clear(); + localStorage.clear(); +}); + +const detail = { + id: "A", + title: "Title A", + type: "movie", + duration_seconds: 3000, + status: "new", + position_seconds: 600, + audio_streams: [{ ord: 0, label: "English", language: "en", default: true }], + subtitle_streams: [], + markers: [], +} as unknown as PlexItemDetail; + +function mount() { + vi.spyOn(api, "plexItem").mockResolvedValue(detail); + vi.spyOn(api, "plexProgress").mockResolvedValue(undefined as never); + const session = vi + .spyOn(api, "plexSession") + .mockResolvedValue({ mode: "hls", url: "/hls/A/index.m3u8", start_s: 600 } as never); + render( + + {}} /> + , + ); + return session; +} + +const lost404 = { + fatal: true, + type: Hls.ErrorTypes.NETWORK_ERROR, + details: "fragLoadError", + response: { code: 404, text: "No active playback session" }, +}; + +describe("an HLS session the server no longer has", () => { + it("is restarted where the viewer is, instead of ending playback", async () => { + // The server reaps a session nobody fetches from (a long pause; a deploy). hls.js never retries + // a 4xx, so the next segment's 404 used to end playback with "couldn't play". + const session = mount(); + await waitFor(() => expect(hlsInstances).toHaveLength(1)); + act(() => hlsInstances[0]!.emit(Hls.Events.MANIFEST_PARSED)); + const v = document.querySelector("video") as Media; + v._t = 700; // 600 + 700 − the HLS lag + act(() => void v.dispatchEvent(new Event("timeupdate"))); + act(() => hlsInstances[0]!.emit(Hls.Events.ERROR, lost404)); + await waitFor(() => expect(session).toHaveBeenCalledTimes(2)); + expect(session.mock.calls[1]![1]).toBeCloseTo(1299, 0); + expect(screen.queryByText("plex.player.errGeneric")).toBeNull(); + }); + + it("gives up and says so when the reclaimed session keeps 404ing", async () => { + // `parsed` cannot bound this: it lives on ONE hls instance and every restart builds a fresh one, + // so a session that parses its manifest and then 404s re-arms the guard every cycle. The server + // also answers 404 for "that segment is not there YET" (a transcode falling behind realtime), + // which is indistinguishable on the wire — so the player respawned ffmpeg forever behind a + // spinner that never became an error (round 5). + const session = mount(); + const v = () => document.querySelector("video") as Media; + // The first session plus the two it is allowed to reclaim; the third 404 has nothing left. + for (let i = 0; i < 3; i++) { + await waitFor(() => expect(hlsInstances).toHaveLength(i + 1)); + act(() => hlsInstances[i]!.emit(Hls.Events.MANIFEST_PARSED)); + v()._t = 700; + act(() => void v().dispatchEvent(new Event("timeupdate"))); + act(() => hlsInstances[i]!.emit(Hls.Events.ERROR, lost404)); + } + await screen.findByText("plex.player.errGeneric"); + // Two reclaims, then the honest answer — not a fourth, fifth, hundredth session. + expect(session).toHaveBeenCalledTimes(3); + }); + + it("keeps giving up even though every reclaimed session plays a fragment first", async () => { + // The budget used to be cleared by the first FRAG_LOADED of any session — and a reclaimed + // session essentially always delivers one, because `start_session` does not return until its + // first segment exists. So it bounded NEITHER case its own comment named: measured at 40 loss + // cycles, the player opened 41 sessions and never showed an error (deep-review round 6). A + // fragment is not proof the session can be served; surviving the window is. + const session = mount(); + const v = () => document.querySelector("video") as Media; + for (let i = 0; i < 3; i++) { + await waitFor(() => expect(hlsInstances).toHaveLength(i + 1)); + act(() => hlsInstances[i]!.emit(Hls.Events.MANIFEST_PARSED)); + v()._t = 700; + act(() => void v().dispatchEvent(new Event("timeupdate"))); + act(() => + hlsInstances[i]!.emit(Hls.Events.FRAG_LOADED, { + frag: { type: "main", duration: 6, stats: { total: 1000 } }, + }), + ); + act(() => hlsInstances[i]!.emit(Hls.Events.ERROR, lost404)); + } + await screen.findByText("plex.player.errGeneric"); + expect(session).toHaveBeenCalledTimes(3); + }); + + it("starts the budget over for a loss long after the last one", async () => { + // The other half: a session reaped once an hour is not a session that cannot be served. The + // window is what separates the two, so a spent budget must not follow the viewer around. + vi.useFakeTimers({ shouldAdvanceTime: true, toFake: ["Date"] }); + try { + const session = mount(); + const v = () => document.querySelector("video") as Media; + for (let i = 0; i < 4; i++) { + await waitFor(() => expect(hlsInstances).toHaveLength(i + 1)); + act(() => hlsInstances[i]!.emit(Hls.Events.MANIFEST_PARSED)); + v()._t = 700; + act(() => void v().dispatchEvent(new Event("timeupdate"))); + act(() => hlsInstances[i]!.emit(Hls.Events.ERROR, lost404)); + act(() => void vi.setSystemTime(Date.now() + 10 * 60_000)); + } + expect(screen.queryByText("plex.player.errGeneric")).toBeNull(); + expect(session).toHaveBeenCalledTimes(5); + } finally { + vi.useRealTimers(); + } + }); + + it("gives up on a session lost over and over on a SLOW cadence too", async () => { + // The window's anchor used to be written only where the window RESET, so it marked the start of + // the window instead of the last loss — a tumbling window, not the rolling one the constant is + // named for. Any cadence slower than half the window then cleared the budget on every second + // loss and the bound never bit: a transcode falling ~70s behind realtime — the case the ERROR + // handler itself names — reclaimed forever (measured: 200 losses 70s apart → 200 reclaims, + // never an error). The two cases above space their losses 0s and 10min apart, i.e. both sit + // outside the 60–120s band where the defect lived (deep-review round 7). + vi.useFakeTimers({ shouldAdvanceTime: true, toFake: ["Date"] }); + try { + const session = mount(); + const v = () => document.querySelector("video") as Media; + for (let i = 0; i < 6 && !screen.queryByText("plex.player.errGeneric"); i++) { + await waitFor(() => expect(hlsInstances).toHaveLength(i + 1)); + act(() => hlsInstances[i]!.emit(Hls.Events.MANIFEST_PARSED)); + v()._t = 700; + act(() => void v().dispatchEvent(new Event("timeupdate"))); + act(() => hlsInstances[i]!.emit(Hls.Events.ERROR, lost404)); + act(() => void vi.setSystemTime(Date.now() + 70_000)); + } + await screen.findByText("plex.player.errGeneric"); + // Two reclaims, each within two minutes of the loss before it — then the honest answer. + expect(session).toHaveBeenCalledTimes(3); + } finally { + vi.useRealTimers(); + } + }); + + it("is a failure, not a restart, when the session never played", async () => { + // Otherwise a session that 404s from the start would restart itself in a loop. + const session = mount(); + await waitFor(() => expect(hlsInstances).toHaveLength(1)); + act(() => hlsInstances[0]!.emit(Hls.Events.ERROR, lost404)); + await screen.findByText("plex.player.errGeneric"); + expect(session).toHaveBeenCalledTimes(1); + }); +}); diff --git a/frontend/src/components/PlexPlayer.session.dom.test.tsx b/frontend/src/components/PlexPlayer.session.dom.test.tsx new file mode 100644 index 00000000..5593d358 --- /dev/null +++ b/frontend/src/components/PlexPlayer.session.dom.test.tsx @@ -0,0 +1,628 @@ +// @vitest-environment jsdom +// +// The Plex player's session lifecycle, rendered for real (deep-review round 3 of bug hunt 2026-09). +// +// jsdom has no media pipeline, so the
)} + {/* Subtitle-extraction notice. Pulling a track out of a big MKV is a full read of the container + (154 s for a 17 GB film, measured), so it happens in the background — around the playhead + first. Until then the is deliberately NOT mounted, and this says why instead of + leaving the viewer to conclude that switching subtitles "does nothing" (bug hunt 2026-09, + item 8). A failure is said even over a mounted window (see `subNotice`). It sits one rung + above the track-switch OSD so the two never overlap. */} +
+ {selectedSub && subNotice && ( +
+ {subNotice !== "preparing" ? ( + {t(`plex.player.subUnavailable.${subNotice}`)} + ) : ( + <> + + {t("plex.player.subPreparing", { label: selectedSub.label })} + + )} +
+ )} +
+ {/* Keyboard track-switch OSD. Sits ABOVE the control bar's own space so it reads even while the bar is showing, and inside the player element so it survives fullscreen. `aria-live` because the whole point is confirming something you cannot otherwise see. @@ -1068,7 +1300,7 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
seekTo(absRef.current - 10)} + onClick={() => seekTo(playhead(absRef, loadStartRef) - 10)} align="start" > @@ -1076,7 +1308,10 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) { {playing ? : } - seekTo(absRef.current + 10)}> + seekTo(playhead(absRef, loadStartRef) + 10)} + > seekTo(0)}> @@ -1191,6 +1426,7 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
)} + {/* The app's Toaster is underneath this full-screen player (and outside it in native + fullscreen), so toasts raised while watching show here instead. See ui/Toaster. */} +
, document.body, ); diff --git a/frontend/src/components/PlexPlaylistAdd.tsx b/frontend/src/components/PlexPlaylistAdd.tsx index 76d37f58..bdd63f76 100644 --- a/frontend/src/components/PlexPlaylistAdd.tsx +++ b/frontend/src/components/PlexPlaylistAdd.tsx @@ -4,6 +4,7 @@ import { useTranslation } from "react-i18next"; import { useQuery, useQueryClient } from "@tanstack/react-query"; import { Check, Minus, Plus } from "lucide-react"; import { api, type PlexPlaylist } from "../lib/api"; +import { notifyActionError } from "../lib/actionErrors"; import { useScrollFade } from "../lib/useScrollFade"; import { Button } from "./ui/Button"; import Modal from "./ui/Modal"; @@ -54,6 +55,11 @@ export default function PlexPlaylistAdd({ return isGroup ? (p.group_in ?? 0) : p.has_item ? 1 : 0; } + // A failure is said, and the list refreshed either way. With no catch, a playlist deleted on + // another device (a 404 `not_found/playlist` — still offered here, from a cache up to 30 s old) + // was an unhandled rejection: `req` leaves 403/404/429 to the caller, so nothing was shown, the + // tick never flipped and the gone row stayed. Round 5 swept the collection editor and its own + // YouTube twin (`AddToPlaylist`) and missed this one (round 6). async function toggle(p: PlexPlaylist) { if (busy) return; setBusy(true); @@ -67,8 +73,13 @@ export default function PlexPlaylistAdd({ else await api.plexPlaylistAddItem(p.id, target.ratingKey); } setOverride((m) => new Map(m).set(p.id, allIn ? 0 : size)); - refresh(); + } catch (e) { + notifyActionError( + e, + t(allIn ? "plex.playlistAdd.removeFailed" : "plex.playlistAdd.addFailed"), + ); } finally { + refresh(); setBusy(false); } } @@ -87,8 +98,12 @@ export default function PlexPlaylistAdd({ setOverride((m) => new Map(m).set(pl.id, 1)); } setNewName(""); - refresh(); + } catch (e) { + // The create may already have landed when the add fails — `refresh()` in `finally` is what + // makes that half-done state visible instead of leaving a playlist the dialog never lists. + notifyActionError(e, t("plex.playlistAdd.createFailed")); } finally { + refresh(); setBusy(false); } } diff --git a/frontend/src/components/PlexPlaylistReorder.dom.test.tsx b/frontend/src/components/PlexPlaylistReorder.dom.test.tsx new file mode 100644 index 00000000..768e5744 --- /dev/null +++ b/frontend/src/components/PlexPlaylistReorder.dom.test.tsx @@ -0,0 +1,130 @@ +// @vitest-environment jsdom +// +// One accepted write, one cache entry that matches it — the Plex half. +// +// `Playlists.tsx` (the YouTube list) got this rule in round 5: the detail cache is what a rollback +// reads and what a re-entry inside the 30 s staleTime renders without refetching, so an accepted +// reorder that is not written there leaves the cache disagreeing with the server. The Plex list got +// the rollback half of that round's fix and not this half (round 6). +// +// Its own file: `@dnd-kit/core` is mocked module-wide so the test can fire `onDragEnd` directly — +// dnd-kit's pointer sensor does not activate under jsdom (no layout, no pointer capture). +import { it, expect, vi, beforeAll, afterEach } from "vitest"; +import { render, cleanup, screen, act, waitFor } from "@testing-library/react"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; + +vi.mock("react-i18next", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }), + }; +}); + +const { dragEnd } = vi.hoisted(() => ({ dragEnd: { fire: (_e: unknown) => {} } })); +vi.mock("@dnd-kit/core", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + DndContext: ({ + children, + onDragEnd, + }: { + children: React.ReactNode; + onDragEnd: (e: unknown) => void; + }) => { + dragEnd.fire = onDragEnd; + return <>{children}; + }, + }; +}); + +import PlexPlaylistView from "./PlexPlaylistView"; +import { api, type PlexCard } from "../lib/api"; +import { qk } from "../lib/queryKeys"; + +beforeAll(() => { + class Stub { + observe() {} + unobserve() {} + disconnect() {} + takeRecords() { + return []; + } + } + globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver; + globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver; +}); +afterEach(() => { + cleanup(); + vi.restoreAllMocks(); +}); + +const movie = (id: string) => + ({ id, type: "movie", title: `Movie ${id}`, year: 2020, thumb: "" }) as unknown as PlexCard; + +it("writes the order the server accepted into the detail cache, without waiting for a refetch", async () => { + // The refetch that same handler schedules (round 7) must not be what makes the cache right: the + // point of the write is that a re-entry inside the 30 s staleTime renders the accepted order + // immediately, with no request at all. So the refetch is HELD OPEN here — if this assertion + // needed it to land, deleting the write would still pass and round 6's fix would be unguarded. + let calls = 0; + vi.spyOn(api, "plexPlaylist").mockImplementation(() => + ++calls === 1 + ? Promise.resolve({ id: 5, title: "Films", items: [movie("1"), movie("2"), movie("3")] }) + : new Promise(() => {}), + ); + const reorder = vi.spyOn(api, "plexReorderPlaylist").mockResolvedValue(undefined as never); + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + render( + + {}} onPlay={() => {}} /> + , + ); + await screen.findByText("Movie 1"); + act(() => dragEnd.fire({ active: { id: "1" }, over: { id: "2" } })); + await waitFor(() => expect(reorder).toHaveBeenCalledWith(5, ["2", "1", "3"])); + // The list key was always invalidated; the DETAIL key — the one `putBack` reads and the one a + // re-entry renders from — kept the pre-reorder order, so leaving and coming back inside the + // staleTime showed the reorder undone, and the next drag persisted that. + await waitFor(() => + expect( + qc.getQueryData<{ items: PlexCard[] }>(qk.plexPlaylist(5))?.items.map((i) => i.id), + ).toEqual(["2", "1", "3"]), + ); +}); + +it("refetches the detail too, so a membership change that landed meanwhile is not clobbered", async () => { + // `next` is the membership of the render that started the drag, and `reorder_playlist` only + // touches the rating_keys it is handed — it never ratifies that list. So writing `next` into the + // detail cache silently drops anything that landed in between (a row removed from another + // surface, an item added elsewhere), and with only the LIST key invalidated nothing corrected it: + // staleTime is 30 s and refetchOnWindowFocus is off, so the phantom row stayed on screen for as + // long as the view was mounted and the next drag persisted an order built from it. The YouTube + // sibling invalidates the detail key right after writing it; this half did not (round 7). + let calls = 0; + vi.spyOn(api, "plexPlaylist").mockImplementation(() => + Promise.resolve( + ++calls === 1 + ? { id: 5, title: "Films", items: [movie("1"), movie("2"), movie("3")] } + : { id: 5, title: "Films", items: [movie("2"), movie("1")] }, // "3" is gone server-side + ), + ); + const reorder = vi.spyOn(api, "plexReorderPlaylist").mockResolvedValue(undefined as never); + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + render( + + {}} onPlay={() => {}} /> + , + ); + await screen.findByText("Movie 3"); + act(() => dragEnd.fire({ active: { id: "1" }, over: { id: "2" } })); + await waitFor(() => expect(reorder).toHaveBeenCalledWith(5, ["2", "1", "3"])); + // The refetch the accepted write schedules brings the real membership back. + await waitFor(() => expect(screen.queryByText("Movie 3")).toBeNull()); + await waitFor(() => + expect( + qc.getQueryData<{ items: PlexCard[] }>(qk.plexPlaylist(5))?.items.map((i) => i.id), + ).toEqual(["2", "1"]), + ); +}); diff --git a/frontend/src/components/PlexPlaylistView.tsx b/frontend/src/components/PlexPlaylistView.tsx index 00505e67..6b3890e3 100644 --- a/frontend/src/components/PlexPlaylistView.tsx +++ b/frontend/src/components/PlexPlaylistView.tsx @@ -31,7 +31,8 @@ import { Trash2, X, } from "lucide-react"; -import { api, type PlexCard } from "../lib/api"; +import { api, HttpError, type PlexCard } from "../lib/api"; +import { notifyActionError } from "../lib/actionErrors"; import { formatRuntime } from "../lib/format"; import { getAccountRaw, LS, setAccountRaw } from "../lib/storage"; import { useConfirm } from "./ConfirmProvider"; @@ -401,14 +402,54 @@ export default function PlexPlaylistView({ qc.invalidateQueries({ queryKey: qk.plexPlaylists() }); }; + // Every edit here is optimistic on the local copy, and none of them used to handle a failure: a + // removal that failed stayed removed on screen (the resync never ran), and a 404 — the playlist + // deleted on another device, now a structured `not_found` — was an unhandled rejection that said + // nothing at all (deep-review round 4; the YouTube side was fixed in round 3). A failure is said, + // and the rows are put back from what the server last sent — explicitly, because a resync that + // returns the same list keeps the same `q.data` and the re-sync effect above never runs. + // Read the cache AT ROLLBACK TIME, not the `q.data` of the render that bound the failing handler: + // a removal completing while a reorder was in flight refreshed the list, and the reorder's stale + // closure then put the removed row back on screen — where the deep-equal refetch could not clear + // it, and the next drag persisted an order containing it (deep-review round 5). `Playlists.tsx`'s + // sibling `restoreFromCache` reads it live for the same reason. + const putBack = () => { + const cur = qc.getQueryData<{ items: PlexCard[] }>(qk.plexPlaylist(playlistId)); + if (cur) setItems(cur.items); + invalidate(); + }; + function commit(next: PlexCard[]) { + if (busy) return; // a reorder racing a removal is what makes a rollback ambiguous setItems(next); api .plexReorderPlaylist( playlistId, next.map((i) => i.id), ) - .then(() => qc.invalidateQueries({ queryKey: qk.plexPlaylists() })); + .then(() => { + // Write the accepted order into the DETAIL cache, not just the list. That cache is what + // `putBack` rolls back to and what a re-entry inside the 30 s staleTime renders without + // refetching — so after an accepted reorder it showed the pre-reorder order again, and the + // next drag from there persisted it, silently undoing the one the server took. The + // YouTube sibling (`Playlists.tsx`) got this in round 5; this half was left behind (round + // 6). One accepted write, one cache entry that matches it. + qc.setQueryData<{ items: PlexCard[] }>(qk.plexPlaylist(playlistId), (old) => + old ? { ...old, items: next } : old, + ); + qc.invalidateQueries({ queryKey: qk.plexPlaylist(playlistId) }); + // …and then REFETCH it, exactly as the sibling does. `next` is this render's membership, and + // the server only reorders the keys it was given — so anything that landed meanwhile (a row + // removed from another surface, an item added elsewhere) is clobbered by that write. With + // only the list key invalidated nothing corrected it: staleTime is 30 s and + // refetchOnWindowFocus is off, so the wrong membership stayed on screen for as long as the + // view was mounted and the next drag persisted an order built from it (round 7). + qc.invalidateQueries({ queryKey: qk.plexPlaylists() }); + }) + .catch((e: unknown) => { + notifyActionError(e, t("plex.playlist.reorderFailed")); + putBack(); + }); } async function onRemove(keys: string[]) { @@ -420,6 +461,9 @@ export default function PlexPlaylistView({ if (keys.length === 1) await api.plexPlaylistRemoveItem(playlistId, keys[0]!); else await api.plexPlaylistRemoveBulk(playlistId, keys); invalidate(); + } catch (e) { + notifyActionError(e, t("plex.playlist.removeFailed")); + putBack(); } finally { setBusy(false); } @@ -428,7 +472,12 @@ export default function PlexPlaylistView({ async function rename() { const title = name.trim(); if (!title) return; - await api.plexRenamePlaylist(playlistId, title); + try { + await api.plexRenamePlaylist(playlistId, title); + } catch (e) { + notifyActionError(e, t("plex.playlist.renameFailed")); // the field stays open to try again + return; + } setRenaming(false); invalidate(); } @@ -440,7 +489,16 @@ export default function PlexPlaylistView({ })) ) return; - await api.plexDeletePlaylist(playlistId); + try { + await api.plexDeletePlaylist(playlistId); + } catch (e) { + // Gone already (deleted on another device): what was asked for is true — leave the dead page. + // Anything else is a real failure, and the page stays. + if (!(e instanceof HttpError && e.status === 404)) { + notifyActionError(e, t("plex.playlist.deleteFailed")); + return; + } + } qc.invalidateQueries({ queryKey: qk.plexPlaylists() }); onBack(); } diff --git a/frontend/src/components/PlexSidebar.tsx b/frontend/src/components/PlexSidebar.tsx index fcdb92d2..cda82882 100644 --- a/frontend/src/components/PlexSidebar.tsx +++ b/frontend/src/components/PlexSidebar.tsx @@ -1,5 +1,6 @@ import { useState } from "react"; import { qk } from "../lib/queryKeys"; +import { usePlexCollectionTitles } from "../lib/usePlexCollectionTitles"; import { useTranslation } from "react-i18next"; import { plexLang } from "../i18n"; import { useQuery, useQueryClient } from "@tanstack/react-query"; @@ -447,6 +448,10 @@ function CollectionPicker({ queryFn: () => api.plexCollections(undefined, searchDeb || undefined), enabled: open, // only fetch while the picker is open }); + // …but a selection restored from a LINK has ids and no titles, and the list above is only fetched + // once the picker is opened — so the token rendered the raw rating_key until then. Resolve just the + // selected ids (see usePlexCollectionTitles). + const resolved = usePlexCollectionTitles(values, titles); const options: ComboboxOption[] = (q.data?.collections ?? []).map((c) => ({ value: c.id, label: c.title, @@ -454,7 +459,7 @@ function CollectionPicker({ })); // Combine the fetched labels with the ones already stored so the trip through onChange keeps every // selected collection's title (the fetched list is narrowed by the current search). - const labelMap: Record = { ...titles }; + const labelMap: Record = { ...resolved }; for (const o of options) if (o.label) labelMap[o.value] = o.label; return ( @@ -463,7 +468,7 @@ function CollectionPicker({ clientFilter={false} // the server already narrows to `search` options={options} selected={values} - selectedLabels={titles} + selectedLabels={resolved} query={search} onQueryChange={setSearch} onOpenChange={setOpen} diff --git a/frontend/src/components/PlexStatsOverlay.tsx b/frontend/src/components/PlexStatsOverlay.tsx index 5fdd1029..0cadd95f 100644 --- a/frontend/src/components/PlexStatsOverlay.tsx +++ b/frontend/src/components/PlexStatsOverlay.tsx @@ -2,7 +2,7 @@ import { type RefObject, useEffect, useState } from "react"; import { useTranslation } from "react-i18next"; import type Hls from "hls.js"; import { X } from "lucide-react"; -import { buildStatRows, type HlsLevelInfo, type StatRow } from "../lib/plexStats"; +import { buildStatRows, type FragMeter, type HlsLevelInfo, type StatRow } from "../lib/plexStats"; // "Stats for nerds" debug overlay (E6 S2). Samples the live HTML5