release: 0.69.0 (improvement/e3-s1-plex-split)

VERSION

@@ -1 +1 @@
-0.68.0
+0.69.0

backend/alembic/versions/0070_fold_maintenance_batch.py

@@ -0,0 +1,47 @@
+"""Fold the maintenance re-validation batch from app_state into sysconfig (B-core-11)
+
+The admin-tunable `maintenance_revalidate_batch` lived in its own `app_state` column with a
+hand-rolled override-or-default reader in `app.state`, duplicating the sysconfig SPECS pattern. It's
+now a sysconfig key (`system_config` store), so every admin knob lives in one place. This migration
+carries any existing override into the sysconfig store, then drops the dead column. Reversible.
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0070_fold_maintenance_batch"
+down_revision: Union[str, None] = "0069_job_index_hygiene"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # Preserve a customized override: copy it into the sysconfig store before the column goes.
+    op.execute(
+        """
+        INSERT INTO system_config (key, value, encrypted)
+        SELECT 'maintenance_revalidate_batch', maintenance_revalidate_batch::text, false
+        FROM app_state
+        WHERE id = 1 AND maintenance_revalidate_batch IS NOT NULL
+        ON CONFLICT (key) DO NOTHING
+        """
+    )
+    op.drop_column("app_state", "maintenance_revalidate_batch")
+
+
+def downgrade() -> None:
+    op.add_column(
+        "app_state",
+        sa.Column("maintenance_revalidate_batch", sa.Integer(), nullable=True),
+    )
+    # Carry the value back onto the row, then drop the sysconfig override (symmetric restore).
+    op.execute(
+        """
+        UPDATE app_state SET maintenance_revalidate_batch =
+            (SELECT value::int FROM system_config WHERE key = 'maintenance_revalidate_batch')
+        WHERE id = 1
+          AND EXISTS (SELECT 1 FROM system_config WHERE key = 'maintenance_revalidate_batch')
+        """
+    )
+    op.execute("DELETE FROM system_config WHERE key = 'maintenance_revalidate_batch'")

backend/app/README.md

@@ -0,0 +1,75 @@
+# Backend conventions
+
+The rules below are already followed across `backend/app` — they lived only in a few module
+docstrings (`demo.py`, `userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
+B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
+I copy".
+
+## Module layering
+
+Three layers, by *who* runs the code — not by feature:
+
+1. **`routes/…` — request-only.** A route module holds HTTP concerns: auth dependencies, request/
+   response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
+   Business logic that a request happens to trigger does **not** live here if anything else runs it too.
+
+2. **`plex/`, `downloads/`, `sync/` — domain logic shared by more than one caller.** Anything the
+   **scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
+   route, the worker, and the scheduler share one definition. Examples from this codebase:
+   `downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
+   run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline.
+
+3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth`, `sysconfig`, `quota`, `audit`,
+   `notifications`, `userscope`, `realtime`, `ratelimit`, `progress`, … are single-file primitives many
+   layers import. They depend on nothing above them (a core module must never import a `routes/…`
+   module).
+
+**Extraction trigger: the SECOND consumer.** Keep new logic inline in its first caller. The moment a
+*second* caller needs it (a worker as well as a route; a scheduler as well as a route), extract it down
+a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
+pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
+
+## Transaction ownership
+
+This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
+(only `quota` owns its session)"** — so read it that way, not as an exception to it. The commit belongs
+to the **operation owner**; anything called *inside* another transaction only stages its change
+(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
+`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
+mutation it records, so an audit entry can never exist without its change (or vice-versa).
+
+- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
+  whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
+  a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
+  the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
+- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
+  loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
+  attribute — see `service.decr_ref_count`.
+
+## Shared counters and status: do it atomically at the DB
+
+- **Never read-modify-write a shared counter in Python** (`x.count = x.count + 1`). Concurrent sessions
+  lose an update. Use a DB-atomic statement — `col = col + 1`, or `greatest(col - 1, 0)` with
+  `RETURNING` so the caller branches on the post-value under the row lock (`service.incr_ref_count` /
+  `decr_ref_count`).
+- **Download job status is a `JobStatus` StrEnum** (`downloads/status.py`), not bare strings. The
+  `HOLDING` set defines which states hold a `MediaAsset` (so release accounting stays correct);
+  `STOPPED_BY_ROUTE` is the route-initiated stop set. A worker child's terminal write goes through
+  `service.finalize_if_running` (an atomic `UPDATE … WHERE status='running'`) so a finishing child can't
+  resurrect a job a route just stopped.
+
+## Config: one home per kind
+
+- **`app.config.settings`** — boot/env config (DB URL, download root, worker concurrency): read once at
+  startup, not user-tunable at runtime.
+- **`sysconfig.SPECS`** — the runtime admin knobs (DB-override-or-default). A new admin-tunable value
+  goes in `SPECS`; don't invent a fourth store. (`state.py`'s `maintenance_batch` predates this and is
+  slated to fold into `sysconfig` — do not add more like it.)
+
+## The gate
+
+Everything below must be green before a change is done (run via `siftlode check`):
+`ruff` (backend) · `tsc` ×3 + `eslint` + `prettier` + `knip` (frontend) · `vitest` · `pytest`. The
+pytest lane runs in `backend/docker-compose.test.yml` — an ephemeral Postgres + the backend tree
+bind-mounted — so the DB-backed integration tests (the `db` fixture) actually run; `SIFTLODE_TEST_
+REQUIRE_DB=1` there turns a would-be silent skip into a hard failure. See `docs/development.md`.

backend/app/config.py

@@ -57,7 +57,7 @@ class Settings(BaseSettings):
     # VPS Caddy front door. A request from any other peer (someone hitting the app port directly)
     # is NOT trusted and its raw socket IP is used, so it can't spoof its rate-limit identity via
     # XFF. Empty = trust no proxy (use the direct peer) — correct for a directly-exposed / local-dev
-    # deploy. Comma-separated. See auth._client_ip + the deploy docs.
+    # deploy. Comma-separated. See routes.auth._client_ip + the deploy docs.
     trusted_proxy_ips: str = ""
 
     # --- Outbound email (onboarding: access-request + approval notices) ---

backend/app/db.py

@@ -12,7 +12,7 @@ class Base(DeclarativeBase):
 
 # Pool sized above the sync-endpoint threadpool (~40) so a burst of short DB sessions can't starve on
 # checkout. The image proxies deliberately DON'T hold a connection during their slow external fetch
-# (see routes/plex.py), so this headroom is only for the many brief sessions, not long ones.
+# (see routes/plex/), so this headroom is only for the many brief sessions, not long ones.
 engine = create_engine(
     settings.database_url,
     pool_pre_ping=True,

backend/app/downloads/jobdb.py

@@ -0,0 +1,73 @@
+"""Short-lived-session DB helpers + staging paths shared by the worker parent and the child pipeline."""
+from pathlib import Path
+
+from sqlalchemy import text
+
+from app.config import settings
+from app.db import SessionLocal
+from app.models import DownloadJob, MediaAsset
+
+
+# --- small DB helpers (short-lived sessions; the download itself holds no transaction) -------
+
+def _set_job(job_id: int, **fields) -> None:
+    with SessionLocal() as db:
+        job = db.get(DownloadJob, job_id)
+        if job is None:
+            return
+        for k, v in fields.items():
+            setattr(job, k, v)
+        db.commit()
+
+
+def _job_status(job_id: int) -> str | None:
+    with SessionLocal() as db:
+        return db.execute(
+            text("SELECT status FROM download_jobs WHERE id = :id"), {"id": job_id}
+        ).scalar()
+
+
+def _touch_job(job_id: int) -> None:
+    """Bump a running job's `updated_at` without changing any value — a stall-watchdog heartbeat for
+    phases that legitimately write no progress. Extraction writes nothing, and a multi-client-set
+    fallback re-writes identical values (progress 0 / phase 'downloading'), so ORM `onupdate` would
+    NOT fire and two slow extraction attempts could otherwise cross the stall timeout and be killed
+    mid-extraction though they'd have succeeded."""
+    with SessionLocal() as db:
+        db.execute(
+            text("UPDATE download_jobs SET updated_at=now() WHERE id=:id AND status='running'"),
+            {"id": job_id},
+        )
+        db.commit()
+
+
+def _requeue_if_running(job_id: int, phase: str) -> None:
+    """Put the job back to 'queued' for a later retry, but ONLY while it is still 'running'. The
+    route flips status to paused/cancelled to stop a job; a blind `status='queued'` here would
+    clobber that and silently revive a job the user just cancelled. The conditional WHERE makes the
+    requeue a no-op unless we still own a running job."""
+    with SessionLocal() as db:
+        db.execute(
+            text(
+                "UPDATE download_jobs SET status='queued', phase=:phase, updated_at=now() "
+                "WHERE id=:id AND status='running'"
+            ),
+            {"id": job_id, "phase": phase},
+        )
+        db.commit()
+
+
+def _reset_asset_pending(asset_id: int) -> None:
+    with SessionLocal() as db:
+        asset = db.get(MediaAsset, asset_id)
+        if asset and asset.status == "downloading":
+            asset.status = "pending"
+            db.commit()
+
+
+def _staging_dir(asset_id: int) -> Path:
+    return Path(settings.download_root) / ".staging" / f"asset-{asset_id}"
+
+
+def _edit_staging_dir(asset_id: int) -> Path:
+    return Path(settings.download_root) / ".staging" / f"edit-{asset_id}"

backend/app/downloads/pipeline.py

@@ -0,0 +1,601 @@
+"""Child-side download/edit pipeline — the yt-dlp download + ffmpeg edit that runs in the spawned subprocess."""
+import logging
+import re
+import shutil
+import time
+from datetime import date, datetime, timedelta, timezone
+from pathlib import Path
+
+import yt_dlp
+
+from app import sysconfig
+from app.config import settings
+from app.db import SessionLocal
+from app.downloads import edit as editmod
+from app.downloads import errors as dlerrors
+from app.downloads import adapters, formats, service, storage
+from app.downloads.jobdb import (
+    _edit_staging_dir,
+    _job_status,
+    _reset_asset_pending,
+    _set_job,
+    _staging_dir,
+    _touch_job,
+)
+from app.downloads.status import JobStatus
+from app.models import DownloadJob, MediaAsset
+from app.notifications import create_notification
+from app.titles import normalize_title
+
+log = logging.getLogger("siftlode.worker")
+
+
+class _Aborted(Exception):
+    """Raised inside the progress hook when the job was paused/canceled mid-download."""
+
+
+# Both progress sources call back FAR faster than the database should be touched: yt-dlp fires its
+# hook per chunk, and ffmpeg's `-progress pipe:1` emits a block of ~12 lines every ~0.5s while
+# run_ffmpeg calls its callbacks once per LINE. Every callback here is a round-trip (`_job_status`
+# opens a session and SELECTs; `_set_job` opens one and UPDATE+COMMITs), so unthrottled they cost
+# ~10^5 queries on a long job. ONE interval, ONE implementation, used by both paths — three
+# hand-rolled copies is how the ffmpeg side ended up unthrottled while the download side was fine.
+_DB_WRITE_INTERVAL_S = 1.0
+_CANCEL_POLL_INTERVAL_S = 1.0
+
+_THUMB_SUFFIXES = (".jpg", ".jpeg", ".png", ".webp")
+# yt-dlp's working files in the staging dir. None of them is ever the finished media, but several
+# CAN be the biggest file there (a video-only per-format stream, an abandoned fragment run), so the
+# staging fallback must exclude them by name — picking one would store e.g. a silent video-only
+# stream as a ready asset.
+_YTDLP_TEMP_RE = re.compile(
+    r"\.part(-Frag\d+)?$"  # in-flight download / fragment
+    r"|\.ytdl$"  # resume state
+    r"|\.temp\.[^.]+$"  # merge/postprocess temp
+    r"|\.f\d+\.[^.]+$",  # per-format stream kept before the merge
+    re.IGNORECASE,
+)
+
+
+def _notify_download_done(
+    db, user_id: int | None, title: str | None, job_id, *, kind: str
+) -> None:
+    """Post a best-effort 'ready' notice to the owner's inbox — downloads/edits used to finish
+    silently while scheduler jobs and playlist-push already notified (R11 feedback symmetry). A
+    notification failure must never mask the completed job, so it is isolated + rolled back (mirrors
+    scheduler._notify_done). `kind` ("download"/"edit") lets the inbox pick accurate copy; the
+    English title here is only a fallback for an unknown type. `user_id`/`title` are captured by the
+    caller BEFORE its commit, so reading them here needs no post-commit refresh of an expired row."""
+    if user_id is None:
+        return
+    try:
+        create_notification(
+            db,
+            user_id=user_id,
+            type="download_done",
+            title=f"Ready to watch: {title}" if title else "Ready to watch",
+            data={"title": title, "kind": kind},
+        )
+    except Exception:
+        db.rollback()
+        log.exception("failed to write download-done notification for job %s", job_id)
+
+
+def _parse_upload_date(raw) -> date | None:
+    if not raw:
+        return None
+    try:
+        return datetime.strptime(str(raw), "%Y%m%d").date()
+    except ValueError:
+        return None
+
+
+def _fill_asset_meta_early(asset_id: int, info: dict) -> None:
+    """Populate an asset's display metadata from yt-dlp's info_dict the first time it's seen
+    (only if not already filled at enqueue from our catalog) — so an ad-hoc URL's row shows a
+    real title + thumbnail while it downloads, not just the URL. Zero extra network cost."""
+    with SessionLocal() as db:
+        asset = db.get(MediaAsset, asset_id)
+        if asset is None or asset.title:
+            return
+        # `or info.get("id")` avoids a blank card while downloading when the cleaned title is empty
+        # (e.g. a title that was only a social engagement prefix); the completion path later sets
+        # the final title with its own source_ref fallback.
+        asset.title = normalize_title(info.get("title")) or info.get("id")
+        asset.uploader = info.get("uploader") or info.get("channel")
+        asset.uploader_url = adapters.resolve_adapter(None, None, info).uploader_url(info)
+        asset.thumbnail_url = info.get("thumbnail")
+        asset.duration_s = int(info["duration"]) if info.get("duration") else None
+        asset.upload_date = _parse_upload_date(info.get("upload_date"))
+        asset.source_webpage_url = info.get("webpage_url")
+        db.commit()
+
+
+def _make_progress_hook(job_id: int, asset_id: int):
+    state = {"meta_done": False}
+    cancelled = _cancel_poll(job_id)          # same policy as the ffmpeg path
+    allow_write = _rate_limiter(_DB_WRITE_INTERVAL_S)
+
+    def hook(d: dict) -> None:
+        # Cooperative pause/cancel: if the job is no longer 'running', abort the download.
+        if cancelled():
+            raise _Aborted
+        if not state["meta_done"] and d.get("info_dict"):
+            state["meta_done"] = True
+            try:
+                _fill_asset_meta_early(asset_id, d["info_dict"])
+            except Exception:  # noqa: BLE001 — metadata is best-effort, never fail a download
+                pass
+        if d.get("status") != "downloading":
+            return
+        if not allow_write():
+            return
+        # yt-dlp downloads the video and audio streams SEPARATELY (each 0→100%), then merges.
+        # Label the phase so the user understands the bar resetting between streams instead of
+        # seeing a mysterious 95%→5% jump.
+        info = d.get("info_dict") or {}
+        vcodec = info.get("vcodec") or "none"
+        acodec = info.get("acodec") or "none"
+        if vcodec != "none" and acodec == "none":
+            phase = "video"
+        elif acodec != "none" and vcodec == "none":
+            phase = "audio"
+        else:
+            phase = "media"
+        total = d.get("total_bytes") or d.get("total_bytes_estimate") or 0
+        done = d.get("downloaded_bytes") or 0
+        pct = int(done * 100 / total) if total else 0
+        _set_job(
+            job_id,
+            progress=min(pct, 99),
+            speed_bps=int(d["speed"]) if d.get("speed") else None,
+            eta_s=int(d["eta"]) if d.get("eta") else None,
+            phase=phase,
+        )
+
+    return hook
+
+
+def _rate_limiter(interval_s: float, *, now=time.monotonic):
+    """`should_run()` → True at most once per `interval_s` (the first call always passes).
+    `now` is injectable so tests drive a clock instead of patching the global one."""
+    # `None`, not 0.0: "never ran" has to be a VALUE, not a falsy timestamp. A truthiness test here
+    # reads a legitimate `now() == 0.0` as "never ran" and lets a second call straight through —
+    # unreachable with real `time.monotonic()`, but `now` exists to be injected, and a test clock
+    # starting at 0 is the obvious choice. That would let a throttle test encode the wrong
+    # behaviour and still pass.
+    state: dict[str, float | None] = {"at": None}
+
+    def should_run() -> bool:
+        t = now()
+        last = state["at"]
+        if last is not None and t - last < interval_s:
+            return False
+        state["at"] = t
+        return True
+
+    return should_run
+
+
+def _cancel_poll(job_id: int, interval_s: float = _CANCEL_POLL_INTERVAL_S, *, now=time.monotonic):
+    """Cooperative "has this job been paused/cancelled?" poll, rate-limited and sticky: once it
+    says yes it never queries again. Shared by the yt-dlp hook and run_ffmpeg so cancel latency is
+    one policy, not one per call site."""
+    allow = _rate_limiter(interval_s, now=now)
+    state = {"cancelled": False}
+
+    def cancelled() -> bool:
+        if not state["cancelled"] and allow():
+            state["cancelled"] = _job_status(job_id) not in ("running", None)
+        return state["cancelled"]
+
+    return cancelled
+
+
+def _ffmpeg_progress_writer(job_id: int, phase: str, *, now=time.monotonic):
+    """`on_progress` for run_ffmpeg: persist a progress row write at most once per interval and only
+    when the whole-percent value changed (the bar shows integers), plus a separate once-per-interval
+    heartbeat while the percent is stuck (so a slow re-encode isn't mistaken for a stall)."""
+    # Progress writes have their OWN 1/interval budget so a heartbeat can never delay a real write (the
+    # round-1 concern). The heartbeat, in turn, is gated on `last_touch` — the last DB write OR beat —
+    # so a progress write (which already refreshes updated_at) suppresses a redundant heartbeat that
+    # same interval, keeping active downloads at ~1 write/interval, not two.
+    allow_write = _rate_limiter(_DB_WRITE_INTERVAL_S, now=now)
+    state = {"pct": -1}
+    last_touch: dict = {"t": None}  # monotonic time of the last DB write/heartbeat
+
+    def write(pct: float) -> None:
+        whole = int(pct)
+        if whole != state["pct"]:
+            if allow_write():
+                state["pct"] = whole
+                _set_job(job_id, progress=whole, phase=phase)
+                last_touch["t"] = now()
+            return
+        # Same integer percent, but ffmpeg IS still working — a slow re-encode (AV1/VP9 -> H.264 or a
+        # crop on the CPU-only host) can sit on one percent for minutes. Heartbeat updated_at so the
+        # stall watchdog doesn't kill a live job (B-sub-4) — but only if no write/beat already touched
+        # the row this interval, so a progressing download doesn't pay a redundant write.
+        t = now()
+        if last_touch["t"] is None or t - last_touch["t"] >= _DB_WRITE_INTERVAL_S:
+            _touch_job(job_id)
+            last_touch["t"] = t
+
+    return write
+
+
+def _pp_phase(pp: str) -> str:
+    """Map a yt-dlp postprocessor class name to a user-facing phase label. ffmpeg post-steps
+    aren't byte-progress operations (no %), so the UI shows the step name + an indeterminate
+    pulse — this makes the long merge/finalize legible instead of a silent 'Processing'."""
+    if "Merg" in pp:
+        return "merging"
+    if "ExtractAudio" in pp:
+        return "audio_extract"
+    if "Thumbnail" in pp:
+        return "thumbnail"
+    if "SponsorBlock" in pp or "ModifyChapters" in pp:
+        return "sponsorblock"
+    if "Metadata" in pp:
+        return "metadata"
+    return "processing"
+
+
+def _make_pp_hook(job_id: int):
+    """Postprocessor hook: surface the current ffmpeg step so the bar shows a concrete label
+    (Merging / Embedding thumbnail / …) instead of freezing at 100% after the streams finish."""
+
+    def hook(d: dict) -> None:
+        if d.get("status") in ("started", "processing"):
+            _set_job(job_id, phase=_pp_phase(d.get("postprocessor") or ""), speed_bps=None, eta_s=None)
+
+    return hook
+
+
+def _find_thumbnail(staging: Path, media: Path) -> Path | None:
+    for p in staging.iterdir():
+        if p != media and p.suffix.lower() in _THUMB_SUFFIXES:
+            return p
+    return None
+
+
+def _produced_file(info: dict, staging: Path) -> Path:
+    """The media file yt-dlp actually wrote.
+
+    `requested_downloads[0].filepath` is the authoritative answer on YouTube, but an arbitrary-URL
+    extractor may not fill it in at all — and `Path("")` is `Path(".")`, which *passes* `.exists()`,
+    so a missing value used to sail past the fallback and blow up two steps later (`with_name` on an
+    empty name: the BBC-news failure). Anything that isn't a real file counts as absent; then fall
+    back to the largest file left in staging that is neither a thumbnail nor one of yt-dlp's own
+    working files (see `_YTDLP_TEMP_RE`)."""
+    req = (info.get("requested_downloads") or [{}])[0]
+    fp = req.get("filepath") or req.get("_filename") or info.get("filepath") or info.get("_filename")
+    if fp:
+        produced = Path(str(fp))
+        if produced.is_file():
+            return produced
+    cands = [
+        p
+        for p in staging.iterdir()
+        if p.is_file()
+        and p.suffix.lower() not in _THUMB_SUFFIXES
+        and not _YTDLP_TEMP_RE.search(p.name)
+    ]
+    if not cands:
+        raise RuntimeError("yt-dlp produced no output file")
+    return max(cands, key=lambda p: p.stat().st_size)
+
+
+def _media_info(info: dict) -> dict:
+    """Unwrap a playlist/multi_video extraction to the entry that carries the download.
+
+    A single non-YouTube page (a news article's video, a multi-rendition embed) can extract as a
+    wrapper whose own dict has no `requested_downloads` — the media lives on an entry. Pick the
+    FIRST entry that actually downloaded something: on a page with a skipped teaser in front of the
+    real video, entries[0] carries no file, and taking it blindly would label the produced file
+    with the wrong id/title/duration."""
+    entries = [e for e in (info.get("entries") or []) if e]
+    if not entries or info.get("requested_downloads"):
+        return info
+    chosen = next((e for e in entries if e.get("requested_downloads")), entries[0])
+    return {**info, **chosen}
+
+
+def _extract_with_fallback(job_id, asset_id, url, spec, staging, adapter):
+    """Download via yt-dlp using the source `adapter`'s client sets + options. For YouTube the adapter
+    supplies the primary player-client set then a POT-backed fallback (retried on a bot/DRM/format
+    error); other sources make a single attempt. Returns the info dict of the successful attempt."""
+    client_sets = adapter.client_sets()
+    outtmpl = str(staging / "%(id)s.%(ext)s")
+    last_exc: Exception | None = None
+    for i, clients in enumerate(client_sets):
+        opts = formats.build_ydl_opts(
+            spec, outtmpl, _make_progress_hook(job_id, asset_id),
+            extra=adapter.ydl_opts_extra(clients),
+        )
+        opts["postprocessor_hooks"] = [_make_pp_hook(job_id)]
+        _touch_job(job_id)  # stall-watchdog heartbeat: extraction writes no progress, so a slow /
+        # multi-client-set fallback would otherwise cross the stall timeout with a stale updated_at.
+        # (Per-ATTEMPT only; a single extract_info that itself runs past the stall timeout is still
+        # killed — acceptable, as socket_timeout + extractor_retries bound one attempt well under it.)
+        try:
+            with yt_dlp.YoutubeDL(opts) as ydl:
+                return ydl.extract_info(url, download=True)
+        except yt_dlp.utils.DownloadError as exc:
+            last_exc = exc
+            if i < len(client_sets) - 1 and adapter.retriable(str(exc)):
+                log.info("job %s: clients %s failed (%s); retrying with fallback", job_id, clients, str(exc)[:70])
+                for p in list(staging.iterdir()):  # drop partials before the retry
+                    try:
+                        p.unlink()
+                    except OSError:
+                        pass
+                _set_job(job_id, progress=0, phase="downloading")
+                continue
+            raise
+    raise last_exc  # unreachable (loop either returns or raises)
+
+
+def _download_settings() -> tuple[str, int]:
+    """Layout + retention days from the DB config (admin-overridable on the Configuration page),
+    falling back to the env defaults. Read here, not from `settings`, so an admin edit takes effect."""
+    with SessionLocal() as db:
+        return (
+            sysconfig.get_str(db, "download_layout"),
+            sysconfig.get_int(db, "download_retention_days"),
+        )
+
+
+def _ensure_browser_playable(job_id: int, produced: Path, spec: dict, info: dict) -> Path:
+    """Make an mp4-compat-profile download play in every browser: H.264 + AAC/MP3 in mp4. If the
+    download (whatever the source shipped) isn't already that, re-encode the offending stream(s) —
+    or just remux when the codecs are fine but the container isn't. **Scope:** only the mp4 compat
+    profile is normalized; an explicit `vcodec` or a non-mp4 custom profile is the user opting out
+    of the guarantee and is left as-is (so a `webm` profile can still ship VP9/Opus). Returns the
+    file to store (converted, or the original) and corrects `info`'s codec fields to match disk."""
+    if spec.get("container") != "mp4" or spec.get("vcodec"):
+        return produced
+    vcodec, acodec = editmod.probe_codecs(produced)
+    is_mp4 = produced.suffix.lower() == ".mp4"
+    needs, reenc_v, reenc_a = editmod.browser_transcode_flags(vcodec, acodec, is_mp4)
+    if not needs:
+        return produced
+
+    dest = produced.with_name(produced.stem + ".browser.mp4")
+    action = "re-encode" if (reenc_v or reenc_a) else "remux"
+    log.info("job %s: %s (v=%s a=%s mp4=%s) -> browser-safe mp4", job_id, action, vcodec, acodec, is_mp4)
+    _set_job(job_id, phase="processing", progress=0, speed_bps=None, eta_s=None)
+    # Prefer the info duration; probe the file when the extractor didn't provide one, so a long
+    # re-encode still shows real progress instead of a frozen 0%.
+    total_s = int(info["duration"]) if info.get("duration") else editmod.probe_duration(produced)
+    editmod.run_ffmpeg(
+        editmod.build_browser_safe_cmd(produced, dest, reenc_v, reenc_a),
+        total_s=total_s,
+        on_progress=_ffmpeg_progress_writer(job_id, "processing"),
+        should_cancel=_cancel_poll(job_id),
+    )
+    produced.unlink(missing_ok=True)
+    # Keep the stored asset metadata truthful about what's now on disk.
+    if reenc_v:
+        info["vcodec"] = "h264"
+    if reenc_a:
+        info["acodec"] = "aac"
+    return dest
+
+
+def _download(job_id: int, asset_id: int, source_kind: str, source_ref: str, spec: dict) -> None:
+    staging = _staging_dir(asset_id)
+    try:
+        if staging.exists():
+            shutil.rmtree(staging, ignore_errors=True)
+        staging.mkdir(parents=True, exist_ok=True)
+
+        _set_job(job_id, phase="downloading", progress=0)
+        adapter = adapters.resolve_adapter(source_kind, source_ref)
+        url = service.source_url(source_kind, source_ref)
+        info = _media_info(_extract_with_fallback(job_id, asset_id, url, spec, staging, adapter))
+        # Re-resolve now that yt-dlp's info reveals the true source (a bare/redirected url only
+        # identifies post-extraction) so the .nfo uniqueid type + uploader_url agree on one source.
+        adapter = adapters.resolve_adapter(source_kind, source_ref, info)
+
+        produced = _produced_file(info, staging)
+        produced = _ensure_browser_playable(job_id, produced, spec, info)
+        ext = produced.suffix.lstrip(".").lower()
+        meta = storage.MediaMeta(
+            video_id=info.get("id") or source_ref,
+            title=normalize_title(info.get("title")) or source_ref,
+            uploader=info.get("uploader") or info.get("channel") or "Unknown Channel",
+            upload_date=_parse_upload_date(info.get("upload_date")),
+            uniqueid_type=adapter.nfo_uniqueid_type(),
+            duration_s=int(info["duration"]) if info.get("duration") else None,
+            description=info.get("description"),
+            thumbnail_url=info.get("thumbnail"),
+        )
+
+        layout, retention_days = _download_settings()
+        rel = storage.rel_path(meta, ext, layout, asset_id=asset_id)
+        thumb = _find_thumbnail(staging, produced)
+        # Convert the raw thumbnail (webp/png/jpeg) to a real JPEG ourselves — robustly — for the
+        # poster + og:image sidecar, now that yt-dlp's fatal thumbnail convertor is gone. On failure
+        # this is None and write_sidecars writes no poster, so ensure_poster cuts a frame instead.
+        poster_jpg = editmod.thumbnail_to_jpg(thumb, staging / "poster.jpg")
+        storage.place_file(produced, settings.download_root, rel)
+        nfo_ok = storage.write_sidecars(settings.download_root, rel, meta, poster_jpg)
+
+        final = storage.abs_path(settings.download_root, rel)
+        size = final.stat().st_size if final.exists() else None
+        expires = datetime.now(timezone.utc) + timedelta(days=retention_days)
+
+        with SessionLocal() as db:
+            asset = db.get(MediaAsset, asset_id)
+            if asset:
+                asset.status = "ready"
+                asset.rel_path = rel
+                asset.size_bytes = size
+                asset.duration_s = meta.duration_s
+                asset.width = int(info["width"]) if info.get("width") else None
+                asset.height = int(info["height"]) if info.get("height") else None
+                asset.container = ext
+                asset.vcodec = info.get("vcodec") if info.get("vcodec") not in (None, "none") else None
+                asset.acodec = info.get("acodec") if info.get("acodec") not in (None, "none") else None
+                asset.title = meta.title
+                asset.uploader = meta.uploader
+                asset.uploader_url = adapter.uploader_url(info)
+                asset.upload_date = meta.upload_date
+                asset.thumbnail_url = meta.thumbnail_url
+                # Record a self-hosted poster for every download: it's the reliable link-preview
+                # (og:image) source — a remote thumbnail, esp. Facebook's signed CDN URL, expires
+                # and can't be relied on cross-origin. write_sidecars already wrote <base>.jpg from
+                # the thumbnail; ensure_poster returns that, or cuts a frame if there was none.
+                asset.poster_path = editmod.ensure_poster(asset, settings.download_root)
+                asset.source_webpage_url = info.get("webpage_url")
+                asset.nfo_written = nfo_ok
+                asset.error = None
+                asset.last_access_at = datetime.now(timezone.utc)
+                asset.expires_at = expires
+            job = db.get(DownloadJob, job_id)
+            # Capture the owner before commit expires the row (avoids a refresh SELECT just to notify).
+            owner_id = job.user_id if job else None
+            fields = {"status": JobStatus.DONE, "progress": 100, "phase": None, "speed_bps": None,
+                      "eta_s": None}
+            if job and not job.display_name and meta.title:
+                # display_name is VARCHAR(255) — clip a long title (see _fill_asset_meta_early).
+                fields["display_name"] = meta.title[:255]
+            # Atomic terminal write: finalizes ONLY while the row is still running, so a child that
+            # finishes just after a route cancel/pause can't resurrect it (see finalize_if_running).
+            finalized = service.finalize_if_running(db, job_id, **fields)
+            db.commit()
+            if finalized:
+                _notify_download_done(db, owner_id, meta.title, job_id, kind="download")
+                log.info("job %s done: %s", job_id, rel)
+
+    except (_Aborted, editmod.EditAborted):
+        # Paused/canceled mid-download or mid-transcode: free the asset for a later retry; keep the
+        # job's status (the route set it to paused/canceled). ref_count is adjusted by the route.
+        _reset_asset_pending(asset_id)
+        log.info("job %s aborted (pause/cancel)", job_id)
+    except Exception as exc:  # noqa: BLE001 — surface any failure to the user, keep worker alive
+        msg = str(exc)[:500]
+        with SessionLocal() as db:
+            asset = db.get(MediaAsset, asset_id)
+            if asset:
+                asset.status = "error"
+                asset.error = msg
+            # Atomic fail: only while still running, so a route-set cancel/pause is never overwritten
+            # with `error` (error also holds the asset → would re-open the double-release class).
+            service.finalize_if_running(
+                db, job_id, status=JobStatus.ERROR, error=msg,
+                error_code=dlerrors.classify(msg), phase=None,
+            )
+            db.commit()
+        log.warning("job %s failed: %s", job_id, msg)
+    finally:
+        shutil.rmtree(staging, ignore_errors=True)
+
+
+def _fail_edit(job_id: int, asset_id: int, msg: str) -> None:
+    with SessionLocal() as db:
+        asset = db.get(MediaAsset, asset_id)
+        if asset:
+            asset.status = "error"
+            asset.error = msg
+        # Atomic fail: only while still running (see the download error path).
+        service.finalize_if_running(
+            db, job_id, status=JobStatus.ERROR, error=msg,
+            error_code=dlerrors.classify(msg), phase=None,
+        )
+        db.commit()
+    log.warning("edit job %s failed: %s", job_id, msg)
+
+
+def _process_edit(job_id: int, asset_id: int) -> None:
+    """Produce a per-user trim/crop clip from the source asset's downloaded file via ffmpeg."""
+    with SessionLocal() as db:
+        job = db.get(DownloadJob, job_id)
+        src = db.get(MediaAsset, job.source_asset_id) if job and job.source_asset_id else None
+        edit_spec = dict(job.edit_spec or {}) if job else {}
+        user_id = job.user_id if job else None
+        src_ready = src is not None and src.status == "ready" and bool(src.rel_path)
+        src_rel = src.rel_path if src else None
+        src_w, src_h, src_dur = (src.width, src.height, src.duration_s) if src else (None, None, None)
+
+    if not src_ready or not src_rel:
+        _fail_edit(job_id, asset_id, "The source download is no longer available.")
+        return
+    src_path = storage.abs_path(settings.download_root, src_rel)
+    if not src_path.exists():
+        _fail_edit(job_id, asset_id, "The source file is missing.")
+        return
+
+    src_ext = src_path.suffix.lstrip(".").lower() or "mp4"
+    crop = editmod.has_crop(edit_spec)
+    out_ext = "mp4" if editmod.needs_reencode(edit_spec) else src_ext
+    total_s = editmod.clip_duration(edit_spec, src_dur) or src_dur or 0
+    staging = _edit_staging_dir(asset_id)
+
+    try:
+        if staging.exists():
+            shutil.rmtree(staging, ignore_errors=True)
+        staging.mkdir(parents=True, exist_ok=True)
+        dest_staging = staging / f"out.{out_ext}"
+
+        _set_job(job_id, phase="editing", progress=0, speed_bps=None, eta_s=None)
+        if edit_spec.get("segments"):
+            cmd, prep = editmod.build_concat_plan(src_path, dest_staging, edit_spec, out_ext, staging)
+            for p, content in prep.items():
+                p.write_text(content, encoding="utf-8")
+        else:
+            cmd = editmod.build_edit_cmd(src_path, dest_staging, edit_spec, out_ext)
+        editmod.run_ffmpeg(
+            cmd,
+            total_s,
+            on_progress=_ffmpeg_progress_writer(job_id, "editing"),
+            should_cancel=_cancel_poll(job_id),
+        )
+        if not dest_staging.exists():
+            raise RuntimeError("ffmpeg produced no output file")
+
+        rel = storage.edit_rel_path(user_id, asset_id, out_ext)
+        storage.place_file(dest_staging, settings.download_root, rel)
+        final = storage.abs_path(settings.download_root, rel)
+        size = final.stat().st_size if final.exists() else None
+        _, retention_days = _download_settings()
+        expires = datetime.now(timezone.utc) + timedelta(days=retention_days)
+
+        with SessionLocal() as db:
+            asset = db.get(MediaAsset, asset_id)
+            if asset:
+                asset.status = "ready"
+                asset.rel_path = rel
+                asset.size_bytes = size
+                asset.container = out_ext
+                asset.duration_s = total_s or asset.duration_s
+                if crop:
+                    asset.width = edit_spec["crop"]["w"]
+                    asset.height = edit_spec["crop"]["h"]
+                else:
+                    asset.width, asset.height = src_w, src_h
+                asset.error = None
+                asset.last_access_at = datetime.now(timezone.utc)
+                asset.expires_at = expires
+            job = db.get(DownloadJob, job_id)
+            # Capture owner + title before commit expires the rows (avoids a refresh SELECT).
+            owner_id = job.user_id if job else None
+            dl_title = asset.title if asset else None
+            # Atomic terminal write (see the download path): don't resurrect a route-stopped job.
+            finalized = service.finalize_if_running(
+                db, job_id, status=JobStatus.DONE, progress=100, phase=None
+            )
+            db.commit()
+            if finalized:
+                _notify_download_done(db, owner_id, dl_title, job_id, kind="edit")
+                log.info("edit job %s done: %s", job_id, rel)
+
+    except editmod.EditAborted:
+        # Paused/canceled mid-encode: free the asset for a later retry (ref_count is the route's job).
+        _reset_asset_pending(asset_id)
+        log.info("edit job %s aborted (pause/cancel)", job_id)
+    except Exception as exc:  # noqa: BLE001 — surface the failure, keep the worker alive
+        _fail_edit(job_id, asset_id, str(exc)[:500])
+    finally:
+        shutil.rmtree(staging, ignore_errors=True)

backend/app/downloads/runner.py

@@ -38,8 +38,8 @@ def main() -> None:
         return
 
     # Imported here (not at module top) so the child pays the yt-dlp import cost only when it has
-    # real work, and to keep the import graph obvious: runner is a thin entry over worker's helpers.
-    from app.worker import _download, _process_edit
+    # real work, and to keep the import graph obvious: runner is a thin entry over the pipeline.
+    from app.downloads.pipeline import _download, _process_edit
 
     if job_kind == "edit":
         _process_edit(job_id, asset_id)

backend/app/downloads/service.py

@@ -112,6 +112,34 @@ def next_queue_pos(db: Session) -> int:
     return (db.execute(select(func.coalesce(func.max(DownloadJob.queue_pos), 0))).scalar() or 0) + 1
 
 
+def incr_ref_count(db: Session, asset_id: int) -> None:
+    """Atomically +1 an asset's hold count AT THE DB. A Python read-modify-write
+    (`asset.ref_count = asset.ref_count + 1`) loses an update when two enqueues of the same
+    (source,format) run in parallel — both read the same value and write the same +1 — leaving the
+    count one short, so the first release can free a file another job still holds (B-sub-5)."""
+    db.execute(
+        update(MediaAsset)
+        .where(MediaAsset.id == asset_id)
+        .values(ref_count=MediaAsset.ref_count + 1)
+        .execution_options(synchronize_session=False)
+    )
+
+
+def decr_ref_count(db: Session, asset_id: int) -> int:
+    """Atomically -1 an asset's hold count (clamped at 0) and RETURN the new value. The caller decides
+    whether to free the asset from the returned count while still holding the row's write lock, so two
+    concurrent releases can't both observe the same pre-decrement value and double-free — Postgres
+    serialises the two UPDATEs on the row lock (B-sub-5). synchronize_session=False: the ORM copy is
+    intentionally left stale; callers read the returned value, not `asset.ref_count`."""
+    return db.execute(
+        update(MediaAsset)
+        .where(MediaAsset.id == asset_id)
+        .values(ref_count=func.greatest(MediaAsset.ref_count - 1, 0))
+        .returning(MediaAsset.ref_count)
+        .execution_options(synchronize_session=False)
+    ).scalar_one()
+
+
 def enqueue(
     db: Session,
     user_id: int,
@@ -143,7 +171,7 @@ def enqueue(
         queue_pos=next_queue_pos(db),
     )
     db.add(job)
-    asset.ref_count = (asset.ref_count or 0) + 1
+    incr_ref_count(db, asset.id)
 
     # Cache hit: the file already exists — no worker round-trip needed.
     if asset.status == "ready":
@@ -212,7 +240,7 @@ def enqueue_edit(
         queue_pos=next_queue_pos(db),
     )
     db.add(job)
-    asset.ref_count = (asset.ref_count or 0) + 1
+    incr_ref_count(db, asset.id)
 
     if asset.status == "ready":  # identical clip already produced for this user → instant
         job.status = "done"
@@ -273,8 +301,83 @@ def release_asset(db: Session, job: DownloadJob) -> None:
     asset = db.get(MediaAsset, job.asset_id)
     if asset is None:
         return
-    asset.ref_count = max(0, (asset.ref_count or 0) - 1)
-    if asset.ref_count == 0 and asset.status == "ready":
+    new_count = decr_ref_count(db, asset.id)
+    if new_count == 0 and asset.status == "ready":
         if asset.rel_path:
             storage.delete_asset_files(settings.download_root, asset.rel_path)
         db.delete(asset)
+
+
+# --- Job state transitions ---------------------------------------------------------------------
+# The single home for the pause/resume/cancel/delete/redownload transitions + their asset
+# bookkeeping (release, failed-asset reset, fresh-asset rebuild), so the routes are thin callers and
+# the worker/GC could share the same definitions (B-sub-8). Each commits its own transition; the
+# route keeps only auth + HTTP mapping + serialization.
+
+
+def pause(db: Session, job: DownloadJob) -> None:
+    """Pause a queued/running job (a running worker aborts cooperatively via the cancel poll); a no-op
+    in any other state."""
+    if job.status in (JobStatus.QUEUED, JobStatus.RUNNING):
+        job.status = JobStatus.PAUSED
+        db.commit()
+
+
+def resume(db: Session, job: DownloadJob) -> None:
+    """Requeue a paused/errored job for a fresh attempt, clearing its error. If the SHARED asset itself
+    failed, reset it to pending so the worker actually re-downloads — otherwise the requeued job would
+    instantly re-inherit the asset's stale error (the worker short-circuits an errored asset)."""
+    if job.status in (JobStatus.PAUSED, JobStatus.ERROR):
+        job.progress = 0
+        job.error = None
+        job.error_code = None
+        if job.asset_id:
+            asset = db.get(MediaAsset, job.asset_id)
+            if asset and asset.status == "error":  # MediaAsset.status (no enum), not a JobStatus
+                asset.status = "pending"
+                asset.error = None
+        job.status = JobStatus.QUEUED
+        db.commit()
+
+
+def cancel(db: Session, job: DownloadJob) -> None:
+    """Cancel a job unless it's already done/canceled, releasing its asset hold WHILE it still counts
+    as holding — `canceled` ∉ HOLDING, so a later delete won't double-release."""
+    if job.status not in (JobStatus.DONE, JobStatus.CANCELED):
+        release_asset(db, job)  # release while the job still counts as holding
+        job.status = JobStatus.CANCELED
+        db.commit()
+
+
+def delete(db: Session, job: DownloadJob) -> None:
+    """Delete a job, releasing its asset hold first (which frees the file iff this was the last holder)."""
+    release_asset(db, job)
+    db.delete(job)
+    db.commit()
+
+
+def redownload(db: Session, job: DownloadJob) -> None:
+    """Rebuild a finished/failed job's file into a FRESH, private asset (the cache sig is salted with
+    the job id + current asset id, so it never reuses the dedup pool) and requeue the SAME job row —
+    so any public share link (DownloadLink -> job_id) keeps resolving to the newly-downloaded file.
+    The old hold is dropped first (freeing the old file iff this job was its last holder). The caller
+    has already rejected edit clips + still-in-progress jobs (HTTP 409)."""
+    spec = formats.normalize(job.profile_snapshot)
+    sig = formats.format_sig(spec, salt=f"{job.id}:{job.asset_id}")
+    asset = get_or_create_asset(db, job.source_kind, job.source_ref, sig)
+    populate_from_catalog(db, asset)
+    # Drop the old hold BEFORE re-pointing the job — release_asset keys off the job's current asset.
+    release_asset(db, job)
+    incr_ref_count(db, asset.id)
+    job.asset_id = asset.id
+    job.status = JobStatus.QUEUED
+    job.progress = 0
+    job.phase = None
+    job.error = None
+    job.speed_bps = None
+    job.eta_s = None
+    # Re-derive the display name from the freshly cleaned title on completion (else an older, un-cleaned
+    # name would survive the rebuild).
+    job.display_name = None
+    job.queue_pos = next_queue_pos(db)  # tail of the queue, not its stale original slot
+    db.commit()

backend/app/models.py

@@ -547,9 +547,6 @@ class AppState(Base):
     sync_paused: Mapped[bool] = mapped_column(
         Boolean, default=False, server_default="false"
     )
-    # Admin override for the maintenance job's rolling re-validation batch (videos re-checked
-    # per run). NULL = use the config/env default (settings.maintenance_revalidate_batch).
-    maintenance_revalidate_batch: Mapped[int | None] = mapped_column(Integer)
     # First-run install wizard. `configured` is False on a fresh instance (the app runs in
     # setup mode: only the wizard works) and flipped True when the wizard finishes. While
     # unconfigured, a one-time setup token (only its SHA-256 hash stored) gates the wizard;

backend/app/plex/query.py

@@ -6,7 +6,7 @@ Pure query fragments — every function returns SQLAlchemy expressions or a para
 from dataclasses import dataclass
 from datetime import datetime, timedelta, timezone
 
-from sqlalchemy import and_, func, or_, type_coerce
+from sqlalchemy import and_, case, func, or_, type_coerce
 from sqlalchemy.dialects.postgresql import JSONB
 
 TS_CONFIG = "public.unaccent_simple"
@@ -189,3 +189,62 @@ class LibraryFilters:
 
 def like_escape(s: str) -> str:
     return s.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
+
+
+# --- Watch-state scope (shared by the /library grid AND the /facets aggregation) ---------------
+#
+# The `show` scope (all | watched | in_progress | unwatched) filters the browse result by per-user
+# watch state. The SAME predicate has to be applied in two places — `unified_library` (the grid) and
+# `facets.wsq` (so the facet counts reflect exactly the visible set) — and they MUST stay identical:
+# a hidden movie kept out of the grid must not inflate the facets either. These helpers are the one
+# home for that predicate so the two call sites can't drift (E3 S1; pinned by test_plex_facets_grid_
+# parity). Callers pass the already-joined state expressions, so this module stays model-agnostic:
+#   * movies — `st` is an aliased PlexState outer-joined on (item_id, user_id).
+#   * shows  — `tot`/`wat`/`inp` are the coalesced totals from `rollup.agg_subquery`.
+
+
+def movie_status_case(st):
+    """The per-movie status label (new | in_progress | watched) from its PlexState alias `st`."""
+    return case(
+        (st.status == "watched", "watched"),
+        (
+            and_(st.position_seconds > 0, or_(st.status.is_(None), st.status != "watched")),
+            "in_progress",
+        ),
+        else_="new",
+    )
+
+
+def show_status_case(tot, wat, inp):
+    """The per-show status label rolled up from its episodes' totals (total/watched/in-progress)."""
+    return case(
+        (and_(tot > 0, wat >= tot), "watched"),
+        (or_(wat > 0, inp > 0), "in_progress"),
+        else_="new",
+    )
+
+
+def apply_movie_state_filter(query, st, show: str):
+    """Restrict a movie query to the `show` scope via its PlexState alias `st`. The default (`all`)
+    excludes hidden movies — the exclusion that must match between grid and facets."""
+    if show == "watched":
+        return query.where(st.status == "watched")
+    if show == "in_progress":
+        return query.where(
+            st.position_seconds > 0, or_(st.status.is_(None), st.status != "watched")
+        )
+    if show == "unwatched":
+        return query.where(or_(st.status.is_(None), st.status.notin_(["watched", "hidden"])))
+    return query.where(or_(st.status.is_(None), st.status != "hidden"))
+
+
+def apply_show_state_filter(query, tot, wat, inp, show: str):
+    """Restrict a show query to the `show` scope via its rollup totals. `all` is a no-op (shows have
+    no hidden concept); the caller only joins the rollup for the three non-`all` scopes."""
+    if show == "watched":
+        return query.where(tot > 0, wat >= tot)
+    if show == "in_progress":
+        return query.where(or_(wat > 0, inp > 0), wat < tot)
+    if show == "unwatched":
+        return query.where(wat == 0, inp == 0)
+    return query

backend/app/plex/sync.py

@@ -5,9 +5,14 @@ Playback is from the local physical file, so this only mirrors METADATA + the ph
 (no ffprobe needed) to classify each item's browser playability. Movies + episodes become playable
 LEAF rows (plex_items); shows/seasons are mirrored for the drill-down + episode ordering.
 
-The sync is a full reconcile of the enabled sections (upsert by rating_key). Pruning of items that
-disappeared from Plex is intentionally left for later (a follow-up can diff rating_keys per library).
-Cast + intro/credit markers are fetched lazily on the item-detail/player path (P2), not here.
+The sync is a full reconcile of the enabled sections (upsert by rating_key) AND prunes rows for
+titles that disappeared from Plex: each section diffs the rating_keys it just listed against the ones
+stored, and deletes the stale ones (their watch-state / playlist links cascade). The prune is guarded
+on a NON-empty listing — a section that returns zero items is left untouched, so a transient Plex
+glitch that yields an empty page can't wipe a library (the metadata is cheap to re-sync; an accidental
+mass-delete is the only real risk). Library-level reconcile (dropping a whole removed section, the
+dead `enabled` column) is a separate follow-up. Cast + intro/credit markers are fetched lazily on the
+item-detail/player path (P2), not here.
 """
 import logging
 from datetime import datetime, timezone
@@ -150,7 +155,7 @@ def sync(db: Session) -> dict:
         return {"skipped": "disabled"}
     stats = {
         "libraries": 0, "movies": 0, "shows": 0, "seasons": 0, "episodes": 0,
-        "collections": 0, "overlaid": 0,
+        "collections": 0, "overlaid": 0, "pruned": 0,
     }
     wanted = _enabled_section_keys(db)
     overlay_map = _overlay_map(db)  # {huKey: enKey}
@@ -279,8 +284,14 @@ def _upsert_library(db: Session, key: str, title: str, kind: str) -> PlexLibrary
     return lib
 
 
-def _paginate(plex: PlexClient, key: str, item_type: int):
+def _paginate(plex: PlexClient, key: str, item_type: int, progress: dict | None = None):
+    """Yield a section's items page by page. If `progress` is given, set `progress["complete"]` True
+    ONLY when the walk reached the server's reported `total` — a truncated fetch (an empty/short page
+    mid-sequence, e.g. a transient Plex hiccup, before `total` is reached) leaves it False so the
+    caller can skip pruning. Pruning on a partial listing would delete rows the listing merely failed
+    to mention (the metadata is re-syncable, but that still cascades watch-state)."""
     start = 0
+    total = None
     while True:
         items, total = plex.section_items(key, item_type, start, _PAGE)
         if not items:
@@ -292,6 +303,10 @@ def _paginate(plex: PlexClient, key: str, item_type: int):
             break
         if len(items) < _PAGE:
             break
+    if progress is not None:
+        # A truthy `total` we reached = a trustworthy complete listing. `total` unknown/zero-with-rows
+        # (a Plex that doesn't report totalSize) is treated as NOT complete, so prune stays conservative.
+        progress["complete"] = bool(total) and start >= total
 
 
 # The section listing exposes at most the top ~20 cast (Plex's own cap); a stored cast longer than this
@@ -351,6 +366,24 @@ def _apply_item(row: PlexItem, lib_id: int, kind: str, meta: dict) -> None:
     row.playable = classify_playable(mf.get("codec_video"), mf.get("codec_audio"), mf.get("container"))
 
 
+def _prune_stale(
+    db: Session, existing: dict, seen: set[str], stats: dict, *, complete: bool
+) -> None:
+    """Delete stored rows whose rating_key is no longer in the freshly-listed `seen` set — the titles
+    that disappeared from Plex (their watch-state / playlist links cascade via the FKs). GUARDED on a
+    COMPLETE, non-empty listing: a truncated fetch (`complete` False — a mid-sequence empty/short page
+    before the server's total) or a zero-item listing prunes NOTHING, so a transient Plex hiccup can
+    never wipe a library over rows it merely failed to list. Callers pass the rating_key -> row map
+    they already built for the upsert."""
+    if not complete or not seen:
+        return
+    for rk, row in existing.items():
+        if rk not in seen:
+            db.delete(row)
+            stats["pruned"] = stats.get("pruned", 0) + 1
+    db.flush()
+
+
 def _sync_movies(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) -> None:
     existing = {
         r.rating_key: r
@@ -358,7 +391,9 @@ def _sync_movies(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) -
             select(PlexItem).where(PlexItem.library_id == lib.id, PlexItem.kind == "movie")
         ).scalars()
     }
-    for meta in _paginate(plex, lib.plex_key, _TYPE_MOVIE):
+    seen: set[str] = set()
+    progress: dict = {}
+    for meta in _paginate(plex, lib.plex_key, _TYPE_MOVIE, progress):
         rk = str(meta.get("ratingKey") or "")
         if not rk:
             continue
@@ -366,8 +401,10 @@ def _sync_movies(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) -
         if row.id is None:
             db.add(row)
         _apply_item(row, lib.id, "movie", meta)
+        seen.add(rk)
         stats["movies"] += 1
     db.flush()
+    _prune_stale(db, existing, seen, stats, complete=progress.get("complete", False))
 
 
 def _sync_shows(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) -> None:
@@ -378,10 +415,13 @@ def _sync_shows(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) ->
             select(PlexShow).where(PlexShow.library_id == lib.id)
         ).scalars()
     }
-    for meta in _paginate(plex, lib.plex_key, _TYPE_SHOW):
+    shows_seen: set[str] = set()
+    shows_progress: dict = {}
+    for meta in _paginate(plex, lib.plex_key, _TYPE_SHOW, shows_progress):
         rk = str(meta.get("ratingKey") or "")
         if not rk:
             continue
+        shows_seen.add(rk)
         sh = shows.get(rk) or PlexShow(rating_key=rk)
         if sh.id is None:
             db.add(sh)
@@ -407,11 +447,14 @@ def _sync_shows(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) ->
             select(PlexSeason).join(PlexShow).where(PlexShow.library_id == lib.id)
         ).scalars()
     }
-    for meta in _paginate(plex, lib.plex_key, _TYPE_SEASON):
+    seasons_seen: set[str] = set()
+    seasons_progress: dict = {}
+    for meta in _paginate(plex, lib.plex_key, _TYPE_SEASON, seasons_progress):
         rk = str(meta.get("ratingKey") or "")
         sid = show_id.get(str(meta.get("parentRatingKey") or ""))
         if not rk or sid is None:
             continue
+        seasons_seen.add(rk)
         se = seasons.get(rk) or PlexSeason(rating_key=rk)
         if se.id is None:
             db.add(se)
@@ -431,10 +474,13 @@ def _sync_shows(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) ->
             select(PlexItem).where(PlexItem.library_id == lib.id, PlexItem.kind == "episode")
         ).scalars()
     }
-    for meta in _paginate(plex, lib.plex_key, _TYPE_EPISODE):
+    episodes_seen: set[str] = set()
+    episodes_progress: dict = {}
+    for meta in _paginate(plex, lib.plex_key, _TYPE_EPISODE, episodes_progress):
         rk = str(meta.get("ratingKey") or "")
         if not rk:
             continue
+        episodes_seen.add(rk)
         row = episodes.get(rk) or PlexItem(rating_key=rk)
         if row.id is None:
             db.add(row)
@@ -446,6 +492,13 @@ def _sync_shows(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) ->
         stats["episodes"] += 1
     db.flush()
 
+    # Prune leaf-first (episodes -> seasons -> shows) so a stale row is deleted directly, before its
+    # parent's cascade would reach it — avoids deleting the same ORM object twice. Each level is
+    # independently guarded on a non-empty listing by _prune_stale.
+    _prune_stale(db, episodes, episodes_seen, stats, complete=episodes_progress.get("complete", False))
+    _prune_stale(db, seasons, seasons_seen, stats, complete=seasons_progress.get("complete", False))
+    _prune_stale(db, shows, shows_seen, stats, complete=shows_progress.get("complete", False))
+
 
 def _sync_overlay(
     db: Session, plex: PlexClient, lib: PlexLibrary, en_lib_id: int, stats: dict

backend/app/routes/auth.py

@@ -10,6 +10,7 @@ from datetime import datetime, timedelta, timezone
 from authlib.integrations.starlette_client import OAuth, OAuthError
 from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
 from fastapi.responses import JSONResponse, RedirectResponse
+from pydantic import BaseModel
 from sqlalchemy import select, update
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import Session
@@ -187,6 +188,36 @@ def _oauth_redirect_uri(request: Request) -> str:
     return settings.oauth_redirect_url
 
 
+
+# --- Request bodies (typed contracts; B-core-6) ---------------------------------------------
+# Fields are `str | None` (not required) so the handlers keep their exact prior behaviour — a
+# missing/null field reads as None and the existing `(... or "")` coercions still apply — while
+# FastAPI now validates the body is a JSON object and publishes the schema.
+
+
+class EmailIn(BaseModel):
+    email: str | None = None
+
+
+class EmailPasswordIn(BaseModel):
+    email: str | None = None
+    password: str | None = None
+
+
+class TokenIn(BaseModel):
+    token: str | None = None
+
+
+class TokenPasswordIn(BaseModel):
+    token: str | None = None
+    password: str | None = None
+
+
+class SetPasswordIn(BaseModel):
+    password: str | None = None
+    current_password: str | None = None
+
+
 @router.get("/login")
 async def login(request: Request, db: Session = Depends(get_db)):
     # access_type=offline ensures a refresh_token on first authorization. We avoid
@@ -448,7 +479,7 @@ async def logout(request: Request):
 
 @router.post("/request-access")
 async def request_access(
-    payload: dict,
+    body: EmailIn,
     request: Request,
     background: BackgroundTasks,
     db: Session = Depends(get_db),
@@ -462,7 +493,7 @@ async def request_access(
     is careful to avoid, cf. demo_login); an already-allowed user can still just sign in with Google.
     Rate-limited per client IP like register/login/reset, and a block returns "pending" too so the
     endpoint can't be probed for throttling either."""
-    email = (payload.get("email") or "").strip().lower()
+    email = (body.email or "").strip().lower()
     if not valid_email(email):
         raise HTTPException(status_code=400, detail="Enter a valid email address.")
     if not _request_access_limiter.allow(_client_ip(request)):
@@ -477,7 +508,7 @@ async def request_access(
 
 
 @router.post("/demo")
-def demo_login(payload: dict, request: Request, db: Session = Depends(get_db)) -> dict:
+def demo_login(body: EmailIn, request: Request, db: Session = Depends(get_db)) -> dict:
     """Hidden demo entry: a whitelisted email logs straight into the shared demo account,
     no Google OAuth. The login page fires this quietly (debounced) as the user types/pastes,
     so there is no visible 'demo login' button. Rate-limited per IP; when blocked it answers
@@ -485,7 +516,7 @@ def demo_login(payload: dict, request: Request, db: Session = Depends(get_db)) -
     enumeration oracle. On a match it sets the session and the client reloads into the app."""
     if not _demo_limiter.allow(_client_ip(request)):
         return {"authenticated": False}
-    email = (payload.get("email") or "").strip().lower()
+    email = (body.email or "").strip().lower()
     if valid_email(email) and is_demo_allowed(db, email):
         demo = get_or_create_demo_user(db)
         # Isolate the demo: drop any real-account wallet on this tab first, so the demo session
@@ -556,7 +587,7 @@ def _consume_token(db: Session, raw: str | None, kind: str) -> User | None:
 
 @router.post("/register")
 def register(
-    payload: dict,
+    body: EmailPasswordIn,
     request: Request,
     background: BackgroundTasks,
     db: Session = Depends(get_db),
@@ -565,8 +596,8 @@ def register(
     verification link, and records an access request for admin approval. Two gates before
     sign-in: verified email AND admin approval. Responses are uniform once input is valid, so
     the endpoint can't be used to discover which emails are registered."""
-    email = (payload.get("email") or "").strip().lower()
-    password = payload.get("password") or ""
+    email = (body.email or "").strip().lower()
+    password = body.password or ""
     # Input validation is safe to reveal (independent of whether the email exists).
     if not valid_email(email):
         raise HTTPException(status_code=400, detail="Enter a valid email address.")
@@ -628,10 +659,10 @@ def _register_account(email: str, password: str) -> None:
 
 
 @router.post("/verify")
-def verify_email_confirm(payload: dict, db: Session = Depends(get_db)) -> dict:
+def verify_email_confirm(body: TokenIn, db: Session = Depends(get_db)) -> dict:
     """Confirm an email-verification token the SPA read from the URL fragment (SB3). Uniform 400
     on any invalid/expired/used token."""
-    user = _consume_token(db, payload.get("token"), "verify")
+    user = _consume_token(db, body.token, "verify")
     if user is None:
         raise HTTPException(
             status_code=400, detail="This verification link is invalid or has expired."
@@ -643,7 +674,7 @@ def verify_email_confirm(payload: dict, db: Session = Depends(get_db)) -> dict:
 
 @router.post("/verify/resend")
 def verify_email_resend(
-    payload: dict,
+    body: EmailIn,
     request: Request,
     background: BackgroundTasks,
 ) -> dict:
@@ -653,7 +684,7 @@ def verify_email_resend(
     account (the lookup runs off the response path), so it can't probe for registered emails."""
     if not _verify_resend_limiter.allow(_client_ip(request)):
         return {"status": "ok"}  # silently throttle; uniform response
-    email = (payload.get("email") or "").strip().lower()
+    email = (body.email or "").strip().lower()
     if valid_email(email):
         background.add_task(_resend_verify_if_eligible, email)
     return {"status": "ok"}
@@ -688,7 +719,7 @@ def verify_email(token: str, db: Session = Depends(get_db)):
 
 @router.post("/password-login")
 def password_login(
-    payload: dict,
+    body: EmailPasswordIn,
     request: Request,
     background: BackgroundTasks,
     db: Session = Depends(get_db),
@@ -700,8 +731,8 @@ def password_login(
         raise HTTPException(
             status_code=429, detail="Too many attempts. Try again shortly."
         )
-    email = (payload.get("email") or "").strip().lower()
-    password = payload.get("password") or ""
+    email = (body.email or "").strip().lower()
+    password = body.password or ""
     user = db.execute(select(User).where(User.email == email)).scalar_one_or_none()
     # Always run the KDF (against a decoy hash for an unknown / passwordless / demo account) so the
     # response time is the same whether or not the email exists — no timing-based enumeration.
@@ -731,7 +762,7 @@ def password_login(
 
 @router.post("/password-reset/request")
 def password_reset_request(
-    payload: dict,
+    body: EmailIn,
     request: Request,
     background: BackgroundTasks,
 ) -> dict:
@@ -739,7 +770,7 @@ def password_reset_request(
     password account (the lookup runs off the response path), so it can't probe for registered emails."""
     if not _reset_limiter.allow(_client_ip(request)):
         return {"status": "ok"}
-    email = (payload.get("email") or "").strip().lower()
+    email = (body.email or "").strip().lower()
     # Do the account lookup + token issue + email OFF the response path (a background task with its
     # own session), so the endpoint takes the same time for any valid email whether or not it has a
     # password account — no timing-based enumeration. Any valid email schedules the same task.
@@ -761,10 +792,10 @@ def _send_reset_if_eligible(email: str) -> None:
 
 
 @router.post("/password-reset/confirm")
-def password_reset_confirm(payload: dict, db: Session = Depends(get_db)) -> dict:
+def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db)) -> dict:
     """Set a new password from a valid reset token, then invalidate any other reset tokens."""
-    token = payload.get("token")
-    new_password = payload.get("password") or ""
+    token = body.token
+    new_password = body.password or ""
     if len(new_password) < PASSWORD_MIN_LEN:
         raise HTTPException(
             status_code=400,
@@ -872,7 +903,7 @@ async def link_google(
 
 @router.post("/set-password")
 def set_password(
-    payload: dict,
+    body: SetPasswordIn,
     request: Request,
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
@@ -884,7 +915,7 @@ def set_password(
         raise HTTPException(
             status_code=403, detail="Not available in the demo account."
         )
-    new_password = payload.get("password") or ""
+    new_password = body.password or ""
     if len(new_password) < PASSWORD_MIN_LEN:
         raise HTTPException(
             status_code=400,
@@ -892,7 +923,7 @@ def set_password(
         )
     if user.password_hash:
         if not verify_password(
-            payload.get("current_password") or "", user.password_hash
+            body.current_password or "", user.password_hash
         ):
             raise HTTPException(
                 status_code=403, detail="Current password is incorrect."
@@ -941,17 +972,6 @@ async def upgrade(
     )
 
 
-@router.get("/me")
-async def me(user: User = Depends(current_user)) -> dict:
-    return {
-        "id": user.id,
-        "email": user.email,
-        "display_name": user.display_name,
-        "avatar_url": user.avatar_url,
-        "role": user.role,
-    }
-
-
 @router.get("/config")
 def auth_config(db: Session = Depends(get_db)) -> dict:
     """Public: which sign-in options this instance offers, so the login page can hide what's off

backend/app/routes/channels.py

@@ -1,6 +1,6 @@
 """Channel manager: the user's subscriptions with their personal overrides (priority,
-hidden, tags) and per-channel sync state. Read + local-write only — YouTube-side writes
-(unsubscribe) live behind the optional write scope and are added in a later phase."""
+hidden, tags) and per-channel sync state. Local reads/writes plus YouTube-side writes
+(subscribe / unsubscribe) that require the optional write scope."""
 import logging
 from datetime import datetime, timezone
 

backend/app/routes/downloads.py

@@ -21,8 +21,7 @@ from app.config import settings
 from app.db import get_db
 from app.downloads import edit as editmod
 from app.downloads import links as linksmod
-from app.downloads import formats, quota, service, storage
-from app.downloads.status import JobStatus
+from app.downloads import quota, service, storage
 from app.models import (
     DownloadJob,
     DownloadLink,
@@ -454,18 +453,12 @@ def update_download_meta(
     return _serialize_job(db, job)
 
 
-def _set_status(db: Session, job: DownloadJob, status: str) -> None:
-    job.status = status
-    db.commit()
-
-
 @router.post("/{job_id}/pause")
 def pause_download(
     job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
 ) -> dict:
     job = _own_job(db, user, job_id)
-    if job.status in ("queued", "running"):
-        _set_status(db, job, "paused")  # a running worker aborts cooperatively via the hook
+    service.pause(db, job)
     return _serialize_job(db, job)
 
 
@@ -474,19 +467,7 @@ def resume_download(
     job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
 ) -> dict:
     job = _own_job(db, user, job_id)
-    if job.status in ("paused", "error"):
-        job.progress = 0
-        job.error = None
-        job.error_code = None
-        # If the shared asset itself failed, reset it to pending so the worker actually
-        # re-downloads — otherwise the requeued job would instantly re-inherit the asset's
-        # stale error (the worker short-circuits a job whose asset is already errored).
-        if job.asset_id:
-            asset = db.get(MediaAsset, job.asset_id)
-            if asset and asset.status == "error":
-                asset.status = "pending"
-                asset.error = None
-        _set_status(db, job, "queued")
+    service.resume(db, job)
     return _serialize_job(db, job)
 
 
@@ -510,33 +491,7 @@ def redownload_download(
         raise HTTPException(status_code=409, detail="Editor clips can't be re-downloaded.")
     if job.status in ("queued", "running", "paused"):
         raise HTTPException(status_code=409, detail="This download is still in progress.")
-
-    # Resolve the fresh private asset FIRST (before touching any file). Salting with the current
-    # asset id makes it distinct from both the dedup pool and this job's current asset, so the
-    # insert never collides — no IntegrityError/rollback can strand us after a file was deleted.
-    spec = formats.normalize(job.profile_snapshot)
-    sig = formats.format_sig(spec, salt=f"{job.id}:{job.asset_id}")
-    asset = service.get_or_create_asset(db, job.source_kind, job.source_ref, sig)
-    service.populate_from_catalog(db, asset)
-
-    # Now drop the old hold — `service.release_asset` deletes the old file + row only if this job was its
-    # last holder (a co-held asset is left intact for its other holders).
-    service.release_asset(db, job)
-
-    asset.ref_count = (asset.ref_count or 0) + 1
-    job.asset_id = asset.id
-    job.status = "queued"
-    job.progress = 0
-    job.phase = None
-    job.error = None
-    job.speed_bps = None
-    job.eta_s = None
-    # Re-derive the display name (card title + Content-Disposition filename) from the freshly
-    # cleaned title — otherwise a name auto-set from an older, un-cleaned title (e.g. still carrying
-    # a "1.6M views · … |" social prefix) would survive the rebuild. The worker fills it on completion.
-    job.display_name = None
-    job.queue_pos = service.next_queue_pos(db)  # tail of the queue, not its stale original slot
-    db.commit()
+    service.redownload(db, job)
     return _serialize_job(db, job)
 
 
@@ -545,10 +500,7 @@ def cancel_download(
     job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
 ) -> dict:
     job = _own_job(db, user, job_id)
-    if job.status not in (JobStatus.DONE, JobStatus.CANCELED):
-        service.release_asset(db, job)  # release while the job still counts as holding
-        job.status = JobStatus.CANCELED
-        db.commit()
+    service.cancel(db, job)
     return _serialize_job(db, job)
 
 
@@ -557,9 +509,7 @@ def delete_download(
     job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
 ) -> dict:
     job = _own_job(db, user, job_id)
-    service.release_asset(db, job)
-    db.delete(job)
-    db.commit()
+    service.delete(db, job)
     return {"deleted": job_id}
 
 

backend/app/routes/playlists.py

@@ -1,6 +1,7 @@
-"""Local playlists: per-user named, ordered collections of videos from the shared
-catalog. Foundation phase = local only (create / rename / delete, add / remove / reorder
-items). YouTube two-way sync (mirror existing playlists, push edits) lands in later phases."""
+"""Playlists: per-user named, ordered collections of videos from the shared catalog. Local
+CRUD (create / rename / delete, add / remove / reorder items) plus YouTube two-way sync —
+mirror the user's own YouTube playlists in (`/sync-youtube`) and push/revert local edits back
+out (`/{id}/push-plan`, `/{id}/revert-youtube`), behind the optional write scope."""
 from fastapi import APIRouter, Depends, HTTPException
 from pydantic import BaseModel
 from sqlalchemy import func, select, and_

backend/app/routes/plex/__init__.py

@@ -10,9 +10,21 @@ wiring expects (`plex_routes.router`), so main.py is unchanged.
 
 from fastapi import APIRouter
 
-from app.routes.plex import admin, browse, playback
+from app.routes.plex import (
+    admin,
+    browse,
+    collections,
+    detail,
+    people,
+    playback,
+    playlists,
+)
 
 router = APIRouter()
 router.include_router(admin.router)
 router.include_router(browse.router)
+router.include_router(collections.router)
+router.include_router(detail.router)
+router.include_router(people.router)
 router.include_router(playback.router)
+router.include_router(playlists.router)

backend/app/routes/plex/_shared.py

@@ -3,10 +3,12 @@
 import logging
 
 from fastapi import HTTPException
+from sqlalchemy import select
 from sqlalchemy.orm import Session
 
 from app.models import (
     PlexItem,
+    PlexLibrary,
 )
 from app.plex.lookups import by_rating_key
 
@@ -18,3 +20,22 @@ def item_or_404(db: Session, rating_key: str) -> PlexItem:
     if it is None:
         raise HTTPException(status_code=404, detail="Unknown Plex item")
     return it
+
+
+def _norm_lang(lang: str | None) -> str:
+    """The overlay has an EN base + an optional HU twin; anything other than `hu` serves EN."""
+    return "hu" if lang == "hu" else "en"
+
+
+def _content_libs(db: Session) -> tuple[list[int], list[int]]:
+    """The BROWSABLE content libraries, partitioned into (movie_lib_ids, show_lib_ids) — the single
+    source of the browse scope. Excludes HU overlay libraries (`overlay_of` set): those are never
+    browsed on their own, they only lend their Hungarian metadata to the EN twin's rows (E5 S5)."""
+    libs = db.execute(
+        select(PlexLibrary).where(
+            PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
+        )
+    ).scalars().all()
+    movie_lib_ids = [lb.id for lb in libs if lb.kind == "movie"]
+    show_lib_ids = [lb.id for lb in libs if lb.kind == "show"]
+    return movie_lib_ids, show_lib_ids

backend/app/routes/plex/collections.py

@@ -0,0 +1,253 @@
+"""Plex collections: browse mirrored collections + the admin manual-collection editor."""
+
+from fastapi import APIRouter, Depends, HTTPException
+from pydantic import BaseModel
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session
+
+from app.auth import admin_user, current_user
+from app.db import get_db
+from app.models import (
+    PlexCollection,
+    PlexLibrary,
+    User,
+)
+from app.plex import sync as plex_sync
+from app.plex.client import PlexClient, PlexError, PlexNotConfigured
+from app.plex.lookups import by_rating_key
+from app.plex.query import like_escape
+
+router = APIRouter(prefix="/api/plex", tags=["plex"])
+
+_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
+
+
+def _collection_source(col: PlexCollection) -> str:
+    first = (col.title or "").strip().split(" ", 1)[0].lower()
+    if first in _STRIP_PROVIDERS:
+        return first
+    if col.smart:
+        return "smart"
+    return "collection"
+
+
+def _collection_editable(c: PlexCollection) -> bool:
+    """Effective editability: the admin marked it editable AND it's a plain manual collection — never
+    smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
+    return bool(c.editable) and not c.smart and _collection_source(c) == "collection"
+
+
+def _collection_card(c: PlexCollection) -> dict:
+    return {
+        "id": c.rating_key,
+        "title": c.title,
+        "summary": c.summary,
+        "thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
+        "child_count": c.child_count,
+        "smart": c.smart,
+        "source": _collection_source(c),
+        "editable": c.editable,
+        "can_edit": _collection_editable(
+            c
+        ),  # whether the edit controls should be offered (admin UI)
+    }
+
+
+@router.get("/collections")
+def collections(
+    library: str | None = None,
+    q: str | None = None,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Mirrored collections as cards (most-populated first), optionally name-filtered by `q`. With a
+    `library` plex_key → that one library's collections (used by the collection editor). WITHOUT one →
+    the UNION across all enabled libraries (the unified-scope sidebar picker, which has no single
+    library selected)."""
+    if library is not None:
+        lib = db.execute(
+            select(PlexLibrary).where(PlexLibrary.plex_key == str(library))
+        ).scalars().first()
+        if lib is None:
+            return {"collections": []}
+        query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
+    else:
+        enabled_ids = list(
+            db.execute(
+                select(PlexLibrary.id).where(
+                    PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
+                )
+            ).scalars()
+        )
+        if not enabled_ids:
+            return {"collections": []}
+        query = select(PlexCollection).where(
+            PlexCollection.library_id.in_(enabled_ids)
+        )
+    term = (q or "").strip()
+    if term:
+        query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
+    rows = db.execute(
+        query.order_by(
+            PlexCollection.child_count.desc().nullslast(),
+            func.lower(PlexCollection.title),
+        )
+    ).scalars().all()
+    return {"collections": [_collection_card(c) for c in rows]}
+
+
+def _editable_collection_or_403(db: Session, rating_key: str) -> PlexCollection:
+    col = by_rating_key(db, PlexCollection, rating_key)
+    if col is None:
+        raise HTTPException(status_code=404, detail="Unknown collection")
+    if not _collection_editable(col):
+        raise HTTPException(status_code=403, detail="This collection is read-only")
+    return col
+
+
+class CreateCollectionIn(BaseModel):
+    library: str = ""
+    title: str = ""
+    item_rating_key: str = ""
+
+
+@router.post("/collections")
+def create_collection(
+    body: CreateCollectionIn,
+    _: User = Depends(admin_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Create a manual collection (seeded with one item — Plex needs at least one). Editable by default."""
+    library = body.library
+    title = body.title.strip()
+    seed = body.item_rating_key
+    if not title or not seed:
+        raise HTTPException(
+            status_code=400, detail="title and item_rating_key are required"
+        )
+    lib = db.execute(
+        select(PlexLibrary).where(PlexLibrary.plex_key == library)
+    ).scalars().first()
+    if lib is None:
+        raise HTTPException(status_code=404, detail="Unknown library")
+    try:
+        with PlexClient(db) as plex:
+            new = plex.create_collection(lib.plex_key, title, seed)
+            col = plex_sync.resync_collection(db, plex, lib, str(new.get("ratingKey")))
+            col.editable = True
+        db.commit()
+    except (PlexError, PlexNotConfigured) as e:
+        db.rollback()
+        raise HTTPException(status_code=422, detail=f"Plex write failed: {e}")
+    return _collection_card(col)
+
+
+@router.post("/collections/{rating_key}/items/{item_rating_key}")
+def collection_add_item(
+    rating_key: str,
+    item_rating_key: str,
+    _: User = Depends(admin_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    col = _editable_collection_or_403(db, rating_key)
+    lib = db.get(PlexLibrary, col.library_id)
+    try:
+        with PlexClient(db) as plex:
+            plex.add_collection_item(rating_key, str(item_rating_key))
+            col = plex_sync.resync_collection(db, plex, lib, rating_key)
+        db.commit()
+    except (PlexError, PlexNotConfigured) as e:
+        db.rollback()
+        raise HTTPException(status_code=422, detail=f"Plex write failed: {e}")
+    return _collection_card(col)
+
+
+@router.delete("/collections/{rating_key}/items/{item_rating_key}")
+def collection_remove_item(
+    rating_key: str,
+    item_rating_key: str,
+    _: User = Depends(admin_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    col = _editable_collection_or_403(db, rating_key)
+    lib = db.get(PlexLibrary, col.library_id)
+    try:
+        with PlexClient(db) as plex:
+            plex.remove_collection_item(rating_key, str(item_rating_key))
+            col = plex_sync.resync_collection(db, plex, lib, rating_key)
+        db.commit()
+    except (PlexError, PlexNotConfigured) as e:
+        db.rollback()
+        raise HTTPException(status_code=422, detail=f"Plex write failed: {e}")
+    return _collection_card(col)
+
+
+class CollectionRenameIn(BaseModel):
+    title: str = ""
+
+
+@router.patch("/collections/{rating_key}")
+def collection_rename(
+    rating_key: str,
+    body: CollectionRenameIn,
+    _: User = Depends(admin_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    col = _editable_collection_or_403(db, rating_key)
+    title = body.title.strip()
+    if not title:
+        raise HTTPException(status_code=400, detail="title is required")
+    lib = db.get(PlexLibrary, col.library_id)
+    try:
+        with PlexClient(db) as plex:
+            plex.rename_collection(lib.plex_key, rating_key, title)
+            col = plex_sync.resync_collection(db, plex, lib, rating_key)
+        db.commit()
+    except (PlexError, PlexNotConfigured) as e:
+        db.rollback()
+        raise HTTPException(status_code=422, detail=f"Plex write failed: {e}")
+    return _collection_card(col)
+
+
+@router.delete("/collections/{rating_key}")
+def collection_delete(
+    rating_key: str, _: User = Depends(admin_user), db: Session = Depends(get_db)
+) -> dict:
+    col = _editable_collection_or_403(db, rating_key)
+    lib = db.get(PlexLibrary, col.library_id)
+    try:
+        with PlexClient(db) as plex:
+            plex.delete_collection(rating_key)
+        plex_sync.delete_collection_local(db, lib, rating_key)
+        db.commit()
+    except (PlexError, PlexNotConfigured) as e:
+        db.rollback()
+        raise HTTPException(status_code=422, detail=f"Plex write failed: {e}")
+    return {"deleted": rating_key}
+
+
+class CollectionEditableIn(BaseModel):
+    editable: bool = False
+
+
+@router.post("/collections/{rating_key}/editable")
+def collection_set_editable(
+    rating_key: str,
+    body: CollectionEditableIn,
+    _: User = Depends(admin_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Mark an existing Plex collection as 'mine to edit' (or unmark). Smart / auto-list collections
+    (IMDb/TMDb/…) can never be made editable — they're regenerated outside Plex."""
+    col = by_rating_key(db, PlexCollection, rating_key)
+    if col is None:
+        raise HTTPException(status_code=404, detail="Unknown collection")
+    want = body.editable
+    if want and (col.smart or _collection_source(col) != "collection"):
+        raise HTTPException(
+            status_code=400,
+            detail="Smart / auto-list collections can't be made editable",
+        )
+    col.editable = want
+    db.commit()
+    return _collection_card(col)

backend/app/routes/plex/detail.py

@@ -0,0 +1,438 @@
+"""Plex show/item detail: the show drill-down + the per-item player payload (markers, cast, nav)."""
+
+import threading
+import time
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime, timezone
+
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session
+
+from app.auth import current_user
+from app.db import get_db
+from app.models import (
+    PlexCollection,
+    PlexItem,
+    PlexLibrary,
+    PlexSeason,
+    PlexShow,
+    PlexState,
+    User,
+)
+from app.plex import detail as plex_detail
+from app.plex import people as plex_people
+from app.plex import rollup, subtitles
+from app.plex.cards import episode_card, movie_card, show_card
+from app.plex.client import PlexClient, PlexError, PlexNotConfigured
+from app.plex.lookups import by_rating_key
+from app.routes.plex._shared import _norm_lang, item_or_404
+from app.routes.plex.collections import _collection_source
+
+router = APIRouter(prefix="/api/plex", tags=["plex"])
+
+
+def _lib_key(db: Session, library_id: int | None) -> str | None:
+    """The Plex section key for a library id — the info/show page passes it to the admin collection
+    editor (in the unified cross-library view there's no single library prop otherwise)."""
+    if library_id is None:
+        return None
+    lib = db.get(PlexLibrary, library_id)
+    return lib.plex_key if lib else None
+
+
+_SHOW_ENRICH_TTL_S = 300
+
+_show_enrich_cache: dict[str, tuple[float, dict, list]] = {}
+
+_show_enrich_lock = threading.Lock()
+
+
+def _show_enrichment(db: Session, rating_key: str) -> tuple[dict, list]:
+    """(meta, related_raw) for a show's live Plex enrichment — served from the per-show TTL cache when
+    warm, otherwise fetched with the two Plex calls (metadata + related) run IN PARALLEL. Returns
+    ({}, []) when Plex is unavailable/unconfigured (not cached, so it retries on the next open)."""
+    now = time.monotonic()
+    with _show_enrich_lock:
+        hit = _show_enrich_cache.get(rating_key)
+        if hit is not None and hit[0] > now:
+            return hit[1], hit[2]
+    try:
+        with PlexClient(db) as plex:
+            with ThreadPoolExecutor(max_workers=2) as ex:
+                meta_f = ex.submit(plex.metadata, rating_key)
+                related_f = ex.submit(
+                    plex.related, rating_key
+                )  # swallows its own errors → []
+                meta = meta_f.result() or {}
+                related_raw = related_f.result()
+    except (PlexError, PlexNotConfigured):
+        return {}, []
+    # Don't cache an EMPTY related list: plex.related() swallows a transient failure as [] (same as a
+    # genuinely no-related show), so caching it would serve an empty "related" section for the whole TTL
+    # even after Plex recovers. Skip the cache in that case (retry next open, as before); shows that DO
+    # have related items — the payload worth caching — still get cached.
+    if related_raw:
+        with _show_enrich_lock:
+            _show_enrich_cache[rating_key] = (
+                now + _SHOW_ENRICH_TTL_S,
+                meta,
+                related_raw,
+            )
+    return meta, related_raw
+
+
+@router.get("/show/{rating_key}")
+def show_detail(
+    rating_key: str,
+    lang: str = "en",
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """A show's seasons + episodes (the drill-down view) with per-user watch state."""
+    hu = _norm_lang(lang) == "hu"
+    sh = by_rating_key(db, PlexShow, rating_key)
+    if sh is None:
+        raise HTTPException(status_code=404, detail="Unknown Plex show")
+    seasons = db.execute(
+        select(PlexSeason)
+        .where(PlexSeason.show_id == sh.id)
+        .order_by(PlexSeason.season_number.asc().nullsfirst())
+    ).scalars().all()
+    eps = db.execute(
+        select(PlexItem)
+        .where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
+        .order_by(
+            PlexItem.season_number.asc().nullsfirst(),
+            PlexItem.episode_number.asc().nullsfirst(),
+        )
+    ).scalars().all()
+    states = {
+        s.item_id: s
+        for s in db.execute(
+            select(PlexState).where(
+                PlexState.user_id == user.id,
+                PlexState.item_id.in_([e.id for e in eps] or [0]),
+            )
+        ).scalars()
+    }
+    by_season: dict[int | None, list] = {}
+    all_cards: list[
+        dict
+    ] = []  # every episode, in season/episode order → show-level rollup
+    for e in eps:
+        card = episode_card(e, states.get(e.id))
+        by_season.setdefault(e.season_id, []).append(card)
+        all_cards.append(card)
+    show_roll = rollup.rollup(all_cards)
+
+    # Cast + IMDb + tagline are served from the DB row (persisted by the cast backfill, E5 S5b). The one
+    # remaining live call is `related` — a Plex recommendation hub we don't mirror — which also carries
+    # the metadata used to lazily fill a not-yet-enriched show. Any live failure just drops `related` and
+    # the lazy fill; the row-served detail below still renders (outage-resilient).
+    related: list[dict] = []
+    try:
+        meta, related_raw = _show_enrichment(db, sh.rating_key)
+        related = _relatedshow_cards(db, user.id, related_raw, exclude=sh.id)
+        if sh.people_enriched_at is None and meta:
+            plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
+            plex_detail.apply_rich_detail(sh, meta)  # imdb/tagline/full_cast for future views
+            sh.people_enriched_at = datetime.now(timezone.utc)
+            try:
+                db.commit()
+            except Exception:
+                db.rollback()
+    except Exception:
+        # Best-effort only — the local episode list is already assembled, so any live-Plex failure
+        # (not configured, HTTP/JSON shape) must degrade, not 500.
+        pass
+
+    imdb_id = sh.imdb_id
+    return {
+        "show": {
+            "id": sh.rating_key,
+            "title": (sh.title_hu or sh.title) if hu else sh.title,
+            "summary": (sh.summary_hu or sh.summary) if hu else sh.summary,
+            "year": sh.year,
+            "thumb": f"/api/plex/image/{sh.rating_key}",
+            "art": f"/api/plex/image/{sh.rating_key}?variant=art",
+            "library": _lib_key(db, sh.library_id),
+            "content_rating": sh.content_rating,
+            "rating": sh.rating,
+            "genres": (sh.genres_hu or sh.genres if hu else sh.genres) or [],
+            "studio": sh.studio,
+            "season_count": sh.child_count,
+            "imdb_rating": sh.imdb_rating,
+            "imdb_id": imdb_id,
+            "imdb_url": f"https://www.imdb.com/title/{imdb_id}/" if imdb_id else None,
+            "cast": sh.full_cast or [],
+            # Show-level rollup for the hero action buttons.
+            "status": show_roll["status"],
+            "resume": show_roll["resume"],
+            "first": show_roll["first"],
+            "episode_count": show_roll["episode_count"],
+            "collection_keys": sh.collection_keys or [],
+        },
+        "seasons": [_season_block(se, by_season.get(se.id, [])) for se in seasons],
+        "related": related,
+    }
+
+
+def _season_block(se: PlexSeason, cards: list[dict]) -> dict:
+    """A season's card for the show page (with its aggregate rollup) plus its episodes (used by the
+    season subpage, read from the same cached show-detail payload)."""
+    roll = rollup.rollup(cards)
+    return {
+        "id": se.rating_key,
+        "season_number": se.season_number,
+        "title": se.title
+        or (f"Season {se.season_number}" if se.season_number else "Season"),
+        "thumb": f"/api/plex/image/{se.rating_key}",
+        "episode_count": roll["episode_count"],
+        "status": roll["status"],
+        "resume": roll["resume"],
+        "first": roll["first"],
+        "episodes": cards,
+    }
+
+
+def _relatedshow_cards(
+    db: Session, user_id: int, related: list[dict], exclude: int
+) -> list[dict]:
+    """Map Plex 'related' metadata to show cards for the shows we actually mirror (so they're openable),
+    excluding the current show. Order preserved; capped."""
+    rks = [str(m.get("ratingKey")) for m in related if m.get("ratingKey")]
+    if not rks:
+        return []
+    shows = {
+        s.rating_key: s
+        for s in db.execute(
+            select(PlexShow).where(
+                PlexShow.rating_key.in_(rks[:60]), PlexShow.id != exclude
+            )
+        ).scalars()
+    }
+    statuses = rollup.state_map(db, user_id, [s.id for s in shows.values()])
+    out: list[dict] = []
+    seen: set[str] = set()
+    for rk in rks:
+        sh = shows.get(rk)
+        if sh is None or rk in seen:
+            continue
+        seen.add(rk)
+        out.append(show_card(sh, statuses.get(sh.id, "new")))
+        if len(out) >= 20:
+            break
+    return out
+
+
+def _collection_strips(db: Session, it: PlexItem, user_id: int) -> list[dict]:
+    """The collections this movie belongs to, each with its member titles (playable cards) — the
+    'Avatar Collection' strip on the info page. Pure local lookup (no Plex call)."""
+    keys = it.collection_keys or []
+    if not keys or it.kind != "movie":
+        return []
+    cols = {
+        c.rating_key: c
+        for c in db.execute(
+            select(PlexCollection).where(PlexCollection.rating_key.in_(keys))
+        ).scalars()
+    }
+    # Smallest (most specific — e.g. a franchise) first; the big auto-lists (IMDb/TMDb Popular) last.
+    ordered = sorted(
+        (c for c in cols.values()), key=lambda c: (c.child_count or 0, c.title)
+    )
+    out = []
+    for col in ordered:
+        rk = col.rating_key
+        members = db.execute(
+            select(PlexItem)
+            .where(
+                PlexItem.library_id == it.library_id,
+                PlexItem.kind == "movie",
+                PlexItem.collection_keys.contains([rk]),
+            )
+            .order_by(PlexItem.year.asc().nullslast(), func.lower(PlexItem.title))
+            .limit(60)
+        ).scalars().all()
+        ids = [m.id for m in members]
+        states = {
+            s.item_id: s
+            for s in db.execute(
+                select(PlexState).where(
+                    PlexState.user_id == user_id, PlexState.item_id.in_(ids or [0])
+                )
+            ).scalars()
+        }
+        out.append(
+            {
+                "id": rk,
+                "title": col.title,
+                "source": _collection_source(col),
+                "items": [movie_card(m, states.get(m.id)) for m in members],
+            }
+        )
+    return out
+
+
+def _episode_nav(db: Session, it: PlexItem) -> tuple[str | None, str | None]:
+    if it.kind != "episode" or it.show_id is None:
+        return None, None
+    keys = list(
+        db.execute(
+            select(PlexItem.rating_key)
+            .where(PlexItem.show_id == it.show_id, PlexItem.kind == "episode")
+            .order_by(
+                PlexItem.season_number.asc().nullsfirst(),
+                PlexItem.episode_number.asc().nullsfirst(),
+            )
+        ).scalars()
+    )
+    try:
+        i = keys.index(it.rating_key)
+    except ValueError:
+        return None, None
+    return (keys[i - 1] if i > 0 else None), (
+        keys[i + 1] if i < len(keys) - 1 else None
+    )
+
+
+@router.get("/item/{rating_key}")
+def item_detail(
+    rating_key: str,
+    lang: str = "en",
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Everything the player needs: metadata, cast, intro/credit markers (from Plex), episode
+    prev/next, and the per-user resume position + watch status."""
+    lang = _norm_lang(lang)
+    it = item_or_404(db, rating_key)
+    st = db.execute(
+        select(PlexState).where(
+            PlexState.user_id == user.id, PlexState.item_id == it.id
+        )
+    ).scalars().first()
+
+    # Markers + audio/subtitle streams are file-specific, so they stay LIVE; the rich detail (cast/IMDb/
+    # tagline/genres) is served from the DB row for movies (persisted below / by the backfill) and only
+    # parsed from this live meta for episodes (never enriched — not in the grid, no HU overlay).
+    meta: dict = {}
+    markers: list[dict] = []
+    audio_streams: list[dict] = []
+    subtitle_streams: list[dict] = []
+    try:
+        with PlexClient(db) as plex:
+            meta = plex.metadata(it.rating_key, markers=True) or {}
+        for m in meta.get("Marker") or []:
+            if m.get("type") in ("intro", "credits"):
+                markers.append(
+                    {
+                        "type": m.get("type"),
+                        "start_s": round((m.get("startTimeOffset") or 0) / 1000, 1),
+                        "end_s": round((m.get("endTimeOffset") or 0) / 1000, 1),
+                    }
+                )
+        # Audio + subtitle tracks (ordinals among their stream type — used to -map the selection).
+        part = ((meta.get("Media") or [{}])[0].get("Part") or [{}])[0]
+        for s in part.get("Stream") or []:
+            if s.get("streamType") == 2:
+                audio_streams.append(
+                    {
+                        "ord": len(audio_streams),
+                        "label": s.get("displayTitle")
+                        or s.get("language")
+                        or f"Audio {len(audio_streams) + 1}",
+                        "language": s.get("languageTag") or s.get("language"),
+                        "default": bool(s.get("selected") or s.get("default")),
+                    }
+                )
+            elif s.get("streamType") == 3:
+                codec = (s.get("codec") or s.get("format") or "").lower()
+                subtitle_streams.append(
+                    {
+                        "ord": len(subtitle_streams),
+                        "label": s.get("displayTitle")
+                        or s.get("language")
+                        or f"Subtitle {len(subtitle_streams) + 1}",
+                        "language": s.get("languageTag") or s.get("language"),
+                        "codec": codec,
+                        # Text subs (srt/ass/…) can be served as WebVTT tracks; image subs (PGS/VobSub)
+                        # can't be turned into text, so the player hides them (burn-in is a later phase).
+                        "text": codec not in subtitles.BITMAP_SUBS,
+                    }
+                )
+    except Exception:
+        pass  # live extras (markers/streams) are best-effort; any live-Plex failure must degrade, not 500 — playback works without them
+
+    # Lazy enrichment (movies only): the live fetch above carries the FULL cast + rich detail (the sync
+    # only stored the listing's top ~3), so fill a not-yet-enriched movie — its cast becomes filterable
+    # and future views serve from the row. Episodes aren't enriched; a show's cast is filled from /show.
+    if it.kind == "movie" and it.people_enriched_at is None and meta:
+        cast = plex_people.cast_from_meta(meta)
+        if cast:
+            plex_people.enrich_row_people(db, it, cast)
+        plex_detail.apply_rich_detail(it, meta)
+        it.people_enriched_at = datetime.now(timezone.utc)
+        try:
+            db.commit()
+        except Exception:
+            db.rollback()
+
+    hu = lang == "hu"
+    # Movies serve rich detail from the DB row; episodes from the live meta just fetched.
+    if it.kind == "movie":
+        _imdb_id = it.imdb_id
+        rich = {
+            "cast": it.full_cast or [],
+            "imdb_rating": it.imdb_rating,
+            "imdb_id": _imdb_id,
+            "imdb_url": f"https://www.imdb.com/title/{_imdb_id}/" if _imdb_id else None,
+            "content_rating": it.content_rating,
+            "genres": (it.genres_hu or it.genres if hu else it.genres) or [],
+            "directors": it.directors or [],
+            "studio": it.studio,
+            "tagline": it.tagline,
+        }
+    else:
+        rich = plex_detail.rich_detail(meta)
+
+    prev_id, next_id = _episode_nav(db, it)
+    show = db.get(PlexShow, it.show_id) if it.kind == "episode" and it.show_id else None
+    return {
+        "id": it.rating_key,
+        "kind": it.kind,
+        "title": (it.title_hu or it.title) if hu else it.title,
+        "summary": (it.summary_hu or it.summary) if hu else it.summary,
+        "year": it.year,
+        "duration_seconds": it.duration_s,
+        "playable": it.playable,
+        "thumb": f"/api/plex/image/{it.rating_key}",
+        "art": f"/api/plex/image/{it.rating_key}?variant=art",
+        "library": _lib_key(db, it.library_id),
+        "cast": rich.get("cast", []),
+        "imdb_rating": rich.get("imdb_rating"),
+        "imdb_id": rich.get("imdb_id"),
+        "imdb_url": rich.get("imdb_url"),
+        # Audience score (Plex audienceRating) served alongside the IMDb score — the UI shows the
+        # IMDb badge when imdb_rating is present, else this. Episodes carry it on the row too.
+        "rating": it.rating,
+        "content_rating": rich.get("content_rating"),
+        "genres": rich.get("genres", []),
+        "directors": rich.get("directors", []),
+        "studio": rich.get("studio"),
+        "tagline": rich.get("tagline"),
+        "markers": markers,
+        "audio_streams": audio_streams,
+        "subtitle_streams": subtitle_streams,
+        "status": st.status if st else "new",
+        "position_seconds": st.position_seconds if st else 0,
+        "show_title": (
+            (show.title_hu or show.title if hu else show.title) if show else None
+        ),
+        "season_number": it.season_number,
+        "episode_number": it.episode_number,
+        "prev_id": prev_id,
+        "next_id": next_id,
+        "collections": _collection_strips(db, it, user.id),
+    }

backend/app/routes/plex/people.py

@@ -0,0 +1,139 @@
+"""Plex people: cast/crew filter-chip cards + a single person's filmography page."""
+
+from fastapi import APIRouter, Depends
+from sqlalchemy import func, or_, select
+from sqlalchemy.orm import Session
+
+from app.auth import current_user
+from app.db import get_db
+from app.models import (
+    PlexItem,
+    PlexPerson,
+    PlexShow,
+    PlexState,
+    User,
+)
+from app.plex import rollup
+from app.plex.cards import movie_card, show_card
+from app.plex.query import split_csv
+from app.routes.plex._shared import _content_libs
+
+router = APIRouter(prefix="/api/plex", tags=["plex"])
+
+
+@router.get("/people/cards")
+def people_cards(
+    names: str,
+    scope: str = "both",
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Cards for the actor filter-chips currently applied: each name + its cached headshot + how many
+    movies/shows (in the given scope) feature them. Photos come from the plex_people cache (populated
+    as items are rich-fetched — info-page view + the rich re-sync); counts are live against cast_names.
+    A name with no cached photo yet still returns a card (the UI shows a placeholder)."""
+    wanted = split_csv(names)[:16]
+    if not wanted:
+        return {"people": []}
+    movie_lib_ids, show_lib_ids = _content_libs(db)
+    want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
+    want_shows = scope in ("show", "both") and bool(show_lib_ids)
+    photos = {
+        p.name: p.photo_thumb
+        for p in db.execute(
+            select(PlexPerson).where(PlexPerson.name.in_(wanted))
+        ).scalars()
+    }
+    out = []
+    for nm in wanted:
+        count = 0
+        if want_movies:
+            count += (
+                db.scalar(
+                    select(func.count(PlexItem.id)).where(
+                        PlexItem.kind == "movie",
+                        PlexItem.library_id.in_(movie_lib_ids),
+                        PlexItem.cast_names.contains([nm]),
+                    )
+                )
+                or 0
+            )
+        if want_shows:
+            count += (
+                db.scalar(
+                    select(func.count(PlexShow.id)).where(
+                        PlexShow.library_id.in_(show_lib_ids),
+                        PlexShow.cast_names.contains([nm]),
+                    )
+                )
+                or 0
+            )
+        out.append({"name": nm, "photo": photos.get(nm), "count": count})
+    return {"people": out}
+
+
+# Cap each filmography section — a prolific person shouldn't return an unbounded list the client then
+# mounts a card per row for (the grid paginates at PAGE; this page renders all at once). Ample for a
+# personal library; the most-recent titles win via the year-desc order.
+_PERSON_SECTION_MAX = 200
+
+
+@router.get("/person/{name}")
+def person_detail(
+    name: str,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """One cast/crew member's filmography: their cached headshot + the movies and shows (in the enabled
+    libraries) whose cast OR directors include this name — the inverse of the cast-name → library-filter
+    link. The name string is the identity key (Plex cast carries no stable per-person id), so distinct
+    people who share a name share a page. Each section is capped at _PERSON_SECTION_MAX. An unknown name
+    simply returns empty sections."""
+    photo = db.scalar(select(PlexPerson.photo_thumb).where(PlexPerson.name == name))
+    movie_lib_ids, show_lib_ids = _content_libs(db)
+
+    movies: list[dict] = []
+    if movie_lib_ids:
+        items = db.execute(
+            select(PlexItem)
+            .where(
+                PlexItem.kind == "movie",
+                PlexItem.library_id.in_(movie_lib_ids),
+                or_(
+                    PlexItem.cast_names.contains([name]),
+                    PlexItem.directors.contains([name]),
+                ),
+            )
+            .order_by(PlexItem.year.desc().nullslast(), func.lower(PlexItem.title))
+            .limit(_PERSON_SECTION_MAX)
+        ).scalars().all()
+        item_ids = [it.id for it in items]
+        states = {
+            s.item_id: s
+            for s in db.execute(
+                select(PlexState).where(
+                    PlexState.user_id == user.id,
+                    PlexState.item_id.in_(item_ids or [0]),
+                )
+            ).scalars()
+        }
+        movies = [movie_card(it, states.get(it.id)) for it in items]
+
+    shows: list[dict] = []
+    if show_lib_ids:
+        show_rows = db.execute(
+            select(PlexShow)
+            .where(
+                PlexShow.library_id.in_(show_lib_ids),
+                or_(
+                    PlexShow.cast_names.contains([name]),
+                    PlexShow.directors.contains([name]),
+                ),
+            )
+            .order_by(PlexShow.year.desc().nullslast(), func.lower(PlexShow.title))
+            .limit(_PERSON_SECTION_MAX)
+        ).scalars().all()
+        statuses = rollup.state_map(db, user.id, [s.id for s in show_rows])
+        shows = [show_card(sh, statuses.get(sh.id, "new")) for sh in show_rows]
+
+    return {"name": name, "photo": photo, "movies": movies, "shows": shows}

backend/app/routes/plex/playlists.py

@@ -0,0 +1,405 @@
+"""Plex playlists: the current user's personal, orderable playlists of playable items."""
+
+from datetime import datetime, timezone
+
+from fastapi import APIRouter, Depends, HTTPException
+from pydantic import BaseModel
+from sqlalchemy import delete, func, select, update
+from sqlalchemy.orm import Session
+
+from app.auth import current_user
+from app.db import get_db
+from app.models import (
+    PlexItem,
+    PlexPlaylist,
+    PlexPlaylistItem,
+    PlexShow,
+    PlexState,
+    User,
+)
+from app.plex.cards import leaf_card
+from app.plex.lookups import by_rating_key
+from app.routes._common import owned_or_404
+
+router = APIRouter(prefix="/api/plex", tags=["plex"])
+
+
+def _own_playlist_or_404(db: Session, pid: int, user: User) -> PlexPlaylist:
+    return owned_or_404(db, PlexPlaylist, pid, user, detail="Playlist not found")
+
+
+def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
+    return {
+        "id": pl.id,
+        "title": pl.title,
+        "item_count": count,
+        "thumb": f"/api/plex/image/{thumb_rk}" if thumb_rk else None,
+    }
+
+
+def _playlist_item_query(pid: int):
+    return (
+        select(PlexPlaylistItem, PlexItem)
+        .join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
+        .where(PlexPlaylistItem.playlist_id == pid)
+        .order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
+    )
+
+
+@router.get("/playlists")
+def playlists(
+    contains: str | None = None,
+    contains_group: str | None = None,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """The current user's playlists (newest-touched first), each with an item count + a cover thumb.
+    `contains`=an item rating_key adds `has_item` per playlist (for the single 'add to playlist' dialog).
+    `contains_group`=a comma-separated list of rating_keys adds `group_in` per playlist (how many of the
+    group it already holds) + a top-level `group_size`, for the 'add a whole season/show' dialog."""
+    contains_id = None
+    if contains:
+        contains_id = db.scalar(
+            select(PlexItem.id).where(PlexItem.rating_key == str(contains))
+        )
+    group_ids: set[int] | None = None
+    if contains_group is not None:
+        rks = [s for s in (contains_group or "").split(",") if s]
+        group_ids = set(
+            db.execute(
+                select(PlexItem.id).where(PlexItem.rating_key.in_(rks or [""]))
+            ).scalars()
+        )
+    out = []
+    for pl in db.execute(
+        select(PlexPlaylist)
+        .where(PlexPlaylist.user_id == user.id)
+        .order_by(PlexPlaylist.updated_at.desc())
+    ).scalars():
+        count = (
+            db.scalar(
+                select(func.count(PlexPlaylistItem.id)).where(
+                    PlexPlaylistItem.playlist_id == pl.id
+                )
+            )
+            or 0
+        )
+        first = db.execute(_playlist_item_query(pl.id)).first()
+        card = _playlist_card(pl, count, first[1].rating_key if first else None)
+        if contains_id is not None:
+            card["has_item"] = (
+                db.execute(
+                    select(PlexPlaylistItem.id).where(
+                        PlexPlaylistItem.playlist_id == pl.id,
+                        PlexPlaylistItem.item_id == contains_id,
+                    )
+                ).first()
+                is not None
+            )
+        if group_ids is not None:
+            card["group_in"] = (
+                db.scalar(
+                    select(func.count(PlexPlaylistItem.id)).where(
+                        PlexPlaylistItem.playlist_id == pl.id,
+                        PlexPlaylistItem.item_id.in_(group_ids or [0]),
+                    )
+                )
+                or 0
+            )
+        out.append(card)
+    resp: dict = {"playlists": out}
+    if group_ids is not None:
+        resp["group_size"] = len(group_ids)
+    return resp
+
+
+class CreatePlexPlaylistIn(BaseModel):
+    title: str = ""
+    item_rating_key: str = ""
+
+
+@router.post("/playlists")
+def create_playlist(
+    body: CreatePlexPlaylistIn,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    title = body.title.strip()
+    if not title:
+        raise HTTPException(status_code=400, detail="title is required")
+    pl = PlexPlaylist(user_id=user.id, title=title)
+    db.add(pl)
+    db.flush()
+    seed = body.item_rating_key
+    count = 0
+    thumb_rk = None
+    if seed:
+        it = by_rating_key(db, PlexItem, seed)
+        if it is not None:
+            db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0))
+            count, thumb_rk = 1, it.rating_key
+    db.commit()
+    return _playlist_card(pl, count, thumb_rk)
+
+
+@router.get("/playlists/{pid}")
+def playlist_detail(
+    pid: int, user: User = Depends(current_user), db: Session = Depends(get_db)
+) -> dict:
+    pl = _own_playlist_or_404(db, pid, user)
+    rows = db.execute(_playlist_item_query(pid)).all()
+    items = [it for (_pi, it) in rows]
+    states = {
+        s.item_id: s
+        for s in db.execute(
+            select(PlexState).where(
+                PlexState.user_id == user.id,
+                PlexState.item_id.in_([it.id for it in items] or [0]),
+            )
+        ).scalars()
+    }
+    shows = {
+        sh.id: sh
+        for sh in db.execute(
+            select(PlexShow).where(
+                PlexShow.id.in_([it.show_id for it in items if it.show_id] or [0])
+            )
+        ).scalars()
+    }
+    cards = [
+        leaf_card(
+            it,
+            states.get(it.id),
+            shows[it.show_id].title if it.show_id in shows else None,
+            shows[it.show_id].rating_key if it.show_id in shows else None,
+        )
+        for it in items
+    ]
+    return {"id": pl.id, "title": pl.title, "items": cards}
+
+
+class RenamePlexPlaylistIn(BaseModel):
+    title: str = ""
+
+
+@router.patch("/playlists/{pid}")
+def rename_playlist(
+    pid: int,
+    body: RenamePlexPlaylistIn,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    pl = _own_playlist_or_404(db, pid, user)
+    title = body.title.strip()
+    if not title:
+        raise HTTPException(status_code=400, detail="title is required")
+    pl.title = title
+    db.commit()
+    count = (
+        db.scalar(
+            select(func.count(PlexPlaylistItem.id)).where(
+                PlexPlaylistItem.playlist_id == pl.id
+            )
+        )
+        or 0
+    )
+    first = db.execute(_playlist_item_query(pl.id)).first()
+    return _playlist_card(pl, count, first[1].rating_key if first else None)
+
+
+@router.delete("/playlists/{pid}")
+def delete_playlist(
+    pid: int, user: User = Depends(current_user), db: Session = Depends(get_db)
+) -> dict:
+    pl = _own_playlist_or_404(db, pid, user)
+    db.delete(pl)
+    db.commit()
+    return {"deleted": pid}
+
+
+class PlexItemRefIn(BaseModel):
+    item_rating_key: str = ""
+
+
+@router.post("/playlists/{pid}/items")
+def playlist_add_item(
+    pid: int,
+    body: PlexItemRefIn,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    pl = _own_playlist_or_404(db, pid, user)
+    it = by_rating_key(db, PlexItem, body.item_rating_key)
+    if it is None:
+        raise HTTPException(status_code=404, detail="Unknown item")
+    exists = db.execute(
+        select(PlexPlaylistItem).where(
+            PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
+        )
+    ).scalars().first()
+    if exists is None:
+        maxpos = db.scalar(
+            select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
+                PlexPlaylistItem.playlist_id == pid
+            )
+        )
+        db.add(
+            PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=int(maxpos) + 1)
+        )
+        pl.updated_at = datetime.now(timezone.utc)
+    db.commit()
+    return {"ok": True}
+
+
+@router.delete("/playlists/{pid}/items/{item_rating_key}")
+def playlist_remove_item(
+    pid: int,
+    item_rating_key: str,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    pl = _own_playlist_or_404(db, pid, user)
+    it = by_rating_key(db, PlexItem, item_rating_key)
+    if it is not None:
+        db.execute(
+            delete(PlexPlaylistItem).where(
+                PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
+            )
+        )
+        pl.updated_at = datetime.now(timezone.utc)
+    db.commit()
+    return {"ok": True}
+
+
+class PlexItemKeysIn(BaseModel):
+    item_rating_keys: list[str] = []
+
+
+def _items_by_rks(db: Session, rks: list) -> list[PlexItem]:
+    """Resolve rating_keys to PlexItems, preserving the input order (drops unknown keys)."""
+    order = [str(k) for k in rks if str(k)]
+    if not order:
+        return []
+    by_rk = {
+        it.rating_key: it
+        for it in db.execute(
+            select(PlexItem).where(PlexItem.rating_key.in_(order))
+        ).scalars()
+    }
+    seen: set[str] = set()
+    out: list[PlexItem] = []
+    for rk in order:
+        it = by_rk.get(rk)
+        if it is not None and rk not in seen:
+            seen.add(rk)
+            out.append(it)
+    return out
+
+
+@router.post("/playlists/{pid}/items/bulk")
+def playlist_add_bulk(
+    pid: int,
+    body: PlexItemKeysIn,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Append many items at once (a whole season/show). Already-present items are skipped; the input
+    order is preserved for the newly-added ones. Returns how many were added + the new total."""
+    pl = _own_playlist_or_404(db, pid, user)
+    items = _items_by_rks(db, body.item_rating_keys)
+    existing = set(
+        db.execute(
+            select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pid)
+        ).scalars()
+    )
+    pos = int(
+        db.scalar(
+            select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
+                PlexPlaylistItem.playlist_id == pid
+            )
+        )
+    )
+    added = 0
+    for it in items:
+        if it.id in existing:
+            continue
+        pos += 1
+        db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=pos))
+        existing.add(it.id)
+        added += 1
+    if added:
+        pl.updated_at = datetime.now(timezone.utc)
+    db.commit()
+    total = (
+        db.scalar(
+            select(func.count(PlexPlaylistItem.id)).where(
+                PlexPlaylistItem.playlist_id == pid
+            )
+        )
+        or 0
+    )
+    return {"added": added, "item_count": total}
+
+
+@router.post("/playlists/{pid}/items/remove-bulk")
+def playlist_remove_bulk(
+    pid: int,
+    body: PlexItemKeysIn,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Remove many items at once (a whole season/show, or a multi-select). Returns how many were
+    removed + the new total."""
+    pl = _own_playlist_or_404(db, pid, user)
+    ids = [it.id for it in _items_by_rks(db, body.item_rating_keys)]
+    removed = 0
+    if ids:
+        removed = db.execute(
+            delete(PlexPlaylistItem).where(
+                PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id.in_(ids)
+            ),
+            execution_options={"synchronize_session": False},
+        ).rowcount
+        if removed:
+            pl.updated_at = datetime.now(timezone.utc)
+    db.commit()
+    total = (
+        db.scalar(
+            select(func.count(PlexPlaylistItem.id)).where(
+                PlexPlaylistItem.playlist_id == pid
+            )
+        )
+        or 0
+    )
+    return {"removed": removed, "item_count": total}
+
+
+@router.put("/playlists/{pid}/order")
+def reorder_playlist(
+    pid: int,
+    body: PlexItemKeysIn,
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Set the playlist order from an explicit list of item rating_keys (0-based positions)."""
+    pl = _own_playlist_or_404(db, pid, user)
+    order = body.item_rating_keys
+    id_by_rk = {
+        it.rating_key: it.id
+        for it in db.execute(
+            select(PlexItem).where(PlexItem.rating_key.in_(order or [""]))
+        ).scalars()
+    }
+    for pos, rk in enumerate(order):
+        iid = id_by_rk.get(rk)
+        if iid is not None:
+            db.execute(
+                update(PlexPlaylistItem)
+                .where(
+                    PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == iid
+                )
+                .values(position=pos)
+            )
+    pl.updated_at = datetime.now(timezone.utc)
+    db.commit()
+    return {"ok": True}

backend/app/routes/scheduler.py

@@ -90,17 +90,13 @@ def set_maintenance_settings(
     db: Session = Depends(get_db),
 ) -> dict:
     """Admin-tune the maintenance job's rolling re-validation batch (videos re-checked per
-    run). Persisted to app_state; the job reads it each run."""
-    value = body.revalidate_batch
-    if not (state.MAINTENANCE_BATCH_MIN <= value <= state.MAINTENANCE_BATCH_MAX):
-        raise HTTPException(
-            status_code=400,
-            detail=(
-                f"revalidate_batch must be between {state.MAINTENANCE_BATCH_MIN} and "
-                f"{state.MAINTENANCE_BATCH_MAX}"
-            ),
-        )
-    new = state.set_maintenance_batch(db, value)
+    run). Persisted via sysconfig; the job reads it each run."""
+    # set_value validates against the spec's min/max (one source of bounds), mirroring the config PATCH.
+    try:
+        sysconfig.set_value(db, "maintenance_revalidate_batch", body.revalidate_batch)
+    except (TypeError, ValueError) as exc:
+        raise HTTPException(status_code=400, detail=str(exc) or "Invalid value")
+    new = sysconfig.get_int(db, "maintenance_revalidate_batch")
     audit.record(db, admin.id, AuditAction.MAINTENANCE_TUNE, summary=f"revalidate batch = {new}", after={"revalidate_batch": new})
     db.commit()
     return {"revalidate_batch": new}
@@ -167,6 +163,7 @@ def get_scheduler(
         "daily_budget": sysconfig.get_int(db, "quota_daily_budget"),
         "backfill_reserve": sysconfig.get_int(db, "backfill_quota_reserve"),
     }
+    maint = sysconfig.spec("maintenance_revalidate_batch")
 
     return {
         **snap,
@@ -174,9 +171,9 @@ def get_scheduler(
         "queue": queue,
         "quota": quota_info,
         "maintenance": {
-            "revalidate_batch": state.get_maintenance_batch(db),
+            "revalidate_batch": sysconfig.get_int(db, "maintenance_revalidate_batch"),
             "revalidate_batch_default": settings.maintenance_revalidate_batch,
-            "min": state.MAINTENANCE_BATCH_MIN,
-            "max": state.MAINTENANCE_BATCH_MAX,
+            "min": maint.min,
+            "max": maint.max,
         },
     }

backend/app/routes/setup.py

@@ -3,7 +3,7 @@
 `GET /api/setup/status` is public (the SPA uses it to decide whether to show the wizard). Every
 mutating step goes through `require_setup`: it rejects once the instance is configured AND requires
 the one-time setup token (printed to the container logs at first boot, carried by the wizard as the
-X-Setup-Token header). The wizard steps themselves are added in epic 6c.
+X-Setup-Token header).
 """
 from typing import Any
 
@@ -96,6 +96,7 @@ def setup_config(settings_map: dict[str, Any], db: Session = Depends(require_set
         except (ValueError, RuntimeError) as exc:
             raise HTTPException(status_code=400, detail=str(exc))
         saved.append(key)
+    db.commit()  # sysconfig.set_value only flushes now — persist the wizard's config in one commit
     return {"saved": saved}
 
 

backend/app/routes/sync.py

@@ -7,30 +7,13 @@ from app.audit import AuditAction
 from app.auth import admin_user, current_user, has_read_scope, require_human
 from app.db import get_db
 from app.models import Channel, Subscription, User, Video
-from app.sync.runner import (
-    estimate_deep_backfill,
-    run_enrich,
-    run_recent_backfill,
-    run_rss_poll,
-)
+from app.sync.runner import estimate_deep_backfill
 from app.sync.subscriptions import import_subscriptions
 from app.scheduler import running_job_ids
 
 router = APIRouter(prefix="/api/sync", tags=["sync"])
 
 
-def _user_channels(db: Session, user: User) -> list[Channel]:
-    return (
-        db.execute(
-            select(Channel)
-            .join(Subscription, Subscription.channel_id == Channel.id)
-            .where(Subscription.user_id == user.id)
-        )
-        .scalars()
-        .all()
-    )
-
-
 @router.post("/subscriptions")
 def sync_subscriptions(
     user: User = Depends(require_human), db: Session = Depends(get_db)
@@ -48,38 +31,6 @@ def sync_subscriptions(
     return result
 
 
-@router.post("/rss")
-def sync_rss(
-    user: User = Depends(require_human), db: Session = Depends(get_db)
-) -> dict:
-    # A partial failure comes back in `failed_feeds` so a manual poll reports it instead of an
-    # innocent-looking "0 new"; a TOTAL outage raises out of run_rss_poll and surfaces as a 500,
-    # which is the honest answer for a poll that reached nothing.
-    result = run_rss_poll(db, _user_channels(db, user))
-    return {"new_videos": result["new"], "failed_feeds": result["failed"]}
-
-
-@router.post("/backfill")
-def sync_backfill(
-    user: User = Depends(require_human),
-    db: Session = Depends(get_db),
-    max_channels: int = 25,
-) -> dict:
-    with quota.measured(db, user.id, quota.QuotaAction.VIDEOS_BACKFILL_RECENT) as m:
-        result = run_recent_backfill(db, _user_channels(db, user), max_channels=max_channels)
-    result["quota_used_estimate"] = m.used
-    return result
-
-
-@router.post("/enrich")
-def sync_enrich(
-    user: User = Depends(require_human), db: Session = Depends(get_db)
-) -> dict:
-    with quota.measured(db, user.id, quota.QuotaAction.VIDEOS_ENRICH) as m:
-        enriched = run_enrich(db)
-    return {"enriched": enriched, "quota_used_estimate": m.used}
-
-
 @router.get("/status")
 def sync_status(
     user: User = Depends(current_user), db: Session = Depends(get_db)

backend/app/routes/tags.py

@@ -4,11 +4,10 @@ from sqlalchemy import func, or_, select
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import Session
 
-from app.auth import admin_user, current_user
+from app.auth import current_user
 from app.db import get_db
 from app.models import ChannelTag, Tag, User
 from app.routes._common import owned_or_404
-from app.sync.autotag import run_autotag_all
 
 router = APIRouter(prefix="/api/tags", tags=["tags"])
 
@@ -113,12 +112,3 @@ def delete_tag(
     db.delete(tag)  # ChannelTag rows cascade on tag delete.
     db.commit()
     return {"deleted": tag_id}
-
-
-@router.post("/recompute")
-def recompute(
-    _: User = Depends(admin_user),
-    db: Session = Depends(get_db),
-    only_missing: bool = False,
-) -> dict:
-    return run_autotag_all(db, only_missing=only_missing)

backend/app/state.py

@@ -4,16 +4,11 @@ import secrets
 
 from sqlalchemy.orm import Session
 
-from app.config import settings
 from app.models import AppState
 from app.security import hash_token
 
 log = logging.getLogger("siftlode.state")
 
-# Bounds for the admin-set maintenance re-validation batch (videos re-checked per run).
-MAINTENANCE_BATCH_MIN = 100
-MAINTENANCE_BATCH_MAX = 100_000
-
 # Once configuration completes in this process it never reverts, so cache it to skip the per-request
 # DB hit in the setup-mode gate (see main.setup_gate). Starts False; refreshed from the DB.
 _configured_cache = False
@@ -33,10 +28,12 @@ def is_sync_paused(db: Session) -> bool:
 
 
 def set_sync_paused(db: Session, paused: bool) -> None:
+    # Flushes, doesn't commit — the caller owns the transaction so the change lands with its audit
+    # row in one transaction (B-core-10).
     row = _row(db)
     row.sync_paused = paused
     db.add(row)
-    db.commit()
+    db.flush()
     log.info("Background sync %s", "paused" if paused else "resumed")
 
 
@@ -90,20 +87,3 @@ def check_setup_token(db: Session, raw: str | None) -> bool:
         return False
     stored = _row(db).setup_token_hash
     return stored is not None and secrets.compare_digest(stored, hash_token(raw))
-
-
-def get_maintenance_batch(db: Session) -> int:
-    """Effective maintenance re-validation batch: the admin override if set, else the
-    env/config default."""
-    return _row(db).maintenance_revalidate_batch or settings.maintenance_revalidate_batch
-
-
-def set_maintenance_batch(db: Session, value: int) -> int:
-    """Clamp and persist the maintenance batch override. Returns the applied value."""
-    value = max(MAINTENANCE_BATCH_MIN, min(MAINTENANCE_BATCH_MAX, int(value)))
-    row = _row(db)
-    row.maintenance_revalidate_batch = value
-    db.add(row)
-    db.commit()
-    log.info("Maintenance re-validation batch set to %s", value)
-    return value

backend/app/sync/maintenance.py

@@ -25,11 +25,10 @@ from datetime import timedelta
 from sqlalchemy import select
 from sqlalchemy.orm import Session
 
-from app import progress, quota
+from app import progress, quota, sysconfig
 from app.config import settings
 from app.models import Playlist, PlaylistItem, Video
 from app.notifications import create_notification
-from app.state import get_maintenance_batch
 from app.sync.runner import get_service_user
 from app.sync.videos import apply_video_details, parse_dt
 from app.utils import now_utc as _now
@@ -182,7 +181,7 @@ def _recheck_flagged(db: Session, yt: YouTubeClient) -> dict:
 def _revalidate_rolling(db: Session, yt: YouTubeClient) -> dict:
     """Phase 2: re-check the least-recently-checked currently-available videos and flag any
     that have become unplayable (hidden immediately; deleted later if they don't recover)."""
-    batch = get_maintenance_batch(db)  # admin override, else config default
+    batch = sysconfig.get_int(db, "maintenance_revalidate_batch")  # admin override, else config default
     flagged_new = 0
     checked = 0
     now = _now()

backend/app/sync/playlists.py

@@ -1,6 +1,6 @@
-"""Read-direction YouTube playlist sync: mirror each user's own YouTube playlists into
-local `source='youtube'` playlists (kept fresh by the scheduler). One-way (YT -> local);
-the write-back direction (local edits -> YouTube) is a later phase.
+"""YouTube playlist sync, both directions. Read: mirror each user's own YouTube playlists
+into local `source='youtube'` playlists (kept fresh by the scheduler). Write-back: push local
+edits out to YouTube (`plan_push` / `push_playlist`), behind the optional write scope.
 
 YouTube's special Watch Later / History playlists are not exposed by the Data API and are
 never synced. Videos in a playlist that aren't in our shared catalog yet are fetched and

backend/app/sysconfig.py

@@ -59,6 +59,10 @@ SPECS: tuple[ConfigSpec, ...] = (
     # --- Batch sizes ---
     ConfigSpec("enrich_batch_size", "int", "batch", "enrich_batch_size", min=1, max=50),
     ConfigSpec("autotag_title_sample", "int", "batch", "autotag_title_sample", min=1, max=1_000),
+    # --- Maintenance (rolling re-validation of already-known videos) ---
+    # Videos re-checked per maintenance run. Folded here from app_state (B-core-11) so every admin
+    # knob lives in one store; read by sync/maintenance.py, tuned from the Scheduler dashboard.
+    ConfigSpec("maintenance_revalidate_batch", "int", "maintenance", "maintenance_revalidate_batch", min=100, max=100_000),
     # --- Discovery cleanup (search + explore ephemeral content) ---
     # Days an explored-but-unsubscribed channel is kept before the cleanup job purges it.
     ConfigSpec("explore_grace_days", "int", "explore", "explore_grace_days", min=0, max=3_650),
@@ -96,8 +100,8 @@ SPECS: tuple[ConfigSpec, ...] = (
     ConfigSpec("plex_overlay_map", "str", "plex", "plex_overlay_map"),
     # (plex_sync_interval_min is tuned from the Scheduler dashboard, not here — it's a scheduler job.)
     ConfigSpec("plex_max_transcodes", "int", "plex", "plex_max_transcodes", min=0, max=16),
-    # NOTE: plex_watch_sync_interval_min (config.py default) is surfaced here in Phase C, when the
-    # incremental watch-sync scheduler job that reads it lands.
+    # NOTE: plex_watch_sync_interval_min is a config.py default read by the watch-sync scheduler job
+    # (scheduler.py); it is intentionally NOT a runtime admin knob, so it stays out of SPECS.
 )
 
 _BY_KEY: dict[str, ConfigSpec] = {s.key: s for s in SPECS}
@@ -182,7 +186,9 @@ def get_bool(db: Session, key: str) -> bool:
 
 
 def set_value(db: Session, key: str, value) -> None:
-    """Validate, coerce, and persist a DB override for `key`. Secrets are encrypted."""
+    """Validate, coerce, and stage a DB override for `key` (secrets are encrypted). Flushes but does
+    NOT commit — the caller owns the transaction, so the change lands together with its audit row
+    (B-core-10). The staged value is visible to a same-session `get()` before the commit."""
     s = _BY_KEY[key]
     if s.type == "int":
         v = int(value)
@@ -208,15 +214,16 @@ def set_value(db: Session, key: str, value) -> None:
         db.add(row)
     row.value = stored
     row.encrypted = encrypted
-    db.commit()
+    db.flush()
 
 
 def reset(db: Session, key: str) -> None:
-    """Remove the DB override for `key` (fall back to the env/config default)."""
+    """Remove the DB override for `key` (fall back to the env/config default). Flushes but does NOT
+    commit — the caller owns the transaction (B-core-10)."""
     row = db.get(SystemConfig, key)
     if row is not None:
         db.delete(row)
-        db.commit()
+        db.flush()
 
 
 def describe(db: Session) -> dict:

backend/app/worker.py

@@ -17,125 +17,40 @@ Design:
 """
 import logging
 import os
-import re
 import shutil
 import signal
 import subprocess
 import sys
 import threading
 import time
-from datetime import date, datetime, timedelta, timezone
-from pathlib import Path
+from datetime import datetime, timezone
 
-import yt_dlp
 from sqlalchemy import text
 
 from app import sysconfig
 from app.config import settings
 from app.db import SessionLocal
 from app.downloads import edit as editmod
-from app.downloads import errors as dlerrors
-from app.downloads import adapters, formats, quota, service, storage
+from app.downloads import quota, service
+from app.downloads.jobdb import (
+    _edit_staging_dir,
+    _requeue_if_running,
+    _reset_asset_pending,
+    _set_job,
+    _staging_dir,
+)
 from app.downloads.status import STOPPED_BY_ROUTE, JobStatus
 from app.models import DownloadJob, MediaAsset
-from app.notifications import create_notification
-from app.titles import normalize_title
 
 log = logging.getLogger("siftlode.worker")
 
-
-def _notify_download_done(
-    db, user_id: int | None, title: str | None, job_id, *, kind: str
-) -> None:
-    """Post a best-effort 'ready' notice to the owner's inbox — downloads/edits used to finish
-    silently while scheduler jobs and playlist-push already notified (R11 feedback symmetry). A
-    notification failure must never mask the completed job, so it is isolated + rolled back (mirrors
-    scheduler._notify_done). `kind` ("download"/"edit") lets the inbox pick accurate copy; the
-    English title here is only a fallback for an unknown type. `user_id`/`title` are captured by the
-    caller BEFORE its commit, so reading them here needs no post-commit refresh of an expired row."""
-    if user_id is None:
-        return
-    try:
-        create_notification(
-            db,
-            user_id=user_id,
-            type="download_done",
-            title=f"Ready to watch: {title}" if title else "Ready to watch",
-            data={"title": title, "kind": kind},
-        )
-    except Exception:
-        db.rollback()
-        log.exception("failed to write download-done notification for job %s", job_id)
-
 _stop = threading.Event()
 
 
-class _Aborted(Exception):
-    """Raised inside the progress hook when the job was paused/canceled mid-download."""
-
-
 def _handle_signal(signum, frame):
     _stop.set()
 
 
-# --- small DB helpers (short-lived sessions; the download itself holds no transaction) -------
-
-def _set_job(job_id: int, **fields) -> None:
-    with SessionLocal() as db:
-        job = db.get(DownloadJob, job_id)
-        if job is None:
-            return
-        for k, v in fields.items():
-            setattr(job, k, v)
-        db.commit()
-
-
-def _job_status(job_id: int) -> str | None:
-    with SessionLocal() as db:
-        return db.execute(
-            text("SELECT status FROM download_jobs WHERE id = :id"), {"id": job_id}
-        ).scalar()
-
-
-def _touch_job(job_id: int) -> None:
-    """Bump a running job's `updated_at` without changing any value — a stall-watchdog heartbeat for
-    phases that legitimately write no progress. Extraction writes nothing, and a multi-client-set
-    fallback re-writes identical values (progress 0 / phase 'downloading'), so ORM `onupdate` would
-    NOT fire and two slow extraction attempts could otherwise cross the stall timeout and be killed
-    mid-extraction though they'd have succeeded."""
-    with SessionLocal() as db:
-        db.execute(
-            text("UPDATE download_jobs SET updated_at=now() WHERE id=:id AND status='running'"),
-            {"id": job_id},
-        )
-        db.commit()
-
-
-def _requeue_if_running(job_id: int, phase: str) -> None:
-    """Put the job back to 'queued' for a later retry, but ONLY while it is still 'running'. The
-    route flips status to paused/cancelled to stop a job; a blind `status='queued'` here would
-    clobber that and silently revive a job the user just cancelled. The conditional WHERE makes the
-    requeue a no-op unless we still own a running job."""
-    with SessionLocal() as db:
-        db.execute(
-            text(
-                "UPDATE download_jobs SET status='queued', phase=:phase, updated_at=now() "
-                "WHERE id=:id AND status='running'"
-            ),
-            {"id": job_id, "phase": phase},
-        )
-        db.commit()
-
-
-def _parse_upload_date(raw) -> date | None:
-    if not raw:
-        return None
-    try:
-        return datetime.strptime(str(raw), "%Y%m%d").date()
-    except ValueError:
-        return None
-
-
 # --- claim + recovery -----------------------------------------------------------------------
 
 def _wait_for_schema(timeout: float = 300.0) -> bool:
@@ -176,7 +91,14 @@ def _recover_orphans() -> None:
 
 
 def _claim_job() -> int | None:
-    """Claim the oldest queued job whose user is under their per-user concurrency limit.
+    """Claim the oldest queued job whose user is under their per-user concurrency limit AND whose
+    shared asset isn't already being downloaded by another worker.
+
+    The asset exclusion (`NOT EXISTS … media_assets.status='downloading'`) stops the same-asset ping-
+    pong (B-sub-3): without it, a second worker claims a queued job that shares an in-flight asset,
+    finds it busy, requeues it, and immediately re-claims it — burning a worker slot behind one
+    in-flight asset and starving everyone else. Excluded here, that job simply waits until its asset
+    settles, so `_process_job` no longer needs to sleep on the busy/lost-race paths.
 
     Users already AT their concurrency limit are excluded IN THE SQL, so one user flooding the
     queue can't monopolize the fetch window and starve everyone behind them. (The old fixed
@@ -193,6 +115,10 @@ def _claim_job() -> int | None:
                 """
                 SELECT id, user_id FROM download_jobs
                 WHERE status='queued'
+                  AND NOT EXISTS (
+                      SELECT 1 FROM media_assets a
+                      WHERE a.id = download_jobs.asset_id AND a.status = 'downloading'
+                  )
                   AND user_id NOT IN (
                       SELECT r.user_id
                       FROM download_jobs r
@@ -463,15 +389,16 @@ def _process_job(job_id: int) -> None:
         _set_job(job_id, status="error", error=asset_error or "Download failed", phase=None)
         return
     if asset_status == "downloading":
-        # Another worker owns this asset; wait and let the job be reclaimed later.
+        # Another worker won this asset between our claim and here (a narrow race — the claim SQL
+        # excludes downloading-asset jobs). Requeue and let it be reclaimed once the asset settles;
+        # no sleep — the claim exclusion keeps it out of the window until then, so no busy-loop.
         _requeue_if_running(job_id, "waiting")
-        time.sleep(2)
         return
 
     # asset_status == "pending": try to win the download / edit.
     if not _claim_asset(asset_id):
+        # Lost the pending->downloading race; requeue (same reasoning — the exclusion prevents a spin).
         _requeue_if_running(job_id, "waiting")
-        time.sleep(2)
         return
 
     # The asset is now ours ('downloading'): run the heavy work in a supervised child process.
@@ -494,543 +421,6 @@ def _finish_from_cache(job_id: int, asset_id: int) -> None:
         db.commit()
 
 
-def _fill_asset_meta_early(asset_id: int, info: dict) -> None:
-    """Populate an asset's display metadata from yt-dlp's info_dict the first time it's seen
-    (only if not already filled at enqueue from our catalog) — so an ad-hoc URL's row shows a
-    real title + thumbnail while it downloads, not just the URL. Zero extra network cost."""
-    with SessionLocal() as db:
-        asset = db.get(MediaAsset, asset_id)
-        if asset is None or asset.title:
-            return
-        # `or info.get("id")` avoids a blank card while downloading when the cleaned title is empty
-        # (e.g. a title that was only a social engagement prefix); the completion path later sets
-        # the final title with its own source_ref fallback.
-        asset.title = normalize_title(info.get("title")) or info.get("id")
-        asset.uploader = info.get("uploader") or info.get("channel")
-        asset.uploader_url = adapters.resolve_adapter(None, None, info).uploader_url(info)
-        asset.thumbnail_url = info.get("thumbnail")
-        asset.duration_s = int(info["duration"]) if info.get("duration") else None
-        asset.upload_date = _parse_upload_date(info.get("upload_date"))
-        asset.source_webpage_url = info.get("webpage_url")
-        db.commit()
-
-
-def _make_progress_hook(job_id: int, asset_id: int):
-    state = {"meta_done": False}
-    cancelled = _cancel_poll(job_id)          # same policy as the ffmpeg path
-    allow_write = _rate_limiter(_DB_WRITE_INTERVAL_S)
-
-    def hook(d: dict) -> None:
-        # Cooperative pause/cancel: if the job is no longer 'running', abort the download.
-        if cancelled():
-            raise _Aborted
-        if not state["meta_done"] and d.get("info_dict"):
-            state["meta_done"] = True
-            try:
-                _fill_asset_meta_early(asset_id, d["info_dict"])
-            except Exception:  # noqa: BLE001 — metadata is best-effort, never fail a download
-                pass
-        if d.get("status") != "downloading":
-            return
-        if not allow_write():
-            return
-        # yt-dlp downloads the video and audio streams SEPARATELY (each 0→100%), then merges.
-        # Label the phase so the user understands the bar resetting between streams instead of
-        # seeing a mysterious 95%→5% jump.
-        info = d.get("info_dict") or {}
-        vcodec = info.get("vcodec") or "none"
-        acodec = info.get("acodec") or "none"
-        if vcodec != "none" and acodec == "none":
-            phase = "video"
-        elif acodec != "none" and vcodec == "none":
-            phase = "audio"
-        else:
-            phase = "media"
-        total = d.get("total_bytes") or d.get("total_bytes_estimate") or 0
-        done = d.get("downloaded_bytes") or 0
-        pct = int(done * 100 / total) if total else 0
-        _set_job(
-            job_id,
-            progress=min(pct, 99),
-            speed_bps=int(d["speed"]) if d.get("speed") else None,
-            eta_s=int(d["eta"]) if d.get("eta") else None,
-            phase=phase,
-        )
-
-    return hook
-
-
-# Both progress sources call back FAR faster than the database should be touched: yt-dlp fires its
-# hook per chunk, and ffmpeg's `-progress pipe:1` emits a block of ~12 lines every ~0.5s while
-# run_ffmpeg calls its callbacks once per LINE. Every callback here is a round-trip (`_job_status`
-# opens a session and SELECTs; `_set_job` opens one and UPDATE+COMMITs), so unthrottled they cost
-# ~10^5 queries on a long job. ONE interval, ONE implementation, used by both paths — three
-# hand-rolled copies is how the ffmpeg side ended up unthrottled while the download side was fine.
-_DB_WRITE_INTERVAL_S = 1.0
-_CANCEL_POLL_INTERVAL_S = 1.0
-
-
-def _rate_limiter(interval_s: float, *, now=time.monotonic):
-    """`should_run()` → True at most once per `interval_s` (the first call always passes).
-    `now` is injectable so tests drive a clock instead of patching the global one."""
-    # `None`, not 0.0: "never ran" has to be a VALUE, not a falsy timestamp. A truthiness test here
-    # reads a legitimate `now() == 0.0` as "never ran" and lets a second call straight through —
-    # unreachable with real `time.monotonic()`, but `now` exists to be injected, and a test clock
-    # starting at 0 is the obvious choice. That would let a throttle test encode the wrong
-    # behaviour and still pass.
-    state: dict[str, float | None] = {"at": None}
-
-    def should_run() -> bool:
-        t = now()
-        last = state["at"]
-        if last is not None and t - last < interval_s:
-            return False
-        state["at"] = t
-        return True
-
-    return should_run
-
-
-def _cancel_poll(job_id: int, interval_s: float = _CANCEL_POLL_INTERVAL_S, *, now=time.monotonic):
-    """Cooperative "has this job been paused/cancelled?" poll, rate-limited and sticky: once it
-    says yes it never queries again. Shared by the yt-dlp hook and run_ffmpeg so cancel latency is
-    one policy, not one per call site."""
-    allow = _rate_limiter(interval_s, now=now)
-    state = {"cancelled": False}
-
-    def cancelled() -> bool:
-        if not state["cancelled"] and allow():
-            state["cancelled"] = _job_status(job_id) not in ("running", None)
-        return state["cancelled"]
-
-    return cancelled
-
-
-def _ffmpeg_progress_writer(job_id: int, phase: str, *, now=time.monotonic):
-    """`on_progress` for run_ffmpeg: persist at most one row write per interval, and only when the
-    whole-percent value actually changed (the bar shows integers)."""
-    allow = _rate_limiter(_DB_WRITE_INTERVAL_S, now=now)
-    state = {"pct": -1}
-
-    def write(pct: float) -> None:
-        whole = int(pct)
-        if whole == state["pct"] or not allow():
-            return
-        state["pct"] = whole
-        _set_job(job_id, progress=whole, phase=phase)
-
-    return write
-
-
-def _pp_phase(pp: str) -> str:
-    """Map a yt-dlp postprocessor class name to a user-facing phase label. ffmpeg post-steps
-    aren't byte-progress operations (no %), so the UI shows the step name + an indeterminate
-    pulse — this makes the long merge/finalize legible instead of a silent 'Processing'."""
-    if "Merg" in pp:
-        return "merging"
-    if "ExtractAudio" in pp:
-        return "audio_extract"
-    if "Thumbnail" in pp:
-        return "thumbnail"
-    if "SponsorBlock" in pp or "ModifyChapters" in pp:
-        return "sponsorblock"
-    if "Metadata" in pp:
-        return "metadata"
-    return "processing"
-
-
-def _make_pp_hook(job_id: int):
-    """Postprocessor hook: surface the current ffmpeg step so the bar shows a concrete label
-    (Merging / Embedding thumbnail / …) instead of freezing at 100% after the streams finish."""
-
-    def hook(d: dict) -> None:
-        if d.get("status") in ("started", "processing"):
-            _set_job(job_id, phase=_pp_phase(d.get("postprocessor") or ""), speed_bps=None, eta_s=None)
-
-    return hook
-
-
-def _staging_dir(asset_id: int) -> Path:
-    return Path(settings.download_root) / ".staging" / f"asset-{asset_id}"
-
-
-_THUMB_SUFFIXES = (".jpg", ".jpeg", ".png", ".webp")
-# yt-dlp's working files in the staging dir. None of them is ever the finished media, but several
-# CAN be the biggest file there (a video-only per-format stream, an abandoned fragment run), so the
-# staging fallback must exclude them by name — picking one would store e.g. a silent video-only
-# stream as a ready asset.
-_YTDLP_TEMP_RE = re.compile(
-    r"\.part(-Frag\d+)?$"  # in-flight download / fragment
-    r"|\.ytdl$"  # resume state
-    r"|\.temp\.[^.]+$"  # merge/postprocess temp
-    r"|\.f\d+\.[^.]+$",  # per-format stream kept before the merge
-    re.IGNORECASE,
-)
-
-
-def _find_thumbnail(staging: Path, media: Path) -> Path | None:
-    for p in staging.iterdir():
-        if p != media and p.suffix.lower() in _THUMB_SUFFIXES:
-            return p
-    return None
-
-
-def _produced_file(info: dict, staging: Path) -> Path:
-    """The media file yt-dlp actually wrote.
-
-    `requested_downloads[0].filepath` is the authoritative answer on YouTube, but an arbitrary-URL
-    extractor may not fill it in at all — and `Path("")` is `Path(".")`, which *passes* `.exists()`,
-    so a missing value used to sail past the fallback and blow up two steps later (`with_name` on an
-    empty name: the BBC-news failure). Anything that isn't a real file counts as absent; then fall
-    back to the largest file left in staging that is neither a thumbnail nor one of yt-dlp's own
-    working files (see `_YTDLP_TEMP_RE`)."""
-    req = (info.get("requested_downloads") or [{}])[0]
-    fp = req.get("filepath") or req.get("_filename") or info.get("filepath") or info.get("_filename")
-    if fp:
-        produced = Path(str(fp))
-        if produced.is_file():
-            return produced
-    cands = [
-        p
-        for p in staging.iterdir()
-        if p.is_file()
-        and p.suffix.lower() not in _THUMB_SUFFIXES
-        and not _YTDLP_TEMP_RE.search(p.name)
-    ]
-    if not cands:
-        raise RuntimeError("yt-dlp produced no output file")
-    return max(cands, key=lambda p: p.stat().st_size)
-
-
-def _media_info(info: dict) -> dict:
-    """Unwrap a playlist/multi_video extraction to the entry that carries the download.
-
-    A single non-YouTube page (a news article's video, a multi-rendition embed) can extract as a
-    wrapper whose own dict has no `requested_downloads` — the media lives on an entry. Pick the
-    FIRST entry that actually downloaded something: on a page with a skipped teaser in front of the
-    real video, entries[0] carries no file, and taking it blindly would label the produced file
-    with the wrong id/title/duration."""
-    entries = [e for e in (info.get("entries") or []) if e]
-    if not entries or info.get("requested_downloads"):
-        return info
-    chosen = next((e for e in entries if e.get("requested_downloads")), entries[0])
-    return {**info, **chosen}
-
-
-def _extract_with_fallback(job_id, asset_id, url, spec, staging, adapter):
-    """Download via yt-dlp using the source `adapter`'s client sets + options. For YouTube the adapter
-    supplies the primary player-client set then a POT-backed fallback (retried on a bot/DRM/format
-    error); other sources make a single attempt. Returns the info dict of the successful attempt."""
-    client_sets = adapter.client_sets()
-    outtmpl = str(staging / "%(id)s.%(ext)s")
-    last_exc: Exception | None = None
-    for i, clients in enumerate(client_sets):
-        opts = formats.build_ydl_opts(
-            spec, outtmpl, _make_progress_hook(job_id, asset_id),
-            extra=adapter.ydl_opts_extra(clients),
-        )
-        opts["postprocessor_hooks"] = [_make_pp_hook(job_id)]
-        _touch_job(job_id)  # stall-watchdog heartbeat: extraction writes no progress, so a slow /
-        # multi-client-set fallback would otherwise cross the stall timeout with a stale updated_at.
-        # (Per-ATTEMPT only; a single extract_info that itself runs past the stall timeout is still
-        # killed — acceptable, as socket_timeout + extractor_retries bound one attempt well under it.)
-        try:
-            with yt_dlp.YoutubeDL(opts) as ydl:
-                return ydl.extract_info(url, download=True)
-        except yt_dlp.utils.DownloadError as exc:
-            last_exc = exc
-            if i < len(client_sets) - 1 and adapter.retriable(str(exc)):
-                log.info("job %s: clients %s failed (%s); retrying with fallback", job_id, clients, str(exc)[:70])
-                for p in list(staging.iterdir()):  # drop partials before the retry
-                    try:
-                        p.unlink()
-                    except OSError:
-                        pass
-                _set_job(job_id, progress=0, phase="downloading")
-                continue
-            raise
-    raise last_exc  # unreachable (loop either returns or raises)
-
-
-def _download_settings() -> tuple[str, int]:
-    """Layout + retention days from the DB config (admin-overridable on the Configuration page),
-    falling back to the env defaults. Read here, not from `settings`, so an admin edit takes effect."""
-    with SessionLocal() as db:
-        return (
-            sysconfig.get_str(db, "download_layout"),
-            sysconfig.get_int(db, "download_retention_days"),
-        )
-
-
-def _ensure_browser_playable(job_id: int, produced: Path, spec: dict, info: dict) -> Path:
-    """Make an mp4-compat-profile download play in every browser: H.264 + AAC/MP3 in mp4. If the
-    download (whatever the source shipped) isn't already that, re-encode the offending stream(s) —
-    or just remux when the codecs are fine but the container isn't. **Scope:** only the mp4 compat
-    profile is normalized; an explicit `vcodec` or a non-mp4 custom profile is the user opting out
-    of the guarantee and is left as-is (so a `webm` profile can still ship VP9/Opus). Returns the
-    file to store (converted, or the original) and corrects `info`'s codec fields to match disk."""
-    if spec.get("container") != "mp4" or spec.get("vcodec"):
-        return produced
-    vcodec, acodec = editmod.probe_codecs(produced)
-    is_mp4 = produced.suffix.lower() == ".mp4"
-    needs, reenc_v, reenc_a = editmod.browser_transcode_flags(vcodec, acodec, is_mp4)
-    if not needs:
-        return produced
-
-    dest = produced.with_name(produced.stem + ".browser.mp4")
-    action = "re-encode" if (reenc_v or reenc_a) else "remux"
-    log.info("job %s: %s (v=%s a=%s mp4=%s) -> browser-safe mp4", job_id, action, vcodec, acodec, is_mp4)
-    _set_job(job_id, phase="processing", progress=0, speed_bps=None, eta_s=None)
-    # Prefer the info duration; probe the file when the extractor didn't provide one, so a long
-    # re-encode still shows real progress instead of a frozen 0%.
-    total_s = int(info["duration"]) if info.get("duration") else editmod.probe_duration(produced)
-    editmod.run_ffmpeg(
-        editmod.build_browser_safe_cmd(produced, dest, reenc_v, reenc_a),
-        total_s=total_s,
-        on_progress=_ffmpeg_progress_writer(job_id, "processing"),
-        should_cancel=_cancel_poll(job_id),
-    )
-    produced.unlink(missing_ok=True)
-    # Keep the stored asset metadata truthful about what's now on disk.
-    if reenc_v:
-        info["vcodec"] = "h264"
-    if reenc_a:
-        info["acodec"] = "aac"
-    return dest
-
-
-def _download(job_id: int, asset_id: int, source_kind: str, source_ref: str, spec: dict) -> None:
-    staging = _staging_dir(asset_id)
-    try:
-        if staging.exists():
-            shutil.rmtree(staging, ignore_errors=True)
-        staging.mkdir(parents=True, exist_ok=True)
-
-        _set_job(job_id, phase="downloading", progress=0)
-        adapter = adapters.resolve_adapter(source_kind, source_ref)
-        url = service.source_url(source_kind, source_ref)
-        info = _media_info(_extract_with_fallback(job_id, asset_id, url, spec, staging, adapter))
-        # Re-resolve now that yt-dlp's info reveals the true source (a bare/redirected url only
-        # identifies post-extraction) so the .nfo uniqueid type + uploader_url agree on one source.
-        adapter = adapters.resolve_adapter(source_kind, source_ref, info)
-
-        produced = _produced_file(info, staging)
-        produced = _ensure_browser_playable(job_id, produced, spec, info)
-        ext = produced.suffix.lstrip(".").lower()
-        meta = storage.MediaMeta(
-            video_id=info.get("id") or source_ref,
-            title=normalize_title(info.get("title")) or source_ref,
-            uploader=info.get("uploader") or info.get("channel") or "Unknown Channel",
-            upload_date=_parse_upload_date(info.get("upload_date")),
-            uniqueid_type=adapter.nfo_uniqueid_type(),
-            duration_s=int(info["duration"]) if info.get("duration") else None,
-            description=info.get("description"),
-            thumbnail_url=info.get("thumbnail"),
-        )
-
-        layout, retention_days = _download_settings()
-        rel = storage.rel_path(meta, ext, layout, asset_id=asset_id)
-        thumb = _find_thumbnail(staging, produced)
-        # Convert the raw thumbnail (webp/png/jpeg) to a real JPEG ourselves — robustly — for the
-        # poster + og:image sidecar, now that yt-dlp's fatal thumbnail convertor is gone. On failure
-        # this is None and write_sidecars writes no poster, so ensure_poster cuts a frame instead.
-        poster_jpg = editmod.thumbnail_to_jpg(thumb, staging / "poster.jpg")
-        storage.place_file(produced, settings.download_root, rel)
-        nfo_ok = storage.write_sidecars(settings.download_root, rel, meta, poster_jpg)
-
-        final = storage.abs_path(settings.download_root, rel)
-        size = final.stat().st_size if final.exists() else None
-        expires = datetime.now(timezone.utc) + timedelta(days=retention_days)
-
-        with SessionLocal() as db:
-            asset = db.get(MediaAsset, asset_id)
-            if asset:
-                asset.status = "ready"
-                asset.rel_path = rel
-                asset.size_bytes = size
-                asset.duration_s = meta.duration_s
-                asset.width = int(info["width"]) if info.get("width") else None
-                asset.height = int(info["height"]) if info.get("height") else None
-                asset.container = ext
-                asset.vcodec = info.get("vcodec") if info.get("vcodec") not in (None, "none") else None
-                asset.acodec = info.get("acodec") if info.get("acodec") not in (None, "none") else None
-                asset.title = meta.title
-                asset.uploader = meta.uploader
-                asset.uploader_url = adapter.uploader_url(info)
-                asset.upload_date = meta.upload_date
-                asset.thumbnail_url = meta.thumbnail_url
-                # Record a self-hosted poster for every download: it's the reliable link-preview
-                # (og:image) source — a remote thumbnail, esp. Facebook's signed CDN URL, expires
-                # and can't be relied on cross-origin. write_sidecars already wrote <base>.jpg from
-                # the thumbnail; ensure_poster returns that, or cuts a frame if there was none.
-                asset.poster_path = editmod.ensure_poster(asset, settings.download_root)
-                asset.source_webpage_url = info.get("webpage_url")
-                asset.nfo_written = nfo_ok
-                asset.error = None
-                asset.last_access_at = datetime.now(timezone.utc)
-                asset.expires_at = expires
-            job = db.get(DownloadJob, job_id)
-            # Capture the owner before commit expires the row (avoids a refresh SELECT just to notify).
-            owner_id = job.user_id if job else None
-            fields = {"status": JobStatus.DONE, "progress": 100, "phase": None, "speed_bps": None,
-                      "eta_s": None}
-            if job and not job.display_name and meta.title:
-                # display_name is VARCHAR(255) — clip a long title (see _fill_asset_meta_early).
-                fields["display_name"] = meta.title[:255]
-            # Atomic terminal write: finalizes ONLY while the row is still running, so a child that
-            # finishes just after a route cancel/pause can't resurrect it (see finalize_if_running).
-            finalized = service.finalize_if_running(db, job_id, **fields)
-            db.commit()
-            if finalized:
-                _notify_download_done(db, owner_id, meta.title, job_id, kind="download")
-                log.info("job %s done: %s", job_id, rel)
-
-    except (_Aborted, editmod.EditAborted):
-        # Paused/canceled mid-download or mid-transcode: free the asset for a later retry; keep the
-        # job's status (the route set it to paused/canceled). ref_count is adjusted by the route.
-        _reset_asset_pending(asset_id)
-        log.info("job %s aborted (pause/cancel)", job_id)
-    except Exception as exc:  # noqa: BLE001 — surface any failure to the user, keep worker alive
-        msg = str(exc)[:500]
-        with SessionLocal() as db:
-            asset = db.get(MediaAsset, asset_id)
-            if asset:
-                asset.status = "error"
-                asset.error = msg
-            # Atomic fail: only while still running, so a route-set cancel/pause is never overwritten
-            # with `error` (error also holds the asset → would re-open the double-release class).
-            service.finalize_if_running(
-                db, job_id, status=JobStatus.ERROR, error=msg,
-                error_code=dlerrors.classify(msg), phase=None,
-            )
-            db.commit()
-        log.warning("job %s failed: %s", job_id, msg)
-    finally:
-        shutil.rmtree(staging, ignore_errors=True)
-
-
-def _reset_asset_pending(asset_id: int) -> None:
-    with SessionLocal() as db:
-        asset = db.get(MediaAsset, asset_id)
-        if asset and asset.status == "downloading":
-            asset.status = "pending"
-            db.commit()
-
-
-# --- editor derivatives (phase 2: trim/crop via ffmpeg) -------------------------------------
-
-def _edit_staging_dir(asset_id: int) -> Path:
-    return Path(settings.download_root) / ".staging" / f"edit-{asset_id}"
-
-
-def _fail_edit(job_id: int, asset_id: int, msg: str) -> None:
-    with SessionLocal() as db:
-        asset = db.get(MediaAsset, asset_id)
-        if asset:
-            asset.status = "error"
-            asset.error = msg
-        # Atomic fail: only while still running (see the download error path).
-        service.finalize_if_running(
-            db, job_id, status=JobStatus.ERROR, error=msg,
-            error_code=dlerrors.classify(msg), phase=None,
-        )
-        db.commit()
-    log.warning("edit job %s failed: %s", job_id, msg)
-
-
-def _process_edit(job_id: int, asset_id: int) -> None:
-    """Produce a per-user trim/crop clip from the source asset's downloaded file via ffmpeg."""
-    with SessionLocal() as db:
-        job = db.get(DownloadJob, job_id)
-        src = db.get(MediaAsset, job.source_asset_id) if job and job.source_asset_id else None
-        edit_spec = dict(job.edit_spec or {}) if job else {}
-        user_id = job.user_id if job else None
-        src_ready = src is not None and src.status == "ready" and bool(src.rel_path)
-        src_rel = src.rel_path if src else None
-        src_w, src_h, src_dur = (src.width, src.height, src.duration_s) if src else (None, None, None)
-
-    if not src_ready or not src_rel:
-        _fail_edit(job_id, asset_id, "The source download is no longer available.")
-        return
-    src_path = storage.abs_path(settings.download_root, src_rel)
-    if not src_path.exists():
-        _fail_edit(job_id, asset_id, "The source file is missing.")
-        return
-
-    src_ext = src_path.suffix.lstrip(".").lower() or "mp4"
-    crop = editmod.has_crop(edit_spec)
-    out_ext = "mp4" if editmod.needs_reencode(edit_spec) else src_ext
-    total_s = editmod.clip_duration(edit_spec, src_dur) or src_dur or 0
-    staging = _edit_staging_dir(asset_id)
-
-    try:
-        if staging.exists():
-            shutil.rmtree(staging, ignore_errors=True)
-        staging.mkdir(parents=True, exist_ok=True)
-        dest_staging = staging / f"out.{out_ext}"
-
-        _set_job(job_id, phase="editing", progress=0, speed_bps=None, eta_s=None)
-        if edit_spec.get("segments"):
-            cmd, prep = editmod.build_concat_plan(src_path, dest_staging, edit_spec, out_ext, staging)
-            for p, content in prep.items():
-                p.write_text(content, encoding="utf-8")
-        else:
-            cmd = editmod.build_edit_cmd(src_path, dest_staging, edit_spec, out_ext)
-        editmod.run_ffmpeg(
-            cmd,
-            total_s,
-            on_progress=_ffmpeg_progress_writer(job_id, "editing"),
-            should_cancel=_cancel_poll(job_id),
-        )
-        if not dest_staging.exists():
-            raise RuntimeError("ffmpeg produced no output file")
-
-        rel = storage.edit_rel_path(user_id, asset_id, out_ext)
-        storage.place_file(dest_staging, settings.download_root, rel)
-        final = storage.abs_path(settings.download_root, rel)
-        size = final.stat().st_size if final.exists() else None
-        _, retention_days = _download_settings()
-        expires = datetime.now(timezone.utc) + timedelta(days=retention_days)
-
-        with SessionLocal() as db:
-            asset = db.get(MediaAsset, asset_id)
-            if asset:
-                asset.status = "ready"
-                asset.rel_path = rel
-                asset.size_bytes = size
-                asset.container = out_ext
-                asset.duration_s = total_s or asset.duration_s
-                if crop:
-                    asset.width = edit_spec["crop"]["w"]
-                    asset.height = edit_spec["crop"]["h"]
-                else:
-                    asset.width, asset.height = src_w, src_h
-                asset.error = None
-                asset.last_access_at = datetime.now(timezone.utc)
-                asset.expires_at = expires
-            job = db.get(DownloadJob, job_id)
-            # Capture owner + title before commit expires the rows (avoids a refresh SELECT).
-            owner_id = job.user_id if job else None
-            dl_title = asset.title if asset else None
-            # Atomic terminal write (see the download path): don't resurrect a route-stopped job.
-            finalized = service.finalize_if_running(
-                db, job_id, status=JobStatus.DONE, progress=100, phase=None
-            )
-            db.commit()
-            if finalized:
-                _notify_download_done(db, owner_id, dl_title, job_id, kind="edit")
-                log.info("edit job %s done: %s", job_id, rel)
-
-    except editmod.EditAborted:
-        # Paused/canceled mid-encode: free the asset for a later retry (ref_count is the route's job).
-        _reset_asset_pending(asset_id)
-        log.info("edit job %s aborted (pause/cancel)", job_id)
-    except Exception as exc:  # noqa: BLE001 — surface the failure, keep the worker alive
-        _fail_edit(job_id, asset_id, str(exc)[:500])
-    finally:
-        shutil.rmtree(staging, ignore_errors=True)
-
-
 # --- loop -----------------------------------------------------------------------------------
 
 def _worker_loop(idx: int) -> None:

backend/tests/test_config_commit_ownership.py

@@ -0,0 +1,38 @@
+"""E3 S3 (B-core-10): the config/state mutators flush but do NOT self-commit, so the caller owns the
+transaction and a change lands together with its audit row. These lock that contract — a rollback
+before the caller commits discards the change (proving no hidden internal commit), while a same-session
+read still sees it (flush visibility). DB lane (`db` fixture) — skipped without Postgres.
+"""
+from app import state, sysconfig
+
+
+def test_set_value_flushes_but_leaves_the_commit_to_the_caller(db):
+    default = sysconfig.get_int(db, "quota_daily_budget")
+    new = default + 4321
+
+    sysconfig.set_value(db, "quota_daily_budget", new)
+    assert sysconfig.get_int(db, "quota_daily_budget") == new  # visible in-session (flushed)
+
+    db.rollback()  # the caller never committed
+    assert sysconfig.get_int(db, "quota_daily_budget") == default  # discarded — no hidden self-commit
+
+
+def test_reset_flushes_but_leaves_the_commit_to_the_caller(db):
+    default = sysconfig.get_int(db, "quota_daily_budget")
+    sysconfig.set_value(db, "quota_daily_budget", default + 10)
+    db.commit()  # a persisted override to reset
+
+    sysconfig.reset(db, "quota_daily_budget")
+    assert sysconfig.get_int(db, "quota_daily_budget") == default  # reset visible in-session
+    db.rollback()
+    assert sysconfig.get_int(db, "quota_daily_budget") == default + 10  # reset discarded until committed
+
+
+def test_set_sync_paused_flushes_but_leaves_the_commit_to_the_caller(db):
+    assert state.is_sync_paused(db) is False
+
+    state.set_sync_paused(db, True)
+    assert state.is_sync_paused(db) is True  # visible in-session
+
+    db.rollback()
+    assert state.is_sync_paused(db) is False  # discarded — the caller owns the commit

backend/tests/test_download_limits.py

@@ -9,13 +9,14 @@
 Uses the `db` fixture (real Postgres) — skipped when no database is reachable. `_claim_job` runs in
 its OWN session, so every test COMMITS its setup before calling it.
 """
-from app.models import DownloadJob, DownloadQuota, User
+from app.models import DownloadJob, DownloadQuota, MediaAsset, User
 from app.worker import _claim_job
 
 
-def _job(db, user_id, queue_pos, status="queued"):
+def _job(db, user_id, queue_pos, status="queued", asset_id=None):
     j = DownloadJob(
         user_id=user_id,
+        asset_id=asset_id,
         source_kind="youtube",
         source_ref="x" * 11,
         status=status,
@@ -26,6 +27,15 @@ def _job(db, user_id, queue_pos, status="queued"):
     return j
 
 
+def _asset(db, status, ref):
+    a = MediaAsset(
+        source_kind="url", source_ref=ref, format_sig="s", status=status, ref_count=1
+    )
+    db.add(a)
+    db.flush()
+    return a
+
+
 def _quota(db, user_id, max_concurrent, unlimited=False):
     db.add(
         DownloadQuota(
@@ -104,6 +114,27 @@ def test_stored_zero_concurrency_is_floored_not_starving(db):
     assert db.get(DownloadJob, claimed).user_id == a.id  # A's own first job runs (floored to 1)
 
 
+def test_job_whose_asset_is_already_downloading_is_skipped(db):
+    """B-sub-3: a queued job whose shared asset is ALREADY `downloading` (another worker owns it) must
+    not be claimed — otherwise a second worker claims it, finds the asset busy, requeues, and ping-
+    pongs on it, starving everyone behind. The claim SQL excludes it, so the job behind it (on a
+    different, free asset) is served instead."""
+    u = User(email="asset-excl@example.com")
+    db.add(u)
+    db.flush()
+    _quota(db, u.id, max_concurrent=5)  # high → the ASSET rule, not the per-user rule, does the excluding
+    busy = _asset(db, status="downloading", ref="busy")
+    free = _asset(db, status="pending", ref="free")
+    _job(db, u.id, queue_pos=1, status="running", asset_id=busy.id)  # the holder actually downloading it
+    competitor = _job(db, u.id, queue_pos=2, asset_id=busy.id)  # same asset — must be skipped
+    behind = _job(db, u.id, queue_pos=3, asset_id=free.id)  # different asset — should be served
+    db.commit()
+
+    assert _claim_job() == behind.id  # the competitor on the busy asset was skipped
+    db.expire_all()
+    assert db.get(DownloadJob, competitor.id).status == "queued"  # still queued, never claimed
+
+
 def test_unlimited_user_is_never_excluded(db):
     u = User(email="unlimited@example.com")
     db.add(u)

backend/tests/test_download_output.py

@@ -6,7 +6,7 @@ with "PosixPath('.') has an empty name" instead of downloading."""
 import pytest
 
 from app.downloads.edit import _concat_quote, parse_probe_streams
-from app.worker import _media_info, _produced_file
+from app.downloads.pipeline import _media_info, _produced_file
 from pathlib import Path, PurePosixPath
 
 

backend/tests/test_download_ref_count.py

@@ -0,0 +1,116 @@
+"""E3 S2 characterization net — MediaAsset.ref_count enqueue/release accounting (B-sub-5).
+
+`ref_count` tracks how many DownloadJobs still hold a shared cache asset; when it hits zero a READY
+asset's file + row are freed. Enqueue +1's it and release_asset -1's it. Today those are Python
+read-modify-writes (`asset.ref_count = (asset.ref_count or 0) ± 1`), which lose an update under two
+concurrent enqueues/releases and can delete a file out from under a live holder. The fix makes the
+increment/decrement atomic at the DB; this net locks the OBSERVABLE single-threaded semantics so the
+conversion is provably behaviour-equivalent (concurrency itself can't be exercised in one session).
+
+DB lane (`db` fixture) — skipped without Postgres.
+"""
+from sqlalchemy import select, text
+
+from app.downloads import service
+from app.models import DownloadJob, MediaAsset, User
+
+
+def _user(db, email="ref@example.com"):
+    u = User(email=email)
+    db.add(u)
+    db.flush()
+    return u
+
+
+def _ref_count(db, asset_id) -> int | None:
+    # Read straight from the row — the atomic-UPDATE fix leaves the ORM copy stale on purpose.
+    return db.execute(
+        text("SELECT ref_count FROM media_assets WHERE id = :id"), {"id": asset_id}
+    ).scalar()
+
+
+def _holding_asset(db, *, status="ready", ref_count=1, rel_path=None) -> MediaAsset:
+    asset = MediaAsset(
+        source_kind="url", source_ref=f"r{ref_count}-{status}", format_sig="sig",
+        status=status, ref_count=ref_count, rel_path=rel_path,
+    )
+    db.add(asset)
+    db.flush()
+    return asset
+
+
+def _job(db, user_id, asset_id, status="done") -> DownloadJob:
+    j = DownloadJob(
+        user_id=user_id, asset_id=asset_id, source_kind="url", source_ref="r",
+        status=status, queue_pos=1, profile_snapshot={},
+    )
+    db.add(j)
+    db.flush()
+    return j
+
+
+def test_two_enqueues_share_one_asset_and_stack_ref_count(db):
+    u = _user(db)
+    j1 = service.enqueue(db, u.id, "url", "http://example.com/v", {})
+    j2 = service.enqueue(db, u.id, "url", "http://example.com/v", {})
+    # One shared cache asset, two jobs on it, ref_count == 2.
+    assert j1.asset_id == j2.asset_id
+    assets = db.execute(select(MediaAsset)).scalars().all()
+    assert len(assets) == 1
+    assert _ref_count(db, j1.asset_id) == 2
+
+
+def test_release_keeps_asset_until_last_holder_then_deletes_row(db):
+    u = _user(db)
+    asset = _holding_asset(db, status="ready", ref_count=2)
+    a = _job(db, u.id, asset.id, status="done")
+    b = _job(db, u.id, asset.id, status="done")
+    db.commit()
+
+    service.release_asset(db, a)
+    db.commit()
+    assert _ref_count(db, asset.id) == 1
+    assert db.get(MediaAsset, asset.id) is not None  # still held by b
+
+    service.release_asset(db, b)
+    db.commit()
+    # Last holder gone → the ready asset's row is freed (rel_path None, so no file to unlink).
+    assert db.execute(select(MediaAsset)).scalars().all() == []
+
+
+def test_release_is_a_noop_for_a_non_holding_job(db):
+    u = _user(db)
+    asset = _holding_asset(db, status="ready", ref_count=1)
+    canceled = _job(db, u.id, asset.id, status="canceled")  # canceled ∉ HOLDING
+    db.commit()
+
+    service.release_asset(db, canceled)
+    db.commit()
+    assert _ref_count(db, asset.id) == 1  # untouched — a canceled job already released while holding
+
+
+def test_release_never_drives_ref_count_negative(db):
+    u = _user(db)
+    # error-status asset (kept at zero, unlike ready) so we isolate the clamp from the delete branch.
+    asset = _holding_asset(db, status="error", ref_count=0)  # already at zero (defensive)
+    j = _job(db, u.id, asset.id, status="error")  # error ∈ HOLDING, so release runs
+    db.commit()
+
+    service.release_asset(db, j)
+    db.commit()
+    # The count clamps at 0 (never negative) even when released with nothing left to release.
+    assert _ref_count(db, asset.id) == 0
+    assert db.get(MediaAsset, asset.id) is not None
+
+
+def test_errored_asset_is_not_deleted_at_zero(db):
+    u = _user(db)
+    asset = _holding_asset(db, status="error", ref_count=1)
+    j = _job(db, u.id, asset.id, status="error")
+    db.commit()
+
+    service.release_asset(db, j)
+    db.commit()
+    # An errored row is cheap (no file) and may be re-enqueued → never deleted here, only ready is.
+    assert _ref_count(db, asset.id) == 0
+    assert db.get(MediaAsset, asset.id) is not None

backend/tests/test_download_transitions.py

@@ -0,0 +1,86 @@
+"""E3 S2 characterization net — download job state transitions (B-sub-8).
+
+pause / resume / cancel / delete / redownload currently live inline in the routes, each hand-rolling
+the asset bookkeeping (release, error-reset, fresh-asset rebuild). This pins their OBSERVABLE HTTP
+behaviour so the logic can move into `service.*` methods (with the routes as thin callers) provably
+behaviour-equivalently. DB lane (`db` fixture) — skipped without Postgres.
+"""
+from sqlalchemy import text
+
+from app.models import DownloadJob, MediaAsset, User
+
+
+def _seed(db, *, job_status, asset_status="ready", ref_count=1, rel_path=None):
+    u = User(email=f"tr-{job_status}-{asset_status}-{ref_count}@example.com")
+    db.add(u)
+    db.flush()
+    asset = MediaAsset(
+        source_kind="url", source_ref=f"r-{u.id}", format_sig="s",
+        status=asset_status, ref_count=ref_count, rel_path=rel_path,
+    )
+    db.add(asset)
+    db.flush()
+    job = DownloadJob(
+        user_id=u.id, asset_id=asset.id, source_kind="url", source_ref="r",
+        status=job_status, queue_pos=1, profile_snapshot={},
+    )
+    db.add(job)
+    db.commit()
+    return u, asset, job
+
+
+def _ref(db, asset_id):
+    return db.execute(
+        text("SELECT ref_count FROM media_assets WHERE id = :id"), {"id": asset_id}
+    ).scalar()
+
+
+def test_cancel_releases_the_asset_and_marks_canceled(db, as_user):
+    u, asset, job = _seed(db, job_status="running", asset_status="downloading", ref_count=1)
+    r = as_user(u).post(f"/api/downloads/{job.id}/cancel")
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    assert db.get(DownloadJob, job.id).status == "canceled"
+    assert _ref(db, asset.id) == 0  # the hold was released while the job still counted as holding
+
+
+def test_pause_then_resume_round_trips_queued(db, as_user):
+    u, asset, job = _seed(db, job_status="queued", asset_status="pending")
+    c = as_user(u)
+    assert c.post(f"/api/downloads/{job.id}/pause").status_code == 200
+    db.expire_all()
+    assert db.get(DownloadJob, job.id).status == "paused"
+    assert c.post(f"/api/downloads/{job.id}/resume").status_code == 200
+    db.expire_all()
+    assert db.get(DownloadJob, job.id).status == "queued"
+
+
+def test_resume_resets_a_failed_shared_asset_to_pending(db, as_user):
+    u, asset, job = _seed(db, job_status="error", asset_status="error")
+    r = as_user(u).post(f"/api/downloads/{job.id}/resume")
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    assert db.get(DownloadJob, job.id).status == "queued"
+    # A failed shared asset is reset so the worker actually re-downloads instead of re-inheriting the error.
+    assert db.get(MediaAsset, asset.id).status == "pending"
+
+
+def test_delete_releases_and_removes_the_job(db, as_user):
+    u, asset, job = _seed(db, job_status="done", asset_status="ready", ref_count=1, rel_path=None)
+    r = as_user(u).request("DELETE", f"/api/downloads/{job.id}")
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    assert db.get(DownloadJob, job.id) is None
+    assert db.get(MediaAsset, asset.id) is None  # last holder gone → ready asset freed
+
+
+def test_redownload_rebuilds_into_a_fresh_asset(db, as_user):
+    u, asset, job = _seed(db, job_status="done", asset_status="ready", ref_count=1, rel_path=None)
+    old_asset_id = asset.id
+    r = as_user(u).post(f"/api/downloads/{job.id}/redownload")
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    j = db.get(DownloadJob, job.id)
+    assert j.status == "queued"
+    assert j.progress == 0
+    assert j.asset_id != old_asset_id  # a fresh private asset, not the old (now-released) one

backend/tests/test_ffmpeg_callbacks.py

@@ -4,20 +4,20 @@ run_ffmpeg calls `on_progress`/`should_cancel` once per ffmpeg output LINE (`-pr
 emits a block of ~12 every half second), and both callbacks are database round-trips: `_set_job`
 opens a session and UPDATE+COMMITs, `_job_status` opens one and SELECTs. Unthrottled, a long
 re-encode is ~10^5 of each."""
-from app import worker
+import app.downloads.pipeline as pipeline
 
 
 class TestProgressWriter:
     def test_writes_the_first_value_immediately(self, monkeypatch):
         writes = []
-        monkeypatch.setattr(worker, "_set_job", lambda job_id, **f: writes.append(f))
-        worker._ffmpeg_progress_writer(7, "processing")(12.4)
+        monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **f: writes.append(f))
+        pipeline._ffmpeg_progress_writer(7, "processing")(12.4)
         assert writes == [{"progress": 12, "phase": "processing"}]
 
     def test_a_burst_of_lines_costs_one_write(self, monkeypatch):
         writes = []
-        monkeypatch.setattr(worker, "_set_job", lambda job_id, **f: writes.append(f))
-        write = worker._ffmpeg_progress_writer(7, "processing")
+        monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **f: writes.append(f))
+        write = pipeline._ffmpeg_progress_writer(7, "processing")
         for pct in range(0, 40):  # what one second of ffmpeg output looks like
             write(pct)
         assert len(writes) == 1
@@ -25,8 +25,8 @@ class TestProgressWriter:
     def test_the_next_second_writes_again(self, monkeypatch):
         writes = []
         clock = {"t": 1000.0}
-        monkeypatch.setattr(worker, "_set_job", lambda job_id, **f: writes.append(f))
-        write = worker._ffmpeg_progress_writer(7, "processing", now=lambda: clock["t"])
+        monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **f: writes.append(f))
+        write = pipeline._ffmpeg_progress_writer(7, "processing", now=lambda: clock["t"])
         write(10)
         write(11)  # same second → dropped
         clock["t"] += 1.5
@@ -36,19 +36,51 @@ class TestProgressWriter:
     def test_an_unchanged_whole_percent_is_never_rewritten(self, monkeypatch):
         writes = []
         clock = {"t": 1000.0}
-        monkeypatch.setattr(worker, "_set_job", lambda job_id, **f: writes.append(f))
-        write = worker._ffmpeg_progress_writer(7, "processing", now=lambda: clock["t"])
+        monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **f: writes.append(f))
+        monkeypatch.setattr(pipeline, "_touch_job", lambda job_id: None)
+        write = pipeline._ffmpeg_progress_writer(7, "processing", now=lambda: clock["t"])
         write(10.1)
         clock["t"] += 5
         write(10.9)  # still 10% — the bar shows integers
         assert [w["progress"] for w in writes] == [10]
 
+    def test_a_slow_same_percent_stretch_heartbeats_the_watchdog(self, monkeypatch):
+        """B-sub-4: a re-encode that sits on one integer percent for minutes must NOT be killed as
+        stalled — the writer touches the row (heartbeat) each interval even though it writes no new
+        progress, so the stall watchdog keeps seeing a fresh updated_at."""
+        writes, touches = [], []
+        clock = {"t": 1000.0}
+        monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **f: writes.append(f))
+        monkeypatch.setattr(pipeline, "_touch_job", lambda job_id: touches.append(job_id))
+        write = pipeline._ffmpeg_progress_writer(7, "processing", now=lambda: clock["t"])
+        write(10.1)  # first real write → progress 10
+        for _ in range(3):  # three more intervals stuck on 10%
+            clock["t"] += 1.5
+            write(10.4)
+        assert [w["progress"] for w in writes] == [10]  # progress never rewritten
+        assert touches == [7, 7, 7]  # …but each interval heartbeats the watchdog
+
+    def test_no_redundant_heartbeat_when_a_progress_write_just_happened(self, monkeypatch):
+        """A progress write already refreshes updated_at, so an unchanged-percent call in the SAME
+        interval must NOT also heartbeat — that would be a redundant DB write (~2x during an active
+        download). The heartbeat only fires once no write/beat touched the row for an interval."""
+        writes, touches = [], []
+        clock = {"t": 1000.0}
+        monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **f: writes.append(f))
+        monkeypatch.setattr(pipeline, "_touch_job", lambda job_id: touches.append(job_id))
+        write = pipeline._ffmpeg_progress_writer(7, "processing", now=lambda: clock["t"])
+        write(6.0)  # progress write at t=1000 → refreshes updated_at
+        clock["t"] += 0.3
+        write(6.4)  # same percent, SAME interval → the write already touched the row, so no heartbeat
+        assert [w["progress"] for w in writes] == [6]
+        assert touches == []
+
 
 class TestCancelPoll:
     def test_asks_once_then_reuses_the_answer(self, monkeypatch):
         calls = []
-        monkeypatch.setattr(worker, "_job_status", lambda job_id: calls.append(job_id) or "running")
-        cancelled = worker._cancel_poll(7)
+        monkeypatch.setattr(pipeline, "_job_status", lambda job_id: calls.append(job_id) or "running")
+        cancelled = pipeline._cancel_poll(7)
         for _ in range(40):
             assert cancelled() is False
         assert len(calls) == 1
@@ -61,8 +93,8 @@ class TestCancelPoll:
             calls.append(job_id)
             return "cancelled"
 
-        monkeypatch.setattr(worker, "_job_status", status)
-        cancelled = worker._cancel_poll(7, now=lambda: clock["t"])
+        monkeypatch.setattr(pipeline, "_job_status", status)
+        cancelled = pipeline._cancel_poll(7, now=lambda: clock["t"])
         assert cancelled() is True
         clock["t"] += 60  # cancellation is sticky: no re-query, and no un-cancelling
         assert cancelled() is True
@@ -71,8 +103,8 @@ class TestCancelPoll:
     def test_a_cancel_arriving_later_is_seen_on_the_next_tick(self, monkeypatch):
         clock = {"t": 1000.0}
         state = {"status": "running"}
-        monkeypatch.setattr(worker, "_job_status", lambda job_id: state["status"])
-        cancelled = worker._cancel_poll(7, now=lambda: clock["t"])
+        monkeypatch.setattr(pipeline, "_job_status", lambda job_id: state["status"])
+        cancelled = pipeline._cancel_poll(7, now=lambda: clock["t"])
         assert cancelled() is False
         state["status"] = "paused"
         clock["t"] += 1.0  # the throttle must not delay a cancel by more than its interval

backend/tests/test_plex_detail.py

@@ -13,7 +13,7 @@ from types import SimpleNamespace
 
 from fastapi.testclient import TestClient
 
-import app.routes.plex.browse as browse
+import app.routes.plex.detail as detail_routes
 import app.state as state
 from app.auth import current_user
 from app.db import get_db
@@ -216,7 +216,7 @@ def test_item_detail_movie_serves_from_row_despite_live_outage(db, monkeypatch):
     user = _user(db, "outage@example.com")
     db.commit()
 
-    monkeypatch.setattr(browse, "PlexClient", lambda db: _StubClient(raise_=True))
+    monkeypatch.setattr(detail_routes, "PlexClient", lambda db: _StubClient(raise_=True))
     resp = _get(db, user, monkeypatch, "/api/plex/item/m1")
     assert resp.status_code == 200, resp.text
     body = resp.json()
@@ -240,7 +240,7 @@ def test_item_detail_movie_lazy_fills_when_unenriched(db, monkeypatch):
     user = _user(db, "lazy@example.com")
     db.commit()
 
-    monkeypatch.setattr(browse, "PlexClient", lambda db: _StubClient(RICH_META))
+    monkeypatch.setattr(detail_routes, "PlexClient", lambda db: _StubClient(RICH_META))
     resp = _get(db, user, monkeypatch, "/api/plex/item/m1")
     assert resp.status_code == 200, resp.text
     assert resp.json()["imdb_id"] == "tt0068646"
@@ -267,7 +267,7 @@ def test_item_detail_episode_serves_rich_from_live(db, monkeypatch):
     user = _user(db, "ep@example.com")
     db.commit()
 
-    monkeypatch.setattr(browse, "PlexClient", lambda db: _StubClient(RICH_META))
+    monkeypatch.setattr(detail_routes, "PlexClient", lambda db: _StubClient(RICH_META))
     resp = _get(db, user, monkeypatch, "/api/plex/item/e1")
     assert resp.status_code == 200, resp.text
     body = resp.json()
@@ -298,7 +298,7 @@ def test_show_detail_serves_from_row(db, monkeypatch):
     def _boom(db_, rk):
         raise RuntimeError("plex unavailable")
 
-    monkeypatch.setattr(browse, "_show_enrichment", _boom)
+    monkeypatch.setattr(detail_routes, "_show_enrichment", _boom)
     resp = _get(db, user, monkeypatch, "/api/plex/show/s1")
     assert resp.status_code == 200, resp.text
     body = resp.json()["show"]
@@ -322,7 +322,7 @@ def test_show_detail_lazy_fills_when_unenriched(db, monkeypatch):
     db.commit()
 
     # _show_enrichment returns (live meta, no related) → the fill branch runs.
-    monkeypatch.setattr(browse, "_show_enrichment", lambda db_, rk: (RICH_META, []))
+    monkeypatch.setattr(detail_routes, "_show_enrichment", lambda db_, rk: (RICH_META, []))
     resp = _get(db, user, monkeypatch, "/api/plex/show/s1")
     assert resp.status_code == 200, resp.text
     assert resp.json()["show"]["imdb_id"] == "tt0068646"

backend/tests/test_plex_facets_grid_parity.py

@@ -0,0 +1,148 @@
+"""E3 S1 characterization net — /library grid ⟷ /facets equivalence across watch-state scopes.
+
+The Plex god-module split (E3 S1) extracts the movie/show watch-state predicates that are currently
+HAND-DUPLICATED in two places: `unified_library` (the grid) and `facets.wsq` (the facet aggregation).
+The facets docstring promises the facets "reflect exactly the visible result set" — i.e. with NO
+metadata filter active (only the `show` state scope), a facet's genre/content-rating counts must equal
+the same counts taken over the grid's visible items. This test derives the expected counts from the
+grid's own response and asserts /facets matches, for every (scope × show) combination.
+
+That locks the two duplicated branch-sets together BEFORE they are merged into one shared helper, so
+the merge is provably behaviour-equivalent (and a drift between grid and facets fails here). DB lane
+(`db` fixture) — skipped without Postgres.
+"""
+from collections import Counter
+from datetime import datetime, timezone
+
+from app.models import (
+    PlexItem,
+    PlexLibrary,
+    PlexSeason,
+    PlexShow,
+    PlexState,
+    User,
+)
+
+SCOPES = ("movie", "show", "both")
+SHOW_STATES = ("all", "watched", "in_progress", "unwatched")
+
+
+def _dt(day: int) -> datetime:
+    return datetime(2024, 1, day, tzinfo=timezone.utc)
+
+
+def _seed(db):
+    """A corpus spanning every watch state, each row carrying genres + a content rating so the facet
+    aggregation has something to count. Movies get per-item state directly; a show's state is the
+    rollup of its episodes' state (new / in-progress / fully watched)."""
+    movie_lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
+    show_lib = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
+    db.add_all([movie_lib, show_lib])
+    db.flush()
+
+    # Movies — new / in-progress / watched / hidden, with overlapping genres.
+    m_new = PlexItem(rating_key="m_new", library_id=movie_lib.id, kind="movie", title="New Movie",
+                     year=2001, added_at=_dt(1), genres=["Drama"], content_rating="PG")
+    m_prog = PlexItem(rating_key="m_prog", library_id=movie_lib.id, kind="movie", title="Prog Movie",
+                      year=2002, added_at=_dt(2), genres=["Drama", "Comedy"], content_rating="PG-13")
+    m_watch = PlexItem(rating_key="m_watch", library_id=movie_lib.id, kind="movie", title="Watched Movie",
+                       year=2003, added_at=_dt(3), genres=["Comedy"], content_rating="R")
+    m_hidden = PlexItem(rating_key="m_hidden", library_id=movie_lib.id, kind="movie", title="Hidden Movie",
+                        year=2004, added_at=_dt(4), genres=["Drama"], content_rating="R")
+
+    # Shows — new / in-progress / watched (state comes from the episodes below).
+    sh_new = PlexShow(rating_key="sh_new", library_id=show_lib.id, title="New Show", year=2005,
+                      added_at=_dt(5), child_count=1, genres=["Sci-Fi"], content_rating="TV-14")
+    sh_prog = PlexShow(rating_key="sh_prog", library_id=show_lib.id, title="Prog Show", year=2006,
+                       added_at=_dt(6), child_count=1, genres=["Sci-Fi", "Drama"], content_rating="TV-MA")
+    sh_watch = PlexShow(rating_key="sh_watch", library_id=show_lib.id, title="Watched Show", year=2007,
+                        added_at=_dt(7), child_count=1, genres=["Drama"], content_rating="TV-PG")
+    db.add_all([m_new, m_prog, m_watch, m_hidden, sh_new, sh_prog, sh_watch])
+    db.flush()
+
+    def _episode(rk, show, day, n):
+        se = PlexSeason(rating_key=f"se_{rk}", show_id=show.id, season_number=1)
+        db.add(se)
+        db.flush()
+        ep = PlexItem(rating_key=rk, library_id=show_lib.id, kind="episode", title=f"Ep {rk}",
+                      show_id=show.id, season_id=se.id, season_number=1, episode_number=n, added_at=_dt(day))
+        db.add(ep)
+        db.flush()
+        return ep
+
+    _episode("e_new", sh_new, 8, 1)  # no state -> show "new"
+    e_prog = _episode("e_prog", sh_prog, 9, 1)  # in-progress episode -> show "in_progress"
+    _episode("e_prog2", sh_prog, 10, 2)  # second episode, untouched (so watched < total)
+    e_watch = _episode("e_watch", sh_watch, 11, 1)  # watched episode -> show "watched"
+
+    user = User(email="plexparity@example.com")
+    db.add(user)
+    db.flush()
+    db.add_all([
+        PlexState(user_id=user.id, item_id=m_prog.id, position_seconds=120),
+        PlexState(user_id=user.id, item_id=m_watch.id, status="watched"),
+        PlexState(user_id=user.id, item_id=m_hidden.id, status="hidden"),
+        PlexState(user_id=user.id, item_id=e_prog.id, position_seconds=90),
+        PlexState(user_id=user.id, item_id=e_watch.id, status="watched"),
+    ])
+    db.commit()
+    # rating_key -> (table, genres, content_rating) so the test can total the grid's visible set.
+    meta = {}
+    for it in (m_new, m_prog, m_watch, m_hidden):
+        meta[it.rating_key] = (it.genres, it.content_rating)
+    for sh in (sh_new, sh_prog, sh_watch):
+        meta[sh.rating_key] = (sh.genres, sh.content_rating)
+    return {"user": user, "meta": meta}
+
+
+def _facet_counts(body):
+    genres = {g["value"]: g["count"] for g in body["genres"]}
+    crs = {c["value"]: c["count"] for c in body["content_ratings"]}
+    return genres, crs
+
+
+def test_facets_mirror_the_grid_visible_set(db, as_user):
+    """For every (scope, show) the facet genre / content-rating counts equal the counts over the
+    grid's visible items — the invariant the extracted shared watch-state predicate must preserve."""
+    seed = _seed(db)
+    meta = seed["meta"]
+    client = as_user(seed["user"])
+
+    for scope in SCOPES:
+        for show in SHOW_STATES:
+            grid = client.get(f"/api/plex/library?scope={scope}&show={show}&limit=100")
+            assert grid.status_code == 200, grid.text
+            ids = [i["id"] for i in grid.json()["items"]]
+
+            exp_genres: Counter = Counter()
+            exp_crs: Counter = Counter()
+            for rk in ids:
+                genres, cr = meta[rk]
+                for g in genres or []:
+                    exp_genres[g] += 1
+                if cr is not None:
+                    exp_crs[cr] += 1
+
+            facets = client.get(f"/api/plex/facets?scope={scope}&show={show}")
+            assert facets.status_code == 200, facets.text
+            got_genres, got_crs = _facet_counts(facets.json())
+
+            ctx = f"scope={scope} show={show} grid_ids={ids}"
+            assert got_genres == dict(exp_genres), f"genre facet != grid ({ctx})"
+            assert got_crs == dict(exp_crs), f"content-rating facet != grid ({ctx})"
+
+
+def test_hidden_movie_absent_from_grid_and_facets(db, as_user):
+    """A hidden movie must not appear in the default grid NOR inflate its facets — the single most
+    important equivalence between the two duplicated 'exclude hidden' branches."""
+    seed = _seed(db)
+    client = as_user(seed["user"])
+
+    grid = client.get("/api/plex/library?scope=movie&show=all&limit=100")
+    assert "m_hidden" not in [i["id"] for i in grid.json()["items"]]
+
+    # Only the hidden movie carries content_rating "R" among the *visible* movies? No — m_watch is
+    # also "R". The hidden one must not add to the "R" count: visible R = {m_watch} = 1.
+    facets = client.get("/api/plex/facets?scope=movie&show=all")
+    crs = {c["value"]: c["count"] for c in facets.json()["content_ratings"]}
+    assert crs.get("R") == 1, crs

backend/tests/test_plex_overlay.py

@@ -11,7 +11,7 @@ import app.plex.sync as syncmod
 from app import sysconfig
 from app.models import PlexItem, PlexLibrary
 from app.plex.sync import _apply_item, _canonical_guid, _sync_overlay
-from app.routes.plex.browse import _content_libs
+from app.routes.plex._shared import _content_libs
 
 
 class _StubPlex:

backend/tests/test_plex_person.py

@@ -19,7 +19,7 @@ from app.models import (
     PlexState,
     User,
 )
-from app.routes.plex.browse import _content_libs
+from app.routes.plex._shared import _content_libs
 
 NAME = "Robert De Niro"
 

backend/tests/test_plex_query_conversion.py

@@ -149,7 +149,9 @@ class TestRollup:
 
 
 def _fake_paginate(by_type):
-    def _p(_plex, _key, typ):
+    def _p(_plex, _key, typ, progress=None):
+        if progress is not None:
+            progress["complete"] = True  # the fake returns the whole list — a complete listing
         return list(by_type.get(typ, []))
     return _p
 

backend/tests/test_plex_sync_prune.py

@@ -0,0 +1,162 @@
+"""E3 S1 catalog-prune (B-sub-6): a sync now deletes rows for titles that disappeared from Plex,
+diffing each section's freshly-listed rating_keys against the stored ones. The prune is GUARDED on a
+non-empty listing so a transient Plex glitch (an empty page) can never wipe a whole library.
+
+DB lane (`db` fixture) — skipped without Postgres. Drives `_sync_movies` / `_sync_shows` directly
+with a stub Plex (the same seam the overlay test uses), so no live server is needed.
+"""
+from sqlalchemy import select
+
+from app.models import (
+    PlexItem,
+    PlexLibrary,
+    PlexSeason,
+    PlexShow,
+    PlexState,
+    User,
+)
+from app.plex.sync import (
+    _TYPE_EPISODE,
+    _TYPE_MOVIE,
+    _TYPE_SEASON,
+    _TYPE_SHOW,
+    _sync_movies,
+    _sync_shows,
+)
+
+
+class _StubPlex:
+    """Returns canned section listings per Plex item type — the current 'truth' the sync reconciles
+    the stored rows against. `_paginate` is the only method the sync calls on it."""
+
+    def __init__(self, by_type: dict[int, list[dict]]):
+        self.by_type = by_type
+
+    def section_items(self, key, item_type, start, size):
+        items = self.by_type.get(item_type, [])
+        return items[start : start + size], len(items)
+
+
+class _TruncatedPlex:
+    """Simulates a truncated listing (a transient Plex hiccup): it reports a large `total` but returns
+    only one short page, so pagination stops BEFORE reaching total. Prune must treat this as an
+    incomplete fetch and delete nothing — the un-listed rows are present, just not mentioned."""
+
+    def __init__(self, items: list[dict], claimed_total: int):
+        self._items = items
+        self._total = claimed_total
+
+    def section_items(self, key, item_type, start, size):
+        if start == 0:
+            return list(self._items), self._total  # one short page, total says there's more
+        return [], self._total
+
+
+def _seed_movies(db):
+    lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
+    db.add(lib)
+    db.flush()
+    m1 = PlexItem(rating_key="m1", library_id=lib.id, kind="movie", title="Kept Movie")
+    m2 = PlexItem(rating_key="m2", library_id=lib.id, kind="movie", title="Gone Movie")
+    db.add_all([m1, m2])
+    user = User(email="prune@example.com")
+    db.add(user)
+    db.flush()
+    # A watch-state on EACH movie: the kept one's must survive, the pruned one's must cascade away.
+    db.add_all([
+        PlexState(user_id=user.id, item_id=m1.id, status="watched"),
+        PlexState(user_id=user.id, item_id=m2.id, status="watched"),
+    ])
+    db.commit()
+    return lib, m1.id, m2.id, user.id
+
+
+def test_movie_prune_deletes_disappeared_titles_and_cascades_state(db):
+    lib, m1_id, _m2_id, _user_id = _seed_movies(db)
+    # Plex now lists only m1 — m2 disappeared.
+    stub = _StubPlex({_TYPE_MOVIE: [{"ratingKey": "m1", "title": "Kept Movie"}]})
+    stats = {"movies": 0, "pruned": 0}
+
+    _sync_movies(db, stub, lib, stats)
+    db.commit()
+
+    live = {r.rating_key for r in db.execute(select(PlexItem)).scalars()}
+    assert live == {"m1"}, "the disappeared movie must be pruned"
+    assert stats["pruned"] == 1
+    # The kept movie's watch-state survives; the pruned movie's cascaded away.
+    states = {s.item_id for s in db.execute(select(PlexState)).scalars()}
+    assert states == {m1_id}
+
+
+def test_empty_listing_prunes_nothing(db):
+    """The safety guard: a section that lists ZERO items (a real empty library, or a transient Plex
+    glitch we can't distinguish) must leave the stored rows untouched — never mass-delete."""
+    lib, _m1_id, _m2_id, _user_id = _seed_movies(db)
+    stub = _StubPlex({_TYPE_MOVIE: []})  # Plex returned nothing this run
+    stats = {"movies": 0, "pruned": 0}
+
+    _sync_movies(db, stub, lib, stats)
+    db.commit()
+
+    live = {r.rating_key for r in db.execute(select(PlexItem)).scalars()}
+    assert live == {"m1", "m2"}, "an empty listing must not prune anything"
+    assert stats["pruned"] == 0
+
+
+def test_truncated_listing_prunes_nothing(db):
+    """A partial fetch (Plex reports 5 items but the listing returns only 1 then stops) must NOT prune
+    the 4 it failed to mention — the completeness guard, not just the empty-listing guard."""
+    lib, m1_id, _m2_id, _user_id = _seed_movies(db)
+    # Only m1 is listed, but total claims 5 → the walk never reaches total → incomplete → no prune.
+    stub = _TruncatedPlex([{"ratingKey": "m1", "title": "Kept Movie"}], claimed_total=5)
+    stats = {"movies": 0, "pruned": 0}
+
+    _sync_movies(db, stub, lib, stats)
+    db.commit()
+
+    live = {r.rating_key for r in db.execute(select(PlexItem)).scalars()}
+    assert live == {"m1", "m2"}, "a truncated listing must not prune the un-listed rows"
+    assert stats["pruned"] == 0
+
+
+def test_show_prune_removes_stale_show_season_and_episode(db):
+    lib = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
+    db.add(lib)
+    db.flush()
+    # Stored: one kept show (s1 -> se1 -> e1) and one stale show (s2 -> se2 -> e2) that Plex dropped.
+    s1 = PlexShow(rating_key="s1", library_id=lib.id, title="Kept Show")
+    s2 = PlexShow(rating_key="s2", library_id=lib.id, title="Gone Show")
+    db.add_all([s1, s2])
+    db.flush()
+    se1 = PlexSeason(rating_key="se1", show_id=s1.id, season_number=1)
+    se2 = PlexSeason(rating_key="se2", show_id=s2.id, season_number=1)
+    db.add_all([se1, se2])
+    db.flush()
+    e1 = PlexItem(rating_key="e1", library_id=lib.id, kind="episode", title="Kept Ep",
+                  show_id=s1.id, season_id=se1.id, season_number=1, episode_number=1)
+    e2 = PlexItem(rating_key="e2", library_id=lib.id, kind="episode", title="Gone Ep",
+                  show_id=s2.id, season_id=se2.id, season_number=1, episode_number=1)
+    db.add_all([e1, e2])
+    db.commit()
+
+    # Plex now lists only s1's subtree.
+    stub = _StubPlex({
+        _TYPE_SHOW: [{"ratingKey": "s1", "title": "Kept Show"}],
+        _TYPE_SEASON: [{"ratingKey": "se1", "parentRatingKey": "s1", "index": 1, "title": "Season 1"}],
+        _TYPE_EPISODE: [{
+            "ratingKey": "e1", "grandparentRatingKey": "s1", "parentRatingKey": "se1",
+            "parentIndex": 1, "index": 1, "title": "Kept Ep",
+        }],
+    })
+    stats = {"shows": 0, "seasons": 0, "episodes": 0, "pruned": 0}
+
+    _sync_shows(db, stub, lib, stats)
+    db.commit()
+
+    shows = {r.rating_key for r in db.execute(select(PlexShow)).scalars()}
+    seasons = {r.rating_key for r in db.execute(select(PlexSeason)).scalars()}
+    eps = {r.rating_key for r in db.execute(select(PlexItem)).scalars()}
+    assert shows == {"s1"}, "the stale show must be pruned"
+    assert seasons == {"se1"}, "the stale season must be pruned"
+    assert eps == {"e1"}, "the stale episode must be pruned"
+    assert stats["pruned"] == 3  # s2 + se2 + e2

docs/development.md

@@ -0,0 +1,61 @@
+# Development
+
+How to run Siftlode locally and pass the gate. Conventions for the backend code itself live in
+[`backend/app/README.md`](../backend/app/README.md).
+
+## Run it locally
+
+The whole stack (Postgres + API + worker) comes up from the localdev compose file:
+
+```bash
+docker compose -f docker-compose.localdev.yml up --build
+```
+
+Browse at **http://localhost:8080** (Google login works — the OAuth redirect is
+`http://localhost:8080/auth/callback`). The API runs `alembic upgrade head` on startup, so the schema
+is always current.
+
+For a fast frontend edit loop, run the Vite dev server with HMR against that API:
+
+```bash
+cd frontend && npm install && npm run dev   # http://localhost:5173, proxying the API on :8080
+```
+
+## The gate
+
+A change is done when all of this is green. CI runs the same checks server-side
+(`.gitea/workflows/check.yml`), so "main is always green".
+
+**Backend**
+
+```bash
+# lint
+python -m ruff check --config backend/ruff.toml backend scripts
+
+# tests — pure-logic + the DB-backed integration lane, against an ephemeral Postgres.
+# The backend tree is bind-mounted, so it always tests your working files.
+docker compose -f backend/docker-compose.test.yml up --build --abort-on-container-exit --exit-code-from test
+docker compose -f backend/docker-compose.test.yml down -v
+```
+
+The test compose sets `SIFTLODE_TEST_REQUIRE_DB=1`, which turns a would-be silent skip (no DB, or no
+ffmpeg) into a hard failure — so the gate can't print green while the DB-backed tests were quietly not
+run. Tests that need a database take the `db` fixture (`backend/tests/conftest.py`); the rest are
+pure-logic and run without one.
+
+**Frontend** (from `frontend/`)
+
+```bash
+npm run typecheck     # tsc x3 (app / node / e2e)
+npm run lint          # eslint
+npm run format:check  # prettier
+npm run knip          # unused exports/deps
+npm run test          # vitest
+```
+
+## Branching
+
+Trunk-based: branch off `main`, keep the branch gate-green, and land the whole change back to `main`
+in one reviewed step. Migrations are Alembic revisions under `backend/alembic/versions`; the API
+applies them on startup, and the DB-lane rebuilds the schema via `alembic upgrade head`, so a broken
+migration chain fails the tests.

frontend/src/lib/releaseNotes.ts

@@ -14,6 +14,19 @@ export interface ReleaseEntry {
 }
 
 export const RELEASE_NOTES: ReleaseEntry[] = [
+  {
+    version: "0.69.0",
+    date: "2026-08-19",
+    summary:
+      "Mostly under-the-hood consolidation of the backend, with a few reliability wins you'll notice in Plex and downloads.",
+    fixes: [
+      "Movies and shows removed from Plex now disappear from your library on the next sync, instead of lingering and failing to play. A safety guard skips the cleanup whenever Plex returns an incomplete listing, so a temporary hiccup can never wipe a library.",
+      "Downloads are more reliable: a slow re-encode is no longer mistaken for a stalled download and stopped; queuing a second copy of the same video while the first is still downloading no longer blocks the queue behind it; and canceling or deleting a download can no longer remove a file another download still shares, even when several run at once.",
+    ],
+    chores: [
+      "A large internal refactor with no behavior change: the Plex browse and download-worker god-modules were split into focused modules, download state transitions and file bookkeeping were centralized, five unused endpoints were removed, admin config settings were unified into one store, and the request contracts and module docs were tidied — all under new automated tests and four clean code-review passes.",
+    ],
+  },
   {
     version: "0.68.0",
     date: "2026-08-18",