release: 0.118.0 (improvement/e59-small-residuals)

`draw.recency` weighs every watch of both affinities: an undated watch
and a future one now have a test (deleting either branch fails it). The
over-the-picture contrast test moves below the form-on-chrome test, so
each sits under the comment that explains it.

.env.example

@@ -68,9 +68,11 @@ SMTP_FROM=
 
 # --- Scheduler ---
 # The background scheduler (subscription sync, backfill, enrichment) runs inside the app.
-# DATABASE_URL is set for you by docker-compose to the bundled `db` service. If you ever run
-# more than one instance against the same database, keep SCHEDULER_ENABLED=true on exactly one
-# of them and false on the rest, to avoid double quota use and write races.
+# DATABASE_URL is set for you by docker-compose to the bundled `db` service. Run ONE app instance
+# per database: a starting instance closes every run still open as interrupted (it assumes the
+# previous process is gone), and "is this job already running" is held in the process. A second
+# instance would cut the first one's runs short and could run the same job twice.
+# SCHEDULER_ENABLED=false keeps the recurring jobs off; "run now" still works.
 SCHEDULER_ENABLED=true
 
 # --- Files library & downloads ---

.gitleaksignore

@@ -0,0 +1,8 @@
+# gitleaks false positives, one fingerprint per finding (commit:file:rule:line). Each says why it is no secret.
+#
+# YouTube's public WEB InnerTube key: embedded in every youtube.com results page, read live and cached;
+# this constant is only the fallback (backend/app/youtube/innertube.py).
+8b37c470ba5d11c45757f0b2e62d660b782e816d:backend/app/youtube/innertube.py:gcp-api-key:23
+# A recorded watch-next page's continuation cursors (continuationCommand.token): page pointers, not credentials.
+8b37c470ba5d11c45757f0b2e62d660b782e816d:backend/tests/fixtures/innertube_next_first.json:generic-api-key:768
+8b37c470ba5d11c45757f0b2e62d660b782e816d:backend/tests/fixtures/innertube_next_first.json:generic-api-key:792

CHANGELOG.md

@@ -7,6 +7,249 @@ edit the release notes and re-run the generator.
 
 Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
 
+## 0.118.0 — 2026-11-08
+
+### New
+
+- Surprise me now leans on the channels and genres you watch, and one pick in three stays a blind draw, so it remains a surprise.
+- On a connection too slow for a Plex video, playback waits and goes on where it was, and says how fast the video needs and how fast it arrives.
+
+### Fixed
+
+- A 24/7 stream's archive no longer adds years to a list's total: a total names the videos it could not count as "+N without length". The Feed sorted by Surprise me opens faster.
+- Plex films the server's GPU cannot decode (VC-1, older DivX or XviD) start in a few seconds instead of about 20. Text over the Plex picture is readable on any frame.
+
+## 0.117.0 — 2026-11-06
+
+### New
+
+- A share link's address now says what it opens: /file/ for a file, /folder/ for a folder, /watch/ for a video. Links you already sent keep working.
+- The upload dialog shows which folder the file goes to, and you can pick another one or make a new folder there. Opening Share copies a link at once.
+- The Channels tag chips show how many followed channels each holds. Continue watching in Plex has a See all, and a Plex item's Info shows the file size.
+
+### Fixed
+
+- An expired share link is marked as expired and is never copied. A crop of a video with wide pixels keeps exactly what its frame showed.
+- A Plex auto-skip waits 3 seconds into the marker before its countdown starts.
+
+## 0.116.3 — 2026-11-04
+
+### Fixed
+
+- If a stylesheet is lost on the way, the app now asks for it again, so a poor connection no longer leaves the page unstyled until a reload.
+
+## 0.116.2 — 2026-11-03
+
+### Fixed
+
+- A Plex video paused or muted in a background tab no longer loses its session after a minute, and if a session is lost anyway, playback comes back at once instead of after half a minute.
+
+## 0.116.1 — 2026-11-03
+
+### Fixed
+
+- A playlist now shows the order the server holds after a revert to YouTube, a sync or a change on another device, and the next drag no longer brings back the old one.
+- An Undo on a removed or moved row puts it back in the right place while another change is still saving.
+
+## 0.116.0 — 2026-11-03
+
+### New
+
+- In a playlist, tick rows with their box, Shift+click or x, then move or remove them together. An Undo puts them back at their places, and a single row's move has one too.
+- Drag rows onto another list in the rail: one row, or all the selected ones, also under a sort or a filter.
+
+### Fixed
+
+- Removing a video from inside the player no longer loses your place in the list when the player closes.
+
+## 0.115.0 — 2026-11-01
+
+### New
+
+- Clear watch progress is on every YouTube video's menu, on Home, in playlists and in the player's related videos. It forgets where you left a started video and changes nothing else.
+- Channels have an Added column, and an About card opens only when the pointer rests on it.
+- In Files, a press anywhere on a row plays the file.
+- The feed's toolbar pills are named by a glyph, with Sort and View on the filters' row.
+
+### Fixed
+
+- In a play queue, Previous and Next pass over a video that can't play, in the direction you were going, and say why when only such videos are left.
+- A Facebook download titled only “Facebook” now takes the post's text as its name.
+- The player's big view keeps its way out in sight on a tablet, and Tab no longer lands on controls hidden under it.
+
+## 0.114.3 — 2026-11-01
+
+### Fixed
+
+- Editing a file that is neither video nor audio now says why it can't be done, instead of a generic error.
+
+## 0.114.2 — 2026-10-30
+
+### Fixed
+
+- Security: a library the sign-in relies on is updated, closing two published vulnerabilities.
+
+## 0.114.1 — 2026-10-30
+
+### Fixed
+
+- Notifications follow a language switch: a notice on screen and every row in the notification center are said again in the new language, their buttons too.
+
+## 0.114.0 — 2026-10-29
+
+### New
+
+- Notifications: what asks for you comes first, then everything else as one stream, newest first. Read notices older than a week fold under Earlier, and Back returns to the row you left.
+
+### Fixed
+
+- Plex: a progress save sent just before an episode ends no longer leaves it unwatched, and a refused finish no longer sticks.
+- After a dialog closes, the keyboard focus returns to where it was, also inside a menu that stays open.
+
+## 0.113.0 — 2026-10-28
+
+### New
+
+- The address names what you see: tabs, list views, sorts and searches (the live YouTube search too), the YouTube player and the file viewer. A reload, a bookmark or a shared link opens the same view.
+- A link to a video reopens the player over its page once; Back closes it. A video the page no longer lists plays on its own and says so.
+
+### Fixed
+
+- Back and Forward step through the views as you saw them, across dialogs, players and reloads: a reload under an open player no longer leaves a Back step that does nothing.
+- Signed out on a Plex page, the sign-in card could show a button with no text.
+- The channel page's address names the channel and its own view, not the page under it.
+- A reload during a Plex binge reopens the episode playing now, not the one it started from.
+
+## 0.112.1 — 2026-10-27
+
+### Fixed
+
+- Security: the sign-in library is updated, closing two published vulnerabilities.
+
+## 0.112.0 — 2026-10-25
+
+### New
+
+- Player: a Related tab lists videos related to the one playing, leaving out what you have already watched (your uploaded YouTube history included). Channels you follow are marked, and you can switch between On topic and Trending.
+- Play a related video in place; Back returns to the one you opened. Admins set a daily per-user limit in Config › Quota (default 200, 0 turns it off).
+- Side panel: the list controls and the watched toggle are glyphs on one line, and Auto and Loop show when they are on, also under the pointer.
+
+### Fixed
+
+- Live search: a search that never reached YouTube no longer uses up a daily slot, a busy moment says so after a few seconds instead of waiting on, and a first page that failed offers Retry.
+- Menus: Play from the beginning wears the same ↺ glyph everywhere, and Reset its own eraser glyph.
+- On the day of this update, API searches made earlier that day stop counting toward the daily search limit, until Pacific midnight.
+
+## 0.111.0 — 2026-10-24
+
+### New
+
+- Channels › Discover from your history: the channels your uploaded YouTube history shows you watching often (five videos of five minutes or more) but not following.
+- Follow one, put it out of the list (with Undo), or open the videos you watched there and play them one by one or all in a row.
+
+### Fixed
+
+- A channel someone explored and then followed on YouTube is now part of the shared Library like any followed channel, and so are its videos; before, they could stay hidden and their watched marks never applied.
+
+## 0.110.0 — 2026-10-23
+
+### New
+
+- YouTube history: upload your Google Takeout export in Settings › Account. What you watched on YouTube is marked watched here (never a Short, never over a state you set), and your YouTube Watch later list joins yours.
+- A video the catalog does not know yet is remembered and marked once it arrives. The import's outcome reaches your inbox, and admins see each run on the Scheduler page.
+
+### Fixed
+
+- Scheduler: a channel's full-history backfill asked for again while one is running now runs after it, instead of being silently dropped.
+
+## 0.109.2 — 2026-10-23
+
+### Fixed
+
+- Plex player: when the server restarts mid-episode (an update), playback carries on from the buffer and picks up where you are once it is back, instead of showing "Playback couldn't start" and stopping where the buffer ended.
+
+## 0.109.1 — 2026-10-23
+
+### New
+
+- Development: the frontend is one folder per page (its page, rail, store and definition), with the layers held by lint and the shared /watch page held apart by a contract test; the header knows no page, and the feed and Plex each keep their own saved views.
+
+### Fixed
+
+- Saved views: a view you applied and then changed is still active after a visit to another page, in the feed and in Plex.
+
+## 0.109.0 — 2026-10-21
+
+### New
+
+- Accounts: a browser is signed in to one account at a time. The account menu's "Sign in as someone else" replaces it; the account list and switcher are gone. After this update everyone signs in once more.
+- Settings follow you: the Home start page, the channel manager and Plex playlist layouts, the discovery sorts and a dismissed onboarding or release banner are saved to your account, on every device. Performance mode stays per device.
+- Development: every key the browser stores declares its kind, held by a contract test; one Google connect route serves the sign-in and the YouTube access grants.
+
+### Fixed
+
+- Accounts: a tab still showing another account than the one signed in now reloads as that one and says why; a failed "Sign in as someone else" says what happened.
+- A link whose path starts with // no longer opens the error screen.
+- A member following a link to an admin page lands on the feed, with the rail and the address bar in step.
+- Google: an unknown access request is refused instead of quietly asking for read access, and a YouTube grant on an instance without Google says so.
+
+## 0.108.1 — 2026-10-20
+
+### New
+
+- Development: the backend is split into four units (downloads, reader, catalog, Plex), each owning its tables. A contract test holds what each one reads and writes, and that every query names whose rows it asks for.
+
+### Fixed
+
+- Watch later: saving the first two videos at once no longer makes two lists. An existing second list is merged into the first, its videos and its comfort-shelf pin with it.
+- Playback: the first resume point of a video or a Plex item no longer fails when it is saved together with the watched mark, a whole season marked at once, or a Plex sync.
+
+## 0.108.0 — 2026-10-18
+
+### New
+
+- Activity: one admin page for background runs, admin actions, the server log and quota, with one time range, account and search. A run opens to its outcome and its log lines, an admin action to its request's lines; old audit and log links open their tab.
+- Scheduler: each job shows its last three runs (a run cut short by a restart is marked) and a button opens all of them; turning on a channel's full history is now a run of its own, listed under Ad-hoc runs.
+- Configuration: how long the run history is kept (Activity › Run history, 180 days by default). Stats keeps your own numbers; the system quota view moved to Activity › Quota.
+
+### Fixed
+
+- Configuration: saving a setting with the value already in force now keeps it, instead of dropping the change.
+- Configuration: the unused SponsorBlock default setting is gone.
+
+## 0.107.3 — 2026-10-17
+
+### New
+
+- Development: the end-to-end suite runs the WCAG A and AA accessibility checks over every page and the player, and every shell script is linted in the gate.
+
+### Fixed
+
+- Readability: muted text in the light themes is a shade darker, and the item count, the labels and the selected Files folder sit on a solid tint, so text on the frosted-glass bars stays readable (4.5:1) in every scheme.
+
+## 0.107.2 — 2026-10-17
+
+### New
+
+- Development: the gate script can list its checks (scripts/check.sh --list) and run just one (--only), and the backend tests run on every CPU core, each worker with its own database.
+
+## 0.107.1 — 2026-10-16
+
+### Fixed
+
+- Downloads: a failed download's explanation now says to press Retry, instead of pointing to a download list that no longer exists.
+
+## 0.107.0 — 2026-10-15
+
+### New
+
+- API tokens: Settings › API tokens gives one machine of yours a key to the Files folders you pick — or, in one click, to a Hub folder for swapping files with it. Signing out your other sessions or changing your password revokes every token.
+- Share by link: a link can now open a whole folder of yours — its page lists what is in it and plays a video above the list.
+
+### Fixed
+
+- Uploads: a finished upload is announced once, and shows in Files at once.
+
 ## 0.106.2 — 2026-10-14
 
 ### Fixed

CLAUDE.md

@@ -12,7 +12,7 @@ router-free navigation provider.
 
 ## The gate — never merge or ship red
 
-The local gate is **ruff (`check` + `format --check`) + basedpyright (backend) + the lock check + typecheck (3 tsc projects) +
+The local gate is **ruff (`check` + `format --check`) + shellcheck + basedpyright (backend) + the lock check + typecheck (3 tsc projects) +
 eslint + prettier --check + knip + vitest + vite build + pytest + pip-audit + npm audit**. It runs
 before every `release` merges and on every `publish`, which **refuses to build on failure — there is deliberately no `--skip`
 flag**; a release also requires the Playwright suite, and the image must pass a Trivy scan before it is pushed. Run it and get it
@@ -55,66 +55,88 @@ how it gets built. Concretely:
 - **Challenge unsound requests.** If a request is technically ill-advised and you can back it with real
   arguments, push back or propose an alternative — don't comply blindly.
 - **Verify UI in the browser before calling it done.** tsc/unit/DOM tests miss visual and interaction
-  bugs; do a real visual check of any UI change, and run DESIGN.md's four-step Conformance check.
+  bugs; do a real visual check of any UI change, and run DESIGN.md's Conformance check (a copy-only
+  change runs its step 4 alone: every state the new text renders in, in both languages).
 - **Be terse everywhere else.** Outside those decision points, lead with the answer or decision. During
   implementation and debugging especially: no restating the same point reworded, no filler.
 
 ## Architecture shape (don't fight it)
 
 `routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
-`downloads/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `quota`,
+`downloads/`, `reader/`, `discover/`, `catalog/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
 `sysconfig`, `state`, `audit`, `notifications`, `userscope`, `ratelimit`, `bodylimit`, `progress`, `collation`,
-`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
-Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
-entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
-components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
-its menus, `Modal`/`Overlay`, `Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`, …). That
-boundary is one-way and **lint-enforced**, not just described: a platform component may not import a
-feature component or a domain API slice, and one layer down `lib/` may not import from `components/` at
-all, not even a type (`no-restricted-imports` in `eslint.config.js`, with its own test). A new generic
-component goes in `ui/`; see [`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
-Data via react-query (`qk` factory), per-account `localStorage`, and server prefs; API through
-`lib/api/*` slices over `core.req<T>`.
+`security`, `logs`, `logsink`, `diagnostics`, `debugsession`, `draw`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
+The domain is four **units**, each owning its tables: Capture (`downloads/`), Reader (`reader/`, `discover/`, the playlist
+sync), Catalog (`catalog/`, the `sync/` ingests) and Plex (`plex/`). A unit's route does no database work: it reads through
+the unit's `queries`, writes through its `service` (which flushes) and commits. A unit's code reads another unit's tables
+only along a declared edge and writes them only through that unit's service; core reaches a unit's tables through its
+`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`) — as far as an AST sees: what it
+cannot (an ORM write on another unit's loaded row, SQL in `text()`, a commit by `begin()`) is review's to catch, listed in
+[`backend/app/README.md`](backend/app/README.md) with the conventions.
+Frontend, by folder under `src/`: `main.tsx` provider tree → `shell/` (`App`, header, nav rail, palette,
+boot; a share link (`/watch`, `/file`, `/folder`) boots from its own entry, `watch.html` → `watchMain.tsx` → `watch/`, held
+app-free by a contract test) → **`modules/<page>/`** (one folder per page: the page, its rail, its store, its tests,
+and its `module.ts` definition — label, icon, gate, the lazy page and rail, its store — collected in nav order by
+`features/navigation/moduleRegistry`, which also mounts every store at the root; a new page is its folder, its id in
+`lib/urlState` and its entry there, plus its panel in `lib/panelLayout` if it has a rail) →
+**`features/<name>/`** (what several pages share: video cards, filters, downloads, the player, navigation) → the
+**platform layer `components/ui/`** (domain-free primitives: the popover engine and its menus, `Modal`/`Overlay`,
+`Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`,
+…) → `lib/`. The direction is one-way and **lint-enforced**, not just described (`local/layers`, with
+its own test): an import only goes down; another module is reached only through its **`index.ts`** (its
+public face — the stores, mounted at the root and read app-wide, which `features/` may read too) or
+lazily by `import()`; a platform component may not import a domain API slice, and `lib/` nothing above
+it, not even a type. A lazily imported page, rail or dialog that a static import drags onto the boot
+path fails the build (`chunkGuard.ts`). A new generic component goes in `ui/`; see
+[`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
+Data via react-query (`qk` factory), the signed-in account's `localStorage` (`storage.ts`), and server
+prefs; API through `lib/api/*` slices over `core.req<T>`.
 
 **Not wanted** (reviewed and rejected as over-engineering for this size): microservices, event bus,
 CQRS, a repository/DI framework, a router library, a generic CRUD-router factory or base serializer.
 
 ## Canonical patterns — when you have a choice, pick THIS
 
-| Area                                 | Use                                                                                                                                                                                               | Not                                                                                                                                                     |
-| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| ORM                                  | SQLAlchemy 2.0 `select()`                                                                                                                                                                         | `db.query(...)`                                                                                                                                         |
-| Request body                         | a Pydantic model                                                                                                                                                                                  | `payload: dict`                                                                                                                                         |
-| Request SIZE                         | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES`                                                                                               | `capped_body` on a route that does not read raw bytes itself                                                                                            |
-| Ownership check                      | `owned_or_404` / `get_or_404`                                                                                                                                                                     | ad-hoc `_own_playlist`/`_owned`/`_own_job`                                                                                                              |
-| Admin gate import                    | `app.auth`                                                                                                                                                                                        | the `app.routes.admin` re-export                                                                                                                        |
-| Upstream failure status              | `422` with JSON detail                                                                                                                                                                            | `502` (reserve 502 for raw byte-proxies)                                                                                                                |
-| User-facing error                    | structured `detail` (`{code, reason}`)                                                                                                                                                            | a bare English string under a bilingual UI                                                                                                              |
-| Mutation response                    | `{"ok": true, …}` (`{"deleted": id}`), test-held                                                                                                                                                  | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok`                                                                                                     |
-| Transaction commit                   | the handler commits; helpers `flush` (test-held list)                                                                                                                                             | helpers that commit (only `quota` owns its session)                                                                                                     |
-| Admin knob storage                   | `sysconfig` (values)                                                                                                                                                                              | env / `app_state` / `scheduler_settings` scatter                                                                                                        |
-| Job status word                      | the `downloads/status.py` StrEnum                                                                                                                                                                 | loose `"canceled"`/`"cancelled"` literals                                                                                                               |
-| Current time                         | inline `datetime.now(timezone.utc)`                                                                                                                                                               | new `now_utc`/`_now` helpers                                                                                                                            |
-| FE per-account state                 | `usePref`                                                                                                                                                                                         | raw `readAccount`/`writeAccount` or bespoke hooks                                                                                                       |
-| FE keyboard shortcut                 | a row in `lib/keymap.ts` + `dispatch()`                                                                                                                                                           | a private `keydown` listener with its own key literals                                                                                                  |
-| FE list/grid keys                    | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open`                                                                                                                                        | a private arrow-key / roving-tabindex handler                                                                                                           |
-| FE view push (history)               | `rememberNavPosition()` right before `history.pushState`                                                                                                                                          | a push that forgets: Back strands focus on `<body>`                                                                                                     |
-| FE static label / button-like link   | `Badge` (flat, no edge) · `LinkButton` / `IconLink`                                                                                                                                               | a glass pill that reads as a chip; an `<a>` hand-painted like a button                                                                                  |
-| FE loading/empty/error               | `QueryState` / `StateMessage`                                                                                                                                                                     | ad-hoc conditionals                                                                                                                                     |
-| FE failed write                      | `meta.failTitle`, else `notifyActionError(err, title)`                                                                                                                                            | a bare `catch {}` or `notify()` with no error attached                                                                                                  |
-| FE player on a native media element  | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE                                                                                  | a private `navigator.mediaSession` / `keydown` / position-saving effect per player                                                                      |
-| FE diagnostics                       | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts`                                                                           | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded                                                                    |
-| FE state that MIRRORS a prop/query   | derive it, or hold a draft (`draft ?? server`)                                                                                                                                                    | a copy an effect re-syncs — that paints the stale value first                                                                                           |
-| FE state reset on a key change       | adjust it DURING render against a remembered key                                                                                                                                                  | `useEffect(() => setX(init), [key])`                                                                                                                    |
-| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback`                                                                                                             | a render-phase `ref.current = x` — the React Compiler skips the component                                                                               |
-| FE hook suppression                  | an inline `eslint-disable-next-line` NAMING the invariant                                                                                                                                         | `eslint-suppressions.json` (deliberately absent — E27 S2)                                                                                               |
-| FE popover / scroll-fade ref         | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}`                                                                                                                    | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips                                         |
-| FE async handler with cleanup        | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
+| Area                                 | Use                                                                                                                                                                                                                                           | Not                                                                                                                                                                                         |
+| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ORM                                  | SQLAlchemy 2.0 `select()`                                                                                                                                                                                                                     | `db.query(...)`                                                                                                                                                                             |
+| Request body                         | a Pydantic model                                                                                                                                                                                                                              | `payload: dict`                                                                                                                                                                             |
+| Request SIZE                         | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES`                                                                                                                                           | `capped_body` on a route that does not read raw bytes itself                                                                                                                                |
+| Ownership check                      | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404`. A composite rule (own or shared, built-in or own) says so in its `queries` | an inline owner filter — `Model.user_id == …`, an alias's, `.in_()`, `filter_by(user_id=…)` (fence-held, every module) — or a `db.get` and a compare where `owned_row` fits                 |
+| Admin gate import                    | `app.auth`                                                                                                                                                                                                                                    | the `app.routes.admin` re-export                                                                                                                                                            |
+| Upstream failure status              | `422` with JSON detail                                                                                                                                                                                                                        | `502` (reserve 502 for raw byte-proxies)                                                                                                                                                    |
+| User-facing error                    | structured `detail` (`{code, reason}`), its sentence named in `lib/api/refusalCodes.ts` (contract-held)                                                                                                                                       | a bare English string under a bilingual UI                                                                                                                                                  |
+| Mutation response                    | `{"ok": true, …}` (`{"deleted": id}`), test-held                                                                                                                                                                                              | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok`                                                                                                                                         |
+| Transaction commit                   | the handler commits; a unit's `service` and every helper `flush` (fence-held, every module)                                                                                                                                                   | a helper that commits — only a listed one (what owns its session, or a core step that must land whatever its caller does next), each with its reason: the fence's `UNIT_COMMITS` / `CORE_COMMITS`, a route's `COMMITTING_HELPERS` (`test_route_conventions`) |
+| Admin knob storage                   | `sysconfig` (values)                                                                                                                                                                                                                          | env / `app_state` / a table of its own (the job intervals had one)                                                                                                                          |
+| Job status word                      | the `downloads/status.py` StrEnum                                                                                                                                                                                                             | loose `"canceled"`/`"cancelled"` literals                                                                                                                                                   |
+| Current time                         | inline `datetime.now(timezone.utc)`                                                                                                                                                                                                           | new `now_utc`/`_now` helpers                                                                                                                                                                |
+| FE browser storage                   | a key in `storage.ts` with its kind (contract-held): a `pref` through `usePref` (the server's, follows the account); `device` / `view` / `cache` this browser's; `public` kept for the /watch page                                            | a `"siftlode.x"` literal; a preference kept only in `localStorage` (it stays on one device)                                                                                                 |
+| FE keyboard shortcut                 | a row in `lib/keymap.ts` + `dispatch()`                                                                                                                                                                                                       | a private `keydown` listener with its own key literals                                                                                                                                      |
+| FE list/grid keys                    | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open`                                                                                                                                                                                    | a private arrow-key / roving-tabindex handler                                                                                                                                               |
+| FE view push (history)               | `rememberNavPosition()` right before `history.pushState`                                                                                                                                                                                      | a push that forgets: Back strands focus on `<body>`                                                                                                                                         |
+| FE view state                        | the module's URL codec: its keys in `urlKeys` (`module.ts`), read and written by `lib/<id>UrlState.ts` through `useUrlMirror` (contract-held, `e2e/bookmark.spec.ts`)                                                                         | `useState` / sessionStorage alone: a reload, a bookmark or a shared link loses the view                                                                                                     |
+| FE a page's header content           | the page renders it into the floating header itself (`PageHeader`, `ui/PageShell`)                                                                                                                                                            | a `page === …` branch in the shell's header, or a bridge carrying a page's state up to it                                                                                                   |
+| FE module store                      | its definition's `Store` (`module.ts`), mounted at the root by `ModuleStores`; it reads no other module's store (contract-held)                                                                                                               | a provider hand-nested in `main.tsx`                                                                                                                                                        |
+| FE saved views                       | the module's own store from `features/filters` (`createSavedViewsStore` + `useSavedViewsStore`)                                                                                                                                               | one store switched by the active page — every visit elsewhere reset it                                                                                                                      |
+| FE static label / button-like link   | `Badge` (flat, no edge) · `LinkButton` / `IconLink`                                                                                                                                                                                           | a glass pill that reads as a chip; an `<a>` hand-painted like a button                                                                                                                      |
+| FE loading/empty/error               | `QueryState` / `StateMessage`                                                                                                                                                                                                                 | ad-hoc conditionals                                                                                                                                                                         |
+| FE failed write                      | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect                                                                                        | a bare `catch {}` or `notify()` with no error attached                                                                                                                                      |
+| FE floating or full-screen layer     | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers")                                                                  | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]`                                                                  |
+| FE player on a native media element  | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE                                                                                                                              | a private `navigator.mediaSession` / `keydown` / position-saving effect per player                                                                                                          |
+| FE diagnostics                       | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts`                                                                                                                       | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded                                                                                                        |
+| FE state that MIRRORS a prop/query   | derive it, or hold a draft (`draft ?? server`)                                                                                                                                                                                                | a copy an effect re-syncs — that paints the stale value first                                                                                                                               |
+| FE state reset on a key change       | adjust it DURING render against a remembered key                                                                                                                                                                                              | `useEffect(() => setX(init), [key])`                                                                                                                                                        |
+| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback`                                                                                                                                                         | a render-phase `ref.current = x` — the React Compiler skips the component                                                                                                                   |
+| FE hook suppression                  | an inline `eslint-disable-next-line` NAMING the invariant                                                                                                                                                                                     | `eslint-suppressions.json` (deliberately absent — E27 S2)                                                                                                                                   |
+| FE popover / scroll-fade ref         | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}`                                                                                                                                                                | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips                                                                             |
+| FE async handler with cleanup        | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper                                             | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these)                                     |
 
 Not repeated here, because a lint rule refuses the wrong one and its message names the right one:
 buttons (`ui/Button`, `IconButton`, `Pressable`, `Chip`, `DragHandle`), text fields and native
 controls (`ui/Input`, `Checkbox`, `Slider`, …), glass surfaces (`SURFACE.*`), a native `<select>`
-(`SegmentedToggle` / `FilterSelect`), a picture (`ui/RemoteImage`, never a bare `<img>`), and network calls (`req` / `uploadReq` / `beacon`, `lib/http.ts`).
+(`SegmentedToggle` / `FilterSelect`), a picture (`ui/RemoteImage`, never a bare `<img>`), a box or font size (the spacing scale or rem,
+never a px-frozen `w-[Npx]` / `text-[Npx]`: the text-size slider scales the root font), and network calls (`req` / `uploadReq` / `beacon`, `lib/http.ts`).
 
 ## House rules that bite (learned from real bugs)
 

VERSION

@@ -1 +1 @@
-0.106.2
+0.118.0

backend/.basedpyright/baseline.json

@@ -185,118 +185,6 @@
                     "lineCount": 1
                 }
             },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 50,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
             {
                 "code": "reportArgumentType",
                 "range": {
@@ -364,25 +252,7 @@
                 }
             }
         ],
-        "./app/downloads/adapters.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 30,
-                    "endColumn": 38,
-                    "lineCount": 1
-                }
-            }
-        ],
         "./app/downloads/backfill.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 34,
-                    "endColumn": 38,
-                    "lineCount": 1
-                }
-            },
             {
                 "code": "reportReturnType",
                 "range": {
@@ -413,14 +283,6 @@
             }
         ],
         "./app/downloads/pipeline.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 34,
-                    "endColumn": 38,
-                    "lineCount": 1
-                }
-            },
             {
                 "code": "reportAttributeAccessIssue",
                 "range": {
@@ -630,41 +492,7 @@
                 }
             }
         ],
-        "./app/routes/admin_logs.py": [
-            {
-                "code": "reportReturnType",
-                "range": {
-                    "startColumn": 15,
-                    "endColumn": 47,
-                    "lineCount": 1
-                }
-            }
-        ],
         "./app/routes/audit.py": [
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 17,
-                    "endColumn": 98,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportAssignmentType",
-                "range": {
-                    "startColumn": 17,
-                    "endColumn": 98,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 22,
-                    "endColumn": 97,
-                    "lineCount": 1
-                }
-            },
             {
                 "code": "reportAttributeAccessIssue",
                 "range": {
@@ -674,48 +502,6 @@
                 }
             }
         ],
-        "./app/routes/channels.py": [
-            {
-                "code": "reportReturnType",
-                "range": {
-                    "startColumn": 11,
-                    "endColumn": 11,
-                    "lineCount": 6
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 15,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 49,
-                    "endColumn": 56,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportAttributeAccessIssue",
-                "range": {
-                    "startColumn": 38,
-                    "endColumn": 44,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportAttributeAccessIssue",
-                "range": {
-                    "startColumn": 33,
-                    "endColumn": 39,
-                    "lineCount": 1
-                }
-            }
-        ],
         "./app/routes/discover.py": [
             {
                 "code": "reportInvalidTypeForm",
@@ -726,277 +512,7 @@
                 }
             }
         ],
-        "./app/routes/downloads.py": [
-            {
-                "code": "reportOptionalMemberAccess",
-                "range": {
-                    "startColumn": 17,
-                    "endColumn": 19,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalMemberAccess",
-                "range": {
-                    "startColumn": 82,
-                    "endColumn": 84,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalMemberAccess",
-                "range": {
-                    "startColumn": 87,
-                    "endColumn": 89,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalMemberAccess",
-                "range": {
-                    "startColumn": 30,
-                    "endColumn": 32,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalMemberAccess",
-                "range": {
-                    "startColumn": 49,
-                    "endColumn": 54,
-                    "lineCount": 1
-                }
-            }
-        ],
-        "./app/routes/playlists.py": [
-            {
-                "code": "reportOptionalOperand",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 9,
-                    "lineCount": 5
-                }
-            },
-            {
-                "code": "reportOptionalOperand",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 9,
-                    "lineCount": 5
-                }
-            },
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 13,
-                    "endColumn": 5,
-                    "lineCount": 7
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 15,
-                    "lineCount": 5
-                }
-            },
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 16,
-                    "endColumn": 5,
-                    "lineCount": 11
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 15,
-                    "lineCount": 9
-                }
-            },
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 13,
-                    "endColumn": 5,
-                    "lineCount": 5
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 15,
-                    "lineCount": 3
-                }
-            },
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 12,
-                    "endColumn": 31,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 23,
-                    "endColumn": 27,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 12,
-                    "endColumn": 34,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 26,
-                    "endColumn": 30,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 12,
-                    "endColumn": 28,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 23,
-                    "endColumn": 27,
-                    "lineCount": 1
-                }
-            }
-        ],
-        "./app/routes/plex/browse.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 26,
-                    "endColumn": 61,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 26,
-                    "endColumn": 61,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 11,
-                    "endColumn": 13,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 25,
-                    "endColumn": 27,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 11,
-                    "endColumn": 13,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 25,
-                    "endColumn": 27,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 15,
-                    "endColumn": 17,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 28,
-                    "endColumn": 30,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 15,
-                    "endColumn": 17,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportOptionalSubscript",
-                "range": {
-                    "startColumn": 28,
-                    "endColumn": 30,
-                    "lineCount": 1
-                }
-            }
-        ],
         "./app/routes/plex/collections.py": [
-            {
-                "code": "reportAssignmentType",
-                "range": {
-                    "startColumn": 11,
-                    "endColumn": 39,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 76,
-                    "endColumn": 80,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 75,
-                    "endColumn": 79,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 80,
-                    "endColumn": 84,
-                    "lineCount": 1
-                }
-            },
             {
                 "code": "reportArgumentType",
                 "range": {
@@ -1054,117 +570,7 @@
                 }
             }
         ],
-        "./app/routes/plex/discover.py": [
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 4,
-                    "endColumn": 63,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 18,
-                    "endColumn": 48,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportCallIssue",
-                "range": {
-                    "startColumn": 4,
-                    "endColumn": 63,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 18,
-                    "endColumn": 48,
-                    "lineCount": 1
-                }
-            }
-        ],
-        "./app/routes/plex/playback.py": [
-            {
-                "code": "reportAttributeAccessIssue",
-                "range": {
-                    "startColumn": 6,
-                    "endColumn": 14,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 51,
-                    "endColumn": 54,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 51,
-                    "endColumn": 54,
-                    "lineCount": 1
-                }
-            }
-        ],
-        "./app/routes/plex/playlists.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 77,
-                    "endColumn": 83,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 77,
-                    "endColumn": 83,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 9,
-                    "lineCount": 5
-                }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 8,
-                    "endColumn": 9,
-                    "lineCount": 5
-                }
-            },
-            {
-                "code": "reportAttributeAccessIssue",
-                "range": {
-                    "startColumn": 10,
-                    "endColumn": 18,
-                    "lineCount": 1
-                }
-            }
-        ],
         "./app/routes/scheduler.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 42,
-                    "endColumn": 55,
-                    "lineCount": 1
-                }
-            },
             {
                 "code": "reportOptionalMemberAccess",
                 "range": {
@@ -1182,42 +588,6 @@
                 }
             }
         ],
-        "./app/routes/search.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 25,
-                    "endColumn": 31,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportAttributeAccessIssue",
-                "range": {
-                    "startColumn": 6,
-                    "endColumn": 14,
-                    "lineCount": 1
-                }
-            }
-        ],
-        "./app/routes/sync.py": [
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 42,
-                    "endColumn": 65,
-                    "lineCount": 1
-                }
-            },
-            {
-                "code": "reportAttributeAccessIssue",
-                "range": {
-                    "startColumn": 42,
-                    "endColumn": 50,
-                    "lineCount": 1
-                }
-            }
-        ],
         "./app/sync/explore.py": [
             {
                 "code": "reportAttributeAccessIssue",

backend/alembic/env.py

@@ -4,7 +4,7 @@ from sqlalchemy import engine_from_config, pool
 from app.config import settings
 from app.db import Base
 from app.logs import configure_logging
-import app.models  # noqa: F401  (import so models register on Base.metadata)
+import app.tables  # noqa: F401  (import so every table registers on Base.metadata)
 
 config = context.config
 config.set_main_option("sqlalchemy.url", settings.database_url)

backend/alembic/versions/0106_api_tokens.py

@@ -0,0 +1,110 @@
+"""Machine access to the Files library: API tokens bound to folders, and the Hub folder
+
+Revision ID: 0106_api_tokens
+Revises: 0105_job_source_key
+Create Date: 2026-10-14
+
+`api_tokens` is a user's token for a machine (a script, the hub): only its SHA-256 is kept, plus the
+first characters to recognise it by, and whether its user was told it is about to run out. `api_token_grants` says what it may do in which folder — list,
+read, write, delete — so a token reaches nothing it was not given, folder by folder.
+
+A third folder kind, `hub`: the one folder a user exchanges files with their hub machine through.
+Like a source folder its name is the client's to render; like an own folder a file may be moved in.
+One per user.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0106_api_tokens"
+down_revision: str | None = "0105_job_source_key"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+_KIND_OLD = (
+    "(kind = 'source' AND source_key IS NOT NULL AND name IS NULL)"
+    " OR (kind = 'own' AND name IS NOT NULL AND source_key IS NULL)"
+)
+_KIND_NEW = _KIND_OLD + " OR (kind = 'hub' AND name IS NULL AND source_key IS NULL)"
+
+
+def upgrade() -> None:
+    op.drop_constraint("ck_file_folder_kind", "file_folders", type_="check")
+    op.create_check_constraint("ck_file_folder_kind", "file_folders", _KIND_NEW)
+    op.create_index(
+        "uq_file_folder_hub",
+        "file_folders",
+        ["user_id"],
+        unique=True,
+        postgresql_where=sa.text("kind = 'hub'"),
+    )
+    op.create_table(
+        "api_tokens",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+            index=True,
+        ),
+        sa.Column("name", sa.String(80), nullable=False),
+        sa.Column("prefix", sa.String(16), nullable=False),
+        sa.Column("token_hash", sa.String(64), nullable=False, unique=True),
+        sa.Column("expires_at", sa.DateTime(timezone=True)),
+        sa.Column("last_used_at", sa.DateTime(timezone=True)),
+        sa.Column("expiry_warned", sa.Boolean(), nullable=False, server_default=sa.text("false")),
+        sa.Column(
+            "created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
+        ),
+    )
+    op.create_table(
+        "api_token_grants",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "token_id",
+            sa.Integer(),
+            sa.ForeignKey("api_tokens.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column(
+            "folder_id",
+            sa.Integer(),
+            sa.ForeignKey("file_folders.id", ondelete="CASCADE"),
+            nullable=False,
+            index=True,
+        ),
+        sa.Column("can_list", sa.Boolean(), nullable=False, server_default=sa.text("false")),
+        sa.Column("can_read", sa.Boolean(), nullable=False, server_default=sa.text("false")),
+        sa.Column("can_write", sa.Boolean(), nullable=False, server_default=sa.text("false")),
+        sa.Column("can_delete", sa.Boolean(), nullable=False, server_default=sa.text("false")),
+        sa.UniqueConstraint("token_id", "folder_id", name="uq_api_token_grant"),
+        sa.CheckConstraint(
+            "can_list OR can_read OR can_write OR can_delete", name="ck_api_token_grant_some"
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("api_token_grants")
+    op.drop_table("api_tokens")
+    op.drop_index("uq_file_folder_hub", table_name="file_folders")
+    # A Hub folder may hold files (their foreign key RESTRICTs a delete): it becomes an own folder,
+    # named "Hub <id>" — or "Hub <id> (2)", … when its user already has a folder of that name,
+    # which `uq_file_folder_own_name` (per user, case-blind) would refuse.
+    conn = op.get_bind()
+    hubs = conn.execute(sa.text("SELECT id, user_id FROM file_folders WHERE kind = 'hub'")).all()
+    for folder_id, user_id in hubs:
+        name, n = f"Hub {folder_id}", 1
+        while conn.execute(
+            sa.text("SELECT 1 FROM file_folders WHERE user_id = :u AND lower(name) = lower(:n)"),
+            {"u": user_id, "n": name},
+        ).first():
+            n += 1
+            name = f"Hub {folder_id} ({n})"
+        conn.execute(
+            sa.text("UPDATE file_folders SET kind = 'own', name = :n WHERE id = :i"),
+            {"n": name, "i": folder_id},
+        )
+    op.drop_constraint("ck_file_folder_kind", "file_folders", type_="check")
+    op.create_check_constraint("ck_file_folder_kind", "file_folders", _KIND_OLD)

backend/alembic/versions/0107_link_token_at_rest.py

@@ -0,0 +1,66 @@
+"""Share-link tokens no longer stored in the clear
+
+Revision ID: 0107_link_token_at_rest
+Revises: 0106_api_tokens
+Create Date: 2026-10-15
+
+A `/watch` link's token is the credential itself, and it sat in `download_links.token` as written,
+so a copy of the database was every shared file. Now a link is found by the token's SHA-256
+(`token_hash`), and the token is kept only Fernet-encrypted (`token_enc`, TOKEN_ENCRYPTION_KEY) so its
+owner can copy the same link again. Without a key the copy is not kept: the link still opens, its
+address just cannot be shown again (`app/downloads/links.py`).
+
+Every existing link is converted here, so the addresses already sent keep working. The downgrade
+writes the tokens back from their encrypted copies; a link with none left (no key, or a key since
+changed) gets a fresh random token, which is the same as revoking it.
+"""
+
+import hashlib
+import secrets
+
+import sqlalchemy as sa
+from alembic import op
+
+from app import security
+
+revision: str = "0107_link_token_at_rest"
+down_revision: str | None = "0106_api_tokens"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.add_column("download_links", sa.Column("token_hash", sa.String(64)))
+    op.add_column("download_links", sa.Column("token_enc", sa.Text()))
+    conn = op.get_bind()
+    sealing = security.can_encrypt()
+    for link_id, token in conn.execute(sa.text("SELECT id, token FROM download_links")).fetchall():
+        conn.execute(
+            sa.text("UPDATE download_links SET token_hash = :h, token_enc = :e WHERE id = :id"),
+            {
+                "id": link_id,
+                "h": hashlib.sha256(token.encode()).hexdigest(),
+                "e": security.encrypt(token) if sealing else None,
+            },
+        )
+    op.alter_column("download_links", "token_hash", nullable=False)
+    op.create_unique_constraint("uq_download_link_token_hash", "download_links", ["token_hash"])
+    op.drop_constraint("uq_download_link_token", "download_links", type_="unique")
+    op.drop_column("download_links", "token")
+
+
+def downgrade() -> None:
+    op.add_column("download_links", sa.Column("token", sa.String(64)))
+    conn = op.get_bind()
+    rows = conn.execute(sa.text("SELECT id, token_enc FROM download_links")).fetchall()
+    for link_id, enc in rows:
+        token = security.decrypt_optional(enc) or secrets.token_urlsafe(24)
+        conn.execute(
+            sa.text("UPDATE download_links SET token = :t WHERE id = :id"),
+            {"id": link_id, "t": token},
+        )
+    op.alter_column("download_links", "token", nullable=False)
+    op.create_unique_constraint("uq_download_link_token", "download_links", ["token"])
+    op.drop_constraint("uq_download_link_token_hash", "download_links", type_="unique")
+    op.drop_column("download_links", "token_enc")
+    op.drop_column("download_links", "token_hash")

backend/alembic/versions/0108_folder_links.py

@@ -0,0 +1,40 @@
+"""A share link may open a folder
+
+Revision ID: 0108_folder_links
+Revises: 0107_link_token_at_rest
+Create Date: 2026-10-15
+
+A `/watch` link opened one file. Now it may open one of its owner's own folders instead: the link's
+page lists what is in the folder when it is opened. `folder_id` sits beside `job_id`, exactly one of
+the two set; a folder's links go with the folder.
+
+The downgrade has nowhere to keep a folder link, so it deletes them: they stop opening.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0108_folder_links"
+down_revision: str | None = "0107_link_token_at_rest"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "download_links",
+        sa.Column("folder_id", sa.Integer(), sa.ForeignKey("file_folders.id", ondelete="CASCADE")),
+    )
+    op.create_index("ix_download_links_folder_id", "download_links", ["folder_id"])
+    op.alter_column("download_links", "job_id", nullable=True)
+    op.create_check_constraint(
+        "ck_download_link_target", "download_links", "(job_id IS NULL) <> (folder_id IS NULL)"
+    )
+
+
+def downgrade() -> None:
+    op.execute("DELETE FROM download_links WHERE folder_id IS NOT NULL")
+    op.drop_constraint("ck_download_link_target", "download_links", type_="check")
+    op.alter_column("download_links", "job_id", nullable=False)
+    op.drop_index("ix_download_links_folder_id", table_name="download_links")
+    op.drop_column("download_links", "folder_id")

backend/alembic/versions/0109_job_runs.py

@@ -0,0 +1,160 @@
+"""Background runs get a durable history
+
+Revision ID: 0109_job_runs
+Revises: 0108_folder_links
+Create Date: 2026-10-17
+
+What a scheduler job last did lived in the API process's memory, gone at every restart; the audit
+log kept a row for a run only when it was manual, failed, or changed something. Now every run is a
+`job_runs` row, written when it starts and when it ends. A run's quota spend points at it
+(`quota_events.job_run_id`), and so do the lines it logged (`log_events.run_id`, no foreign key: the
+log writer runs behind the event).
+
+Runs are not admin actions, so the audit log stops carrying them: the existing `job_run` audit rows
+MOVE into `job_runs` (start = end = when the row was written, the only time it kept), and leave the
+audit log. The downgrade moves every run back into the audit log as such a row (one still running as
+interrupted, what the audit row has no column for in its `after`), then drops the table.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision: str = "0109_job_runs"
+down_revision: str | None = "0108_folder_links"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "job_runs",
+        sa.Column("id", sa.BigInteger(), primary_key=True),
+        sa.Column("job", sa.String(40), nullable=False),
+        sa.Column("trigger", sa.String(16), nullable=False),
+        sa.Column("actor_id", sa.Integer(), sa.ForeignKey("users.id", ondelete="SET NULL")),
+        sa.Column("target", sa.String(128)),
+        sa.Column("status", sa.String(16), nullable=False),
+        sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("finished_at", sa.DateTime(timezone=True)),
+        sa.Column("summary", sa.Text()),
+        sa.Column("result", postgresql.JSONB()),
+        sa.Column("error", sa.Text()),
+        sa.Column("failure", postgresql.JSONB()),
+        sa.Column("quota_units", sa.Integer(), nullable=False, server_default="0"),
+        sa.CheckConstraint(
+            "status IN ('running', 'ok', 'error', 'skipped', 'interrupted')",
+            name="ck_job_runs_status",
+        ),
+        sa.CheckConstraint(
+            "trigger IN ('scheduled', 'manual', 'ad_hoc')", name="ck_job_runs_trigger"
+        ),
+        sa.CheckConstraint(
+            "(status = 'running') = (finished_at IS NULL)", name="ck_job_runs_finished"
+        ),
+    )
+    op.create_index("ix_job_runs_job_started", "job_runs", ["job", "started_at"])
+    op.create_index("ix_job_runs_actor_id", "job_runs", ["actor_id"])
+    op.create_index("ix_job_runs_started_at", "job_runs", ["started_at"])
+
+    op.add_column(
+        "quota_events",
+        sa.Column("job_run_id", sa.BigInteger(), sa.ForeignKey("job_runs.id", ondelete="SET NULL")),
+    )
+    op.create_index("ix_quota_events_job_run_id", "quota_events", ["job_run_id"])
+
+    op.add_column("log_events", sa.Column("run_id", sa.BigInteger()))
+    op.create_index(
+        "ix_log_events_run",
+        "log_events",
+        ["run_id"],
+        postgresql_where=sa.text("run_id IS NOT NULL"),
+    )
+
+    # The audit trail's runs move over: the three statuses the scheduler wrote, and `interrupted`,
+    # which a downgrade writes back (as it does a run's trigger, target, spend and start, which the
+    # audit row has no column for: its actor alone cannot say "manual" once that admin is deleted),
+    # so a rollback and a redeploy keep every run as it was. A status the run table does
+    # not know would fail the CHECK.
+    op.execute(
+        """
+        INSERT INTO job_runs (job, trigger, actor_id, target, status, started_at, finished_at,
+                              summary, result, error, failure, quota_units)
+        SELECT target_id,
+               coalesce(after::jsonb ->> 'trigger',
+                        CASE WHEN actor_id IS NULL THEN 'scheduled' ELSE 'manual' END),
+               actor_id,
+               after::jsonb ->> 'target',
+               after::jsonb ->> 'status',
+               coalesce((after::jsonb ->> 'started_at')::timestamptz, created_at),
+               created_at,
+               CASE WHEN after::jsonb ->> 'status' <> 'error' THEN after::jsonb ->> 'summary' END,
+               CASE WHEN after::jsonb ->> 'status' <> 'error' THEN after::jsonb -> 'result' END,
+               CASE WHEN after::jsonb ->> 'status' = 'error' THEN after::jsonb ->> 'summary' END,
+               CASE WHEN after::jsonb ->> 'status' = 'error' THEN after::jsonb -> 'result' END,
+               coalesce((after::jsonb ->> 'quota_units')::int, 0)
+          FROM audit_log
+         WHERE action = 'job_run'
+           AND target_id IS NOT NULL
+           AND after::jsonb ->> 'status' IN ('ok', 'error', 'skipped', 'interrupted')
+         ORDER BY id
+        """
+    )
+    op.execute(
+        """
+        DELETE FROM audit_log
+         WHERE action = 'job_run'
+           AND target_id IS NOT NULL
+           AND after::jsonb ->> 'status' IN ('ok', 'error', 'skipped', 'interrupted')
+        """
+    )
+
+
+def downgrade() -> None:
+    # A run still `running` is one the rollback stopped: it goes back as interrupted. What the audit
+    # row's own columns cannot say rides in `after` (its trigger, target, spend and start), so the
+    # upgrade restores them.
+    op.execute(
+        """
+        INSERT INTO audit_log (actor_id, action, target_type, target_id, summary, after, created_at)
+        SELECT actor_id,
+               'job_run',
+               'job',
+               job,
+               left(job || ': ' || st
+                    || coalesce(' — ' || CASE WHEN st = 'error' THEN error ELSE summary END, ''),
+                    255),
+               (jsonb_build_object(
+                    'status', st,
+                    'summary', CASE WHEN st = 'error' THEN error ELSE summary END)
+                || CASE
+                     WHEN st = 'error' AND failure IS NOT NULL
+                       THEN jsonb_build_object('result', failure)
+                     WHEN st <> 'error' AND result IS NOT NULL
+                       THEN jsonb_build_object('result', result)
+                     ELSE '{}'::jsonb
+                   END
+                || jsonb_build_object('trigger', trigger, 'quota_units', units,
+                                      'started_at', started_at)
+                || CASE WHEN target IS NOT NULL
+                     THEN jsonb_build_object('target', target) ELSE '{}'::jsonb END)::json,
+               coalesce(finished_at, started_at)
+          FROM (SELECT *, CASE WHEN status = 'running' THEN 'interrupted' ELSE status END AS st,
+                       -- A run stopped mid-way has not summed its spend yet: sum it as the startup
+                       -- sweep would, before the link to it is dropped.
+                       CASE WHEN status = 'running'
+                         THEN (SELECT coalesce(sum(q.units), 0) FROM quota_events q
+                                WHERE q.job_run_id = job_runs.id)
+                         ELSE quota_units END AS units
+                  FROM job_runs) AS r
+         ORDER BY id
+        """
+    )
+    op.drop_index("ix_log_events_run", table_name="log_events")
+    op.drop_column("log_events", "run_id")
+    op.drop_index("ix_quota_events_job_run_id", table_name="quota_events")
+    op.drop_column("quota_events", "job_run_id")
+    op.drop_index("ix_job_runs_started_at", table_name="job_runs")
+    op.drop_index("ix_job_runs_actor_id", table_name="job_runs")
+    op.drop_index("ix_job_runs_job_started", table_name="job_runs")
+    op.drop_table("job_runs")

backend/alembic/versions/0110_intervals_in_sysconfig.py

@@ -0,0 +1,79 @@
+"""Job intervals move into sysconfig; the unread SponsorBlock default goes
+
+Revision ID: 0110_intervals_in_sysconfig
+Revises: 0109_job_runs
+Create Date: 2026-10-18
+
+An admin's interval override lived in its own table, `scheduler_settings`, beside the one store every
+other admin knob uses. Each override becomes the `job_interval_<job>` sysconfig key and the table
+goes. Only the jobs that still exist are copied: a row for any other id never did anything.
+
+The `sponsorblock_default` setting was offered on the Configuration page but nothing ever read it, so
+its key is gone and its stored value with it — the downgrade cannot bring that value back.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0110_intervals_in_sysconfig"
+down_revision: str | None = "0109_job_runs"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+# The interval jobs as of this revision (a migration does not read the live code's list).
+_JOBS = (
+    "rss_poll",
+    "enrich",
+    "backfill",
+    "autotag",
+    "shorts",
+    "subscriptions",
+    "playlist_sync",
+    "maintenance",
+    "explore_cleanup",
+    "download_gc",
+    "plex_sync",
+    "plex_watch_sync",
+    "plex_watch_reconcile",
+    "tmdb_enrich",
+    "people_enrich",
+    "audit_gc",
+    "log_gc",
+)
+_PREFIX = "job_interval_"
+
+
+def upgrade() -> None:
+    op.execute(
+        sa.text(
+            "INSERT INTO system_config (key, value, encrypted)"
+            " SELECT :prefix || job_id, interval_minutes::text, false FROM scheduler_settings"
+            " WHERE interval_minutes IS NOT NULL AND job_id IN :jobs"
+            " ON CONFLICT (key) DO NOTHING"
+        ).bindparams(sa.bindparam("jobs", expanding=True), prefix=_PREFIX, jobs=list(_JOBS))
+    )
+    op.drop_table("scheduler_settings")
+    op.execute("DELETE FROM system_config WHERE key = 'sponsorblock_default'")
+
+
+def downgrade() -> None:
+    op.create_table(
+        "scheduler_settings",
+        sa.Column("job_id", sa.String(40), primary_key=True),
+        sa.Column("interval_minutes", sa.Integer(), nullable=False),
+    )
+    op.execute(
+        sa.text(
+            "INSERT INTO scheduler_settings (job_id, interval_minutes)"
+            " SELECT substr(key, :start), value::int FROM system_config WHERE key IN :keys"
+        ).bindparams(
+            sa.bindparam("keys", expanding=True),
+            start=len(_PREFIX) + 1,
+            keys=[_PREFIX + j for j in _JOBS],
+        )
+    )
+    op.execute(
+        sa.text("DELETE FROM system_config WHERE key IN :keys").bindparams(
+            sa.bindparam("keys", expanding=True), keys=[_PREFIX + j for j in _JOBS]
+        )
+    )

backend/alembic/versions/0111_audit_request_id.py

@@ -0,0 +1,28 @@
+"""An audit row keeps the request that made it
+
+Revision ID: 0111_audit_request_id
+Revises: 0110_intervals_in_sysconfig
+Create Date: 2026-10-18
+
+The activity page opens an admin action's log lines from its row: `audit_log.request_id` is the
+request the change was made in, the same id `log_events.request_id` carries. Older rows have none —
+the request that made them was never recorded.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0111_audit_request_id"
+down_revision: str | None = "0110_intervals_in_sysconfig"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.add_column("audit_log", sa.Column("request_id", sa.String(64)))
+    op.create_index("ix_audit_log_request_id", "audit_log", ["request_id"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_audit_log_request_id", table_name="audit_log")
+    op.drop_column("audit_log", "request_id")

backend/alembic/versions/0112_one_watch_later.py

@@ -0,0 +1,70 @@
+"""One Watch later list per user, held by the database
+
+Revision ID: 0112_one_watch_later
+Revises: 0111_audit_request_id
+Create Date: 2026-10-20
+
+Watch later was made on a user's first save by a read then an insert, so two first saves at once
+each made a list, and the reader then picked between them at random. A partial unique index holds
+"one per user" now. An instance that already has a second list keeps the oldest: the others' videos
+move into it (after its own, in their order; one it already holds is not added twice), a Home pin
+of one of them moves to it (unless it is pinned itself; of several, the first on the shelf), then
+they go — a pin left on them goes with them, as the kept list already holds one.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0112_one_watch_later"
+down_revision: str | None = "0111_audit_request_id"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+_EXTRA = """
+    SELECT p.id, k.keep_id
+    FROM playlists p
+    JOIN (
+        SELECT user_id, min(id) AS keep_id FROM playlists WHERE kind = 'watch_later' GROUP BY user_id
+    ) k ON k.user_id = p.user_id
+    WHERE p.kind = 'watch_later' AND p.id <> k.keep_id
+"""
+
+
+def upgrade() -> None:
+    op.execute(
+        f"""
+        INSERT INTO playlist_items (playlist_id, video_id, position, added_at)
+        SELECT e.keep_id, i.video_id,
+               (SELECT coalesce(max(position), -1) FROM playlist_items WHERE playlist_id = e.keep_id)
+               + row_number() OVER (PARTITION BY e.keep_id ORDER BY i.playlist_id, i.position, i.id),
+               i.added_at
+        FROM playlist_items i
+        JOIN ({_EXTRA}) e ON e.id = i.playlist_id
+        ON CONFLICT ON CONSTRAINT uq_playlist_video DO NOTHING
+        """
+    )
+    op.execute(
+        f"""
+        UPDATE comfort_pins c SET playlist_id = m.keep_id
+        FROM (
+            SELECT DISTINCT ON (e.keep_id) p.id AS pin_id, e.keep_id
+            FROM comfort_pins p
+            JOIN ({_EXTRA}) e ON e.id = p.playlist_id
+            WHERE NOT EXISTS (SELECT 1 FROM comfort_pins k WHERE k.playlist_id = e.keep_id)
+            ORDER BY e.keep_id, p.position, p.id
+        ) m
+        WHERE c.id = m.pin_id
+        """
+    )
+    op.execute(f"DELETE FROM playlists WHERE id IN (SELECT id FROM ({_EXTRA}) e)")
+    op.create_index(
+        "uq_playlist_watch_later",
+        "playlists",
+        ["user_id"],
+        unique=True,
+        postgresql_where=sa.text("kind = 'watch_later'"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("uq_playlist_watch_later", table_name="playlists")

backend/alembic/versions/0113_takeout_items.py

@@ -0,0 +1,51 @@
+"""A user's Google Takeout export: what they watched on YouTube, and their Watch later list
+
+Revision ID: 0113_takeout_items
+Revises: 0112_one_watch_later
+Create Date: 2026-10-23
+
+One row per video and kind. A video the catalog does not know yet waits with no outcome, so the id
+is not a foreign key; the partial index is what a newly known video looks its waiting rows up by.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0113_takeout_items"
+down_revision: str | None = "0112_one_watch_later"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "takeout_items",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column("kind", sa.String(12), nullable=False),
+        sa.Column("video_id", sa.String(16), nullable=False),
+        sa.Column("channel_id", sa.String(32), nullable=True),
+        sa.Column("at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("outcome", sa.String(12), nullable=True),
+        sa.Column(
+            "created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
+        ),
+        sa.UniqueConstraint("user_id", "kind", "video_id", name="uq_takeout_item"),
+        sa.CheckConstraint("kind IN ('watched', 'watch_later')", name="ck_takeout_item_kind"),
+    )
+    op.create_index(
+        "ix_takeout_items_waiting",
+        "takeout_items",
+        ["video_id"],
+        postgresql_where=sa.text("outcome IS NULL"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_takeout_items_waiting", table_name="takeout_items")
+    op.drop_table("takeout_items")

backend/alembic/versions/0114_video_unmarks.py

@@ -0,0 +1,44 @@
+"""When a user last took a video back to "new"
+
+Revision ID: 0114_video_unmarks
+Revises: 0113_takeout_items
+Create Date: 2026-10-23
+
+Un-marking or resetting a video leaves no `video_states` row, so this is what tells an import of the
+user's YouTube history a video they un-marked here from one never touched.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0114_video_unmarks"
+down_revision: str | None = "0113_takeout_items"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "video_unmarks",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column(
+            "video_id",
+            sa.String(16),
+            sa.ForeignKey("videos.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column("at", sa.DateTime(timezone=True), nullable=False),
+        sa.UniqueConstraint("user_id", "video_id", name="uq_video_unmark"),
+    )
+    op.create_index("ix_video_unmarks_video_id", "video_unmarks", ["video_id"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_video_unmarks_video_id", table_name="video_unmarks")
+    op.drop_table("video_unmarks")

backend/alembic/versions/0115_takeout_channels.py

@@ -0,0 +1,49 @@
+"""A watched Takeout video's length, and the channels a user put out of their list
+
+Revision ID: 0115_takeout_channels
+Revises: 0114_video_unmarks
+Create Date: 2026-10-24
+
+The "channels you watch but do not follow" list counts a user's watched videos over five minutes
+by channel. The length is asked of YouTube for that list and kept on the Takeout row; a channel the
+user does not want on the list is kept apart.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0115_takeout_channels"
+down_revision: str | None = "0114_video_unmarks"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "takeout_items", sa.Column("looked_up_at", sa.DateTime(timezone=True), nullable=True)
+    )
+    op.add_column("takeout_items", sa.Column("duration_seconds", sa.Integer(), nullable=True))
+    op.create_table(
+        "takeout_channel_dismissals",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column("channel_id", sa.String(32), nullable=False),
+        sa.Column(
+            "created_at",
+            sa.DateTime(timezone=True),
+            server_default=sa.func.now(),
+            nullable=False,
+        ),
+        sa.UniqueConstraint("user_id", "channel_id", name="uq_takeout_channel_dismissal"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("takeout_channel_dismissals")
+    op.drop_column("takeout_items", "duration_seconds")
+    op.drop_column("takeout_items", "looked_up_at")

backend/alembic/versions/0116_followed_explored_channels.py

@@ -0,0 +1,34 @@
+"""An explored channel someone follows is the catalog's own, and so are its videos
+
+Revision ID: 0116_followed_explored_channels
+Revises: 0115_takeout_channels
+Create Date: 2026-10-24
+
+A follow pulled in from YouTube (the subscription import) left an explored channel flagged as an
+exploration's, and a subscribe landing while an exploration's videos came in left those videos
+flagged on a channel that no longer was one. Both are held in the code now; this clears what they
+left: the channel of a subscription is not an exploration's, and no video is an exploration's on a
+channel that is not one. Data only, so the downgrade has nothing to undo.
+"""
+
+from alembic import op
+
+revision: str = "0116_followed_explored_channels"
+down_revision: str | None = "0115_takeout_channels"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.execute(
+        "UPDATE channels SET from_explore = false WHERE from_explore"
+        " AND EXISTS (SELECT 1 FROM subscriptions s WHERE s.channel_id = channels.id)"
+    )
+    op.execute(
+        "UPDATE videos SET via_explore = false WHERE via_explore"
+        " AND channel_id IN (SELECT id FROM channels WHERE NOT from_explore)"
+    )
+
+
+def downgrade() -> None:
+    pass

backend/alembic/versions/0117_videos_length_sort_indexes.py

@@ -0,0 +1,76 @@
+"""videos: the duration sorts ride an index on the video's length, not on the raw column
+
+Revision ID: 0117_videos_length_sort_indexes
+Revises: 0116_followed_explored_channels
+Create Date: 2026-11-07
+
+A YouTube length past a day is not a length (`catalog.models.LENGTH_MAX_S`): a 24/7 stream's archive
+reports the years it has run. The feed's duration sorts now order by `Video.length_seconds`, which
+is NULL for such a video, so "longest first" no longer opens on a stream archive. Migration 0096's
+two duration indexes are on the raw column and no longer match that ORDER BY; they are replaced by
+the same indexes over the length expression. The expression is spelled out here as SQLAlchemy emits
+it, with the limit as a literal (an index matches a query's constant, never a bound parameter);
+tests/test_feed_sort_indexes.py fails if the two drift apart.
+
+CONCURRENTLY (autocommit_block), re-runnable, as in 0096.
+"""
+
+from alembic import op
+from sqlalchemy import text
+
+revision: str = "0117_videos_length_sort_indexes"
+down_revision: str | None = "0116_followed_explored_channels"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+# Frozen with this migration: `catalog.models.LENGTH_MAX_S` is 24 h.
+LENGTH = "(CASE WHEN (duration_seconds > 86400) THEN NULL ELSE duration_seconds END)"
+
+NEW = {
+    "ix_videos_length_desc": f"{LENGTH} DESC NULLS LAST, id",
+    "ix_videos_length_asc": f"{LENGTH} ASC NULLS LAST, id",
+}
+OLD = {
+    "ix_videos_duration_desc": "duration_seconds DESC NULLS LAST, id",
+    "ix_videos_duration_asc": "duration_seconds ASC NULLS LAST, id",
+}
+
+
+def _valid(name: str) -> bool | None:
+    """The index's `indisvalid`, or None when there is no index of that name."""
+    row = (
+        op.get_bind()
+        .execute(
+            text(
+                "SELECT i.indisvalid FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid "
+                "WHERE c.relname = :name"
+            ),
+            {"name": name},
+        )
+        .first()
+    )
+    return None if row is None else bool(row[0])
+
+
+def _build(name: str, columns: str) -> None:
+    """CREATE INDEX CONCURRENTLY, re-runnable: an INVALID leftover of a failed build is dropped first
+    (see 0096 `_build`)."""
+    if _valid(name) is False:
+        op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")
+    op.execute(f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {name} ON videos ({columns})")
+
+
+def upgrade() -> None:
+    with op.get_context().autocommit_block():
+        for name, columns in NEW.items():
+            _build(name, columns)
+        for name in OLD:
+            op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")
+
+
+def downgrade() -> None:
+    with op.get_context().autocommit_block():
+        for name, columns in OLD.items():
+            _build(name, columns)
+        for name in NEW:
+            op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")

backend/alembic/versions/0118_videos_shuffle_key.py

@@ -0,0 +1,40 @@
+"""videos: a random key per video, so a seeded shuffle walks an index instead of sorting the scope
+
+Revision ID: 0118_videos_shuffle_key
+Revises: 0117_videos_length_sort_indexes
+Create Date: 2026-11-07
+
+The feed's Shuffle sort and the Home surprise ordered the whole feed scope by md5(id || seed) on every
+call: ~350 ms on prod for a scope of ~213k videos, and a Shuffle page deeper in the list cost more
+(767 ms on page 7 on localdev, against 4 ms for Newest). Each video now carries a random number drawn
+once (`Video.shuffle_key`); a seed picks a point on that ring and the draw reads the index from there
+(`feedquery.shuffle_walk`).
+
+One ADD COLUMN with a volatile default rewrites the table and fills every row with its own value. On
+localdev (278k videos) that took 28 s, against 84 s for an UPDATE of a nullable column, so the rewrite
+it is: the table is locked for that time, and a deploy has the app down while it migrates anyway.
+The index is (shuffle_key, id): the walk's keyset cursor compares the pair.
+"""
+
+from alembic import op
+
+revision: str = "0118_videos_shuffle_key"
+down_revision: str | None = "0117_videos_length_sort_indexes"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.execute(
+        "ALTER TABLE videos ADD COLUMN IF NOT EXISTS shuffle_key double precision "
+        "NOT NULL DEFAULT random()"
+    )
+    with op.get_context().autocommit_block():
+        op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_shuffle_key")
+        op.execute("CREATE INDEX CONCURRENTLY ix_videos_shuffle_key ON videos (shuffle_key, id)")
+
+
+def downgrade() -> None:
+    with op.get_context().autocommit_block():
+        op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_shuffle_key")
+    op.execute("ALTER TABLE videos DROP COLUMN IF EXISTS shuffle_key")

backend/alembic/versions/0119_plex_transcode_gpu.py

@@ -0,0 +1,65 @@
+"""plex_items.transcode_hw: the facet says whether the GPU ENCODES, under the measured codec sets
+
+Measured on prod (2026-11-07): the iGPU has no VC-1 Advanced decode, so a VC-1 film never ran on the
+GPU although the facet said it did; VC-1, XviD and DivX now take the CPU-decode → GPU-encode path; and an
+HDR film runs on the GPU through the OpenCL tonemap although the facet said software. The rule is
+`plex.sync.transcode_hw_capable`; the two codec sets below are pinned to `plex.sync` by
+tests/test_plex_playback_class.py.
+
+Rewritten from the stored codec and bit depth. HDR is not stored, so a 10-bit HW-decodable item is set to
+software here and its probe watermark cleared: the bit-depth backfill (`plex.sync._probe_media_batch`,
+a bounded batch per Plex sync) reads its HDR flag again and moves an HDR one back to the GPU.
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+revision: str = "0119_plex_transcode_gpu"
+down_revision: Union[str, None] = "0118_videos_shuffle_key"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+HW_DECODABLE_CODECS = ("h264", "hevc", "h265", "vp9", "mpeg2video")
+CPU_DECODE_CODECS = ("vc1", "mpeg4", "msmpeg4v3")
+
+
+def _in(codecs: tuple[str, ...]) -> str:
+    return ", ".join(f"'{c}'" for c in codecs)
+
+
+def upgrade() -> None:
+    op.execute(
+        f"""
+        UPDATE plex_items
+           SET transcode_hw = CASE
+               WHEN lower(codec_video) IN ({_in(HW_DECODABLE_CODECS)})
+               THEN coalesce(video_bit_depth, 8) <= 8
+               WHEN lower(codec_video) IN ({_in(CPU_DECODE_CODECS)})
+               THEN coalesce(video_bit_depth, 8) <= 8
+               ELSE false
+           END
+         WHERE playable = 'transcode'
+        """
+    )
+    op.execute(
+        f"""
+        UPDATE plex_items
+           SET media_probed_at = NULL
+         WHERE playable = 'transcode'
+           AND lower(codec_video) IN ({_in(HW_DECODABLE_CODECS)})
+           AND video_bit_depth > 8
+        """
+    )
+
+
+def downgrade() -> None:
+    # The old rule: a decodable codec (VC-1 counted) at 8 bits; HDR and every other codec software.
+    op.execute(
+        f"""
+        UPDATE plex_items
+           SET transcode_hw = lower(codec_video) IN ({_in(HW_DECODABLE_CODECS + ("vc1",))})
+                              AND coalesce(video_bit_depth, 8) <= 8
+         WHERE playable = 'transcode'
+        """
+    )

backend/app/README.md

@@ -1,9 +1,8 @@
 # Backend conventions
 
 The rules below are already followed across `backend/app` — they lived only in a few module
-docstrings (`userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
-B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
-I copy".
+docstrings (`userscope.py`, `routes/_common.py`) until this file collected them. When you add code,
+match these so the next reader doesn't have to guess "which pattern do I copy".
 
 ## Module layering
 
@@ -13,18 +12,21 @@ Three layers, by *who* runs the code — not by feature:
    response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
    Business logic that a request happens to trigger does **not** live here if anything else runs it too.
 
-2. **`plex/`, `downloads/`, `sync/`, `issues/` — domain logic shared by more than one caller.** Anything the
-   **scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
-   route, the worker, and the scheduler share one definition. Examples from this codebase:
-   `downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
-   run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline;
-   `issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
+2. **The units and the other domain packages — logic shared by more than one caller.** A unit's
+   database work (below), and anything the **scheduler or the worker** runs (not just a request), lives
+   in a domain package, so the route, the worker, and the scheduler share one definition. Examples from
+   this codebase: `downloads/service.py` owns enqueue + the job state transitions (routes call it);
+   `plex/sync.py` is run by both the admin route and the scheduler; `downloads/pipeline.py` is the
+   worker's child pipeline; `issues/bundle.py` is the one definition of a bug report's files, for the
+   zip export and the tracker.
 
-3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth`, `sysconfig`, `quota`, `audit`,
+3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's
+   folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every
+   background run's durable row, its words in `runstatus`),
    `notifications`, `userscope`, `ratelimit`, `progress`, `logs` (the one logging
    configuration), `logsink` (its `log_events` table), `diagnostics` (a user's consented log
-   level), `debugsession` (a recording that tags its user's lines, E16) and `collation` (A→Z in the
-   reader's alphabet, E30), … are single-file primitives many layers import. They depend on nothing above them (a core
+   level), `debugsession` (a recording that tags its user's lines, E16), `collation` (A→Z in the
+   reader's alphabet, E30) and `draw` (the weighted, seeded draw "Surprise me" deals from), … are single-file primitives many layers import. They depend on nothing above them (a core
    module must never import a `routes/…` module).
 
 **Extraction trigger: the SECOND consumer.** Keep new logic inline in its first caller. The moment a
@@ -32,19 +34,59 @@ Three layers, by *who* runs the code — not by feature:
 a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
 pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
 
+## Units
+
+The domain is four units, each owning its tables (`models.py` in its package):
+
+| Unit | Package | Its routes |
+| --- | --- | --- |
+| Capture | `downloads/`, the worker | `downloads.py`, `files.py`, `public.py` |
+| Reader | `reader/`, `discover/`, `feedquery.py`, the playlist sync and tag suggestions in `sync/` | `feed.py`, `playlists.py`, `saved_views.py`, `discover.py`, `tags.py` |
+| Catalog | `catalog/`, the ingests in `sync/` | `channels.py`, `search.py`, `sync.py` |
+| Plex | `plex/` | `plex/` |
+
+- **A unit's route does no database work.** It validates, reads through the unit's `queries.py`,
+  writes through its `service.py`, serializes, and commits. It may name a unit's model as a type
+  (under `TYPE_CHECKING`), never use one as a value, and it passes a kind (`"movie"`), not a model, to
+  a query.
+- **A unit's code keeps to its edges.** It reads another unit's tables only along a declared edge
+  (Capture and Reader read Catalog's and Plex's: the feed joins `Video` with `VideoState` in one
+  query) and writes them only through the owner's service. Core code reaches a unit's tables through
+  that unit's `queries`.
+- **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id,
+  user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the
+  same answer as a missing one). A unit's `queries.own_x` returns None and its route answers
+  `not_found`; a core route calls `owned_or_404`. A rule `owned_row` cannot say — a job its owner
+  shared, a built-in profile nobody owns, a folder read under `FOR SHARE` — is written out in the
+  unit's own code, beside the reason.
+
+All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's
+packages and edges (`UNIT_PACKAGES`, `READS`) and what may commit (`UNIT_COMMITS`, `CORE_COMMITS`)
+are listed there. It reads the code's AST, so it holds the forms written here: an owner column
+compared inline (a model's, an import alias's or an `aliased()` one's, by `==`, `!=`, `.in_()`, or a
+`filter_by(user_id=…)`), a model built or an `insert`/`update`/`delete` (under any import name, of a
+model or its `__table__`) across units, and a `.commit()` call off the lists all fail it. What an AST
+cannot tell is review's to catch: an ORM write on another unit's loaded row (an attribute, a
+`db.delete`), SQL in `text()`, a column or model reached another way (bound to a variable, through
+`__table__.c` or a module path), and a commit made by a `begin()` block or a `commit` handed in.
+
 ## Transaction ownership
 
-This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
-(only `quota` owns its session)"** — so read it that way, not as an exception to it. The commit belongs
-to the **operation owner**; anything called *inside* another transaction only stages its change
+This follows CLAUDE.md's canonical rule — **the handler commits; a unit's `service` and every helper
+flush** — so read it that way, not as an exception to it. The commit belongs to the **operation
+owner**; anything called *inside* another transaction only stages its change
 (`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
 `audit.record()` is the reference: it adds the audit row and the caller commits it together with the
 mutation it records, so an audit entry can never exist without its change (or vice-versa).
 
-- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
-  whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
-  a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
-  the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
+- The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it
+  makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the
+  request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, the
+  scheduler's jobs, a unit's jobs (the worker's `jobdb` writes, the syncs and ingests that commit per
+  page or batch so a later failure loses nothing already stored), and a few core steps that must land
+  whatever their caller does next (a token's last-used stamp, a refreshed YouTube grant) — the fence's
+  `UNIT_COMMITS` and `CORE_COMMITS` name each with its reason, `test_route_conventions` a route's own
+  committing helpers. Everything else flushes.
 - `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
   loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
   attribute — see `service.decr_ref_count`.
@@ -66,8 +108,9 @@ mutation it records, so an audit entry can never exist without its change (or vi
 - **`app.config.settings`** — boot/env config (DB URL, download root, worker concurrency): read once at
   startup, not user-tunable at runtime.
 - **`sysconfig.SPECS`** — the runtime admin knobs (DB-override-or-default). A new admin-tunable value
-  goes in `SPECS`; don't invent a fourth store. (`state.py`'s `maintenance_batch` predates this and is
-  slated to fold into `sysconfig` — do not add more like it.)
+  goes in `SPECS`; don't invent a fourth store. Each spec states when a change takes effect
+  (`applies`, required): `live` keys are read where they are used, never at import (test-held). The
+  scheduler's job intervals are `SPECS` keys too, edited on the Scheduler page, not the Configuration page.
 
 ## The gate
 

backend/app/apitokens.py

@@ -0,0 +1,276 @@
+"""Machine access to the Files library: API tokens, and what each one may do where.
+
+A user makes a token for a machine of theirs — their hub, a script — and it reaches their library as
+`Authorization: Bearer sfl_…`. Only on `/api/files`: the routes there take `files_caller` instead of
+`current_user`, and nothing else does (a test holds the route table to that), so a token can never
+manage the account, the tokens, or anything outside the library. And only in the folders its grants
+name, with the rights they give: list, read, write, delete. Every route asks the one question,
+`FilesCaller.may(folder_id, right)`; a signed-in user may everything in their own folders.
+
+The token is `sfl_` + 32 random bytes in base62 + a CRC32 of them: a mistyped or cut-off token is
+refused without a database read, and a secret scanner can tell one by its shape. Only its SHA-256 is
+kept (`security.hash_token`) — the user sees it once, when it is made — with its first characters to
+tell tokens apart by. It is looked up by that hash on every request: no signed claims, so a deleted
+token stops working at once. A token about to run out warns its user once, a week before
+(`warn_expiring`), so a machine does not lose its key unannounced.
+"""
+
+import secrets
+import zlib
+from collections.abc import Mapping
+from dataclasses import dataclass
+from datetime import datetime, timedelta, timezone
+from enum import StrEnum
+
+from fastapi import Depends, Request
+from sqlalchemy import delete, or_, select, update
+from sqlalchemy.orm import Session
+
+from app.userscope import owned_by
+from app import audit, debugsession, logs
+from app.audit import AuditAction
+from app.auth import current_user
+from app.db import get_db
+from app.models import ApiToken, ApiTokenGrant, User
+from app.notifications import create_notification
+from app.ratelimit import RateLimiter, client_key
+from app.refusal import refused
+from app.security import hash_token
+from app.utils import iso
+
+log = logs.get_logger(__name__)
+
+PREFIX = "sfl_"
+_ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
+_BODY_BYTES = 32
+_BODY_LEN = 43  # 62**43 > 2**256: every 32-byte value fits
+_CHECK_LEN = 6  # 62**6 > 2**32: every CRC32 fits
+TOKEN_LEN = len(PREFIX) + _BODY_LEN + _CHECK_LEN
+# What a token is shown by once it is made: enough to tell a user's tokens apart, far too little to
+# narrow a guess (8 of its 43 random characters).
+SHOWN_LEN = len(PREFIX) + 8
+
+# A machine that runs away (a loop in a script) is held back per token; a caller trying tokens is
+# held back per client, counting only the refusals.
+_calls = RateLimiter(600, 60)
+_bad = RateLimiter(20, 300)
+# How stale `last_used_at` may get before a request writes it again: a busy script costs one write a
+# minute, not one per request.
+_TOUCH_EVERY = timedelta(minutes=1)
+# How long before a token runs out its user is told (`warn_expiring`); the token list marks it
+# "expires soon" for the same span.
+EXPIRY_WARNING = timedelta(days=7)
+
+
+class Right(StrEnum):
+    LIST = "list"
+    READ = "read"
+    WRITE = "write"
+    DELETE = "delete"
+
+
+# Each right's column on a grant. A dict over the enum, so a new right without a column fails here.
+GRANT_COLUMNS = {
+    Right.LIST: "can_list",
+    Right.READ: "can_read",
+    Right.WRITE: "can_write",
+    Right.DELETE: "can_delete",
+}
+
+
+def _b62(value: int, width: int) -> str:
+    out = []
+    for _ in range(width):
+        value, digit = divmod(value, 62)
+        out.append(_ALPHABET[digit])
+    return "".join(reversed(out))
+
+
+def _check(body: str) -> str:
+    return _b62(zlib.crc32(body.encode()), _CHECK_LEN)
+
+
+def mint() -> str:
+    """A new token, in full. The caller stores its hash and shows it to the user, once."""
+    body = _b62(int.from_bytes(secrets.token_bytes(_BODY_BYTES)), _BODY_LEN)
+    return PREFIX + body + _check(body)
+
+
+def well_formed(raw: str) -> bool:
+    """Whether `raw` has a token's shape and its checksum adds up."""
+    if len(raw) != TOKEN_LEN or not raw.startswith(PREFIX):
+        return False
+    body, check = raw[len(PREFIX) : -_CHECK_LEN], raw[-_CHECK_LEN:]
+    # The check too: compare_digest refuses (TypeError, a 500) a str with anything non-ASCII in it.
+    if any(c not in _ALPHABET for c in body + check):
+        return False
+    return secrets.compare_digest(_check(body), check)
+
+
+def grant_rights(grant: ApiTokenGrant) -> frozenset[Right]:
+    return frozenset(r for r, col in GRANT_COLUMNS.items() if getattr(grant, col))
+
+
+@dataclass(frozen=True)
+class FilesCaller:
+    """Who is asking the Files library, and where they may do what. `grants` is None for a signed-in
+    user (everything in their own folders); for a token, folder id → its rights there."""
+
+    user: User
+    token_id: int | None = None
+    token_prefix: str | None = None
+    grants: Mapping[int, frozenset[Right]] | None = None
+
+    def may(self, folder_id: int | None, right: Right) -> bool:
+        if self.grants is None:
+            return True
+        return folder_id is not None and right in self.grants.get(folder_id, frozenset())
+
+    def folders_with(self, right: Right) -> set[int] | None:
+        """The folders `right` is granted in; None = any of the user's own."""
+        if self.grants is None:
+            return None
+        return {f for f, rights in self.grants.items() if right in rights}
+
+    @property
+    def via(self) -> str | None:
+        """How an audit row says a token did it, None when the user did."""
+        return None if self.token_prefix is None else f"api token {self.token_prefix}…"
+
+
+def bearer_token(request: Request) -> str | None:
+    """The token of an `Authorization: Bearer` header; None when the request carries none (another
+    scheme counts as none — it is not ours to judge)."""
+    scheme, _, value = request.headers.get("authorization", "").partition(" ")
+    if scheme.lower() != "bearer":
+        return None
+    return value.strip()
+
+
+def session_caller(request: Request, db: Session = Depends(get_db)) -> User | None:
+    """The signed-in user, or None when the request brings a token instead: a token wins over a
+    cookie, so a machine's request is judged by its grants alone, whatever else it carries."""
+    if bearer_token(request) is not None:
+        return None
+    return current_user(request, db)
+
+
+def files_caller(
+    request: Request,
+    db: Session = Depends(get_db),
+    user: User | None = Depends(session_caller),
+) -> FilesCaller:
+    """The `/api/files` caller: a signed-in user, or a valid token's user with its grants."""
+    if user is not None:
+        return FilesCaller(user=user)
+    return _token_caller(request, db, bearer_token(request) or "")
+
+
+def _refuse_token(key: str, reason: str) -> Exception:
+    _bad.allow(key)
+    log.info(
+        "a request brought an API token that does not hold",
+        extra=logs.event("apitoken.refused", reason=reason),
+    )
+    return refused(401, "not_authenticated")
+
+
+def _token_caller(request: Request, db: Session, raw: str) -> FilesCaller:
+    key = client_key(request)
+    if _bad.exhausted(key):
+        raise refused(429, "api_token_rate_limited")
+    if not well_formed(raw):
+        raise _refuse_token(key, "malformed")
+    token = db.execute(
+        select(ApiToken).where(ApiToken.token_hash == hash_token(raw))
+    ).scalar_one_or_none()
+    if token is None:
+        raise _refuse_token(key, "unknown")
+    now = datetime.now(timezone.utc)
+    if token.expires_at is not None and token.expires_at <= now:
+        raise _refuse_token(key, "expired")
+    user = db.get(User, token.user_id)
+    if user is None or not user.is_active or user.is_suspended:
+        raise _refuse_token(key, "account")
+    if not _calls.allow(str(token.id)):
+        raise refused(429, "api_token_rate_limited")
+    grants = {
+        g.folder_id: grant_rights(g)
+        for g in db.execute(select(ApiTokenGrant).where(ApiTokenGrant.token_id == token.id))
+        .scalars()
+        .all()
+    }
+    caller = FilesCaller(user=user, token_id=token.id, token_prefix=token.prefix, grants=grants)
+    db.execute(
+        update(ApiToken)
+        .where(
+            ApiToken.id == token.id,
+            or_(ApiToken.last_used_at.is_(None), ApiToken.last_used_at < now - _TOUCH_EVERY),
+        )
+        .values(last_used_at=now)
+        .execution_options(synchronize_session=False)
+    )
+    # Committed here, ahead of the route: a read-only route never commits, and a write route must
+    # not carry this along into a rollback of its own.
+    db.commit()
+    logs.set_user(user.id, *debugsession.level_for(user, None))
+    request.state.user_id = user.id
+    return caller
+
+
+def revoke_all(db: Session, user_id: int) -> int:
+    """Revoke every token of `user_id`, each with its audit row (NOT committed). A token is a
+    credential like a session, so whatever ends every session of an account — a password reset or
+    change, "log out everywhere", a revoked untrusted password — ends its tokens too: one minted
+    from a stolen session must not outlive the owner's answer to that theft. Returns how many."""
+    gone = (
+        db.execute(
+            delete(ApiToken).where(owned_by(ApiToken.user_id, user_id)).returning(ApiToken.id)
+        )
+        .scalars()
+        .all()
+    )
+    for token_id in gone:
+        audit.record(
+            db,
+            user_id,
+            AuditAction.API_TOKEN_REVOKE,
+            target_type="api_token",
+            target_id=token_id,
+            summary="revoked with every session of the account",
+        )
+    return len(gone)
+
+
+def expiring(token: ApiToken, now: datetime) -> bool:
+    """Whether `token` runs out within the warning span (and has not yet)."""
+    return token.expires_at is not None and now < token.expires_at <= now + EXPIRY_WARNING
+
+
+def warn_expiring(db: Session) -> int:
+    """Tell each user whose token runs out within `EXPIRY_WARNING` — once per token: the mark and the
+    notice commit together, and a token made again (a new row) warns afresh. Returns how many."""
+    now = datetime.now(timezone.utc)
+    hits = db.execute(
+        update(ApiToken)
+        .where(
+            ApiToken.expiry_warned.is_(False),
+            ApiToken.expires_at.is_not(None),
+            ApiToken.expires_at > now,
+            ApiToken.expires_at <= now + EXPIRY_WARNING,
+        )
+        .values(expiry_warned=True)
+        .returning(ApiToken.id, ApiToken.user_id, ApiToken.name, ApiToken.expires_at)
+        .execution_options(synchronize_session=False)
+    ).all()
+    for token_id, user_id, name, expires_at in hits:
+        create_notification(
+            db,
+            user_id,
+            "api_token_expiring",
+            title=f"API token {name} expires soon",  # English fallback; the client names it from data
+            data={"token_id": token_id, "name": name, "expires_at": iso(expires_at)},
+            commit=False,
+        )
+    db.commit()
+    return len(hits)

backend/app/audit.py

@@ -1,9 +1,9 @@
-"""Admin/scheduler audit trail: an append-only record of who changed what and when.
+"""Admin audit trail: an append-only record of who changed what and when.
 
-Design (locked): log at ACTION / job-run granularity, NEVER per-row — a scheduler run that
-touches thousands of videos is ONE audit row summarising the outcome. Automatic job runs are
-only logged when they actually changed something (or errored); all admin actions and all manual
-runs are always logged. Secrets are never stored (config changes log the key name only).
+Design (locked): log at ACTION granularity, NEVER per-row — an action that touches thousands of
+rows is ONE audit row summarising the outcome. Background runs are not actions: they have their
+own history (app/jobruns.py). Secrets are never stored (config changes log the key name only).
+Each row keeps the request that made it, so the request's log lines are one click away.
 
 `record()` only db.add()s the row — the caller commits it in the SAME transaction as the
 mutation it describes, so the trail can't drift from reality (and a rolled-back mutation drops
@@ -15,6 +15,7 @@ from datetime import datetime, timedelta, timezone
 from sqlalchemy import delete
 from sqlalchemy.orm import Session
 
+from app import logs
 from app.models import AuditLog
 
 
@@ -48,8 +49,6 @@ class AuditAction:
     MAINTENANCE_TUNE = "maintenance_tune"
     SYNC_PAUSE = "sync_pause"
     SYNC_RESUME = "sync_resume"
-    # Scheduler job outcome / run-summary (actor_id set = a manual "run now"; NULL = automatic)
-    JOB_RUN = "job_run"
     # Per-user download cookie/auth credential set or removed (the value is NEVER recorded)
     DOWNLOAD_CREDENTIAL_SET = "download_credential_set"
     DOWNLOAD_CREDENTIAL_DELETE = "download_credential_delete"
@@ -69,6 +68,16 @@ class AuditAction:
     FILE_FOLDER_RENAME = "file_folder_rename"
     FILE_FOLDER_RETENTION = "file_folder_retention"
     FILE_FOLDER_DELETE = "file_folder_delete"
+    # A machine's token to the library (`routes.tokens`): made (its folders and rights) or revoked.
+    # What a token then did is the file rows above, their summary naming it.
+    API_TOKEN_CREATE = "api_token_create"
+    API_TOKEN_REVOKE = "api_token_revoke"
+    # A public `/watch` link to a file or a folder (`routes.downloads`): made, its settings changed,
+    # revoked.
+    # Never its token or password — only whether it has one.
+    DOWNLOAD_LINK_CREATE = "download_link_create"
+    DOWNLOAD_LINK_UPDATE = "download_link_update"
+    DOWNLOAD_LINK_REVOKE = "download_link_revoke"
 
 
 def record(
@@ -94,16 +103,16 @@ def record(
             summary=(summary or None) and summary[:255],
             before=before,
             after=after,
+            request_id=logs.current_request_id(),
         )
     )
 
 
 def gc(db: Session, retention_days: int) -> int:
     """Delete audit rows older than `retention_days`. Returns the number removed. A
-    retention_days <= 0 disables pruning (keep forever)."""
+    retention_days <= 0 disables pruning (keep forever). Flushes; the job commits."""
     if retention_days <= 0:
         return 0
     cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
     result = db.execute(delete(AuditLog).where(AuditLog.created_at < cutoff))
-    db.commit()
     return result.rowcount or 0

backend/app/auth.py

@@ -4,6 +4,7 @@ from sqlalchemy import delete, func, select
 from sqlalchemy.orm import Session
 from starlette.requests import HTTPConnection
 
+from app.userscope import owned_by
 from app import debugsession
 from app import email as email_mod
 from app import logs
@@ -24,8 +25,6 @@ def operator_contact() -> str | None:
     return next(iter(sorted(settings.admin_email_set)), None)
 
 
-MAX_SESSION_ACCOUNTS = 6
-
 # OAuth scope vocabulary — the atoms + the request bundles, kept together as core. has_*_scope check
 # the atoms against a user's stored grant; the *_SCOPES bundles are what the login/callback endpoints
 # request (imported by app/routes/auth.py).
@@ -230,7 +229,7 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
     # request's access line, a queued row, a later background task would otherwise put rows back
     # under the id after this DELETE (logs.forget_user).
     logs.forget_user(user_id)
-    db.execute(delete(LogEvent).where(LogEvent.user_id == user_id))
+    db.execute(delete(LogEvent).where(owned_by(LogEvent.user_id, user_id)))
     # Every canonically-equal invite goes, not just the literal spelling: is_allowed matches on the
     # canonical form, so leaving a sibling row behind would keep granting access to a mailbox whose
     # account was just erased — and prod really does carry approved pairs for one mailbox.
@@ -252,94 +251,63 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
     background.add_task(email_mod.send_account_deleted, email, lang, operator_contact())
 
 
-ACTIVE_ACCOUNT_HEADER = "x-siftlode-account"
+def session_user_id(conn: HTTPConnection) -> int | None:
+    """The account this browser session signed in as, read from the signed cookie alone (no DB).
+    One account per session: signing in as another replaces it (`routes.auth._establish_session`)."""
+    return conn.session.get("user_id")
 
 
-def resolved_user_id(conn: HTTPConnection, *, strict: bool = True) -> tuple[int | None, bool]:
-    """Which account this connection acts as, and whether that's the session's default account.
-    Takes an HTTPConnection rather than a Request: it reads only the session and the headers.
-
-    The signed cookie holds the *wallet* (`account_ids`, every account signed into this browser)
-    plus a default `user_id`. A caller may name its account with the X-Siftlode-Account header (or
-    `?account=`), but ONLY an account in the wallet — so a tab can't impersonate an account that
-    never authenticated here. A connection that names no account acts as the cookie default.
-
-    A connection naming an account NOT in the wallet acts as nobody (`None`) — typically a tab still
-    pinned to an account that was signed out in another tab. It must not fall back to the default:
-    that tab would go on acting as an account it never showed, its player checkpoints and pref saves
-    landing there and its Sign out signing THAT account out. Only the bootstrap /api/me probe passes
-    `strict=False` and gets the default, which is how such a tab learns who it now is (the frontend
-    re-pins it and reloads). An account whose session ended while it stayed in the wallet (deleted,
-    suspended, revoked) leaves the wallet when it is refused (`_session_user`), so a tab pinned to it
-    takes the same way back.
-    """
-    default_id = conn.session.get("user_id")
-    wallet = conn.session.get("account_ids") or []
-    # Header for normal XHR; ?account= for a context that can't set headers (a plain <a> file
-    # download). Both are wallet-gated below.
-    hdr = conn.headers.get(ACTIVE_ACCOUNT_HEADER) or conn.query_params.get("account")
-    if hdr:
-        try:
-            hid = int(hdr)
-        except ValueError:
-            # Under `strict` this becomes a 401 and the tab authenticates as nobody, which reads
-            # in the browser exactly like a session that expired. The value itself stays out —
-            # it came from a request, and `wallet` says which accounts were actually on offer.
-            log.debug(
-                "the active-account header could not be read as an id",
-                extra=logs.event("auth.account_header.invalid", accounts=len(wallet)),
-            )
-            hid = None
-        if hid is not None and hid in wallet:
-            return hid, hid == default_id
-        if strict:
-            return None, False
-    return default_id, True
+def stamp_session(request: Request, user: User) -> None:
+    """Record the account's current `session_epoch` in the signed cookie, so `current_user` can later
+    reject the cookie once the epoch moves (password reset/change, or "log out everywhere")."""
+    request.session["epoch"] = user.session_epoch or 0
 
 
-def drop_from_wallet(session: dict, user_id: int) -> None:
-    """Take one account out of this browser's wallet (`account_ids`); the session default is untouched."""
-    session["account_ids"] = [a for a in (session.get("account_ids") or []) if a != user_id]
+# The account a tab was built for, sent with every request it makes (`identityHeaders`, the
+# frontend). The cookie alone says who is asking, and one browser holds one account: a sign-in as
+# someone else in another tab replaces it under every open tab. A tab still showing the previous
+# account would otherwise act as the new one — its player checkpoints, pref saves, Sign out and
+# Delete account landing there — until it noticed. It never selects an account; a mismatch is
+# refused, and the tab reloads as whoever is signed in now.
+EXPECTED_ACCOUNT_HEADER = "x-siftlode-expect-account"
 
 
-def _end_account_session(request: Request, user_id: int, is_default: bool) -> None:
-    """An account whose session here is no longer valid. Only nuke the whole session when it is the
-    SESSION DEFAULT; a per-tab account just leaves the wallet (other tabs' default may still be valid).
-    Leaving is what lets a tab pinned to it recover: its next /api/me names an account no longer signed
-    in here and gets the default. Left in, it answered "signed out" — the landing page, with the default
-    still signed in, and no way off it but closing the tab."""
-    if is_default:
-        request.session.clear()
-    else:
-        drop_from_wallet(request.session, user_id)
+def built_for_another_account(user_id: int, expected: str | None) -> bool:
+    """Whether a tab naming `expected` is showing another account than the one signed in. No name is
+    no claim (a caller from before this check, a script): only a mismatch is refused, and the
+    session stays — it is the other tab's sign-in, and it holds. The one comparison for every way a
+    tab names its account: this header on a request, `expect` on a navigation."""
+    if expected is None or expected == str(user_id):
+        return False
+    log.info(
+        "a request from a tab built for another account was refused",
+        extra=logs.event("auth.session.account_changed", user_id=user_id),
+    )
+    return True
 
 
 def current_user(request: Request, db: Session = Depends(get_db)) -> User:
-    return _session_user(request, db, *resolved_user_id(request))
+    user = _session_user(request, db, session_user_id(request))
+    if built_for_another_account(user.id, request.headers.get(EXPECTED_ACCOUNT_HEADER)):
+        raise refused(401, "account_changed")
+    return user
 
 
-def _session_user(request: Request, db: Session, user_id: int | None, is_default: bool) -> User:
+def _session_user(request: Request, db: Session, user_id: int | None) -> User:
     if not user_id:
         raise refused(401, "not_authenticated")
     user = db.get(User, user_id)
     if user is None or not user.is_active or user.is_suspended:
         # Suspended/deactivated mid-session → end it so the block takes effect at once.
-        _end_account_session(request, user_id, is_default)
+        request.session.clear()
         raise refused(401, "not_authenticated")
-    # SA4: reject a cookie whose recorded epoch is behind the account's current one (revoked by a
-    # password reset/change or "log out everywhere"). A cookie with no recorded epoch (pre-SA4, or an
-    # account signed in before this shipped) is treated as epoch 0 — so the FIRST bump revokes it too.
-    epochs = request.session.get("epochs") or {}
-    if int(epochs.get(str(user_id), 0)) != (user.session_epoch or 0):
-        _end_account_session(request, user_id, is_default)
+    # A cookie whose epoch is behind the account's (revoked by a password reset/change or "log out
+    # everywhere") is refused — and so is one with no epoch at all, which only a build before the
+    # one-account session wrote: its holder signs in once more.
+    epoch = request.session.get("epoch")
+    if epoch is None or int(epoch) != (user.session_epoch or 0):
+        request.session.clear()
         raise refused(401, "not_authenticated")
-    # Always keep the session default in the switchable list — covers sessions created before
-    # multi-session existed (their account_ids was never seeded), so the switcher isn't empty.
-    default_id = request.session.get("user_id")
-    accounts = request.session.get("account_ids") or []
-    if default_id and default_id not in accounts:
-        accounts.append(default_id)
-        request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
     # Attribute this request's log lines to the account. Here, not in the request middleware:
     # this is the one place a session becomes an account, and the middleware runs outside both the
     # session middleware and this task's context. `request.state` is shared across middleware layers
@@ -357,25 +325,17 @@ def _session_user(request: Request, db: Session, user_id: int | None, is_default
 def optional_current_user(request: Request, db: Session = Depends(get_db)) -> User | None:
     """Like `current_user` but returns None instead of raising 401 when there's no valid session.
     Used by the bootstrap /api/me probe so the logged-out landing doesn't log a failed request to
-    the browser console (a non-2xx fetch is a console error no matter how the app handles it).
-    The one lenient resolution (`strict=False`): a tab naming an account no longer signed in here gets
-    the default account, which is how it learns who it now is — see `resolved_user_id`. That includes an
-    account refused right here, which `_session_user` has just taken out of the wallet."""
-    user_id, is_default = resolved_user_id(request, strict=False)
+    the browser console (a non-2xx fetch is a console error no matter how the app handles it)."""
+    user_id = session_user_id(request)
+    if user_id is None:
+        return None
     try:
-        return _session_user(request, db, user_id, is_default)
-    # silent-ok: this is the anonymous case as much as the refused one — every landing page hit
-    # comes through here. The second attempt below is the handling, and it reports.
+        return _session_user(request, db, user_id)
     except HTTPException:
-        if is_default:
-            return None
-    try:
-        return _session_user(request, db, *resolved_user_id(request, strict=False))
-    except HTTPException:
-        # Both the pinned account AND the default one were refused, so the tab is signed out
-        # without being told why — and it got here from a session that named a real account.
+        # A session that named an account, refused: the browser is signed out without being told
+        # why, so say it here.
         log.info(
-            "a session was resolved to nobody: neither the named nor the default account held",
+            "a session was refused: its account no longer holds",
             extra=logs.event("auth.session.probe_rejected", user_id=user_id),
         )
         return None

backend/app/bodylimit.py

@@ -72,6 +72,13 @@ DEFAULT_MAX_BODY_BYTES = 1024 * 1024
 # A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
 # `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
 # default, so listing it could only widen what it accepts.
+# A Google Takeout file (`reader.takeout`). A real watch history measured 3.7 MB for 9500 views, ~390
+# bytes a view, so 32 MiB holds ~86k views. YouTube keeps the history 36 months by default on an
+# account made since mid-2020 and for good on an older one, so a heavy viewer's can pass 16 MiB. The
+# file's text is decoded a window at a time and read an entry at a time (`takeout._json_list`), so
+# it costs its bytes and a window, not a tree of it; the route takes two at once across every user.
+TAKEOUT_MAX_BYTES = 32 * 1024 * 1024
+
 ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
     # Ingesting a local file. The body IS the file now — no multipart envelope to allow for —
     # so this ceiling and the route's own cap count exactly the same bytes. `sysconfig`'s
@@ -95,6 +102,7 @@ ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
     # sending it at all, off the `upload_max_bytes` that `GET /api/files/usage` reports.
     ("POST", "/api/files/upload"): settings.download_upload_max_bytes,
     ("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES,
+    ("POST", "/api/takeout"): TAKEOUT_MAX_BYTES,
 }
 
 # The refusal, byte-for-byte what `capped_body`'s `HTTPException(413, {"code": "too_large"})` renders

backend/app/catalog/models.py

@@ -0,0 +1,319 @@
+"""Catalog's tables: the shared YouTube world every user reads — channels and their videos — and
+each user's footprint in it: what they subscribe to, block, explored or found by search. Only this
+unit writes them; another unit reads them along a declared edge, or asks this unit's service."""
+
+from datetime import datetime
+from typing import TYPE_CHECKING
+
+from sqlalchemy import (
+    BigInteger,
+    Boolean,
+    Computed,
+    DateTime,
+    Float,
+    ForeignKey,
+    Integer,
+    JSON,
+    String,
+    Text,
+    UniqueConstraint,
+    case,
+    func,
+    literal_column,
+    null,
+)
+from sqlalchemy.dialects.postgresql import TSVECTOR
+from sqlalchemy.ext.hybrid import hybrid_property
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+from sqlalchemy.sql.elements import ColumnElement
+
+from app.db import Base
+from app.models import TimestampMixin
+
+if TYPE_CHECKING:
+    from app.models import User
+
+
+class Channel(Base, TimestampMixin):
+    """A YouTube channel. Shared across all users (one channel's videos are the same
+    for everyone), so its expensive metadata is fetched and stored only once."""
+
+    __tablename__ = "channels"
+
+    id: Mapped[str] = mapped_column(String(32), primary_key=True)  # "UC..." channel id
+    title: Mapped[str | None] = mapped_column(String(255))
+    handle: Mapped[str | None] = mapped_column(String(255))
+    description: Mapped[str | None] = mapped_column(Text)
+    thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
+    uploads_playlist_id: Mapped[str | None] = mapped_column(String(64))
+    subscriber_count: Mapped[int | None] = mapped_column(BigInteger)
+    video_count: Mapped[int | None] = mapped_column(BigInteger)
+    # Channel "About" metadata (fetched with the same channels.list call, no extra quota):
+    # lifetime view count, the channel's join date, banner image, and external links.
+    total_view_count: Mapped[int | None] = mapped_column(BigInteger)
+    published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    banner_url: Mapped[str | None] = mapped_column(String(1024))
+    external_links: Mapped[list | None] = mapped_column(JSON)
+    topic_categories: Mapped[list | None] = mapped_column(JSON)
+    default_language: Mapped[str | None] = mapped_column(String(16))
+    country: Mapped[str | None] = mapped_column(String(8))
+    # Creator keyword tags (brandingSettings.channel.keywords) — a single space-separated string,
+    # multi-word tags quoted; the client parses it into chips for the About tab.
+    keywords: Mapped[str | None] = mapped_column(Text)
+
+    # Sync bookkeeping.
+    details_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    recent_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    last_rss_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # When the newest uploads were last read through the Data API because the RSS feed would not
+    # answer (`sync/runner.py` `run_feed_fallback`): the once-per-`rss_fallback_hours` clock.
+    feed_fallback_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    backfill_cursor: Mapped[str | None] = mapped_column(String(128))
+    backfill_done: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # True when this channel only entered the catalog because a live YouTube search
+    # surfaced one of its videos (nobody subscribes to it / it has no organic upload sync).
+    # Lets the Library hide search-only "noise" by default.
+    from_search: Mapped[bool] = mapped_column(
+        Boolean, default=False, server_default="false", index=True
+    )
+    # True when this channel exists only because a user EXPLORED it (opened its page without
+    # subscribing) — an ephemeral, per-explorer addition. After a grace period the cleanup job
+    # hard-deletes its via_explore videos nobody keeps (no subscription, no live ExploredChannel
+    # row, no per-user state), and then the channel row itself once it holds no videos — unless a
+    # Takeout watch history shows it watched: the "watched but not followed" list names it from
+    # that row. Cleared when someone subscribes, here or on YouTube (the "keep" signal promotes it
+    # to organic).
+    from_explore: Mapped[bool] = mapped_column(
+        Boolean, default=False, server_default="false", index=True
+    )
+
+    videos: Mapped[list["Video"]] = relationship(back_populates="channel")
+    subscriptions: Mapped[list["Subscription"]] = relationship(back_populates="channel")
+
+
+class Subscription(Base, TimestampMixin):
+    """Per-user link to a channel, with the user's own priority/visibility overrides."""
+
+    __tablename__ = "subscriptions"
+    __table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_channel"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    channel_id: Mapped[str] = mapped_column(
+        ForeignKey("channels.id", ondelete="CASCADE"), index=True
+    )
+    # YouTube's subscription resource id, needed to unsubscribe via the API.
+    yt_subscription_id: Mapped[str | None] = mapped_column(String(128))
+    priority: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    hidden: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # Per-user opt-in to full-history (deep) backfill of this channel. Default off so a
+    # new user's unique channels don't trigger a big shared-quota burst; the deep
+    # scheduler only picks up channels at least one user has requested.
+    deep_requested: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    subscribed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+    user: Mapped["User"] = relationship(back_populates="subscriptions")
+    channel: Mapped["Channel"] = relationship(back_populates="subscriptions")
+
+
+# Video.live_status values that mean "currently a live/upcoming broadcast" — transient states
+# hidden from feeds/search and revisited by the refresh job until they settle into a VOD.
+LIVE_OR_UPCOMING = ("live", "upcoming")
+
+# A YouTube length past a day is not a length. A 24/7 stream's archive (`was_live`) reports the years
+# the stream has run (measured on prod, 2026-11-06: 122,962,914 s), and one such video made a Watch
+# later total read 44,073 hours. YouTube caps an upload at 12 hours, so nothing real is lost. Such a
+# length is unknown wherever a length is used: the API sends none (`Video.length_seconds`), sums leave
+# it out (a playlist's counts it as `without_length`), and the duration filter, sort and Home budget never
+# match it. The raw `duration_seconds` stays for what it does prove — that the video is not a Short —
+# and for the jobs that ask whether YouTube has answered yet (test_video_length holds who reads it).
+LENGTH_MAX_S = 24 * 3600
+
+
+def known_length(seconds: int | None) -> int | None:
+    """`seconds` as a video's length, or None when it is not one (see LENGTH_MAX_S)."""
+    return seconds if seconds is not None and seconds <= LENGTH_MAX_S else None
+
+
+class Video(Base, TimestampMixin):
+    """A YouTube video. Shared across users; per-user state lives elsewhere."""
+
+    __tablename__ = "videos"
+
+    id: Mapped[str] = mapped_column(String(16), primary_key=True)  # video id
+    # No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
+    # (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
+    # serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
+    # sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
+    # Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
+    # Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
+    channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
+    title: Mapped[str | None] = mapped_column(Text)  # normalized for display (see app.titles)
+    original_title: Mapped[str | None] = mapped_column(Text)  # raw YouTube title, preserved
+    description: Mapped[str | None] = mapped_column(Text)
+    # NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
+    # forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
+    # that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
+    # PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
+    # the real date); the ix_videos_published_at index stays for the `oldest` sort.
+    published_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
+    )
+    thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
+    # The feed's views / duration / title sorts each ride a migration-defined index in their exact
+    # ORDER BY, one per direction (and per ICU collation for title): migration 0096 (the duration
+    # pair since 0117, on `length_seconds`), held to `feedquery.order_by` by test_feed_sort_indexes.
+    duration_seconds: Mapped[int | None] = mapped_column(Integer)
+    view_count: Mapped[int | None] = mapped_column(BigInteger)
+    like_count: Mapped[int | None] = mapped_column(BigInteger)
+    category_id: Mapped[int | None] = mapped_column(Integer)
+    topic_categories: Mapped[list | None] = mapped_column(JSON)
+    default_language: Mapped[str | None] = mapped_column(String(16))
+    detected_language: Mapped[str | None] = mapped_column(String(16))
+    # Creator-supplied keyword tags (snippet.tags joined by spaces) — fetched for free with the
+    # snippet we already request. Indexed into search_vector so the feed search matches the
+    # uploader's own keywords, not just words in the title.
+    keywords: Mapped[str | None] = mapped_column(Text)
+    # Accumulated distinct live-search queries that surfaced this video (across all users) —
+    # folds YouTube's own relevance judgement into the local search: a video YouTube returned
+    # for "ti amo magyarul" becomes findable by that query locally even if its title lacks it.
+    search_terms: Mapped[str | None] = mapped_column(Text)
+    # No btree here or on the flag columns below: migration 0069 (PERF-3) dropped the never-picked
+    # boolean btrees and replaced the useful ones with partial indexes over the target subset
+    # (via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — migration-defined, like the
+    # search_vector GIN.
+    is_short: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # Whether the youtube.com/shorts/<id> probe has run for this video.
+    shorts_probed: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # True when this video first entered the catalog via a live YouTube search (not an
+    # organic channel/playlist sync). Used to hide search-discovered "noise" from the
+    # Library by default; left False by the organic ingest paths.
+    via_search: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # True when this video entered the catalog because a user EXPLORED its (un-subscribed)
+    # channel. Like via_search it's hidden from the shared Library by default, but it's
+    # additionally PER-USER private: a via_explore video is only visible to users who
+    # explored or subscribe to its channel (a gate in the feed query), so one user's
+    # curiosity never floods everyone else's catalog. Left False by the organic ingest paths.
+    via_explore: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # none | live | upcoming | premiere | was_live
+    live_status: Mapped[str] = mapped_column(
+        String(16), default="none", server_default="none", index=True
+    )
+    enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # videos.list?part=status fields (populated during enrichment); used by the
+    # maintenance/validation job to decide whether a video is playable anywhere.
+    embeddable: Mapped[bool | None] = mapped_column(Boolean)
+    privacy_status: Mapped[str | None] = mapped_column(String(16))  # public|unlisted|private
+    upload_status: Mapped[str | None] = mapped_column(
+        String(16)
+    )  # processed|failed|rejected|deleted
+    # Maintenance/validation lifecycle. last_checked_at drives the rolling re-validation
+    # (least-recently-checked first); unavailable_since marks a video as currently
+    # unplayable (hidden from the feed immediately, hard-deleted after a grace period);
+    # unavailable_reason records why (deleted|private|paywalled|abandoned|stuck_live).
+    # last_checked_at: no plain btree — migration 0069 (PERF-18) replaced it with the partial
+    # ix_videos_recheck_order (last_checked_at NULLS FIRST WHERE unavailable_since IS NULL) that the
+    # revalidate job's exact ORDER BY rides; unavailable_since is a partial WHERE IS NOT NULL (PERF-3).
+    last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    unavailable_reason: Mapped[str | None] = mapped_column(String(24))
+    # A random number in [0, 1), drawn once when the row is written: the ring a seeded shuffle walks
+    # (`feedquery.shuffle_walk`, index ix_videos_shuffle_key on (shuffle_key, id), migration 0118).
+    shuffle_key: Mapped[float] = mapped_column(Float, nullable=False, server_default=func.random())
+
+    @hybrid_property
+    def length_seconds(self) -> int | None:
+        """The video's length, or None when it is unknown — not fetched yet, or not a length at all
+        (`LENGTH_MAX_S`). What every reader of a length reads."""
+        return known_length(self.duration_seconds)
+
+    @length_seconds.inplace.expression
+    @classmethod
+    def _length_seconds_expression(cls) -> ColumnElement[int | None]:
+        # The limit is a literal, not a bound parameter: the duration sort indexes (migration 0117)
+        # are built on this very expression, and Postgres matches an index to a query's constant only.
+        return case(
+            (cls.duration_seconds > literal_column(str(LENGTH_MAX_S)), null()),
+            else_=cls.duration_seconds,
+        )
+
+    @hybrid_property
+    def without_length(self) -> bool:
+        """A video whose length is unknown — live, an archive past a day, or not fetched yet: what a
+        sum of lengths leaves out and names as "+N without length". Every such video, so the count
+        and the sum never disagree about what was left out."""
+        return self.length_seconds is None
+
+    @without_length.inplace.expression
+    @classmethod
+    def _without_length_expression(cls) -> ColumnElement[bool]:
+        return cls.length_seconds.is_(None)
+
+    # Weighted full-text search document (DB-generated, never written by the app): title (A) >
+    # creator keywords + search queries (B) > truncated description (C). ts_rank honours the
+    # weights so title matches outrank description matches. Backed by a GIN index (migration
+    # 0032). The accent-insensitive `unaccent_simple` config comes from migration 0031.
+    search_vector: Mapped[object | None] = mapped_column(
+        TSVECTOR,
+        Computed(
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(keywords, '') || ' ' || "
+            "coalesce(search_terms, '')), 'B') || "
+            "setweight(to_tsvector('public.unaccent_simple', left(coalesce(description, ''), 1000)), 'C')",
+            persisted=True,
+        ),
+    )
+
+    channel: Mapped["Channel"] = relationship(back_populates="videos")
+
+
+class SearchFind(Base, TimestampMixin):
+    """Per-user record of a video the user surfaced via their own live YouTube search.
+
+    Lets "videos I searched for" appear in that user's own (Mine) feed and the Mine Source
+    filter — distinct from the GLOBAL `Video.via_search` provenance, which marks a video as
+    search-discovered by *anyone* and drives the shared Library's Source filter."""
+
+    __tablename__ = "search_finds"
+    __table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_search_video"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
+
+
+class ExploredChannel(Base, TimestampMixin):
+    """Per-user record of a channel the user opened the page of WITHOUT subscribing.
+
+    Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
+    user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
+    explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
+    purge: of a from_explore channel with no live ExploredChannel row (and no subscription) the
+    cleanup job hard-deletes the videos without per-user state, then the channel once it holds none
+    and no Takeout watch history shows it watched. Subscribing is the "keep"
+    signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
+    (from TimestampMixin) is the grace clock; a re-visit refreshes it."""
+
+    __tablename__ = "explored_channels"
+    __table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_explored_channel"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    channel_id: Mapped[str] = mapped_column(
+        ForeignKey("channels.id", ondelete="CASCADE"), index=True
+    )
+
+
+class BlockedChannel(Base, TimestampMixin):
+    """Per-user channel blocklist. Videos from a blocked channel are excluded from this user's
+    live YouTube search (and not ingested on their behalf) and hidden from their feed / explore
+    views — a personal "never show me this channel again" that doesn't affect other users."""
+
+    __tablename__ = "blocked_channels"
+    __table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_blocked_channel"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    channel_id: Mapped[str] = mapped_column(
+        ForeignKey("channels.id", ondelete="CASCADE"), index=True
+    )

backend/app/catalog/queries.py

@@ -0,0 +1,310 @@
+"""Catalog's reads: channels and videos, and each user's footprint in them (subscriptions, blocks,
+explorations), for the HTTP layer and for the core code that asks about them.
+
+A user's row is read through `userscope`, so someone else's is the same answer as a missing one.
+Catalog does not read another unit's tables: what it needs from Reader (a playlist's channels, a
+user's tag links) it asks `reader.queries`.
+"""
+
+from datetime import datetime
+
+from sqlalchemy import Select, and_, case, func, or_, select
+from sqlalchemy.orm import Session
+
+from app.catalog.models import (
+    LIVE_OR_UPCOMING,
+    BlockedChannel,
+    Channel,
+    ExploredChannel,
+    Subscription,
+    Video,
+)
+from app.collation import collated
+from app.models import User
+from app.sync.explore import followed_channel
+from app.userscope import owned_by
+
+# --- channels --------------------------------------------------------------------------------
+
+
+def channel(db: Session, channel_id: str) -> Channel | None:
+    return db.get(Channel, channel_id)
+
+
+def channel_title(db: Session, channel_id: str) -> str | None:
+    return db.scalar(select(Channel.title).where(Channel.id == channel_id))
+
+
+def channel_titles(db: Session, ids: set[str]) -> dict[str, str]:
+    """Each channel's title by id; a channel with no title yet is left out."""
+    if not ids:
+        return {}
+    rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(ids)))
+    return {cid: title for cid, title in rows.tuples() if title is not None}
+
+
+def channel_aggregates(channel_ids: list[str]) -> Select:
+    """Per-channel stored count, last upload and total duration (the lengths it knows: a video with
+    none is left out, `Video.length_seconds`), in one grouped pass. Every column it
+    reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
+    alone; a test holds that (test_channel_list_index_only)."""
+    return (
+        select(
+            Video.channel_id,
+            func.count(Video.id),
+            func.max(Video.published_at),
+            func.coalesce(func.sum(Video.length_seconds), 0),
+        )
+        .where(Video.channel_id.in_(channel_ids))
+        .group_by(Video.channel_id)
+    )
+
+
+def aggregates(db: Session, channel_ids: list[str]) -> list[tuple[str, int, datetime | None, int]]:
+    """`channel_aggregates`, run: (channel id, stored videos, last upload, total seconds)."""
+    if not channel_ids:
+        return []
+    return list(db.execute(channel_aggregates(channel_ids)).tuples())
+
+
+def video_counts(db: Session, channel_id: str) -> tuple[int, int, int]:
+    """A channel's stored videos, its Shorts and its live/upcoming ones, in one grouped pass."""
+    n_all, n_short, n_live = db.execute(
+        select(
+            func.count(Video.id),
+            func.count(Video.id).filter(Video.is_short.is_(True)),
+            func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
+        ).where(Video.channel_id == channel_id)
+    ).one()
+    return int(n_all or 0), int(n_short or 0), int(n_live or 0)
+
+
+def deep_queued(db: Session, channel_ids: list[str]) -> set[str]:
+    """Which of these channels at least one user (any user) has asked full history for."""
+    if not channel_ids:
+        return set()
+    return set(
+        db.execute(
+            select(Subscription.channel_id)
+            .where(
+                Subscription.channel_id.in_(channel_ids),
+                Subscription.deep_requested.is_(True),
+            )
+            .distinct()
+        ).scalars()
+    )
+
+
+def followed(db: Session, channel_id: str) -> bool:
+    """Whether anyone subscribes to the channel."""
+    return (
+        db.execute(
+            select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
+        ).first()
+        is not None
+    )
+
+
+# --- a user's subscriptions, blocks and explorations ----------------------------------------
+
+
+def subscriptions(db: Session, user: User, lang: str) -> list[tuple[Subscription, Channel]]:
+    """The user's subscriptions with their channels, highest priority first, then by title in the
+    reader's alphabet."""
+    return list(
+        db.execute(
+            select(Subscription, Channel)
+            .join(Channel, Channel.id == Subscription.channel_id)
+            .where(owned_by(Subscription.user_id, user))
+            .order_by(Subscription.priority.desc(), collated(Channel.title, lang))
+        ).tuples()
+    )
+
+
+def subscription(db: Session, user: User, channel_id: str) -> Subscription | None:
+    return db.execute(
+        select(Subscription).where(
+            owned_by(Subscription.user_id, user), Subscription.channel_id == channel_id
+        )
+    ).scalar_one_or_none()
+
+
+def subscribed_channels(user: User):
+    """The channel ids the user subscribes to, as a subquery."""
+    return select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
+
+
+def exploration(db: Session, user: User, channel_id: str) -> ExploredChannel | None:
+    return db.execute(
+        select(ExploredChannel).where(
+            owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
+        )
+    ).scalar_one_or_none()
+
+
+def is_blocked(db: Session, user: User, channel_id: str) -> bool:
+    return (
+        db.execute(
+            select(BlockedChannel.id).where(
+                owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
+            )
+        ).first()
+        is not None
+    )
+
+
+def blocked_ids(db: Session, user: User) -> set[str]:
+    return set(
+        db.scalars(select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user)))
+    )
+
+
+def followed_among(db: Session, user: User, channel_ids: set[str]) -> set[str]:
+    """Which of these channels the user subscribes to."""
+    if not channel_ids:
+        return set()
+    return set(
+        db.scalars(
+            select(Subscription.channel_id).where(
+                owned_by(Subscription.user_id, user), Subscription.channel_id.in_(channel_ids)
+            )
+        )
+    )
+
+
+def blocked_channels(db: Session, user: User, lang: str) -> list[Channel]:
+    return list(
+        db.execute(
+            select(Channel)
+            .join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
+            .where(owned_by(BlockedChannel.user_id, user))
+            .order_by(collated(Channel.title, lang))
+        ).scalars()
+    )
+
+
+# --- sync progress ---------------------------------------------------------------------------
+
+
+def catalog_totals(db: Session, user: User) -> dict:
+    """The instance's catalog size and this user's subscription count (the sync status card)."""
+    return {
+        "subscriptions": db.scalar(
+            select(func.count())
+            .select_from(Subscription)
+            .where(owned_by(Subscription.user_id, user))
+        ),
+        "channels_total": db.scalar(select(func.count()).select_from(Channel)),
+        "channels_backfilling": db.scalar(
+            select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
+        ),
+        "videos_total": db.scalar(select(func.count()).select_from(Video)),
+        "pending_enrich": db.scalar(
+            select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
+        ),
+    }
+
+
+def my_progress(db: Session, user: User) -> tuple[int, int, int, int]:
+    """Over the channels this user subscribes to: how many, how many have details, how many their
+    recent uploads, how many their whole history."""
+    total, details_synced, recent_synced, all_deep_done = db.execute(
+        select(
+            func.count(Channel.id),
+            func.count(Channel.details_synced_at),
+            func.count(Channel.recent_synced_at),
+            func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
+        )
+        .select_from(Channel)
+        .join(
+            Subscription,
+            and_(Subscription.channel_id == Channel.id, owned_by(Subscription.user_id, user)),
+        )
+    ).one()
+    return total or 0, details_synced or 0, recent_synced or 0, int(all_deep_done or 0)
+
+
+def my_deep_requested(db: Session, user: User) -> list[Channel]:
+    """The channels this user asked full history for."""
+    return list(
+        db.execute(
+            select(Channel).join(
+                Subscription,
+                and_(
+                    Subscription.channel_id == Channel.id,
+                    owned_by(Subscription.user_id, user),
+                    Subscription.deep_requested.is_(True),
+                ),
+            )
+        ).scalars()
+    )
+
+
+def my_video_count(db: Session, user: User) -> int:
+    """The videos of the channels this user subscribes to and has not hidden."""
+    return (
+        db.scalar(
+            select(func.count(Video.id)).join(
+                Subscription,
+                and_(
+                    Subscription.channel_id == Video.channel_id,
+                    owned_by(Subscription.user_id, user),
+                    Subscription.hidden.is_(False),
+                ),
+            )
+        )
+        or 0
+    )
+
+
+def video_total(db: Session) -> int:
+    return db.scalar(select(func.count(Video.id))) or 0
+
+
+def deep_backlog(db: Session) -> list[Channel]:
+    """Every channel someone asked full history for that does not have it yet."""
+    deep_requested = (
+        select(Subscription.id)
+        .where(Subscription.channel_id == Channel.id, Subscription.deep_requested.is_(True))
+        .exists()
+    )
+    return list(
+        db.execute(
+            select(Channel).where(Channel.backfill_done.is_(False), deep_requested)
+        ).scalars()
+    )
+
+
+def work_queue(db: Session) -> dict:
+    """What the catalog's jobs still have queued, instance-wide (the scheduler card)."""
+    return {
+        # The same selection run_recent_backfill works through, or the card counts channels the
+        # job will never touch and never reaches zero.
+        "channels_recent_pending": db.scalar(
+            select(func.count())
+            .select_from(Channel)
+            .where(Channel.recent_synced_at.is_(None), followed_channel())
+        )
+        or 0,
+        "videos_pending_enrich": db.scalar(
+            select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
+        )
+        or 0,
+        "videos_pending_shorts": db.scalar(
+            select(func.count())
+            .select_from(Video)
+            .where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
+        )
+        or 0,
+        "videos_live_refresh": db.scalar(
+            select(func.count())
+            .select_from(Video)
+            .where(
+                or_(
+                    Video.live_status.in_(("live", "upcoming")),
+                    and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
+                )
+            )
+        )
+        or 0,
+    }

backend/app/catalog/search.py

@@ -0,0 +1,244 @@
+"""Live YouTube search, the catalog's side: what a page of results leaves in the shared catalog.
+
+A result becomes a catalog row so it renders with the normal feed cards and the in-app player and
+gains per-user state. The ingest runs in steps with a YouTube call between each, and each step's
+rows are committed before the next call: an enrich that fails rolls back to the stubs, which stay
+(the FK targets for what follows, backfilled later). Results are flagged (`videos.via_search`,
+`channels.from_search`) so the Library can hide this search-discovered noise; Shorts and
+live/upcoming videos are never kept. The HTTP layer (`routes/search.py`) runs the paging, the caps
+and the refusals.
+"""
+
+from collections.abc import Sequence
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime, timezone
+
+from sqlalchemy import delete, func, select, update
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+from sqlalchemy.engine import CursorResult
+from sqlalchemy.orm import Session
+
+from app import logs, quota, sysconfig
+from app.catalog import queries
+from app.catalog.models import LIVE_OR_UPCOMING, Channel, SearchFind, Subscription, Video
+from app.models import User
+from app.reader import queries as reader_queries
+from app.sync.subscriptions import apply_channel_details
+from app.sync.videos import apply_video_details, insert_stubs, parse_dt
+from app.userscope import owned_by
+from app.utils import like_escape
+from app.youtube.client import YouTubeClient, YouTubeError
+from app.youtube.shorts import make_client, probe_is_short
+
+log = logs.get_logger(__name__)
+
+
+def classify_shorts(db: Session, videos: Sequence[Video]) -> None:
+    """Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
+
+    Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
+    synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
+    duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
+    quota). Undetermined probes are left for the scheduler to retry."""
+    probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
+    to_probe: list[Video] = []
+    for v in videos:
+        if v.shorts_probed:
+            continue
+        if v.enriched_at is None:
+            # Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
+            # so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
+            # (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
+            continue
+        if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
+            v.is_short = False
+            v.shorts_probed = True
+        else:
+            to_probe.append(v)
+    if to_probe:
+        with make_client() as client:
+
+            def work(v: Video):
+                return v, probe_is_short(client, v.id)
+
+            with ThreadPoolExecutor(max_workers=16) as pool:
+                for v, result in pool.map(work, to_probe):
+                    if result is None:
+                        continue
+                    v.is_short = result
+                    v.shorts_probed = True
+    db.commit()
+
+
+def ingest_candidates(
+    db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
+) -> list[str]:
+    """Materialise one page of search candidates into the catalog (channel stubs + enrich,
+    video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
+    video ids — non-Short, non-live — in the input relevance order. A videos.list that fails
+    raises its YouTubeError: with nothing enriched, nothing could be kept."""
+    ids = [it["id"] for it in candidates]
+    if not ids:
+        return []
+
+    # 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
+    #    then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
+    channel_ids = list({it["channel_id"] for it in candidates})
+    existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
+    new_channel_ids = [c for c in channel_ids if c not in existing]
+    if new_channel_ids:
+        titles = {it["channel_id"]: it["channel_title"] for it in candidates}
+        for cid in new_channel_ids:
+            db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
+        db.commit()
+        try:
+            with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
+                details = yt.get_channels(new_channel_ids)
+            apply_channel_details(db, details, new_channel_ids)
+            db.commit()
+        except YouTubeError:
+            # The stubs remain and the details can be backfilled later — but until they are, those
+            # channels sit in the results with no thumbnail and no subscriber count, looking like
+            # channels that have neither.
+            log.warning(
+                "the channels found by a search could not be enriched",
+                extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
+            )
+            db.rollback()  # details can be backfilled later; the stubs remain
+
+    # 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
+    #    NOTHING), so an organically-synced video stays organic.
+    #    published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
+    #    (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
+    #    enrich in step 3 overwrites it with the video's real snippet date.
+    now = datetime.now(timezone.utc)
+    insert_stubs(
+        db,
+        [
+            {
+                "id": it["id"],
+                "channel_id": it["channel_id"],
+                "title": it["title"],
+                "published_at": parse_dt(it["published_at"]) or now,
+                "thumbnail_url": it["thumbnail_url"],
+                "via_search": True,
+            }
+            for it in candidates
+        ],
+    )
+
+    # 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
+    videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
+    try:
+        with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
+            detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
+    except YouTubeError:
+        # An unenriched video cannot be judged live or Short, so none of this page could be kept:
+        # swallowed, the failure answered as a page with nothing on it — "no results" for a search
+        # that never ran (the shared budget spent, no API key). The stubs stay (`insert_stubs`
+        # committed them), as the channels' do above; the caller answers the failure (its refusal
+        # says which).
+        log.warning(
+            "the videos found by a search could not be enriched",
+            extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
+        )
+        raise
+    enriched_at = datetime.now(
+        timezone.utc
+    )  # when THIS enrich ran (distinct from the stub placeholder)
+    for v in videos:
+        item = detail_by_id.get(v.id)
+        if item is None:
+            continue
+        apply_video_details(v, item)
+        v.enriched_at = enriched_at
+    db.commit()
+
+    # 4) Confirm/deny Shorts (no quota) so none enter via search.
+    classify_shorts(db, videos)
+
+    keep = {
+        v.id
+        for v in videos
+        if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
+    }
+    return [vid for vid in ids if vid in keep]
+
+
+def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
+    """Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
+
+    "Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
+    video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
+    wildcard — either way the term was never folded in, and local search lost that relevance."""
+    padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
+    db.execute(
+        update(Video)
+        .where(Video.id.in_(video_ids))
+        .where(padded.notilike(f"% {like_escape(term)} %"))
+        .values(
+            search_terms=func.trim(
+                func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
+            )
+        )
+    )
+
+
+def new_first(db: Session, user: User, ids: list[str]) -> list[str]:
+    """Reorder results so genuinely-new discoveries come first and things the user already has
+    (a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
+    to the back — keeping the relevance order within each group."""
+    if not ids:
+        return ids
+    have = set(
+        db.scalars(
+            select(Video.id).where(
+                Video.id.in_(ids), Video.channel_id.in_(queries.subscribed_channels(user))
+            )
+        )
+    ) | reader_queries.held_by(db, user, ids)
+    return [i for i in ids if i not in have] + [i for i in ids if i in have]
+
+
+def record_finds(db: Session, user: User, video_ids: list[str], term: str) -> None:
+    """These results as THIS user's search finds (the Mine feed, the Source filter), and the term
+    folded into each one's `search_terms` so local full-text search inherits YouTube's relevance.
+    Idempotent."""
+    db.execute(
+        pg_insert(SearchFind)
+        .values([{"user_id": user.id, "video_id": vid} for vid in video_ids])
+        .on_conflict_do_nothing(index_elements=["user_id", "video_id"])
+    )
+    _fold_term(db, video_ids, term)
+
+
+def clear(db: Session, user: User, video_ids: list[str]) -> int:
+    """Discard these results "as if never added": drop this user's finds of them, then delete the
+    ones nobody kept — a `via_search` video no reader holds (`reader.queries.kept_by_a_reader`),
+    whose channel nobody subscribes to, and that no other user still has as a find. Returns how
+    many videos went.
+
+    Only the ones this user found: a video in the catalog with no find at all (a related list's,
+    which records none) is in someone's list right now, and an id any user could name would let
+    them delete it from under that list."""
+    mine = list(
+        db.scalars(
+            delete(SearchFind)
+            .where(owned_by(SearchFind.user_id, user), SearchFind.video_id.in_(video_ids))
+            .returning(SearchFind.video_id)
+        )
+    )
+    if not mine:
+        return 0
+    subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
+    other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
+    result = db.execute(
+        delete(Video).where(
+            Video.id.in_(mine),
+            Video.via_search.is_(True),
+            ~reader_queries.kept_by_a_reader(Video.id),
+            ~subscribed,
+            ~other_find,
+        )
+    )
+    return result.rowcount if isinstance(result, CursorResult) else 0

backend/app/catalog/service.py

@@ -0,0 +1,144 @@
+"""Catalog's writes a request makes: a user's subscription settings, what they explore or block, a
+subscription recorded or dropped, a channel's sync markers reset.
+
+Every function here flushes and leaves the commit to its caller (the handler). The YouTube side of a
+subscribe or an unsubscribe is the handler's (it needs the user's client and its refusals); what
+lands here is the local record of it. The ingest jobs (`sync/`, `catalog/search`) write the
+catalog's shared rows themselves.
+"""
+
+from datetime import datetime, timezone
+
+from sqlalchemy import delete, update
+from sqlalchemy.engine import CursorResult
+from sqlalchemy.orm import Session
+
+from app.catalog import queries
+from app.catalog.models import BlockedChannel, Channel, ExploredChannel, Subscription, Video
+from app.models import User
+from app.userscope import owned_by
+
+
+def remember_own_channel(user: User, channel_id: str) -> None:
+    """The user's own YouTube channel, so discovery never offers it to them."""
+    user.yt_channel_id = channel_id
+
+
+def update_subscription(
+    db: Session,
+    sub: Subscription,
+    *,
+    priority: int | None = None,
+    hidden: bool | None = None,
+    deep_requested: bool | None = None,
+) -> bool:
+    """Change the settings given (None leaves one as it is). Returns whether full history was just
+    turned ON — the caller then pulls the channel now rather than at the scheduler's next tick."""
+    turned_on = False
+    if priority is not None:
+        sub.priority = priority
+    if hidden is not None:
+        sub.hidden = hidden
+    if deep_requested is not None:
+        # Turning it off won't undo already-fetched videos; it stops further deep paging if nobody
+        # else still wants it.
+        turned_on = deep_requested and not sub.deep_requested
+        sub.deep_requested = deep_requested
+    db.flush()
+    return turned_on
+
+
+def set_deep_all(db: Session, user: User, on: bool) -> int:
+    """Every one of the user's channels in (or out of) full-history backfill; how many rows moved."""
+    result = db.execute(
+        update(Subscription).where(owned_by(Subscription.user_id, user)).values(deep_requested=on)
+    )
+    return result.rowcount if isinstance(result, CursorResult) else 0
+
+
+def explore(db: Session, user: User, channel: Channel) -> None:
+    """Record (or refresh the grace clock of) the user's exploration of a channel, and flag the
+    channel ephemeral unless someone follows it — the cleanup job's keep rule reads both."""
+    existing = queries.exploration(db, user, channel.id)
+    if existing is None:
+        db.add(ExploredChannel(user_id=user.id, channel_id=channel.id))
+    else:
+        existing.created_at = datetime.now(timezone.utc)
+    if not channel.from_explore and not queries.followed(db, channel.id):
+        channel.from_explore = True
+    db.flush()
+
+
+def reset_backfill(db: Session, channel: Channel) -> None:
+    """Fetch the channel again from scratch: its backfill markers cleared, and full history asked
+    for on EVERY subscription to it — the deep scheduler gates on an EXISTS over those, so the admin
+    who resets it need not subscribe themselves."""
+    channel.backfill_done = False
+    channel.backfill_cursor = None
+    channel.recent_synced_at = None
+    db.execute(
+        update(Subscription)
+        .where(Subscription.channel_id == channel.id)
+        .values(deep_requested=True)
+    )
+    db.flush()
+
+
+def record_subscription(db: Session, user: User, channel: Channel, yt_subscription_id: str) -> None:
+    """The subscription YouTube now holds, recorded here. Subscribing is the "keep" signal: an
+    explored channel becomes an organic one, so the cleanup job leaves it alone and its videos join
+    the shared Library like any subscribed channel's uploads."""
+    db.add(
+        Subscription(
+            user_id=user.id,
+            channel_id=channel.id,
+            yt_subscription_id=yt_subscription_id or None,
+            subscribed_at=datetime.now(timezone.utc),
+        )
+    )
+    db.flush()
+    settle_explored(db, channel)
+
+
+def settle_explored(db: Session, channel: Channel) -> None:
+    """Hold the rule that a channel's videos are an exploration's only while nobody follows it,
+    decided by who follows it now, not by the flag held in memory: a subscribe in another request
+    may have cleared that while YouTube answered this one, and the videos that came in meanwhile
+    would stay flagged. A channel someone follows, however the follow came (here or pulled in from
+    YouTube), is the catalog's own, and so are its videos."""
+    if channel.from_explore and not queries.followed(db, channel.id):
+        return
+    channel.from_explore = False
+    db.execute(
+        update(Video)
+        .where(Video.channel_id == channel.id, Video.via_explore.is_(True))
+        .values(via_explore=False)
+    )
+    db.flush()
+
+
+def drop_subscription(db: Session, sub: Subscription) -> None:
+    db.delete(sub)
+    db.flush()
+
+
+def block(db: Session, user: User, channel_id: str) -> None:
+    """Block a channel for this user (idempotent), and end any exploration of it so the cleanup job
+    can reclaim it."""
+    if queries.is_blocked(db, user, channel_id):
+        return
+    db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
+    db.execute(
+        delete(ExploredChannel).where(
+            owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
+        )
+    )
+    db.flush()
+
+
+def unblock(db: Session, user: User, channel_id: str) -> None:
+    db.execute(
+        delete(BlockedChannel).where(
+            owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
+        )
+    )

backend/app/config.py

@@ -113,6 +113,11 @@ class Settings(BaseSettings):
     # each search.list costs 100 units, so the shared budget only affords ~80-90/day total; with
     # the zero-quota scrape source it's a pure anti-abuse rate limit. Admin-tunable.
     search_daily_limit_per_user: int = 70
+    # Per-user cap on related-video requests per day — a list opened and each "Show more" (the
+    # InnerTube watch page, no quota; its enrich is up to ~8 units: up to 4 pages, each a videos.list
+    # and a channels.list). A failed one gives its slot back. 0 turns related videos off.
+    # Admin-tunable.
+    related_daily_limit_per_user: int = 200
     # Source for live YouTube search: "scrape" (InnerTube, no API quota) or "api" (search.list,
     # 100 units/page). Scrape is the default; flip to "api" if scraping is blocked/breaks.
     search_source: SearchSource = "scrape"
@@ -268,10 +273,12 @@ class Settings(BaseSettings):
     # YouTube quota history (quota_events): days kept (0 = forever). A year, so the admin usage view's
     # longest window (365 days) stays whole; the same daily cleanup job prunes it.
     quota_retention_days: int = 365
+    # Background-run history (job_runs): days kept (0 = forever). The audit log's 180, which is how
+    # long a run lived there before it moved here: ~1.8k rows a day at the default intervals
+    # (measured 2026-10-17), ~320k at the limit. The same daily cleanup job prunes it.
+    job_run_retention_days: int = 180
     # On-disk layout: "plex" (Channel/Season YYYY/… [id].ext + .nfo/poster) or "flat".
     download_layout: DownloadLayout = "plex"
-    # Default SponsorBlock (skip sponsor segments) for new custom profiles.
-    sponsorblock_default: bool = False
 
     # --- Diagnostics / logging ---
     # BOOTSTRAP level only: the running level is the `log_level` admin knob (sysconfig), applied at

backend/app/debugsession.py

@@ -26,6 +26,7 @@ from uuid import uuid4
 from sqlalchemy import and_, delete, func, not_, select, update
 from sqlalchemy.orm import Session, defer
 
+from app.userscope import owned_by, owned_row
 from app import diagnostics
 from app.models import DebugScreenshot, DebugSession, LogEvent, User
 
@@ -76,7 +77,9 @@ def open_session(db: Session, user: User) -> DebugSession | None:
 
 def _unclosed(db: Session, user: User) -> DebugSession | None:
     return db.scalar(
-        select(DebugSession).where(DebugSession.user_id == user.id, DebugSession.ended_at.is_(None))
+        select(DebugSession).where(
+            owned_by(DebugSession.user_id, user), DebugSession.ended_at.is_(None)
+        )
     )
 
 
@@ -126,12 +129,8 @@ def from_header(db: Session, user: User, value: str | None) -> DebugSession | No
     """The session a request names, if it is the caller's own and still live; else None."""
     if not value or len(value) != _ID_LEN or not all(c in "0123456789abcdef" for c in value):
         return None
-    session = db.get(DebugSession, value)
-    if (
-        session is None
-        or session.user_id != user.id
-        or not is_live(session, datetime.now(timezone.utc))
-    ):
+    session = owned_row(db, DebugSession, value, user)
+    if session is None or not is_live(session, datetime.now(timezone.utc)):
         return None
     return session
 
@@ -354,7 +353,7 @@ def stored_screenshot_bytes(db: Session, user_id: int) -> int:
         db.scalar(
             select(func.coalesce(func.sum(DebugScreenshot.size), 0))
             .join(DebugSession, DebugSession.id == DebugScreenshot.session_id)
-            .where(DebugSession.user_id == user_id)
+            .where(owned_by(DebugSession.user_id, user_id))
         )
         or 0
     )

backend/app/discover/cards.py

@@ -22,7 +22,8 @@ to the client, which has both the native payload and the translations.
 """
 
 from app.discover.budget import remaining_seconds
-from app.models import Channel, Playlist
+from app.catalog.models import Channel
+from app.reader.models import Playlist
 
 # Plex artwork is a poster for the things you browse and a landscape still for the things you play.
 _POSTER_KINDS = ("movie", "show")

backend/app/discover/pins.py

@@ -22,10 +22,14 @@ from sqlalchemy import func, select
 from sqlalchemy.orm import Session
 
 from app.discover import cards, queries
-from app.models import Channel, ComfortPin, PlexItem, PlexShow, Playlist, User, Video
+from app.models import User
+from app.plex.models import PlexItem, PlexShow
+from app.catalog.models import Channel, Video
+from app.reader.models import ComfortPin, Playlist
 from app.plex import rollup
 from app.plex.cards import SHOW_NEW, leaf_card, show_card
 from app.plex.lookups import by_rating_key
+from app.userscope import owned_by, owned_row
 
 # kind -> the ComfortPin column that holds it. The client's vocabulary and the schema's, in one place.
 PIN_COLUMNS: dict[str, str] = {
@@ -68,11 +72,11 @@ def resolve_ref(db: Session, user: User, kind: str, ref: str) -> tuple[str, obje
     # such pin target", which is the same answer this function gives for one that does not exist.
     except ValueError:
         return None
-    pl = db.get(Playlist, pid)
     # A playlist is per-user, unlike the other four (which are shared catalog rows whose PRIVACY
     # lives in the state tables). Someone else's list must read as "no such thing", not as a 403 —
     # the same reason `owned_or_404` 404s rather than 403s.
-    return (col, pl.id) if pl is not None and pl.user_id == user.id else None
+    pl = owned_row(db, Playlist, pid, user)
+    return (col, pl.id) if pl is not None else None
 
 
 def reorder(db: Session, user: User, ids: list[int]) -> None:
@@ -92,7 +96,7 @@ def reorder(db: Session, user: User, ids: list[int]) -> None:
     # old slots — and one global lock order means two reorders cannot deadlock.
     locked = db.execute(
         select(ComfortPin)
-        .where(ComfortPin.user_id == user.id)
+        .where(owned_by(ComfortPin.user_id, user.id))
         .order_by(ComfortPin.id)
         .with_for_update()
         .execution_options(populate_existing=True)
@@ -117,7 +121,7 @@ def list_pins(db: Session, user: User) -> list[ComfortPin]:
     return list(
         db.execute(
             select(ComfortPin)
-            .where(ComfortPin.user_id == user.id)
+            .where(owned_by(ComfortPin.user_id, user.id))
             .order_by(ComfortPin.position.asc(), ComfortPin.id.asc())
         )
         .scalars()
@@ -132,17 +136,44 @@ def next_position(db: Session, user: User) -> int:
     deletes the row, and every FK cascade silently removes a pin when the thing it points at goes
     away — so past the first hole a count under-counts and would drop the new pin into the MIDDLE of
     the shelf, or onto a position another pin already holds (nothing in the schema forbids that).
-    Same idiom as `routes/playlists._next_playlist_position`."""
+    Same idiom as a new playlist's place in the rail (`reader.service`)."""
     return (
         db.scalar(
             select(func.coalesce(func.max(ComfortPin.position), -1)).where(
-                ComfortPin.user_id == user.id
+                owned_by(ComfortPin.user_id, user.id)
             )
         )
         + 1
     )
 
 
+def own_pin(db: Session, user: User, pin_id: int) -> ComfortPin | None:
+    return owned_row(db, ComfortPin, pin_id, user)
+
+
+def pin_of(db: Session, user: User, column: str, value: object) -> ComfortPin | None:
+    """The user's pin of this thing (`column`/`value` from `resolve_ref`), if it is on the shelf."""
+    return db.execute(
+        select(ComfortPin).where(
+            owned_by(ComfortPin.user_id, user), getattr(ComfortPin, column) == value
+        )
+    ).scalar_one_or_none()
+
+
+def add(db: Session, user: User, column: str, value: object) -> ComfortPin:
+    """A new pin at the end of the shelf. A second pin of the same thing (another tab, at the same
+    moment) raises `IntegrityError` at the flush: the unique constraint is the guard."""
+    pin = ComfortPin(user_id=user.id, position=next_position(db, user), **{column: value})
+    db.add(pin)
+    db.flush()
+    return pin
+
+
+def remove(db: Session, pin: ComfortPin) -> None:
+    db.delete(pin)
+    db.flush()
+
+
 @dataclass
 class _Batch:
     """What the shelf resolves for every pinned channel and playlist in one go."""

backend/app/discover/queries.py

@@ -6,20 +6,29 @@ Two rules shape this module:
 **Reuse the query layer, not the route.** The YouTube half calls `feedquery.filtered_query` /
 `sort_keys`, so the Home surface inherits the feed's own definitions of subscribed, blocked,
 explore-visible and in-progress for free, and cannot drift from them. The Plex half calls
-`plex.rollup` and the `plex.cards` serializers rather than `routes/plex/discover.py`: that route
+`plex.rollup` and the `plex.cards` serializers rather than `plex/discover.py`: that read-model
 narrows EVERY rail by the Plex sidebar's `LibraryFilters`, which is right for a surface that HAS a
 sidebar and wrong for a landing page that does not.
 
-**One seed drives both halves.** The feed orders a shuffle by `md5(id || seed)` so a keyset cursor
-stays valid across pages; the Plex surprise rail uses `ORDER BY random()`, which is fine for a single
-un-paged draw but cannot be reproduced. A cross-source reroll needs both halves to move together —
-otherwise "Mást kérek" re-draws the Plex picks while the YouTube ones stand still — so the Plex side
-here takes the feed's seeded-md5 idiom instead of `random()`.
+**One seed drives both halves.** The YouTube half starts on the ring of stored random keys at the
+seed's point and shuffles that window by `md5(id || seed)`; the Plex surprise rail uses
+`ORDER BY random()`, which is fine for a single un-paged draw but cannot be reproduced. A
+cross-source reroll needs both halves to move together — otherwise "Mást kérek" re-draws the Plex
+picks while the YouTube ones stand still — so the Plex side here orders by the same seeded md5
+instead of `random()`.
 """
 
-from sqlalchemy import and_, func, or_, select
+from datetime import datetime, timezone
+
+from sqlalchemy import (
+    and_,
+    func,
+    or_,
+    select,
+)
 from sqlalchemy.orm import Session, aliased
 
+from app import draw
 from app.discover.budget import band
 from app.feedquery import (
     feed_columns,
@@ -27,24 +36,20 @@ from app.feedquery import (
     not_blocked_cond,
     order_by,
     serialize,
+    shuffle_walk,
     sort_keys,
 )
-from app.models import (
-    Channel,
-    PlaylistItem,
-    PlexItem,
-    PlexShow,
-    PlexState,
-    User,
-    Video,
-    VideoState,
-)
-from app.plex import rollup
+from app.models import User
+from app.plex.models import PlexItem, PlexShow, PlexState
+from app.catalog.models import Channel, Video
+from app.reader.models import PlaylistItem, VideoState
+from app.plex import affinity, rollup
 from app.plex.cards import display_title, leaf_card, movie_card, show_card
 from app.plex.query import apply_show_state_filter, content_libs
+from app.userscope import owned_by
 
 # How many rows a single rail may carry — Home's rails and the Plex module's Discover rails alike
-# (`routes/plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
+# (`plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
 # is a rail. One definition, because a comment was all that held two copies in step.
 RAIL_LIMIT = 20
 
@@ -72,7 +77,8 @@ _FEED_BASE: dict = {
 
 
 def _seeded(col, seed: int):
-    """The feed's deterministic shuffle expression, reused for Plex so one seed moves both halves."""
+    """The Home surprise's seeded order (md5 of the key and the seed), the same for Plex so one
+    seed moves both halves."""
     return func.md5(func.concat(col, str(seed)))
 
 
@@ -98,7 +104,7 @@ def yt_in_progress(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[tup
     q = (
         select(*feed_columns(user, st), st.progress_updated_at.label("progressed_at"))
         .join(Channel, Channel.id == Video.channel_id)
-        .join(st, and_(st.video_id == Video.id, st.user_id == user.id))
+        .join(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
         .where(
             st.position_seconds > 0,
             st.status.notin_(("watched", "hidden")),
@@ -117,17 +123,75 @@ def yt_surprise(db: Session, user: User, budget: str, seed: int, limit: int) ->
     Shorts are opened up only for the shortest band: "I have ten minutes" is the one answer a Short is
     actually a good response to, and the feed's own default keeps them out otherwise."""
     lo, hi = band(budget)
-    params = dict(_FEED_BASE, min_duration=lo, max_duration=hi, include_shorts=budget == "short")
-    query, rank_expr = filtered_query(db, user, **params)
-    keys = sort_keys("shuffle", seed, rank_expr)
-    # Over-fetch, then drop anything already started. The feed's `unwatched` scope keeps watched and
-    # hidden out but deliberately keeps in-progress rows in (they belong in the feed). Here they must
-    # go: the resume tier has already had its chance at them, and it may have passed precisely because
-    # the REMAINING time did not fit the budget — offering the same item back as a surprise, whose
-    # runtime does fit, would label a half-finished thing "Meglepetés" and contradict the question.
-    rows = db.execute(query.order_by(*order_by(keys)).limit(limit * 4)).all()
-    out = [serialize(r) for r in rows]
-    return [r for r in out if not r["position_seconds"]][:limit]
+    params: dict = {
+        **_FEED_BASE,
+        "min_duration": lo,
+        "max_duration": hi,
+        "include_shorts": budget == "short",
+    }
+    query, _rank = filtered_query(db, user, **params)
+
+    # A window of the ring, not the whole scope: the seed's SURPRISE_WINDOW videos read along the
+    # stored random key (`shuffle_walk`, a millisecond), then shuffled by the seed. Dealing straight
+    # from the ring would favour for good every video that sits after a wide gap of it (the first
+    # video past a random point is the one whose gap the point fell into); shuffling the window makes
+    # each video's chance the window's share, even to within a few percent. A scope no larger than
+    # the window is the whole scope, in exactly the md5 order the draw always had.
+    #
+    # Anything already started is dropped. The feed's `unwatched` scope keeps watched and hidden out
+    # but deliberately keeps in-progress rows in (they belong in the feed). Here they must go: the
+    # resume tier has already had its chance at them, and it may have passed precisely because the
+    # REMAINING time did not fit the budget — offering the same item back as a surprise, whose runtime
+    # does fit, would label a half-finished thing "Meglepetés" and contradict the question.
+    window = [
+        r
+        for r in map(serialize, shuffle_walk(db, query, seed, SURPRISE_WINDOW))
+        if not r["position_seconds"]
+    ]
+    blind = draw.shuffled(window, seed, key=_video_id)
+    channels = _channel_affinity(db, user)
+    if not channels:
+        return blind[:limit]
+
+    def weight(row: dict) -> float:
+        return 1.0 + CHANNEL_WEIGHT * channels.get(row["channel_id"], 0.0)
+
+    leaning = draw.leaning(window, seed, key=_video_id, weight=weight)
+    return draw.blend(leaning, blind, seed, limit, key=_video_id)
+
+
+# How many videos of the ring a surprise draws from: wide enough that a channel the user watches has
+# a few in it, small enough to read in a millisecond or two.
+SURPRISE_WINDOW = 500
+
+
+def _video_id(row: dict) -> str:
+    return row["id"]
+
+
+# How much a channel the user watches most outweighs one they never watch (1 + this), and how far back
+# the watched history counts: the latest WATCHED_RECENT watches, the recent ones weighing more.
+CHANNEL_WEIGHT = 3.0
+WATCHED_RECENT = 300
+WATCHED_HALF_LIFE_DAYS = 90
+
+
+def _channel_affinity(db: Session, user: User) -> dict[str, float]:
+    """Each channel the user watched lately, scored 0..1 against the most watched: the sum of its
+    watches, each weighing half as much every WATCHED_HALF_LIFE_DAYS. Empty with no history."""
+    rows = db.execute(
+        select(Video.channel_id, VideoState.watched_at)
+        .join(Video, Video.id == VideoState.video_id)
+        .where(owned_by(VideoState.user_id, user), VideoState.status == "watched")
+        .order_by(VideoState.watched_at.desc().nullslast())
+        .limit(WATCHED_RECENT)
+    ).all()
+    now = datetime.now(timezone.utc)
+    scores: dict[str, float] = {}
+    for channel_id, watched_at in rows:
+        r = draw.recency(watched_at, now, WATCHED_HALF_LIFE_DAYS)
+        scores[channel_id] = scores.get(channel_id, 0.0) + r
+    return draw.scaled(scores)
 
 
 def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
@@ -141,7 +205,7 @@ def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
     row = db.execute(
         select(*feed_columns(user, st))
         .join(Channel, Channel.id == Video.channel_id)
-        .outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
+        .outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
         .where(Video.id == video_id)
     ).first()
     return serialize(row) if row is not None else None
@@ -184,7 +248,7 @@ def playlists_summary(
         select(*feed_columns(user, st), PlaylistItem.playlist_id.label("_playlist"))
         .join(PlaylistItem, PlaylistItem.video_id == Video.id)
         .join(Channel, Channel.id == Video.channel_id)
-        .outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
+        .outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
         .where(PlaylistItem.playlist_id.in_(playlist_ids))
         .order_by(PlaylistItem.playlist_id, PlaylistItem.position.asc())
     ).all()
@@ -209,7 +273,7 @@ def _unseen_movie_conds(uid: int):
         .select_from(PlexState)
         .where(
             PlexState.item_id == PlexItem.id,
-            PlexState.user_id == uid,
+            owned_by(PlexState.user_id, uid),
             or_(
                 PlexState.status.in_(("watched", "hidden")),
                 # Started counts as "not a surprise" too — see the note in `yt_surprise`.
@@ -226,7 +290,7 @@ def plex_in_progress(
 ) -> list[tuple[object, dict]]:
     """In-progress movies + partially-watched shows as (progressed_at, card).
 
-    Mirrors `routes/plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
+    Mirrors `plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
     NULL status still counts as started, so `notin_` alone would drop it), and the same shared
     `rollup.agg_subquery` + `apply_show_state_filter` for shows, so the two rails cannot disagree on
     what "in progress" means. What it does NOT inherit is that route's `LibraryFilters` narrowing:
@@ -237,7 +301,9 @@ def plex_in_progress(
     if movie_libs:
         mq = (
             select(PlexItem, PlexState)
-            .join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
+            .join(
+                PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
+            )
             .where(
                 PlexItem.kind == "movie",
                 PlexItem.library_id.in_(movie_libs),
@@ -292,21 +358,24 @@ def plex_surprise(
     if not movie_libs:
         return []
     lo, hi = band(budget)
-    q = (
-        select(PlexItem)
-        .where(
-            PlexItem.kind == "movie",
-            PlexItem.library_id.in_(movie_libs),
-            _unseen_movie_conds(uid),
-        )
-        .order_by(_seeded(PlexItem.rating_key, seed))
-        .limit(limit)
+    q = select(PlexItem).where(
+        PlexItem.kind == "movie",
+        PlexItem.library_id.in_(movie_libs),
+        _unseen_movie_conds(uid),
     )
     if lo is not None:
         q = q.where(PlexItem.duration_s >= lo)
     if hi is not None:
         q = q.where(PlexItem.duration_s <= hi)
-    items = db.execute(q).scalars().all()
+    shuffled = _seeded(PlexItem.rating_key, seed)
+    blind = db.execute(q.order_by(shuffled, PlexItem.id).limit(limit)).scalars().all()
+    leaning = affinity.for_user(db, uid)
+    if leaning:
+        key = draw.weighted_key(draw.seeded_u(PlexItem.rating_key, seed), leaning.weight(PlexItem))
+        weighted = db.execute(q.order_by(key, shuffled, PlexItem.id).limit(limit)).scalars().all()
+        items = draw.blend(list(weighted), list(blind), seed, limit, key=lambda it: it.id)
+    else:
+        items = list(blind)
     states = rollup.item_state_map(db, uid, [it.id for it in items])
     return [movie_card(it, states.get(it.id), lang=lang) for it in items]
 
@@ -331,7 +400,9 @@ def next_episode(db: Session, uid: int, show_id: int, *, lang: str) -> dict | No
     )
     base = (
         select(PlexItem, PlexState)
-        .outerjoin(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
+        .outerjoin(
+            PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
+        )
         .where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
         .limit(1)
     )

backend/app/discover/rails.py

@@ -11,7 +11,8 @@ from sqlalchemy.orm import Session
 from app import sysconfig
 from app.discover import cards, queries
 from app.discover.queries import RAIL_LIMIT
-from app.models import PlexShow, User
+from app.models import User
+from app.plex.models import PlexShow
 from app.plex.lookups import by_rating_key
 from app.plex.rollup import NO_TS
 
@@ -75,7 +76,7 @@ def surprise(
 ) -> list[dict]:
     """The hero's draw pool: unseen things that fit the budget, from both sources under ONE seed.
 
-    Interleaved rather than concatenated — the same alternation `routes/plex/discover.py` uses for its
+    Interleaved rather than concatenated — the same alternation `plex/discover.py` uses for its
     movie/show split — so a short pool never comes back all-YouTube or all-Plex just because one side
     happened to be queried first.
 

backend/app/downloads/adapters.py

@@ -14,10 +14,10 @@ Resolution works both before extraction (from source_kind / URL host) and after
 import re
 from urllib.parse import urlparse
 
-import yt_dlp
 from yt_dlp.networking.impersonate import ImpersonateTarget
 
 from app.config import settings
+from app.downloads import egress
 from app.downloads.stated import StatedFailure
 
 # Facebook's "Cannot parse data": the page yt-dlp got was a wall, not the content. Both the Chrome retry
@@ -33,6 +33,9 @@ class SourceAdapter:
     """Generic source: yt-dlp handles everything and the ref is already a fetchable URL."""
 
     key = "generic"
+    # SEC-17: every fetch of this source's download goes through the egress gate (`egress`). A
+    # source's own host list is no guarantee — its pages, redirects and formats can point anywhere.
+    egress_guarded = True
 
     # --- extraction (worker._extract_with_fallback / backfill) ---------------------------------
     def resolve_url(self, url: str, ydl_opts: dict) -> str:
@@ -66,6 +69,11 @@ class SourceAdapter:
         """The download's raw title, before `titles.normalize_title` cleans it. Generic: yt-dlp's."""
         return info.get("title")
 
+    def is_placeholder_title(self, title: str | None) -> bool:
+        """Whether `title` is a stand-in the source gives in place of the real one (worth one more,
+        signed-out read of the page). Generic: never."""
+        return False
+
     def uploader_url(self, info: dict) -> str | None:
         """The channel/uploader page URL. Generic: whatever yt-dlp exposes directly (YouTube gives
         `channel_url`), else None (reddit / a raw media URL have no derivable channel page)."""
@@ -87,6 +95,9 @@ class SourceAdapter:
 
 class YouTubeAdapter(SourceAdapter):
     key = "youtube"
+    # Not through the gate: YouTube's fetches go to Google's fixed hosts, and the PO-token provider
+    # it needs is the sidecar on our internal network, which the gate exists to refuse.
+    egress_guarded = False
     # Errors worth retrying with a different player-client set (YouTube per-client flakiness).
     _RETRIABLE = (
         "not a bot",
@@ -174,7 +185,7 @@ class FacebookAdapter(SourceAdapter):
         post_id = facebook_group_post_id(url)
         if post_id is None or not ydl_opts.get("cookiefile"):
             return url
-        with yt_dlp.YoutubeDL(ydl_opts) as ydl:
+        with egress.open_ydl(ydl_opts) as ydl:
             raw = ydl.urlopen(url).read(self._POST_PAGE_MAX_BYTES)
         page = raw.decode("utf-8", "replace")
         # Measured: a refused or expired cookie gets the login page (~450 KB, a login
@@ -201,11 +212,17 @@ class FacebookAdapter(SourceAdapter):
     # Measured (2026-09-14): a SIGNED-IN read gives the placeholder "Video" (plain and as Chrome), and an
     # 18+ reel only readable as Chrome gives "Facebook"; the post's text is still in `description`. A
     # signed-out read titles the page with that same text behind the engagement counts, cut short.
+    # Measured (2026-11-01, yt-dlp 2026.08.19): since 2026-10-28 every signed-in reel read gives
+    # "Facebook" and NO description, while a signed-out read of the same reel still carries the text —
+    # so a placeholder left with nothing behind it gets one signed-out read (`backfill.public_title_info`).
     _PLACEHOLDER_TITLES = frozenset({"video", "facebook"})
 
+    def is_placeholder_title(self, title: str | None) -> bool:
+        return (title or "").strip().lower() in self._PLACEHOLDER_TITLES
+
     def title(self, info: dict) -> str | None:
         raw = info.get("title")
-        if (raw or "").strip().lower() not in self._PLACEHOLDER_TITLES:
+        if not self.is_placeholder_title(raw):
             return raw
         line = next(
             (ln.strip() for ln in (info.get("description") or "").splitlines() if ln.strip()), ""

backend/app/downloads/backfill.py

@@ -5,7 +5,10 @@ auto-detected channel becomes a clickable link) and `poster_path` (an ffmpeg fra
 thumbnail-less sources, so the card / watch page / link preview still shows an image). Best-effort
 and safe to re-run: it only touches assets still missing the field, and only re-fetches YouTube /
 Facebook sources (a direct media URL like a reddit HLS manifest carries no channel and its signed
-URL may have expired — those just get a locally-generated poster instead).
+URL may have expired — those just get a locally-generated poster instead). A Facebook download
+titled with the source's placeholder ("Facebook", "Video") is retitled from a signed-out read of its
+page, and so is each download name that only repeated the placeholder. That page read
+(`public_title_info`) also titles every new download, from the worker.
 
 Run once per environment:
     docker exec siftlode-worker-1 python -c "from app.downloads import backfill; print(backfill.run())"
@@ -13,21 +16,23 @@ Run once per environment:
 
 from urllib.parse import urlsplit
 
-import yt_dlp
-from sqlalchemy import select
+from sqlalchemy import select, update
+from sqlalchemy.orm import Session
 
 from app import logs
 from app.config import settings
 from app.db import SessionLocal
 from app.downloads import adapters
 from app.downloads import edit as editmod
-from app.downloads import formats, service
-from app.models import MediaAsset
+from app.downloads import egress, formats, service
+from app.downloads.models import DownloadJob, MediaAsset
+from app.models import DISPLAY_TEXT_MAX
+from app.titles import normalize_title
 
 log = logs.get_logger(__name__)
 
 
-def _extract_meta(url: str, adapter: adapters.SourceAdapter, asset_id: int) -> dict | None:
+def extract_meta(url: str, adapter: adapters.SourceAdapter, asset_id: int) -> dict | None:
     """Metadata-only yt-dlp extraction, using the source adapter's client sets + options (YouTube
     needs the POT/player-client strategy even just to extract)."""
     with formats.hiding(url):
@@ -37,11 +42,15 @@ def _extract_meta(url: str, adapter: adapters.SourceAdapter, asset_id: int) -> d
 def _extract_meta_scoped(url: str, adapter: adapters.SourceAdapter, asset_id: int) -> dict | None:
     for clients in adapter.client_sets():
         opts = formats.build_ydl_opts(
-            {}, "%(id)s.%(ext)s", lambda d: None, extra=adapter.ydl_opts_extra(clients)
+            {},
+            "%(id)s.%(ext)s",
+            lambda d: None,
+            extra=adapter.ydl_opts_extra(clients),
+            egress=adapter.egress_guarded,
         )
         opts["skip_download"] = True
         try:
-            with yt_dlp.YoutubeDL(opts) as ydl:
+            with egress.open_ydl(opts) as ydl:
                 info = ydl.extract_info(url, download=False)
             if info:
                 return info
@@ -62,8 +71,43 @@ def _extract_meta_scoped(url: str, adapter: adapters.SourceAdapter, asset_id: in
     return None
 
 
+def public_title_info(info: dict, url: str, adapter: adapters.SourceAdapter, asset_id: int) -> dict:
+    """`info` to title a download from: itself, unless its title is a source's placeholder with
+    nothing behind it (a signed-in Facebook read: "Facebook", no description). Then one signed-out,
+    metadata-only read of the page lends its text — the description, else its title — as the
+    description the adapter takes a placeholder's title from. Nothing better there: `info` as it was."""
+    if not adapter.is_placeholder_title(adapter.title(info)):
+        return info
+    public = extract_meta(url, adapter, asset_id)
+    text = public and (public.get("description") or public.get("title"))
+    return {**info, "description": text} if text else info
+
+
+def _retitle(db: Session, a: MediaAsset, adapter: adapters.SourceAdapter, url: str) -> bool:
+    """Swap a placeholder title kept from an earlier download for the page's text, and the download
+    names that only repeated it. A name its owner gave (anything else) stays."""
+    if not adapter.is_placeholder_title(a.title):
+        return False
+    info = public_title_info({"title": a.title, "description": None}, url, adapter, a.id)
+    named = normalize_title(adapter.title(info))
+    if not named or adapter.is_placeholder_title(named):
+        return False
+    # One conditional write, not a read and a compare: a name its owner gives while the page is read
+    # is not overwritten by what was read before it. The asset first, as every other writer of the
+    # two locks them (`service.lock_own_job`): the session does not flush by itself.
+    placeholder = a.title
+    a.title = named
+    db.flush()
+    db.execute(
+        update(DownloadJob)
+        .where(DownloadJob.asset_id == a.id, DownloadJob.display_name == placeholder)
+        .values(display_name=named[:DISPLAY_TEXT_MAX])
+    )
+    return True
+
+
 def run() -> dict:
-    filled_url = filled_poster = 0
+    filled_url = filled_poster = retitled = 0
     with SessionLocal() as db:
         assets = (
             db.execute(
@@ -78,7 +122,7 @@ def run() -> dict:
             adapter = adapters.resolve_adapter(a.source_kind, a.source_ref)
             if a.uploader_url is None and adapter.wants_metadata_backfill():
                 url = a.source_webpage_url or service.source_url(a.source_kind, a.source_ref)
-                info = _extract_meta(url, adapter, a.id)
+                info = extract_meta(url, adapter, a.id)
                 uu = adapter.uploader_url(info) if info else None
                 if uu:
                     a.uploader_url = uu
@@ -86,6 +130,14 @@ def run() -> dict:
                     log.info(
                         "asset %s uploader_url filled", a.id
                     )  # the URL names what they downloaded
+            if adapter.wants_metadata_backfill():
+                url = a.source_webpage_url or service.source_url(a.source_kind, a.source_ref)
+                if _retitle(db, a, adapter, url):
+                    # Before the poster step: it can cut a frame for 30 s, and these locks held
+                    # across it block a rename and deadlock a delete.
+                    db.commit()
+                    retitled += 1
+                    log.info("asset %s retitled", a.id)  # the title names what they downloaded
             if a.poster_path is None:
                 # Record a self-hosted poster for every download (the reliable og:image source):
                 # ensure_poster returns the existing <base>.jpg thumbnail sidecar, or cuts a frame.
@@ -99,6 +151,7 @@ def run() -> dict:
         "assets": len(assets),
         "filled_uploader_url": filled_url,
         "filled_poster": filled_poster,
+        "retitled": retitled,
     }
     log.info("backfill done: %s", result)
     return result

backend/app/downloads/clip.py

@@ -22,6 +22,7 @@ import json
 import math
 import re
 from pathlib import Path
+from typing import Literal
 
 from app.downloads.edit import AAC_ARGS, EVEN_FRAME, INPUT_GUARD, X264_ARGS
 from app.plex.sync import HDR_TRANSFERS
@@ -57,12 +58,27 @@ _CROP_ASPECTS = (0.25, 4.0)
 _EDGE_SLACK = 1e-3
 
 
+# Every reason a clip is refused; the UI names each one (`clip.errors.<reason>`, held by the frontend's
+# backendIds contract, which reads this Literal).
+ClipReason = Literal[
+    "bad_source",
+    "bad_range",
+    "too_short",
+    "too_long",
+    "bad_audio",
+    "bad_crop",
+    "not_found",
+    "media_missing",
+    "past_end",
+]
+
+
 class ClipError(Exception):
     """A clip request that cannot be cut; `reason` is an i18n-mappable key, `params` the numbers its
     message names (the cap itself, so the client never says a stale one)."""
 
-    def __init__(self, reason: str, **params: float):
-        self.reason = reason
+    def __init__(self, reason: ClipReason, **params: float):
+        self.reason: ClipReason = reason
         self.params = params
         super().__init__(reason)
 
@@ -185,6 +201,16 @@ def _ratio(v: object) -> float | None:
     return n / d if n > 0 and d > 0 else None
 
 
+def quarter_turned(probe: dict) -> bool | None:
+    """Whether the first video stream of an ffprobe `-show_streams` JSON is stored turned a quarter
+    (ffmpeg stands it upright before any filter, so its filters see width and height swapped). None
+    when there is no video stream."""
+    for s in probe.get("streams") or []:
+        if s.get("codec_type") == "video":
+            return _quarter_turned(s)
+    return None
+
+
 def _quarter_turned(stream: dict) -> bool:
     rotation: object = (stream.get("tags") or {}).get("rotate")
     for sd in stream.get("side_data_list") or []:

backend/app/downloads/credentials.py

@@ -25,6 +25,7 @@ Security controls enforced here:
 
 import os
 from pathlib import Path
+from typing import Literal
 
 from sqlalchemy import select
 from sqlalchemy.orm import Session
@@ -33,8 +34,9 @@ from app import logs
 from app.downloads import clip as clipmod
 from app.downloads import formats
 from app.downloads.adapters import _host_is
-from app.models import DownloadCredential, MediaAsset
+from app.downloads.models import DownloadCredential, MediaAsset
 from app.security import decrypt_optional, encrypt
+from app.userscope import owned_by
 
 log = logs.get_logger(__name__)
 
@@ -157,14 +159,23 @@ def salt_for_enqueue(db: Session, user_id: int, url: str | None) -> str | None:
 def get_credential(db: Session, user_id: int, host_key: str) -> DownloadCredential | None:
     return db.execute(
         select(DownloadCredential).where(
-            DownloadCredential.user_id == user_id, DownloadCredential.host_key == host_key
+            owned_by(DownloadCredential.user_id, user_id), DownloadCredential.host_key == host_key
         )
     ).scalar_one_or_none()
 
 
+# Every reason a pasted cookies.txt is refused; the UI names each one (`downloads.errors.cookies.<reason>`,
+# held by the frontend's backendIds contract, which reads this Literal).
+CookieReason = Literal["empty", "format"]
+
+
 class InvalidCookies(ValueError):
     """The pasted cookies aren't a usable Netscape cookies.txt — surfaced as a 400 by the route."""
 
+    def __init__(self, reason: CookieReason):
+        super().__init__(reason)
+        self.reason: CookieReason = reason
+
 
 def validate_cookies(raw: str) -> str:
     """Sanity-check a pasted cookies.txt (Netscape format) and return the cleaned text. yt-dlp reads
@@ -214,7 +225,7 @@ def status_list(db: Session, user_id: int) -> list[dict]:
     have = {
         c.host_key: c
         for c in db.execute(
-            select(DownloadCredential).where(DownloadCredential.user_id == user_id)
+            select(DownloadCredential).where(owned_by(DownloadCredential.user_id, user_id))
         ).scalars()
     }
     out = []

backend/app/downloads/edit.py

@@ -11,7 +11,9 @@ with the worker branching on the asset's `source_kind` to run ffmpeg instead of
                        (the `-ss` seeks first), so a short clip is cheap even from a long source
   * crop (± trim)    → always re-encode (a filter can't be stream-copied), frame-accurate
 
-`edit_spec = {trim?: {start_s, end_s}, crop?: {x, y, w, h}, accurate?: bool}`. `accurate` is the
+`edit_spec = {trim?: {start_s, end_s}, crop?: {x, y, w, h, frame_w?, frame_h?}, accurate?: bool}`. A
+crop with `frame_w`/`frame_h` counts the picture as it is SHOWN, measured on a frame of that size
+(`crop_filter`); one without them is in the file's stored pixels (its older form). `accurate` is the
 per-edit user choice for a trim-only cut (ignored when a crop forces a re-encode anyway). `split`
 is a frontend concept: the
 UI fans a split out into N trim jobs, so there is one output file per job here.
@@ -60,6 +62,46 @@ class EditAborted(Exception):
 _MAX_SEGMENTS = 100
 
 
+def _crop_frame(crop: dict) -> dict | None:
+    """The picture size a crop was measured on, or None when it has none it could be scaled by (a
+    spec from before it was sent, which `crop_filter` cuts in the file's own pixels)."""
+    try:
+        w, h = int(crop["frame_w"]), int(crop["frame_h"])
+    # silent-ok: no frame is a crop's older form, and it is still the crop that was asked for. An
+    # infinite number (JSON `1e400`) is OverflowError, and it is no more a frame than "x" is.
+    except (KeyError, TypeError, ValueError, OverflowError):
+        return None
+    return {"frame_w": w, "frame_h": h} if w > 0 and h > 0 else None
+
+
+def crop_filter(crop: dict) -> str:
+    """The ffmpeg `crop` of an edit. Its numbers count the picture as the browser reports it, which
+    is its SHOWN size: a video with wide pixels (320 stored, 640 shown) would be cut off to one side
+    in ffmpeg's stored ones, so they are scaled to `iw` / `ih` by the size they were measured on. A
+    rotated file needs nothing more: both the browser and ffmpeg's filters see it upright."""
+    frame_w, frame_h = crop.get("frame_w"), crop.get("frame_h")
+    if not frame_w or not frame_h:
+        return f"crop={crop['w']}:{crop['h']}:{crop['x']}:{crop['y']}"
+    return (
+        f"crop=iw*{crop['w']}/{frame_w}:ih*{crop['h']}/{frame_h}"
+        f":iw*{crop['x']}/{frame_w}:ih*{crop['y']}/{frame_h}"
+    )
+
+
+def crop_stored_size(crop: dict, src_w: int | None, src_h: int | None) -> tuple[int, int] | None:
+    """The size `crop_filter` cuts out of a `src_w` x `src_h` frame, evened as `EVEN_FRAME` evens it —
+    the clip's size when ffprobe cannot tell. The frame is the one ffmpeg's filters see: a rotated
+    file's stood upright, its stored size turned (the caller's to turn). A crop counted in the shown
+    picture is a fraction of that frame, not its own pixel count; None when the size is unknown."""
+    if crop.get("frame_w") and crop.get("frame_h"):
+        if not src_w or not src_h:
+            return None
+        w, h = src_w * crop["w"] // crop["frame_w"], src_h * crop["h"] // crop["frame_h"]
+    else:
+        w, h = crop["w"], crop["h"]
+    return w // 2 * 2, h // 2 * 2
+
+
 def normalize_edit_spec(spec: dict | None) -> dict:
     """Canonicalize a raw edit spec: clamp/round trim, coerce crop to ints, drop empties.
 
@@ -72,7 +114,7 @@ def normalize_edit_spec(spec: dict | None) -> dict:
     if crop:
         try:
             c = {k: int(round(float(crop[k]))) for k in ("x", "y", "w", "h")}
-        except (KeyError, TypeError, ValueError) as exc:
+        except (KeyError, TypeError, ValueError, OverflowError) as exc:
             # The clip still renders — uncropped. The user asked for a crop, got a whole frame,
             # and nothing anywhere called it a failure; this line is the only trace there is.
             log.warning(
@@ -81,7 +123,8 @@ def normalize_edit_spec(spec: dict | None) -> dict:
             )
             c = None
         if c and c["w"] > 0 and c["h"] > 0 and c["x"] >= 0 and c["y"] >= 0:
-            out["crop"] = c
+            frame = _crop_frame(crop)
+            out["crop"] = {**c, **frame} if frame else c
 
     # Multi-segment JOIN (cut-list → one concatenated file). Takes precedence over `trim`.
     segs = spec.get("segments")
@@ -268,7 +311,7 @@ def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]
     if reencode:
         # Input -ss + re-encode is frame-accurate (decodes from the prior keyframe, discards up to
         # the exact cut). A crop adds the filter; an accurate trim only evens the frame.
-        cropf = f"crop={crop['w']}:{crop['h']}:{crop['x']}:{crop['y']}," if crop else ""
+        cropf = f"{crop_filter(crop)}," if crop else ""
         cmd += ["-vf", cropf + EVEN_FRAME, *X264_ARGS, *AAC_ARGS]
     else:
         cmd += ["-c", "copy", "-avoid_negative_ts", "make_zero"]
@@ -313,7 +356,7 @@ def build_concat_plan(src: Path, dest: Path, spec: dict, out_ext: str, staging:
     ]
 
     if needs_reencode(spec):
-        cropf = f",crop={crop['w']}:{crop['h']}:{crop['x']}:{crop['y']}" if crop else ""
+        cropf = f",{crop_filter(crop)}" if crop else ""
         parts, labels = [], []
         for i, s in enumerate(segs):
             st, en = s["start_s"], s["end_s"]

backend/app/downloads/egress.py

@@ -0,0 +1,297 @@
+"""The egress gate: every fetch of a user-supplied download goes through here, and this is where the
+address is judged — at the moment of the connection, not when the URL was typed (SEC-17).
+
+`urlguard` judges the URL the user typed, at enqueue and again in the worker. What that cannot see is
+every request yt-dlp makes after it: a public host answering 3xx with an internal `Location`, a page
+whose formats point inside, a name that resolves public for the check and private for the fetch (DNS
+rebinding). So a guarded download runs with yt-dlp's `proxy` set to this gate, a small HTTP proxy on
+the worker's loopback. For each request it resolves the host ONCE, refuses the connection if any
+address the name has is one the server must not fetch from (`urlguard.is_forbidden`), and otherwise
+connects to the very address it vetted — there is no second lookup for a rebinding answer to slip into.
+
+Measured (yt-dlp 2026.8.19, 2026-11-06) on the worker image, with a logging proxy: the `proxy` option
+reaches every path a download takes — the Requests handler, the CurlCFFI one an impersonated request
+uses, the native HLS downloader and an ffmpeg HLS download (segments included). ffmpeg itself also
+speaks rtmp, rtsp, ftp, tcp and more, which no HTTP proxy covers, so a guarded download keeps only the
+formats it fetches over http(s) (`HttpOnlyFormats`, added by `open_ydl`) and fetches HLS natively.
+
+A refused connection is answered `403` before anything is dialled, so a live internal port and a dead
+one read the same; `errors.user_error_text` then stores the one redacted sentence for both.
+
+Not used for YouTube: those downloads talk to Google's fixed hosts and to the PO-token sidecar, which
+lives on the internal network on purpose (`adapters.YouTubeAdapter.egress_guarded`).
+"""
+
+import ipaddress
+import select
+import socket
+import socketserver
+import threading
+from collections.abc import Callable
+from typing import Any, cast
+from urllib.parse import urlsplit
+
+import yt_dlp
+from yt_dlp.postprocessor.common import PostProcessor
+
+from app import logs
+from app.downloads import urlguard
+
+log = logs.get_logger(__name__)
+
+Resolver = Callable[..., list]
+Forbidden = Callable[[ipaddress.IPv4Address | ipaddress.IPv6Address], bool]
+
+_BLOCKED = b"HTTP/1.1 403 Forbidden\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
+_BAD_GATEWAY = b"HTTP/1.1 502 Bad Gateway\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
+_BAD_REQUEST = b"HTTP/1.1 400 Bad Request\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
+_MAX_HEAD = 64 * 1024
+# A tunnel with no traffic either way for this long is closed. Above yt-dlp's own socket_timeout
+# (30 s, `formats.build_ydl_opts`), so the gate never ends a transfer yt-dlp would still wait for.
+_IDLE_S = 120
+# Hop-by-hop headers a proxy must not pass on; `Connection: close` replaces them, so one connection
+# carries one request and a reused client connection can never be steered to a second host unvetted.
+_HOP = (b"proxy-", b"connection:", b"keep-alive:")
+
+
+def vetted_address(
+    host: str, port: int, *, resolve: Resolver = socket.getaddrinfo, forbidden: Forbidden
+) -> tuple[str, int] | None:
+    """The address to connect to for `host:port`, or None when it must not be fetched: the name has
+    no answer, or ANY of its addresses is forbidden (a round-robin answer is not a coin flip). IPv4
+    only, as every download is (`source_address` 0.0.0.0 in `formats.build_ydl_opts`)."""
+    try:
+        infos = resolve(host, port, socket.AF_INET, socket.SOCK_STREAM)
+    except (socket.gaierror, UnicodeError, OSError):
+        # silent-ok: "no answer" is a refusal, and the caller's refusal is the line that is logged.
+        return None
+    addrs = [info[4] for info in infos]
+    if not addrs:
+        return None
+    for addr in addrs:
+        try:
+            ip = ipaddress.ip_address(addr[0])
+        except ValueError:
+            # silent-ok: an answer that is not an address is refused like a forbidden one, and the
+            # caller logs the refusal.
+            return None
+        if forbidden(ip):
+            return None
+    return addrs[0][0], addrs[0][1]
+
+
+def _read_head(sock: socket.socket) -> bytes | None:
+    buf = b""
+    while b"\r\n\r\n" not in buf:
+        if len(buf) > _MAX_HEAD:
+            return None
+        chunk = sock.recv(8192)
+        if not chunk:
+            return None
+        buf += chunk
+    return buf
+
+
+def _pipe(a: socket.socket, b: socket.socket) -> None:
+    try:
+        _relay(a, b)
+    except OSError:
+        # silent-ok: one side hung up mid-transfer (yt-dlp closes a connection once it has read a
+        # file's size, or gives up on a slow host); whichever side cared reports it itself.
+        return
+
+
+def _relay(a: socket.socket, b: socket.socket) -> None:
+    socks = [a, b]
+    while True:
+        ready, _, _ = select.select(socks, [], [], _IDLE_S)
+        if not ready:
+            return
+        for s in ready:
+            data = s.recv(65536)
+            if not data:
+                return
+            (b if s is a else a).sendall(data)
+
+
+class _Handler(socketserver.BaseRequestHandler):
+    def handle(self) -> None:
+        client: socket.socket = self.request
+        head = _read_head(client)
+        if head is None:
+            return
+        head_part, _, body = head.partition(b"\r\n\r\n")
+        lines = head_part.split(b"\r\n")
+        try:
+            method, target, version = lines[0].decode("latin1").split(" ", 2)
+        except ValueError:
+            # silent-ok: the only client is yt-dlp on this loopback; a malformed request is answered
+            # 400 and reaches its own error, which the job records.
+            client.sendall(_BAD_REQUEST)
+            return
+        if method == "CONNECT":
+            host, _, port_s = target.rpartition(":")
+            host = host.strip("[]")
+            upstream = self._dial(host, port_s)
+            if upstream is None:
+                return
+            with upstream:
+                client.sendall(b"HTTP/1.1 200 Connection established\r\n\r\n")
+                if body:
+                    upstream.sendall(body)
+                _pipe(client, upstream)
+            return
+        u = urlsplit(target)
+        if u.scheme != "http" or not u.hostname:
+            client.sendall(_BLOCKED)
+            return
+        try:
+            port = u.port or 80
+        except ValueError:
+            # silent-ok: as for a malformed request line above — answered 400, recorded by the job.
+            client.sendall(_BAD_REQUEST)
+            return
+        upstream = self._dial(u.hostname, str(port))
+        if upstream is None:
+            return
+        path = (u.path or "/") + (f"?{u.query}" if u.query else "")
+        headers = [h for h in lines[1:] if not h.lower().startswith(_HOP)]
+        request = b"\r\n".join(
+            [f"{method} {path} {version}".encode("latin1"), *headers, b"Connection: close"]
+        )
+        with upstream:
+            upstream.sendall(request + b"\r\n\r\n" + body)
+            _pipe(client, upstream)
+
+    def _dial(self, host: str, port_s: str) -> socket.socket | None:
+        """Connect to the address `host` was vetted at, or answer the client and return None."""
+        client: socket.socket = self.request
+        try:
+            port = int(port_s)
+        except ValueError:
+            # silent-ok: as for a malformed request line above — answered 400, recorded by the job.
+            client.sendall(_BAD_REQUEST)
+            return None
+        gate = cast("_Gate", self.server)
+        addr = vetted_address(host, port, resolve=gate.resolve, forbidden=gate.forbidden)
+        if addr is None:
+            log.warning(
+                "egress refused a connection to %s:%s",
+                host,  # log-pii-ok: the host a download tried to reach, never its path or query
+                port,
+                extra=logs.event("download.egress.refused", port=port),
+            )
+            client.sendall(_BLOCKED)
+            return None
+        try:
+            return socket.create_connection(addr, timeout=_IDLE_S)
+        except OSError:
+            # silent-ok: a public host that does not answer; yt-dlp reports it as the job's own
+            # network failure, which the job records.
+            client.sendall(_BAD_GATEWAY)
+            return None
+
+
+class _Gate(socketserver.ThreadingTCPServer):
+    daemon_threads = True
+    allow_reuse_address = True
+
+    def __init__(self, resolve: Resolver, forbidden: Forbidden):
+        self.resolve = resolve
+        self.forbidden = forbidden
+        super().__init__(("127.0.0.1", 0), _Handler)
+
+    @property
+    def url(self) -> str:
+        return f"http://127.0.0.1:{self.server_address[1]}"
+
+    def handle_error(self, request, client_address) -> None:
+        # socketserver's default prints a bare traceback to stderr, outside the log; a connection the
+        # gate could not finish is a line with its code instead.
+        log.exception("egress gate dropped a connection", extra=logs.event("download.egress.error"))
+
+
+def start_gate(
+    *, resolve: Resolver = socket.getaddrinfo, forbidden: Forbidden = urlguard.is_forbidden
+) -> _Gate:
+    """A gate serving on its own thread. The worker uses the one `proxy_url` starts; a test starts
+    its own, with a resolver and a judgement it controls."""
+    gate = _Gate(resolve, forbidden)
+    threading.Thread(target=gate.serve_forever, name="egress-gate", daemon=True).start()
+    return gate
+
+
+# The options key `formats.build_ydl_opts` marks a guarded download with; `open_ydl` reads it. yt-dlp
+# keeps unknown keys in its params and never acts on them.
+GUARDED = "siftlode_egress_guarded"
+
+
+def open_ydl(opts: dict) -> yt_dlp.YoutubeDL:
+    """The one way this package opens yt-dlp (a contract test holds it): a guarded download also
+    gets `HttpOnlyFormats`, which yt-dlp takes only as an instance, not through the options dict."""
+    # yt-dlp's stub types its params as a TypedDict; these are the plain dict `formats.build_ydl_opts`
+    # assembles, the same every caller handed yt-dlp before.
+    ydl = yt_dlp.YoutubeDL(opts)  # type: ignore[arg-type]
+    if opts.get(GUARDED):
+        ydl.add_post_processor(HttpOnlyFormats(), when="pre_process")
+    return ydl
+
+
+_gate: _Gate | None = None
+_gate_lock = threading.Lock()
+
+
+def proxy_url() -> str:
+    """This process's gate, started on first use (the worker's jobs share it)."""
+    global _gate
+    with _gate_lock:
+        if _gate is None:
+            _gate = start_gate()
+        return _gate.url
+
+
+# The protocols fetched over http(s) through the proxy option — the only ones the gate sees: yt-dlp's
+# own fetchers (http, f4m, ism, the DASH segments and their generator, native HLS — which a bare m3u8
+# also gets, as a guarded download sets hls_prefer_native) and ffmpeg's http input (a live stream or
+# a clip), which takes the proxy too. Everything else (rtmp, rtsp, mms, ws) ffmpeg or another tool
+# would dial directly.
+_HTTP_PROTOCOLS = frozenset(
+    {
+        "http",
+        "https",
+        "f4m",
+        "ism",
+        "m3u8",
+        "m3u8_native",
+        "http_dash_segments",
+        "http_dash_segments_generator",
+    }
+)
+
+
+class HttpOnlyFormats(PostProcessor):
+    """Drops every format a guarded download would fetch past the gate. Runs as `pre_process`, which
+    yt-dlp re-reads its formats after ("The pre-processors may have modified the formats"), so the
+    selector never sees them. With nothing left, yt-dlp reports no formats — a refusal, not a leak."""
+
+    def set_downloader(self, downloader):
+        # Not the job's postprocessor hooks: they would report this filter as a "processing" step
+        # before a byte is fetched (yt-dlp attaches them here, once per call).
+        self._downloader = downloader
+
+    def run(self, information):
+        # yt-dlp's stub types the info dict as a TypedDict with optional keys; it is a dict here.
+        info = cast("dict[str, Any]", information)
+        if info.get("formats"):
+            info["formats"] = [f for f in info["formats"] if _over_http(f)]
+        elif info.get("url") and not _over_http(info):
+            info["url"] = None
+        return [], information
+
+
+def _over_http(fmt: dict) -> bool:
+    url = fmt.get("manifest_url") or fmt.get("url") or ""
+    if urlsplit(url).scheme not in ("http", "https"):
+        return False
+    protocol = fmt.get("protocol") or urlsplit(fmt.get("url") or "").scheme
+    return all(p in _HTTP_PROTOCOLS for p in protocol.split("+"))

backend/app/downloads/folders.py

@@ -1,26 +1,30 @@
 """The folders of the Files library — where each of a user's files sits.
 
 A folder is a row, never a directory: on disk a file stays where the download layout put it (one
-shared asset can be held by several users), so a rename or a move is a column update. Two kinds:
+shared asset can be held by several users), so a rename or a move is a column update. Three kinds:
 
 - a SOURCE folder is made automatically, one per origin a user has files from (YouTube, Plex clips,
   uploads, one per other site). Its `source_key` names the origin; the client renders its name
   from the key, in the reader's language.
 - an OWN folder is the user's, with the name they gave it.
+- the HUB folder (one per user, made when they first give a machine a token for it, `hub_folder`) is
+  where they swap files with that machine. Its name, like a source folder's, is the client's; a file
+  may be moved in like into an own folder, and it keeps a file until it is deleted.
 
 Every file of a user is in exactly one folder. The folder a new file lands in is decided here
 (`source_key_for` + `source_folder`), at every place a job is created, so the rule lives once. A file
-moves only into one of the user's own folders or back to its own source folder (`move`): a YouTube
-file in the TikTok folder would be a lie the folder's name tells.
+moves only into one of the user's own folders, their Hub folder, or back to its own source folder
+(`move`): a YouTube file in the TikTok folder would be a lie the folder's name tells.
 
 Retention is the folder's: a source folder keeps a file `download_retention_days` (the value when the
-folder was made; the user can change it), Uploads and own folders keep it until it is deleted. A
+folder was made; the user can change it), Uploads, own folders and the Hub keep it until it is deleted. A
 done file carries its own `expires_at`, stamped by `done_expiry` when it finishes — unless the user
 kept it — and its `ready_at`, the moment it finished (`stamp_done`).
 """
 
 from datetime import datetime, timedelta, timezone
 from enum import StrEnum
+from typing import Literal
 from urllib.parse import urlsplit
 
 from sqlalchemy import select
@@ -29,12 +33,14 @@ from sqlalchemy.orm import Session
 
 from app import sysconfig
 from app.downloads.status import JobStatus
-from app.models import DownloadJob, FileFolder, MediaAsset
+from app.downloads.models import DownloadJob, FileFolder, MediaAsset
+from app.userscope import owned_by
 
 
 class FolderKind(StrEnum):
     SOURCE = "source"
     OWN = "own"
+    HUB = "hub"
 
 
 class SourceKey(StrEnum):
@@ -102,7 +108,9 @@ def source_key_for(db: Session, job: DownloadJob) -> str:
 def source_folder(db: Session, user_id: int, key: str) -> FileFolder:
     """The user's source folder for `key`, made on first use. A race on the unique (user, key) is
     settled by re-selecting, inside a SAVEPOINT so the caller's own pending writes survive."""
-    stmt = select(FileFolder).where(FileFolder.user_id == user_id, FileFolder.source_key == key)
+    stmt = select(FileFolder).where(
+        owned_by(FileFolder.user_id, user_id), FileFolder.source_key == key
+    )
     folder = db.execute(stmt).scalar_one_or_none()
     if folder is not None:
         return folder
@@ -123,6 +131,26 @@ def source_folder(db: Session, user_id: int, key: str) -> FileFolder:
     return folder
 
 
+def hub_folder(db: Session, user_id: int) -> FileFolder:
+    """The user's Hub folder, made on first use; a race on its one-per-user index is settled like
+    `source_folder`'s."""
+    stmt = select(FileFolder).where(
+        owned_by(FileFolder.user_id, user_id), FileFolder.kind == FolderKind.HUB
+    )
+    folder = db.execute(stmt).scalar_one_or_none()
+    if folder is not None:
+        return folder
+    folder = FileFolder(user_id=user_id, kind=FolderKind.HUB)
+    try:
+        with db.begin_nested():
+            db.add(folder)
+            db.flush()
+    # silent-ok: another request made the Hub folder first; re-selecting it is the whole handling.
+    except IntegrityError:
+        folder = db.execute(stmt).scalar_one()
+    return folder
+
+
 def place(db: Session, job: DownloadJob) -> None:
     """Put a new job's file in its source folder, and record that origin on it. Called wherever a
     job is created; the job must carry its `user_id`, `source_kind` and source refs already."""
@@ -154,12 +182,22 @@ def stamp_done(db: Session, job: DownloadJob) -> None:
     job.expiry_warned = False
 
 
+# Every folder change the rules refuse, answered as a generic refusal; the UI names each one
+# (`errors.refused.<reason>`, held by the frontend's backendIds contract, which reads this Literal).
+FolderReason = Literal[
+    "file_folder_gone",
+    "file_folder_name_taken",
+    "file_folder_not_its_source",
+    "file_folder_not_own",
+]
+
+
 class FolderError(Exception):
     """A folder change the rules refuse; `reason` is the refusal the route answers with."""
 
-    def __init__(self, reason: str) -> None:
+    def __init__(self, reason: FolderReason) -> None:
         super().__init__(reason)
-        self.reason = reason
+        self.reason: FolderReason = reason
 
 
 def own_folder(db: Session, user_id: int, folder_id: int, *, hold: bool = True) -> FileFolder:
@@ -178,8 +216,9 @@ def own_folder(db: Session, user_id: int, folder_id: int, *, hold: bool = True)
 
 
 def upload_target(db: Session, user_id: int, folder_id: int, *, hold: bool = True) -> FileFolder:
-    """The folder an upload asked to land in, if it may: one of the user's own folders, or their
-    Uploads folder. A file put in a site's folder would claim an origin it does not have."""
+    """The folder an upload asked to land in, if it may: one of the user's own folders, their Hub
+    folder, or their Uploads folder. A file put in a site's folder would claim an origin it does not
+    have."""
     folder = own_folder(db, user_id, folder_id, hold=hold)
     if folder.kind == FolderKind.SOURCE and folder.source_key != SourceKey.UPLOAD:
         raise FolderError("file_folder_not_its_source")
@@ -201,7 +240,8 @@ def set_kept(db: Session, job: DownloadJob, kept: bool) -> None:
 
 
 def move(db: Session, job: DownloadJob, folder: FileFolder) -> None:
-    """Put a file in `folder` — one of its user's own folders, or the source folder it came from.
+    """Put a file in `folder` — one of its user's own folders, their Hub folder, or the source folder
+    it came from.
     The caller has checked that `folder` is the job's user's."""
     if folder.kind == FolderKind.SOURCE and folder.source_key != source_key_for(db, job):
         raise FolderError("file_folder_not_its_source")
@@ -284,3 +324,33 @@ def delete_own(db: Session, folder: FileFolder) -> int:
     db.flush()  # the files leave before the folder goes: its foreign key RESTRICTs the delete
     db.delete(folder)
     return len(jobs)
+
+
+def create_own(db: Session, user_id: int, name: str) -> FileFolder:
+    """A new folder of the user's own; FolderError("file_folder_name_taken") when they already have
+    one by that name (the unique (user, lower(name)))."""
+    folder = FileFolder(user_id=user_id, kind=FolderKind.OWN, name=name)
+    try:
+        with db.begin_nested():
+            db.add(folder)
+            db.flush()
+    except IntegrityError as e:
+        raise FolderError("file_folder_name_taken") from e
+    return folder
+
+
+def renamable(folder: FileFolder) -> bool:
+    """Only a folder of the user's own making takes a name: a source folder's is its origin's."""
+    return folder.kind == FolderKind.OWN
+
+
+def rename(db: Session, folder: FileFolder, name: str) -> None:
+    """Rename one of the user's own folders (a source folder's name is its origin's)."""
+    if not renamable(folder):
+        raise FolderError("file_folder_not_own")
+    folder.name = name
+    try:
+        with db.begin_nested():
+            db.flush()
+    except IntegrityError as e:
+        raise FolderError("file_folder_name_taken") from e

backend/app/downloads/formats.py

@@ -28,6 +28,7 @@ import urllib.parse
 from collections.abc import Iterator
 
 from app import logs
+from app.downloads import egress as egress_gate
 
 log = logs.get_logger(__name__)
 
@@ -294,6 +295,8 @@ def build_ydl_opts(
     extra: dict | None = None,
     cookiefile: str | None = None,
     max_bytes: int | None = None,
+    *,
+    egress: bool,
 ) -> dict:
     """Assemble yt-dlp options for a single download to `outtmpl` (a full path template).
 
@@ -309,7 +312,11 @@ def build_ydl_opts(
     a file announced larger than it rather than filling the disk and only then being caught by the
     quota accounting. Its ffmpeg downloader (a `download_ranges` fetch) does not read it, nor does
     a fragmented download (YouTube's formats: measured, 718 MB fetched with 1 MB left) — so the
-    pipeline also holds every produced file to what is left by its real size."""
+    pipeline also holds every produced file to what is left by its real size.
+
+    `egress` (SEC-17) routes every fetch through the worker's egress gate, which refuses a connection
+    into our own network, and keeps only the formats fetched over http(s). Required, with no default:
+    a new caller has to say which kind of download it builds (`SourceAdapter.egress_guarded`)."""
     s = normalize(spec)
     h = s["max_height"]
     opts: dict = {
@@ -364,6 +371,13 @@ def build_ydl_opts(
     if cookiefile:
         opts["cookiefile"] = cookiefile
 
+    # SEC-17. AFTER the adapter merge, like the cookie: an adapter must not be able to point a
+    # guarded download past the gate.
+    if egress:
+        opts["proxy"] = egress_gate.proxy_url()
+        opts["hls_prefer_native"] = True
+        opts[egress_gate.GUARDED] = True
+
     if s["mode"] == "a":
         opts["format"] = "bestaudio/best"
         opts["postprocessors"].append(

backend/app/downloads/gc.py

@@ -1,6 +1,6 @@
 """Download retention garbage collector (a scheduler job).
 
-Five passes each run:
+Six passes each run:
   1. Pre-expiry warning — a done file whose `expires_at` (its folder's retention, `folders`) falls
      within the grace window, and that its user did not keep, warns its user once (`expiry_warned`).
   2. Expiry — such a file past its `expires_at` leaves the library: the job is deleted and its hold
@@ -14,6 +14,8 @@ Five passes each run:
      re-download renames an asset, e.g. old accented channel dirs, or an upload's stranded file).
      It skips the dot-prefixed system trees (all but `.uploads`) and anything newer than an hour,
      so an in-flight download is never touched.
+  6. Token expiry warning — a machine's token for the library about to run out warns its user once
+     (`apitokens.warn_expiring`): the library's other clock.
 
 Runs in the API process (which mounts FILES_ROOT). Pure disk/DB work — no YouTube quota.
 """
@@ -25,11 +27,11 @@ from pathlib import Path
 from sqlalchemy import func, or_, select, update
 from sqlalchemy.orm import Session
 
-from app import logs, sysconfig
+from app import apitokens, logs, sysconfig
 from app.config import settings
 from app.downloads import quota, service, storage
 from app.downloads.status import JobStatus
-from app.models import DownloadJob, MediaAsset
+from app.downloads.models import DownloadJob, MediaAsset
 from app.notifications import create_notification
 
 log = logs.get_logger(__name__)
@@ -366,12 +368,16 @@ def run_download_gc(db: Session) -> dict:
     #    posters). Runs last so the keep-set reflects the assets that survived passes 2-4.
     swept = _sweep_orphan_files(db, settings.files_root)
 
+    # 6. A machine's token about to run out.
+    tokens_warned = apitokens.warn_expiring(db)
+
     result = {
         "warned": warned,
         "expired": expired,
         "evicted": evicted,
         "reaped": reaped,
         "swept": swept,
+        "tokens_warned": tokens_warned,
     }
     log.info("download_gc %s", result)
     return result

backend/app/downloads/jobdb.py

@@ -6,7 +6,7 @@ from sqlalchemy import text
 
 from app.config import settings
 from app.db import SessionLocal
-from app.models import DownloadJob, MediaAsset
+from app.downloads.models import DownloadJob, MediaAsset
 
 
 # --- small DB helpers (short-lived sessions; the download itself holds no transaction) -------

backend/app/downloads/links.py

@@ -1,27 +1,91 @@
 """Public "share by link" helpers for the download center.
 
-A DownloadLink is a capability URL: the unguessable `token` grants anyone read access to one
-download's file on the login-free `/watch/{token}` page. Password-protected links additionally
-require a short-lived signed **grant** on the file request — because a `<video src>` can't send a
-header, the password is exchanged once at `/unlock` for an HMAC grant appended to the file URL,
-so the raw password never rides in the media URL (or server logs).
+A DownloadLink is a capability URL: the unguessable token grants anyone read access to one
+download's file, or to the files in one of its owner's own folders (`folder_files`), on the
+login-free `/watch/{token}` page. So the token is never stored as written:
+the link is found by its SHA-256, and the owner's copy is Fernet-encrypted — a database copy alone
+opens nothing. Without that copy (no TOKEN_ENCRYPTION_KEY, or a key since changed) the link still
+opens; only its address cannot be shown again, and the owner makes a new link to share.
+
+Password-protected links additionally require a short-lived signed **grant** on the file request —
+because a `<video src>` can't send a header, the password is exchanged once at `/unlock` for an HMAC
+grant appended to the file URL, so the raw password never rides in the media URL (or server logs).
 """
 
 import hashlib
 import hmac
 import secrets
-from datetime import datetime, timezone
+from datetime import datetime, timedelta, timezone
 
+from sqlalchemy import select
+from sqlalchemy import update as sql_update
+from sqlalchemy.orm import Session
+
+from app import audit, security
 from app.config import settings
-from app.models import DownloadLink
+from app.downloads import storage
+from app.downloads.status import JobStatus
+from app.downloads.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
+from app.models import User
+from app.utils import iso
+from app.userscope import owned_by
 
 _GRANT_TTL = 6 * 3600  # a watch session's grant is good for 6h, then the viewer re-unlocks
+# The most files a folder link lists: as many as the owner's own Files list shows
+# (`queries.MAX_LIST`; a test holds the two equal — `queries` imports this module).
+FOLDER_MAX = 500
 
 
 def new_token() -> str:
     return secrets.token_urlsafe(24)  # ~32 chars, 192 bits
 
 
+def seal(link: DownloadLink, token: str) -> None:
+    """Store `token` on `link` the only ways it is kept: its hash, and its encrypted copy."""
+    link.token_hash = security.hash_token(token)
+    link.token_enc = security.encrypt(token) if security.can_encrypt() else None
+
+
+def find(db: Session, token: str) -> DownloadLink | None:
+    """The link a `/watch` token names, by its hash."""
+    return db.execute(
+        select(DownloadLink).where(DownloadLink.token_hash == security.hash_token(token))
+    ).scalar_one_or_none()
+
+
+def recover(link: DownloadLink) -> str | None:
+    """The link's token from its encrypted copy, or None when there is none this key can read."""
+    return security.decrypt_optional(link.token_enc)
+
+
+def in_folder(folder: FileFolder) -> tuple:
+    """Which jobs a link to `folder` reaches: the done files in it NOW, the folder owner's. A file
+    moved out stops opening through the link at its next request; one moved in starts."""
+    return (
+        DownloadJob.folder_id == folder.id,
+        owned_by(DownloadJob.user_id, folder.user_id),
+        DownloadJob.status == JobStatus.DONE,
+    )
+
+
+def folder_files(db: Session, folder: FileFolder) -> list[tuple[DownloadJob, MediaAsset]]:
+    """What a link to `folder` shows — its page lists exactly these, and its link card counts them:
+    each file `in_folder` whose asset is ready and whose bytes are on disk (so the list never offers
+    a dead row), newest first, at most `FOLDER_MAX`."""
+    rows = db.execute(
+        select(DownloadJob, MediaAsset)
+        .join(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
+        .where(*in_folder(folder), MediaAsset.status == "ready")
+        .order_by(DownloadJob.ready_at.desc().nulls_last(), DownloadJob.id.desc())
+        .limit(FOLDER_MAX)
+    ).all()
+    return [
+        (job, asset)
+        for job, asset in rows
+        if storage.safe_abs_path(settings.files_root, asset.rel_path) is not None
+    ]
+
+
 def is_expired(link: DownloadLink) -> bool:
     return link.expires_at is not None and link.expires_at <= datetime.now(timezone.utc)
 
@@ -55,18 +119,43 @@ def check_grant(token: str, grant: str | None, password_version: int) -> bool:
         return False
     if exp < datetime.now(timezone.utc).timestamp():
         return False
-    return hmac.compare_digest(sig, _sign(f"{token}.{exp}.{password_version}"))
+    # As bytes: compare_digest refuses (TypeError, a 500) a str with anything non-ASCII in it, and
+    # `sig` is whatever the URL carried.
+    return hmac.compare_digest(sig.encode(), _sign(f"{token}.{exp}.{password_version}").encode())
 
 
 # --- serialization -------------------------------------------------------------------------
 
 
-def owner_view(link: DownloadLink, base_url: str = "") -> dict:
-    """What the file owner sees when managing their links."""
+def opens(db: Session, link: DownloadLink) -> str:
+    """What a link opens, in the words its address and its preview use: "folder", "video" or
+    "file". A file still being made reads as the video it mostly is."""
+    from app.downloads import mediakind
+
+    if link.folder_id is not None:
+        return "folder"
+    job = db.get(DownloadJob, link.job_id)
+    asset = db.get(MediaAsset, job.asset_id) if job and job.asset_id else None
+    kind = asset.media_kind if asset else None
+    return "file" if kind and kind != mediakind.VIDEO else "video"
+
+
+def share_path(db: Session, link: DownloadLink, token: str) -> str:
+    """The address a link is given out on: its path says what it opens (`opens`). Only a word for the
+    reader, the token is the whole key: every prefix of `main.SHARE_PREFIXES` opens every link, so
+    the links already sent on /watch/ work for good."""
+    prefix = {"folder": "folder", "file": "file", "video": "watch"}[opens(db, link)]
+    return f"/{prefix}/{token}"
+
+
+def owner_view(db: Session, link: DownloadLink, token: str | None = None) -> dict:
+    """What the file owner sees when managing their links. `token` is the one just made (it is in
+    hand even where no copy is kept); else it is read back, and `url` is None when it cannot be."""
+    token = token or recover(link)
     return {
         "id": link.id,
-        "token": link.token,
-        "url": f"{base_url}/watch/{link.token}",
+        "token": token,
+        "url": share_path(db, link, token) if token else None,
         "allow_download": link.allow_download,
         "has_password": link.password_hash is not None,
         "expires_at": link.expires_at.isoformat() if link.expires_at else None,
@@ -106,3 +195,117 @@ def public_meta(
         "file_url": file_url,
         "poster_url": poster_url,
     }
+
+
+# --- the owner's writes --------------------------------------------------------------------
+# Each flushes and records its audit row; the handler commits.
+
+
+def expiry(days: int | None) -> datetime | None:
+    """None/0/negative → no expiry; otherwise N days out, capped at ~10 years."""
+    if not days or days <= 0:
+        return None
+    return datetime.now(timezone.utc) + timedelta(days=min(days, 3650))
+
+
+def audit_view(link: DownloadLink) -> dict:
+    """A link's settings as the audit records them: whether it has a password, never which."""
+    return {
+        "allow_download": link.allow_download,
+        "has_password": link.password_hash is not None,
+        "expires_at": iso(link.expires_at),
+    }
+
+
+def create(
+    db: Session,
+    user: User,
+    *,
+    password: str,
+    allow_download: bool,
+    expires_days: int | None,
+    **target: int,
+) -> tuple[DownloadLink, str]:
+    """A link to `target` (`job_id=` or `folder_id=`, already checked to be the user's), and the
+    token it is opened by — which is in hand only now."""
+    token = new_token()
+    link = DownloadLink(
+        **target,
+        created_by=user.id,
+        allow_download=allow_download,
+        password_hash=security.hash_password(password) if password else None,
+        expires_at=expiry(expires_days),
+    )
+    seal(link, token)
+    db.add(link)
+    db.flush()
+    audit.record(
+        db,
+        user.id,
+        audit.AuditAction.DOWNLOAD_LINK_CREATE,
+        target_type="download_link",
+        target_id=link.id,
+        after={**target, **audit_view(link)},
+    )
+    return link, token
+
+
+def update(
+    db: Session,
+    user: User,
+    link: DownloadLink,
+    *,
+    allow_download: bool | None = None,
+    expires_days: int | None = None,
+    password: str | None = None,
+    sent: frozenset[str] = frozenset(),
+) -> None:
+    """Change what the caller sent (`sent` names the fields, so `expires_days=None` can mean "never
+    expires"). Setting, clearing or changing the password bumps its version, so any grant issued
+    under the previous one stops working now rather than living out its TTL."""
+    before = audit_view(link)
+    if "allow_download" in sent:
+        link.allow_download = bool(allow_download)
+    if "expires_days" in sent:
+        link.expires_at = expiry(expires_days)
+    if "password" in sent:
+        pw = (password or "").strip()
+        link.password_hash = security.hash_password(pw) if pw else None
+        link.password_version += 1
+    after = audit_view(link)
+    if after != before or "password" in sent:
+        audit.record(
+            db,
+            user.id,
+            audit.AuditAction.DOWNLOAD_LINK_UPDATE,
+            target_type="download_link",
+            target_id=link.id,
+            before=before,
+            after=after,
+        )
+    db.flush()
+
+
+def revoke(db: Session, user: User, link: DownloadLink) -> None:
+    link_id = link.id
+    db.delete(link)
+    audit.record(
+        db,
+        user.id,
+        audit.AuditAction.DOWNLOAD_LINK_REVOKE,
+        target_type="download_link",
+        target_id=link_id,
+        after={"job_id": link.job_id} if link.job_id is not None else {"folder_id": link.folder_id},
+    )
+    db.flush()
+
+
+def count_view(db: Session, link: DownloadLink) -> None:
+    """One more opening of the link. In SQL, not `+= 1` on the loaded row: two opens at once would
+    both write the same count."""
+    db.execute(
+        sql_update(DownloadLink)
+        .where(DownloadLink.id == link.id)
+        .values(view_count=DownloadLink.view_count + 1)
+        .execution_options(synchronize_session=False)
+    )

backend/app/downloads/models.py

@@ -0,0 +1,387 @@
+"""Capture's tables: the downloaded files (`MediaAsset`), each user's jobs over them and the folders
+they sit in, the shares and public links that hand them on, a user's quota, cookies and format
+profiles, and where they stopped playing one. Only this unit writes them; another unit reads them
+along a declared edge, or asks this unit's service."""
+
+from datetime import date, datetime, timezone
+
+from sqlalchemy import (
+    BigInteger,
+    Boolean,
+    CheckConstraint,
+    Date,
+    DateTime,
+    ForeignKey,
+    Index,
+    Integer,
+    JSON,
+    String,
+    Text,
+    UniqueConstraint,
+    func,
+    text,
+)
+from sqlalchemy.orm import Mapped, mapped_column, validates
+
+from app.db import Base
+from app.models import DISPLAY_TEXT_MAX, TimestampMixin, UpdatedAtMixin
+
+
+class LibraryState(Base):
+    """Where a user stopped in a library item (their own download, or one shared with them) — the
+    library player's resume point. Keyed by the JOB, not the file: a job is the user's library entry,
+    a re-download keeps it, and deleting the job deletes the row; the shared file under it may outlive
+    both. A revoked share leaves its recipient's row behind, unread — only a job the user can still
+    reach is ever read or written — and it is back if the item is shared again. Rows exist only for a
+    position worth keeping (see `routes._common.kept_position`); no row means "from the start"."""
+
+    __tablename__ = "library_states"
+    __table_args__ = (UniqueConstraint("user_id", "job_id", name="uq_library_user_job"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    job_id: Mapped[int] = mapped_column(
+        ForeignKey("download_jobs.id", ondelete="CASCADE"), index=True
+    )
+    # int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
+    position_seconds: Mapped[int] = mapped_column(Integer, nullable=False)
+    progress_updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+
+
+class MediaAsset(Base, TimestampMixin):
+    """A physical downloaded media file — the SHARED cache layer of the download center.
+
+    Mirrors the app's "shared catalog, private per-user state" model: one row per unique
+    ``(source_kind, source_ref, format_sig)`` is the global cache. When a second user requests
+    the same video with a format that hashes to the same `format_sig`, their `download_job`
+    links to this ready asset instantly — no re-download, no cost. Only the SOURCE download is
+    shared; per-user edited derivatives (phase 2) are never cached here.
+
+    `rel_path` is relative to `settings.files_root` (the mount), so the physical filename is
+    system-decided and bound to the source id — never renamed (the user's custom name is display
+    metadata on `download_job`, applied only when downloading to their own machine). The naming
+    fields (`title`/`uploader`/`upload_date`) are denormalized so the Plex-tree path can be built
+    uniformly for both catalog videos and ad-hoc URLs. `ref_count` (active jobs pointing here) frees
+    the file with its last holder, and `last_access_at` orders the GC's LRU eviction. How long a file
+    stays is not the asset's: each holder's job carries its own `expires_at` (its library folder's)."""
+
+    __tablename__ = "media_assets"
+    __table_args__ = (
+        UniqueConstraint(
+            "source_kind", "source_ref", "format_sig", name="uq_media_asset_cache_key"
+        ),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    source_kind: Mapped[str] = mapped_column(String(16))  # "youtube" | "url"
+    source_ref: Mapped[str] = mapped_column(String(512))  # video id, or full URL for ad-hoc
+    format_sig: Mapped[str] = mapped_column(String(64))  # hash of the output-affecting spec
+    status: Mapped[str] = mapped_column(
+        String(16), default="pending", server_default="pending", index=True
+    )  # "pending" | "ready" | "error"
+    rel_path: Mapped[str | None] = mapped_column(Text)  # relative to files_root
+    storyboard_path: Mapped[str | None] = mapped_column(Text)  # phase-2 filmstrip mosaic
+    # A locally-generated poster frame (ffmpeg), set only when the source had NO thumbnail so the
+    # card / watch page / link preview can still show an image. Relative to files_root.
+    poster_path: Mapped[str | None] = mapped_column(Text)
+    size_bytes: Mapped[int | None] = mapped_column(BigInteger)
+    duration_s: Mapped[int | None] = mapped_column(Integer)
+    width: Mapped[int | None] = mapped_column(Integer)
+    height: Mapped[int | None] = mapped_column(Integer)
+    container: Mapped[str | None] = mapped_column(String(16))
+    vcodec: Mapped[str | None] = mapped_column(String(32))
+    acodec: Mapped[str | None] = mapped_column(String(32))
+    # "video" | "audio" | "file" — the media-kind discriminator that drives which library actions
+    # apply (play/edit for media; download-only for a generic uploaded file). Null while an asset is
+    # still pending (kind is known once the file is probed at finalize). It is the single stored home
+    # of the invariant `downloads.mediakind.derive` used to compute — never re-derived in prose (an
+    # earlier ad-hoc `vcodec is None and acodec` check lived only in links.py; consolidated here).
+    media_kind: Mapped[str | None] = mapped_column(String(16))
+    nfo_written: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # Denormalized naming/metadata for the Plex tree + ad-hoc display.
+    title: Mapped[str | None] = mapped_column(Text)
+    uploader: Mapped[str | None] = mapped_column(String(255))
+    # yt-dlp's channel/uploader page URL (channel_url|uploader_url) — lets an auto-filled channel
+    # render as a clickable link. Null for sources without one (e.g. reddit) or older rows.
+    uploader_url: Mapped[str | None] = mapped_column(Text)
+    upload_date: Mapped[date | None] = mapped_column(Date)
+    thumbnail_url: Mapped[str | None] = mapped_column(Text)
+    # yt-dlp's canonical page URL for the source (webpage_url) — the clean "downloaded from"
+    # reference shown in the Files library. Null until the worker extracts it (older/queued rows
+    # fall back to a URL derived from source_kind+source_ref).
+    source_webpage_url: Mapped[str | None] = mapped_column(Text)
+    error: Mapped[str | None] = mapped_column(Text)
+    ref_count: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    last_access_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # The file's SHA-256 (hex), so a machine fetching it can check what it got. Hashed while an upload
+    # streams in, otherwise on the first ask (`routes.files`); NULL = not hashed yet.
+    sha256: Mapped[str | None] = mapped_column(String(64))
+
+    @validates("rel_path")
+    def _forget_hash(self, _key: str, rel_path: str | None) -> str | None:
+        # A hash names the bytes at `rel_path`: a new file there (a conversion, a retry) is not them.
+        self.sha256 = None
+        return rel_path
+
+
+class DownloadProfile(Base, TimestampMixin):
+    """A named download preset — a reusable format profile.
+
+    `user_id` NULL + `is_builtin` marks the shipped presets (Best MP4, 1080p, audio-only, …);
+    a user's saved custom profiles carry their `user_id`. `spec` is the format definition
+    (mode av/v/a, max_height, container, audio_format, codec prefs, embed subs/chapters/
+    thumbnail, sponsorblock) that `downloads.formats` turns into a yt-dlp selector + a cache
+    `format_sig`."""
+
+    __tablename__ = "download_profiles"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int | None] = mapped_column(
+        ForeignKey("users.id", ondelete="CASCADE"), index=True
+    )  # NULL for a builtin preset
+    name: Mapped[str] = mapped_column(String(80))
+    spec: Mapped[dict] = mapped_column(JSON)
+    is_builtin: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    sort_order: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+
+
+class FileFolder(Base, TimestampMixin):
+    """A folder of the Files library (`app.downloads.folders`): a row, never a directory.
+
+    A SOURCE folder (`source_key` set: youtube / plex / upload / site:<host>) is made the first time a
+    user gets a file from that origin; an OWN folder (`name` set) is one the user made; the HUB folder
+    (neither set, one per user) is where they swap files with a machine of theirs. Every file —
+    a `DownloadJob` — sits in exactly one (`DownloadJob.folder_id`). `retention_days` is how long a
+    file stays once done (NULL = until deleted); a file's own `expires_at` is stamped from it."""
+
+    __tablename__ = "file_folders"
+    __table_args__ = (
+        UniqueConstraint("user_id", "source_key", name="uq_file_folder_source"),
+        Index(
+            "uq_file_folder_own_name",
+            "user_id",
+            func.lower(text("name")),
+            unique=True,
+            postgresql_where=text("name IS NOT NULL"),
+        ),
+        # One Hub folder per user.
+        Index("uq_file_folder_hub", "user_id", unique=True, postgresql_where=text("kind = 'hub'")),
+        CheckConstraint(
+            "(kind = 'source' AND source_key IS NOT NULL AND name IS NULL)"
+            " OR (kind = 'own' AND name IS NOT NULL AND source_key IS NULL)"
+            " OR (kind = 'hub' AND name IS NULL AND source_key IS NULL)",
+            name="ck_file_folder_kind",
+        ),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+    kind: Mapped[str] = mapped_column(String(16))  # folders.FolderKind
+    # "site:" + a host name (DNS caps one at 253 characters).
+    source_key: Mapped[str | None] = mapped_column(String(260))
+    name: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
+    retention_days: Mapped[int | None] = mapped_column(Integer)
+
+
+def _ready_on_insert(context) -> datetime | None:
+    """A job inserted already done (an upload kept as it came) was ready the moment it was made; one
+    that becomes done later is stamped then (`folders.stamp_done`, `service.finalize_if_running`)."""
+    done = context.get_current_parameters().get("status") == "done"
+    return datetime.now(timezone.utc) if done else None
+
+
+class DownloadJob(Base, TimestampMixin, UpdatedAtMixin):
+    """A user's request to download one source — the per-user layer over `MediaAsset`.
+
+    The worker claims `queued` jobs with ``FOR UPDATE SKIP LOCKED`` and writes live
+    `progress`/`speed_bps`/`eta_s`/`phase` here (the frontend polls via useLiveQuery: there is
+    no server push channel, and the worker is a separate process anyway). `profile_snapshot`
+    freezes the spec at enqueue so later edits/deletion of the profile don't change this job.
+    `display_name` is the user's own name for the file (display + the Content-Disposition on
+    local download); the on-disk asset is never renamed. `asset_id` links to the shared file
+    (SET NULL if the asset is GC'd — the job then shows as expired).
+
+    Phase 2 (editor): `job_kind='edit'` jobs derive a new per-user clip from an already-downloaded
+    ready job. They still hang off a `MediaAsset` (with `source_kind='edit'`, a per-user cache key)
+    so all the file-serve/ref-count/GC/quota machinery is reused; the worker branches on the
+    asset's `source_kind` and runs ffmpeg instead of yt-dlp. `source_job_id`/`source_asset_id`
+    point at the parent download the clip was cut from; `edit_spec` is the (trim/crop) recipe."""
+
+    __tablename__ = "download_jobs"
+    __table_args__ = (
+        Index("ix_download_jobs_claim", "status", "queue_pos"),
+        # Every way a job becomes done stamps when (`folders.stamp_done`, `finalize_if_running`).
+        CheckConstraint(
+            "status <> 'done' OR ready_at IS NOT NULL", name="ck_download_jobs_ready_at"
+        ),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+    asset_id: Mapped[int | None] = mapped_column(
+        ForeignKey("media_assets.id", ondelete="SET NULL"), index=True
+    )
+    source_kind: Mapped[str] = mapped_column(String(16))  # "youtube" | "url"
+    source_ref: Mapped[str] = mapped_column(String(512))
+    # "download" (default), "edit" (a per-user trim/crop derivative of another job), "upload" (a
+    # user's own file, converted by the worker when it is not phone-safe — see `downloads.upload`) or
+    # "clip" (a range cut straight from a YouTube video or a Plex item — see `downloads.clip`).
+    job_kind: Mapped[str] = mapped_column(String(16), default="download", server_default="download")
+    # For edit jobs: the parent download this clip is cut from (SET NULL if the parent is deleted).
+    source_job_id: Mapped[int | None] = mapped_column(
+        ForeignKey("download_jobs.id", ondelete="SET NULL")
+    )
+    source_asset_id: Mapped[int | None] = mapped_column(
+        ForeignKey("media_assets.id", ondelete="SET NULL")
+    )
+    # For edit jobs: the trim/crop recipe {trim?:{start_s,end_s}, crop?:{x,y,w,h,frame_w?,frame_h?}}, its
+    # crop in the SHOWN picture's pixels when it carries the frame it was measured on (`edit.crop_filter`),
+    # else in the file's stored ones. For clip
+    # jobs: the clip spec {source, ref, start_s, end_s, audio_track?, crop?:{x,y,w,h,aspect}}, its crop
+    # in fractions of the box it was drawn in (`downloads.clip.normalize`).
+    edit_spec: Mapped[dict | None] = mapped_column(JSON)
+    profile_id: Mapped[int | None] = mapped_column(
+        ForeignKey("download_profiles.id", ondelete="SET NULL")
+    )
+    profile_snapshot: Mapped[dict] = mapped_column(JSON)
+    display_name: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
+    # Per-user display overrides + extras (like display_name — metadata only, never the file).
+    # `display_uploader`/`display_uploader_url` override the asset's auto channel name/link;
+    # `extra_links` is an optional list of extra reference URLs shown on the card + watch page.
+    display_uploader: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
+    display_uploader_url: Mapped[str | None] = mapped_column(Text)
+    extra_links: Mapped[list | None] = mapped_column(JSON)
+    status: Mapped[str] = mapped_column(
+        String(16), default="queued", server_default="queued", index=True
+    )  # queued | running | paused | done | error | canceled
+    progress: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    speed_bps: Mapped[int | None] = mapped_column(BigInteger)
+    eta_s: Mapped[int | None] = mapped_column(Integer)
+    phase: Mapped[str | None] = mapped_column(String(32))
+    error: Mapped[str | None] = mapped_column(Text)  # raw yt-dlp/ffmpeg text (technical detail)
+    error_code: Mapped[str | None] = mapped_column(
+        String(24)
+    )  # category, see downloads.errors.classify
+    # The host-specific failure-catalog entry on top of the category (`facebook.login_wall`), or NULL.
+    error_detail: Mapped[str | None] = mapped_column(String(48))  # see downloads.errors.diagnose
+    queue_pos: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    # The Files library: the folder this file sits in (`downloads.folders.place` at creation; an own
+    # folder is emptied back into the source folders before it is deleted, hence RESTRICT), whether
+    # the user keeps it past its folder's retention, and when it goes (stamped once done; NULL =
+    # never). `expiry_warned` marks the one warning the GC sends before it deletes the file.
+    folder_id: Mapped[int | None] = mapped_column(
+        ForeignKey("file_folders.id", ondelete="RESTRICT"), index=True
+    )
+    # The origin the file came from (the source folder's key), recorded where it is placed: an edit
+    # traces its origin through its parent, and that parent may be deleted or expire later — the
+    # file must still know which source folder it may go back to. NULL only for a file placed before
+    # the column existed outside a source folder; `folders.source_key_for` works those out.
+    source_key: Mapped[str | None] = mapped_column(String(260))
+    kept: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
+    expiry_warned: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # When the file became done — what the library's "Added" means: a download that failed for days
+    # and then went through is new to its user, so it sorts as new. `created_at` stays when it was
+    # asked for (the queue's order). Set again when a file is made anew (a retry, a re-download).
+    ready_at: Mapped[datetime | None] = mapped_column(
+        DateTime(timezone=True), default=_ready_on_insert
+    )
+
+
+class DownloadShare(Base, TimestampMixin):
+    """A user→user grant: `to_user` may access `from_user`'s downloaded file (private by
+    default). Since assets are already deduped/shared on disk, a share is just an ACL row."""
+
+    __tablename__ = "download_shares"
+    __table_args__ = (UniqueConstraint("job_id", "to_user_id", name="uq_download_share"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    job_id: Mapped[int] = mapped_column(ForeignKey("download_jobs.id", ondelete="CASCADE"))
+    from_user_id: Mapped[int] = mapped_column(
+        ForeignKey("users.id", ondelete="CASCADE"), index=True
+    )
+    to_user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+
+
+class DownloadQuota(Base, UpdatedAtMixin):
+    """Per-user download limits (admin-managed). A row exists only when an admin customizes a
+    user; otherwise the sysconfig `download_default_*` values apply. `unlimited` bypasses the
+    byte cap entirely (e.g. for the admin's own account)."""
+
+    __tablename__ = "download_quotas"
+
+    user_id: Mapped[int] = mapped_column(
+        ForeignKey("users.id", ondelete="CASCADE"), primary_key=True
+    )
+    max_bytes: Mapped[int] = mapped_column(BigInteger)
+    max_concurrent: Mapped[int] = mapped_column(Integer)
+    max_jobs: Mapped[int] = mapped_column(Integer)
+    unlimited: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+
+
+class DownloadLink(Base, TimestampMixin):
+    """A public capability link to one download, or to one of its owner's own folders — the "share
+    by link" surface.
+
+    Anyone holding the unguessable token can watch the file on the login-free `/watch/{token}`
+    player page (Google-Drive style); a folder link opens the folder's list there, with what is in
+    the folder at the time it is opened. The token itself is never stored as written: a link is
+    found by its SHA-256 (`token_hash`), and `token_enc` is the Fernet-encrypted copy its owner
+    copies the link from again — absent without a key (`downloads/links.py`). Optional per-link
+    controls: `expires_at`, `allow_download` (stream-only vs downloadable), and a `password_hash`
+    (argon2, like a user password). Revoke = delete the row. Distinct from `DownloadShare`, which
+    grants a *registered* user access via the in-app "Shared with me" list."""
+
+    __tablename__ = "download_links"
+    __table_args__ = (
+        # A link opens exactly one thing: a file or a folder.
+        CheckConstraint("(job_id IS NULL) <> (folder_id IS NULL)", name="ck_download_link_target"),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    job_id: Mapped[int | None] = mapped_column(
+        ForeignKey("download_jobs.id", ondelete="CASCADE"), index=True
+    )
+    folder_id: Mapped[int | None] = mapped_column(
+        ForeignKey("file_folders.id", ondelete="CASCADE"), index=True
+    )
+    token_hash: Mapped[str] = mapped_column(String(64), unique=True)
+    token_enc: Mapped[str | None] = mapped_column(Text)  # Fernet ciphertext — never returned/logged
+    created_by: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    allow_download: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    password_hash: Mapped[str | None] = mapped_column(Text)
+    # Bumped whenever the password changes; folded into the signed watch grant so rotating the
+    # password immediately invalidates any grants issued under the old one (instead of them living
+    # out the 6h grant TTL).
+    password_version: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    view_count: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+
+
+class DownloadCredential(Base, TimestampMixin, UpdatedAtMixin):
+    """A user's stored login cookies for ONE auth-walled download host (Facebook, Instagram, …).
+
+    Per-user, per-host: each user pastes their OWN browser cookies.txt (Netscape format) for a host
+    where they're logged in (e.g. a closed Facebook group they belong to), and a download of a
+    yt-dlp-supported, auth-walled link on that host is retried with the ACTING user's cookie. The
+    blob is Fernet-encrypted at rest (`cookies_enc`) and is NEVER returned by any API/log/audit —
+    the routes are write-only (host + updated_at only). See `downloads.credentials`.
+
+    Two load-bearing security controls live around this row, NOT on it:
+      * **cache privacy** — a cookie-authed download SALTS its MediaAsset `format_sig` with the
+        user id, so the authed asset is private to that user and can never be cache-served to
+        another user (`downloads.credentials.cookie_salt`).
+      * **host scope (SEC-12)** — a cookie is attached ONLY when the download URL's netloc matches
+        this row's `host_key` (`downloads.credentials.host_key_for_url`), so an arbitrary `?url=`
+        can't exfiltrate the user's session to a look-alike host.
+
+    `host_key` is a short canonical key from the supported-host registry (e.g. "facebook"), not the
+    raw netloc — one credential can cover several domains of the same site (facebook.com/fb.watch)."""
+
+    __tablename__ = "download_credentials"
+    __table_args__ = (UniqueConstraint("user_id", "host_key", name="uq_download_credential"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+    host_key: Mapped[str] = mapped_column(String(64))
+    cookies_enc: Mapped[str] = mapped_column(Text)  # Fernet ciphertext — never returned/logged

backend/app/downloads/og.py

@@ -1,4 +1,4 @@
-"""Server-rendered Open Graph tags for the public /watch/{token} page.
+"""Server-rendered Open Graph tags for the public page of a share link (/watch/, /file/, /folder/).
 
 The watch page is a client-side SPA, so a social crawler (facebookexternalhit, Twitterbot, …) that
 fetches /watch/{token} would otherwise only see the generic index.html — a blank/greyed link card.
@@ -11,13 +11,12 @@ import html
 import re
 from pathlib import Path
 
-from sqlalchemy import select
 from sqlalchemy.orm import Session
 
 from app.config import settings
 from app.downloads import links as linksmod
 from app.downloads import storage
-from app.models import DownloadJob, DownloadLink, MediaAsset
+from app.downloads.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
 
 _OG_BLOCK = re.compile(r"<!--OG:START-->.*?<!--OG:END-->", re.DOTALL)
 
@@ -74,27 +73,35 @@ def _jpeg_size(path: Path) -> tuple[int, int] | None:
 
 def _watch_tags(token: str, db: Session) -> str | None:
     """Build the OG/Twitter meta block for a watch token, or None to keep the generic card."""
-    link = db.execute(select(DownloadLink).where(DownloadLink.token == token)).scalar_one_or_none()
+    link = linksmod.find(db, token)
     if link is None or linksmod.is_expired(link):
         return None
-    url = f"{settings.app_base}/watch/{token}"
+    url = f"{settings.app_base}{linksmod.share_path(db, link, token)}"
+    # What the card calls it, as its address does.
+    kind = linksmod.opens(db, link)
+    folder, video = kind == "folder", kind == "video"
     if link.password_hash:
         # Password-gated: don't leak the title/thumbnail in the unfurl.
         return "\n    ".join(
             [
                 "<title>Siftlode</title>",
-                _tag("og:title", "Password-protected video"),
+                _tag(
+                    "og:title",
+                    "Password-protected " + ("folder" if folder else "video" if video else "file"),
+                ),
                 _tag("og:description", "Shared via Siftlode"),
-                _tag("og:type", "video.other"),
+                _tag("og:type", "video.other" if video else "website"),
                 _tag("og:url", url),
                 _tag("og:site_name", "Siftlode"),
             ]
         )
+    if folder:
+        return _folder_tags(db, link, url)
     job = db.get(DownloadJob, link.job_id)
     asset = db.get(MediaAsset, job.asset_id) if job and job.asset_id else None
     if job is None or asset is None or asset.status != "ready":
         return None
-    title = (job.display_name or asset.title or "Video").strip()
+    title = (job.display_name or asset.title or ("Video" if video else "File")).strip()
     channel = (job.display_uploader or asset.uploader or "").strip()
     # Pick the link-preview image. Facebook's scraper REJECTS our own poster ("Corrupted Image" in the
     # Sharing Debugger): the poster is an ffmpeg-produced JPEG whose non-standard marker order FB's
@@ -116,7 +123,7 @@ def _watch_tags(token: str, db: Session) -> str | None:
         f"<title>{html.escape(title)}</title>",
         _tag("og:title", title),
         _tag("og:description", channel or "Shared via Siftlode"),
-        _tag("og:type", "video.other"),
+        _tag("og:type", "video.other" if video else "website"),
         _tag("og:url", url),
         _tag("og:site_name", "Siftlode"),
         _tag("twitter:title", title, attr="name"),
@@ -147,6 +154,30 @@ def _watch_tags(token: str, db: Session) -> str | None:
     return "\n    ".join(tags)
 
 
+def _folder_tags(db: Session, link: DownloadLink, url: str) -> str | None:
+    """A folder link's card: the folder's name and how many files it holds — never their names,
+    which the page shows only to whoever opens it."""
+    folder = db.get(FileFolder, link.folder_id)
+    if folder is None or not folder.name:
+        return None
+    count = len(linksmod.folder_files(db, folder))  # the page's own list, so the two agree
+    title = folder.name.strip()
+    about = f"{count} file{'' if count == 1 else 's'} · Shared via Siftlode"
+    return "\n    ".join(
+        [
+            f"<title>{html.escape(title)}</title>",
+            _tag("og:title", title),
+            _tag("og:description", about),
+            _tag("og:type", "website"),
+            _tag("og:url", url),
+            _tag("og:site_name", "Siftlode"),
+            _tag("twitter:title", title, attr="name"),
+            _tag("twitter:description", about, attr="name"),
+            _tag("twitter:card", "summary", attr="name"),
+        ]
+    )
+
+
 def render_watch_html(token: str, page_html: Path, db: Session) -> str | None:
     """Return the watch page (watch.html) with the OG block replaced for this token, or None to
     serve the generic page unchanged (invalid/expired/not-ready link, or a template without the

backend/app/downloads/pipeline.py

@@ -7,6 +7,7 @@ from datetime import date, datetime, timezone
 from pathlib import Path
 
 import yt_dlp
+from sqlalchemy import update
 from yt_dlp.utils import download_range_func
 
 from app import logs, sysconfig
@@ -16,6 +17,7 @@ from app.downloads import clip as clipmod
 from app.downloads import edit as editmod
 from app.downloads import errors as dlerrors
 from app.downloads import adapters, credentials, formats, mediakind, playable, probe, quota
+from app.downloads import backfill, egress
 from app.downloads import service, storage
 from app.downloads import urlguard
 from app.downloads.jobdb import (
@@ -28,7 +30,9 @@ from app.downloads.jobdb import (
 )
 from app.downloads.stated import StatedFailure
 from app.downloads.status import JobStatus
-from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset, PlexItem
+from app.models import DISPLAY_TEXT_MAX
+from app.plex.models import PlexItem
+from app.downloads.models import DownloadJob, MediaAsset
 from app.plex import lookups as plex_lookups
 from app.plex import paths as plex_paths
 from app.notifications import create_notification
@@ -390,6 +394,7 @@ def _extract_with_fallback(
             _make_progress_hook(job_id, asset_id),
             cookiefile=cookiefile,
             max_bytes=max_bytes,
+            egress=adapter.egress_guarded,
         ),
     )
     last_exc: Exception | None = None
@@ -401,6 +406,7 @@ def _extract_with_fallback(
             extra=adapter.ydl_opts_extra(clients),
             cookiefile=cookiefile,
             max_bytes=max_bytes,
+            egress=adapter.egress_guarded,
         )
         opts["postprocessor_hooks"] = [_make_pp_hook(job_id)]
         if section is not None:
@@ -411,7 +417,7 @@ def _extract_with_fallback(
         # (Per-ATTEMPT only; a single extract_info that itself runs past the stall timeout is still
         # killed — acceptable, as socket_timeout + extractor_retries bound one attempt well under it.)
         try:
-            with yt_dlp.YoutubeDL(opts) as ydl:
+            with egress.open_ydl(opts) as ydl:
                 return ydl.extract_info(target, download=True)
         except yt_dlp.utils.DownloadError as exc:
             last_exc = exc
@@ -583,9 +589,14 @@ def _download(
         # missing duration hid the trim action. Probed before the sidecars so the .nfo runtime has it
         # too. Same probe the upload ingest uses.
         vcodec, acodec, width, height, duration_s = probe.resolve_media_facts(produced, info)
+        # A placeholder title with nothing behind it ("Facebook" from a signed-in read) borrows the
+        # page's text from one signed-out read; the download itself stays the signed-in one. The page
+        # yt-dlp read, not the link queued: a group post's link is rewritten to its video, and
+        # signed out the post itself is the login wall.
+        titled = backfill.public_title_info(info, info.get("webpage_url") or url, adapter, asset_id)
         meta = storage.MediaMeta(
             video_id=info.get("id") or source_ref,
-            title=normalize_title(adapter.title(info)) or source_ref,
+            title=normalize_title(adapter.title(titled)) or source_ref,
             uploader=info.get("uploader") or info.get("channel") or "Unknown Channel",
             upload_date=_parse_upload_date(info.get("upload_date")),
             uniqueid_type=adapter.nfo_uniqueid_type(),
@@ -828,6 +839,18 @@ def _fail_edit(
     )
 
 
+def _filter_frame(
+    path: Path, width: int | None, height: int | None
+) -> tuple[int | None, int | None]:
+    """A file's frame as ffmpeg's filters see it: the stored size, turned when the stream is stored
+    turned a quarter (ffmpeg stands it upright first). Unknown when the file cannot be read."""
+    data = probe.probe_json(path)
+    turned = clipmod.quarter_turned(data) if data else None
+    if turned is None:
+        return None, None
+    return (height, width) if turned else (width, height)
+
+
 def _process_edit(job_id: int, asset_id: int) -> None:
     """Produce a per-user trim/crop clip from the source asset's downloaded file via ffmpeg."""
     with SessionLocal() as db:
@@ -896,6 +919,12 @@ def _process_edit(job_id: int, asset_id: int) -> None:
         size = final.stat().st_size if final.exists() else None
         # The clip's real frame: a re-encode evens an odd crop or source (`editmod.EVEN_FRAME`).
         _, _, out_w, out_h, _ = probe.probe_media(final)
+        # Unsized, a crop's own size stands in, cut from the frame the filters saw.
+        cut = (
+            editmod.crop_stored_size(edit_spec["crop"], *_filter_frame(src_path, src_w, src_h))
+            if crop and not (out_w and out_h)
+            else None
+        )
 
         with SessionLocal() as db:
             asset = db.get(MediaAsset, asset_id)
@@ -907,11 +936,10 @@ def _process_edit(job_id: int, asset_id: int) -> None:
                 asset.duration_s = total_s or asset.duration_s
                 if out_w and out_h:
                     asset.width, asset.height = out_w, out_h
-                elif crop:
-                    asset.width = edit_spec["crop"]["w"]
-                    asset.height = edit_spec["crop"]["h"]
+                elif cut:
+                    asset.width, asset.height = cut
                 else:
-                    asset.width, asset.height = src_w, src_h
+                    asset.width, asset.height = (None, None) if crop else (src_w, src_h)
                 asset.error = None
                 asset.last_access_at = datetime.now(timezone.utc)
             job = db.get(DownloadJob, job_id)
@@ -1114,14 +1142,20 @@ def _process_clip(job_id: int, asset_id: int) -> None:
             fields: dict = {"status": JobStatus.DONE, "progress": 100, "phase": None}
             # A video outside the catalog was queued as "Clip (clip)": the fetch has its title now.
             placeholder = service.clip_title(None, None)
+            named = None
             if asset and fetched_title and asset.title == placeholder:
                 named = service.clip_title(fetched_title, None)
                 asset.title = named
-                if job_row := db.get(DownloadJob, job_id):
-                    if job_row.display_name == placeholder:
-                        fields["display_name"] = named[:DISPLAY_TEXT_MAX]
             title = asset.title if asset else None
             finalized = service.finalize_if_running(db, job_id, **fields)
+            if finalized and named:
+                # Only a name that still repeats the placeholder, in one conditional write: one its
+                # owner gave meanwhile stays.
+                db.execute(
+                    update(DownloadJob)
+                    .where(DownloadJob.id == job_id, DownloadJob.display_name == placeholder)
+                    .values(display_name=named[:DISPLAY_TEXT_MAX])
+                )
             db.commit()
             if finalized:
                 _notify_download_done(db, user_id, title, job_id, kind="edit")

backend/app/downloads/queries.py

@@ -0,0 +1,404 @@
+"""Capture's reads: what the library, the download tray, the share dialogs and the public watch pages
+show. Each returns rows or plain values and writes nothing; whose a row is goes through
+`userscope.owned_by` / `owned_row`, so a row of someone else's reads as a missing one.
+
+The HTTP layer (`routes.downloads`, `routes.files`, `routes.public`) asks here and serializes; the
+writes are the service's (`service`, `folders`, `links`, `quota`)."""
+
+from collections.abc import Sequence
+from datetime import datetime, timezone
+
+from sqlalchemy import func, or_, select
+from sqlalchemy.orm import Session
+
+from app.apitokens import FilesCaller, Right
+from app.collation import collated
+from app.downloads import folders, links, quota
+from app.downloads.models import (
+    DownloadJob,
+    DownloadLink,
+    DownloadProfile,
+    DownloadQuota,
+    DownloadShare,
+    FileFolder,
+    LibraryState,
+    MediaAsset,
+)
+from app.downloads.status import FINISHED, JobStatus
+from app.models import User
+from app.userscope import owned_by, owned_row
+from app.utils import like_escape
+
+# Cap a listing so a heavy library can't return an unbounded result set (the old Download Center's
+# own cap; far beyond any real library).
+MAX_LIST = 500
+
+
+# --- jobs and their files ------------------------------------------------------------------
+
+
+def own_job(db: Session, user: User, job_id: int) -> DownloadJob | None:
+    return owned_row(db, DownloadJob, job_id, user)
+
+
+def job(db: Session, job_id: int) -> DownloadJob | None:
+    """A job by id, whoever's — for a public link, which already names its job."""
+    return db.get(DownloadJob, job_id)
+
+
+def accessible_job(db: Session, user: User, job_id: int) -> DownloadJob | None:
+    """A file the caller may read: their own, or one shared with them."""
+    row = db.get(DownloadJob, job_id)
+    if row is None:
+        return None
+    if row.user_id == user.id or shared_with(db, job_id, user) is not None:
+        return row
+    return None
+
+
+def readable_job(db: Session, caller: FilesCaller, job_id: int) -> DownloadJob | None:
+    """A file `caller` may read: as `accessible_job` for the user; for a token, one of the user's
+    own in a folder it may read."""
+    if caller.grants is None:
+        return accessible_job(db, caller.user, job_id)
+    row = own_job(db, caller.user, job_id)
+    if row is None or not caller.may(row.folder_id, Right.READ):
+        return None
+    return row
+
+
+def asset_of(db: Session, job: DownloadJob | None) -> MediaAsset | None:
+    """The file under a job, if it has one yet."""
+    return db.get(MediaAsset, job.asset_id) if job is not None and job.asset_id else None
+
+
+def assets_for(db: Session, jobs: Sequence[DownloadJob]) -> dict[int, MediaAsset]:
+    ids = {j.asset_id for j in jobs if j.asset_id}
+    if not ids:
+        return {}
+    rows = db.execute(select(MediaAsset).where(MediaAsset.id.in_(ids))).scalars()
+    return {a.id: a for a in rows}
+
+
+def positions_for(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, int]:
+    """`user`'s resume point in each of `jobs` that has one (the library player's `library_states`)."""
+    ids = [j.id for j in jobs]
+    if not ids:
+        return {}
+    rows = db.execute(
+        select(LibraryState.job_id, LibraryState.position_seconds).where(
+            owned_by(LibraryState.user_id, user), LibraryState.job_id.in_(ids)
+        )
+    ).all()
+    return {job_id: pos for job_id, pos in rows}
+
+
+def source_keys(db: Session, jobs: Sequence[DownloadJob]) -> dict[int, str]:
+    """Where each file came from (its source folder's key), for the list's Source column, the
+    "back to its source folder" move and the download tray's site: a file in a source folder has its
+    folder's key; one moved into an own folder (or someone else's, shared) is traced back by
+    `folders.source_key_for`."""
+    ids = {j.folder_id for j in jobs if j.folder_id is not None}
+    by_id = (
+        {f.id: f for f in db.execute(select(FileFolder).where(FileFolder.id.in_(ids))).scalars()}
+        if ids
+        else {}
+    )
+    out: dict[int, str] = {}
+    for j in jobs:
+        folder = by_id.get(j.folder_id) if j.folder_id is not None else None
+        if folder is not None and folder.kind == folders.FolderKind.SOURCE and folder.source_key:
+            out[j.id] = folder.source_key
+        else:
+            out[j.id] = folders.source_key_for(db, j)
+    return out
+
+
+def queue(db: Session, user: User, recent: datetime) -> list[DownloadJob]:
+    """The download tray: what is waiting, running, paused or failed, and what finished (done or
+    canceled) since `recent`. A done job is timed from when it became done, never from its last
+    write: a rename, a move or the GC's expiry warning touches an old file's row, and the tray would
+    announce it as just finished."""
+    return list(
+        db.execute(
+            select(DownloadJob)
+            .where(
+                owned_by(DownloadJob.user_id, user),
+                DownloadJob.status.not_in(FINISHED)
+                | ((DownloadJob.status == JobStatus.DONE) & (DownloadJob.ready_at >= recent))
+                | ((DownloadJob.status == JobStatus.CANCELED) & (DownloadJob.updated_at >= recent)),
+            )
+            .order_by(DownloadJob.created_at.desc(), DownloadJob.id.desc())
+            .limit(MAX_LIST)
+        ).scalars()
+    )
+
+
+def statuses_by_source(db: Session, user: User, statuses: Sequence[str]) -> list[tuple[str, str]]:
+    """(source_ref, status) of each of `user`'s jobs in one of `statuses`."""
+    return list(
+        db.execute(
+            select(DownloadJob.source_ref, DownloadJob.status).where(
+                owned_by(DownloadJob.user_id, user), DownloadJob.status.in_(tuple(statuses))
+            )
+        ).tuples()
+    )
+
+
+def library(
+    db: Session,
+    user: User,
+    *,
+    folder_id: int | None,
+    listable: set[int] | None,
+    shared: bool,
+    term: str,
+    sort: str,
+    order: str,
+    lang: str,
+) -> list[DownloadJob]:
+    """The caller's done files (optionally one folder's, or only in the folders a token may list),
+    or the ones shared with them; searched by name and channel, sorted by when they were added, by
+    name (in the reader's alphabet) or by size. A file is added to its owner's list when it is done,
+    not when it was asked for (a download that failed for days and then went through is new); one
+    shared with the caller, when it was shared."""
+    stmt = (
+        select(DownloadJob)
+        .join(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
+        .where(DownloadJob.status == JobStatus.DONE)
+    )
+    if shared:
+        stmt = stmt.join(DownloadShare, DownloadShare.job_id == DownloadJob.id).where(
+            owned_by(DownloadShare.to_user_id, user)
+        )
+    else:
+        stmt = stmt.where(owned_by(DownloadJob.user_id, user))
+        if folder_id is not None:
+            stmt = stmt.where(DownloadJob.folder_id == folder_id)
+        elif listable is not None:
+            stmt = stmt.where(DownloadJob.folder_id.in_(listable))
+    name = func.coalesce(DownloadJob.display_name, MediaAsset.title, "")
+    if term:
+        like = f"%{like_escape(term)}%"
+        stmt = stmt.where(
+            or_(
+                name.ilike(like),
+                func.coalesce(DownloadJob.display_uploader, MediaAsset.uploader, "").ilike(like),
+            )
+        )
+    key = {
+        "added": DownloadShare.created_at if shared else DownloadJob.ready_at,
+        "name": collated(name, lang),
+        "size": MediaAsset.size_bytes,
+    }[sort]
+    first = key.asc() if order == "asc" else key.desc()
+    tie = DownloadJob.id.asc() if order == "asc" else DownloadJob.id.desc()
+    return list(db.execute(stmt.order_by(first.nulls_last(), tie).limit(MAX_LIST)).scalars())
+
+
+def recent_jobs(db: Session) -> list[DownloadJob]:
+    """Everyone's newest jobs, for the admin's list."""
+    return list(
+        db.execute(
+            select(DownloadJob)
+            .order_by(DownloadJob.created_at.desc(), DownloadJob.id.desc())
+            .limit(MAX_LIST)
+        ).scalars()
+    )
+
+
+# --- shares --------------------------------------------------------------------------------
+
+
+def shared_with(db: Session, job_id: int, user: User | int) -> DownloadShare | None:
+    """The grant that hands this job to `user`, if there is one."""
+    return db.execute(
+        select(DownloadShare).where(
+            DownloadShare.job_id == job_id, owned_by(DownloadShare.to_user_id, user)
+        )
+    ).scalar_one_or_none()
+
+
+def shared_at(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, datetime]:
+    rows = db.execute(
+        select(DownloadShare.job_id, DownloadShare.created_at).where(
+            owned_by(DownloadShare.to_user_id, user),
+            DownloadShare.job_id.in_([j.id for j in jobs]),
+        )
+    ).all()
+    return {job_id: at for job_id, at in rows}
+
+
+# --- folders -------------------------------------------------------------------------------
+
+
+def own_folder(db: Session, user: User, folder_id: int) -> FileFolder | None:
+    return owned_row(db, FileFolder, folder_id, user)
+
+
+def folders_by_id(db: Session, ids: set[int]) -> dict[int, FileFolder]:
+    """These folders, by id — for a reader that already knows which ones (a token's grants)."""
+    if not ids:
+        return {}
+    return {f.id: f for f in db.execute(select(FileFolder).where(FileFolder.id.in_(ids))).scalars()}
+
+
+def folders_with_totals(
+    db: Session, user: User, listable: set[int] | None
+) -> list[tuple[FileFolder, int, int]]:
+    """The user's folders (only `listable` ones, when given) with how many done files each holds and
+    their size. A source folder its files have all left stays (count 0)."""
+    done = (DownloadJob.folder_id == FileFolder.id) & (DownloadJob.status == JobStatus.DONE)
+    stmt = select(
+        FileFolder,
+        func.count(DownloadJob.id),
+        func.coalesce(func.sum(MediaAsset.size_bytes), 0),
+    ).where(owned_by(FileFolder.user_id, user))
+    if listable is not None:
+        stmt = stmt.where(FileFolder.id.in_(listable))
+    rows = db.execute(
+        stmt.outerjoin(DownloadJob, done)
+        .outerjoin(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
+        .group_by(FileFolder.id)
+        .order_by(FileFolder.kind.desc(), FileFolder.source_key, FileFolder.name, FileFolder.id)
+    ).tuples()
+    return [(f, count, int(size or 0)) for f, count, size in rows]
+
+
+def folder_totals(db: Session, folder_id: int) -> tuple[int, int]:
+    count, size = db.execute(
+        select(func.count(DownloadJob.id), func.coalesce(func.sum(MediaAsset.size_bytes), 0))
+        .select_from(DownloadJob)
+        .outerjoin(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
+        .where(DownloadJob.folder_id == folder_id, DownloadJob.status == JobStatus.DONE)
+    ).one()
+    return count, int(size)
+
+
+def linked_folders(db: Session, folder_ids: list[int]) -> set[int]:
+    """Which of these folders a live public link opens (one not past its expiry)."""
+    now = datetime.now(timezone.utc)
+    rows = db.execute(
+        select(DownloadLink.folder_id).where(
+            DownloadLink.folder_id.in_(folder_ids),
+            or_(DownloadLink.expires_at.is_(None), DownloadLink.expires_at > now),
+        )
+    ).scalars()
+    return {folder_id for folder_id in rows if folder_id is not None}
+
+
+# --- links ---------------------------------------------------------------------------------
+
+
+def own_link(db: Session, user: User, link_id: int) -> DownloadLink | None:
+    """A link to one of the user's files or folders."""
+    link = db.get(DownloadLink, link_id)
+    if link is None:
+        return None
+    target = (
+        owned_row(db, DownloadJob, link.job_id, user)
+        if link.job_id is not None
+        else owned_row(db, FileFolder, link.folder_id, user)
+    )
+    return link if target is not None else None
+
+
+def links_to(
+    db: Session, *, job_id: int | None = None, folder_id: int | None = None
+) -> list[DownloadLink]:
+    """The links to one file or one folder, newest first."""
+    target = (
+        DownloadLink.job_id == job_id if job_id is not None else DownloadLink.folder_id == folder_id
+    )
+    return list(
+        db.execute(select(DownloadLink).where(target).order_by(DownloadLink.id.desc())).scalars()
+    )
+
+
+def link_folder(db: Session, link: DownloadLink) -> FileFolder | None:
+    return db.get(FileFolder, link.folder_id) if link.folder_id is not None else None
+
+
+def folder_job(db: Session, folder: FileFolder, job_id: int) -> DownloadJob | None:
+    """One of the files a link to `folder` reaches (`links.in_folder`), while it is in it."""
+    return db.execute(
+        select(DownloadJob).where(DownloadJob.id == job_id, *links.in_folder(folder))
+    ).scalar_one_or_none()
+
+
+def owner_suspended(db: Session, job: DownloadJob) -> bool:
+    """The job's owner is suspended (or gone): the worker leaves their queued jobs where they are."""
+    owner = db.get(User, job.user_id)
+    return owner is None or owner.is_suspended
+
+
+# --- profiles ------------------------------------------------------------------------------
+
+
+def profiles_for(db: Session, user: User) -> list[DownloadProfile]:
+    """The built-in format profiles, then the user's own."""
+    return list(
+        db.execute(
+            select(DownloadProfile)
+            .where(DownloadProfile.is_builtin.is_(True) | owned_by(DownloadProfile.user_id, user))
+            .order_by(
+                DownloadProfile.is_builtin.desc(), DownloadProfile.sort_order, DownloadProfile.id
+            )
+        ).scalars()
+    )
+
+
+def own_profile(db: Session, user: User, profile_id: int) -> DownloadProfile | None:
+    """One of the user's own profiles (a built-in is no one's to change)."""
+    p = owned_row(db, DownloadProfile, profile_id, user)
+    return None if p is None or p.is_builtin else p
+
+
+def usable_profile(db: Session, user: User, profile_id: int) -> DownloadProfile | None:
+    """A profile the user may enqueue with: a built-in, or their own."""
+    p = db.get(DownloadProfile, profile_id)
+    if p is None or (not p.is_builtin and p.user_id != user.id):
+        return None
+    return p
+
+
+def default_profile(db: Session) -> DownloadProfile | None:
+    """The first built-in, for an enqueue that named no format."""
+    return db.execute(
+        select(DownloadProfile)
+        .where(DownloadProfile.is_builtin.is_(True))
+        .order_by(DownloadProfile.sort_order)
+        .limit(1)
+    ).scalar_one_or_none()
+
+
+# --- storage and quota (admin) -------------------------------------------------------------
+
+
+def storage_totals(db: Session) -> tuple[int, int, int]:
+    """(files on disk, their bytes, every asset row). On disk is `quota.ON_DISK`: ready files, and
+    uploads waiting for or failed in their conversion — the same disk the footprint and the GC cap
+    count."""
+    on_disk = db.execute(
+        select(func.count(), func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(quota.ON_DISK)
+    ).one()
+    total_assets = db.execute(select(func.count()).select_from(MediaAsset)).scalar_one()
+    return on_disk[0], int(on_disk[1]), total_assets
+
+
+def done_files_per_user(db: Session) -> dict[int, int]:
+    """How many finished files each user's Files lists."""
+    return dict(
+        db.execute(
+            select(DownloadJob.user_id, func.count())
+            .where(DownloadJob.status == JobStatus.DONE)
+            .group_by(DownloadJob.user_id)
+        )
+        .tuples()
+        .all()
+    )
+
+
+def custom_quota(db: Session, user_id: int) -> DownloadQuota | None:
+    """The admin's own limits for this user, if any were set."""
+    return db.get(DownloadQuota, user_id)

backend/app/downloads/quota.py

@@ -16,17 +16,20 @@ That is a per-job ceiling, not an aggregate ledger: concurrent jobs of one user
 remainder (in-flight bytes are not `ready`, so `footprint` cannot see them), so a wave of
 max_concurrent downloads can still land over the cap — the deliberate trade, since a download's size
 is unknown until it finishes. CLIP jobs consult it before their fetch or encode and again with
-their output's real size before it is placed. Edit jobs do not consult it at all (tracked as SEC-19).
+their output's real size before it is placed. Edit jobs consult it with their output's real size
+before it is placed (`test_edit_cap.py`).
 """
 
 from dataclasses import dataclass
+from typing import Literal
 
 from sqlalchemy import and_, func, or_, select
 from sqlalchemy.orm import Session
 
 from app import sysconfig
 from app.downloads.status import JobKind, JobStatus
-from app.models import DownloadJob, DownloadQuota, MediaAsset
+from app.downloads.models import DownloadJob, DownloadQuota, MediaAsset
+from app.userscope import owned_by
 
 # Jobs a user is considered to be "holding" (occupy a job slot / footprint). Terminal failures
 # (error, canceled) don't count.
@@ -49,11 +52,16 @@ class QuotaLimits:
     unlimited: bool
 
 
-class QuotaExceeded(Exception):
-    """Raised by check_enqueue; `reason` is one of max_jobs|max_bytes (i18n key on the client)."""
+# Every cap a download can hit; the UI names each one (`downloads.errors.quota.<reason>`, held by the
+# frontend's backendIds contract, which reads this Literal).
+QuotaReason = Literal["max_jobs", "max_bytes"]
 
-    def __init__(self, reason: str, limit: int, current: int):
-        self.reason = reason
+
+class QuotaExceeded(Exception):
+    """Raised by check_enqueue; `reason` names the cap it hit (an i18n key on the client)."""
+
+    def __init__(self, reason: QuotaReason, limit: int, current: int):
+        self.reason: QuotaReason = reason
         self.limit = limit
         self.current = current
         super().__init__(f"download quota exceeded: {reason} ({current}/{limit})")
@@ -85,7 +93,7 @@ def footprint(db: Session, user_id: int) -> int:
     held_assets = (
         select(DownloadJob.asset_id)
         .where(
-            DownloadJob.user_id == user_id,
+            owned_by(DownloadJob.user_id, user_id),
             or_(
                 DownloadJob.status.in_(_HOLDING),
                 and_(DownloadJob.job_kind == JobKind.UPLOAD, DownloadJob.status == JobStatus.ERROR),
@@ -109,7 +117,7 @@ def _count(db: Session, user_id: int, states) -> int:
         db.execute(
             select(func.count())
             .select_from(DownloadJob)
-            .where(DownloadJob.user_id == user_id, DownloadJob.status.in_(states))
+            .where(owned_by(DownloadJob.user_id, user_id), DownloadJob.status.in_(states))
         ).scalar()
         or 0
     )
@@ -201,3 +209,46 @@ def usage(db: Session, user_id: int) -> dict:
         "max_jobs": lim.max_jobs,
         "unlimited": lim.unlimited,
     }
+
+
+def set_custom(
+    db: Session,
+    user_id: int,
+    *,
+    max_bytes: int | None = None,
+    max_concurrent: int | None = None,
+    max_jobs: int | None = None,
+    unlimited: bool | None = None,
+) -> None:
+    """The admin's own limits for a user: starts from what applies now, then each value given (a
+    size floored at 0, a count at 1). The handler commits."""
+    row = db.get(DownloadQuota, user_id)
+    if row is None:
+        cur = resolve(db, user_id)
+        row = DownloadQuota(
+            user_id=user_id,
+            max_bytes=cur.max_bytes,
+            max_concurrent=cur.max_concurrent,
+            max_jobs=cur.max_jobs,
+            unlimited=cur.unlimited,
+        )
+        db.add(row)
+    if max_bytes is not None:
+        row.max_bytes = max(0, max_bytes)
+    if max_concurrent is not None:
+        row.max_concurrent = max(1, max_concurrent)
+    if max_jobs is not None:
+        row.max_jobs = max(1, max_jobs)
+    if unlimited is not None:
+        row.unlimited = unlimited
+    db.flush()
+
+
+def reset_custom(db: Session, user_id: int) -> bool:
+    """Back to the defaults; False when the user had no limits of their own."""
+    row = db.get(DownloadQuota, user_id)
+    if row is None:
+        return False
+    db.delete(row)
+    db.flush()
+    return True

backend/app/downloads/runner.py

@@ -16,6 +16,7 @@ import shutil
 import subprocess
 import sys
 
+import app.tables  # noqa: F401 — a fresh interpreter: every table, before the first query
 from app import logs
 from app import logsink
 from app.config import settings
@@ -23,7 +24,7 @@ from app.db import SessionLocal
 from app.downloads import edit as editmod
 from app.downloads import formats
 from app.downloads.status import JobKind
-from app.models import DownloadJob
+from app.downloads.models import DownloadJob
 
 log = logs.get_logger(__name__)
 

backend/app/downloads/service.py

@@ -1,16 +1,20 @@
-"""Enqueue + cache resolution for the download center (the per-user layer over MediaAsset).
+"""Capture's writes: enqueue + cache resolution, the job transitions, and what the library's
+dialogs change (profiles, resume points, shares, a file's display fields).
 
 `enqueue` is the single entry point used by the API: it resolves (or creates) the shared
 MediaAsset for the requested source+format, links a per-user DownloadJob to it, and — on a
 cache hit (asset already ready) — marks the job done instantly with no re-download. The worker
-handles everything else. State transitions used by the routes (pause/resume/cancel/delete) and
-the worker live in `worker.py` / the routes; this module owns enqueue + asset bookkeeping.
+handles everything else. Every function here flushes and leaves the commit to its caller (the
+handler); the folder, link and quota writes are `folders`, `links` and `quota`.
 """
 
 import uuid
 from datetime import datetime, timezone
+from typing import Literal
 
+from sqlalchemy import delete as delete_rows
 from sqlalchemy import func, select, update
+from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import Session
 
@@ -21,24 +25,31 @@ from app.downloads import credentials
 from app.downloads import edit as editmod
 from app.downloads import folders, formats, mediakind, quota, storage
 from app.downloads.status import HOLDING, JobKind, JobStatus
-from app.models import (
-    DISPLAY_TEXT_MAX,
-    Channel,
+from app.models import DISPLAY_TEXT_MAX, User
+from app.plex.models import PlexItem, PlexShow
+from app.catalog.models import Channel, Video
+from app.downloads.models import (
     DownloadJob,
+    DownloadProfile,
+    DownloadShare,
+    LibraryState,
     MediaAsset,
-    PlexItem,
-    PlexShow,
-    Video,
 )
 from app.plex import lookups as plex_lookups
 from app.plex import paths as plex_paths
+from app.userscope import owned_by, owned_row
+
+
+# Every reason an edit is refused; the UI names each one (`downloads.errors.edit.<reason>`, held by
+# the frontend's backendIds contract, which reads this Literal).
+EditReason = Literal["source_not_ready", "not_media", "empty_edit"]
 
 
 class EditError(Exception):
-    """Bad edit request (empty spec / source not ready); `reason` is an i18n-mappable key."""
+    """Bad edit request (empty spec / source not ready / not media); `reason` names the sentence."""
 
-    def __init__(self, reason: str):
-        self.reason = reason
+    def __init__(self, reason: EditReason):
+        self.reason: EditReason = reason
         super().__init__(reason)
 
 
@@ -142,7 +153,7 @@ def populate_from_catalog(db: Session, asset: MediaAsset) -> bool:
         return False
     asset.title = video.title
     asset.thumbnail_url = video.thumbnail_url
-    asset.duration_s = video.duration_seconds
+    asset.duration_s = video.length_seconds
     if video.published_at:
         asset.upload_date = video.published_at.date()
     channel = db.get(Channel, video.channel_id)
@@ -230,8 +241,7 @@ def enqueue(
         folders.stamp_done(db, job)
         asset.last_access_at = datetime.now(timezone.utc)
 
-    db.commit()
-    db.refresh(job)
+    db.flush()
     return job
 
 
@@ -310,8 +320,7 @@ def enqueue_edit(
         folders.stamp_done(db, job)
         asset.last_access_at = datetime.now(timezone.utc)
 
-    db.commit()
-    db.refresh(job)
+    db.flush()
     return job
 
 
@@ -328,7 +337,7 @@ def _clip_source_facts(db: Session, spec: dict) -> dict:
             "title": video.title,
             "uploader": channel.title if channel else None,
             "thumbnail_url": video.thumbnail_url,
-            "length_s": video.duration_seconds,
+            "length_s": video.length_seconds,
         }
     item = plex_lookups.by_rating_key(db, PlexItem, spec["ref"])
     if item is None:
@@ -399,8 +408,7 @@ def enqueue_clip(
         folders.stamp_done(db, job)
         asset.last_access_at = datetime.now(timezone.utc)
 
-    db.commit()
-    db.refresh(job)
+    db.flush()
     return job
 
 
@@ -508,8 +516,8 @@ def release_asset(db: Session, job: DownloadJob) -> None:
 # --- Job state transitions ---------------------------------------------------------------------
 # The single home for the pause/resume/cancel/delete/redownload transitions + their asset
 # bookkeeping (release, failed-asset reset, fresh-asset rebuild), so the routes are thin callers and
-# the worker/GC could share the same definitions (B-sub-8). Each commits its own transition; the
-# route keeps only auth + HTTP mapping + serialization.
+# the worker/GC could share the same definitions (B-sub-8). Each flushes its transition and the caller
+# commits it; the route keeps only auth + HTTP mapping + serialization.
 
 
 def pause(db: Session, job: DownloadJob) -> None:
@@ -517,7 +525,7 @@ def pause(db: Session, job: DownloadJob) -> None:
     in any other state."""
     if job.status in (JobStatus.QUEUED, JobStatus.RUNNING):
         job.status = JobStatus.PAUSED
-        db.commit()
+        db.flush()
 
 
 def resume(db: Session, job: DownloadJob) -> None:
@@ -541,7 +549,7 @@ def resume(db: Session, job: DownloadJob) -> None:
                 asset.status = "pending"
                 asset.error = None
         job.status = JobStatus.QUEUED
-        db.commit()
+        db.flush()
 
 
 def _follow_the_cookie(db: Session, job: DownloadJob, asset: MediaAsset) -> MediaAsset:
@@ -576,14 +584,14 @@ def cancel(db: Session, job: DownloadJob) -> None:
     if job.status not in (JobStatus.DONE, JobStatus.CANCELED):
         release_asset(db, job)  # release while the job still counts as holding
         job.status = JobStatus.CANCELED
-        db.commit()
+        db.flush()
 
 
 def delete(db: Session, job: DownloadJob) -> None:
     """Delete a job, releasing its asset hold first (which frees the file iff this was the last holder)."""
     release_asset(db, job)
     db.delete(job)
-    db.commit()
+    db.flush()
 
 
 def redownload(db: Session, job: DownloadJob) -> None:
@@ -632,4 +640,181 @@ def redownload(db: Session, job: DownloadJob) -> None:
     job.queue_pos = next_queue_pos(db)  # tail of the queue, not its stale original slot
     job.expires_at = None  # stamped again when the rebuild is done
     job.expiry_warned = False
-    db.commit()
+    db.flush()
+
+
+# --- the library's writes ------------------------------------------------------------------
+# What the HTTP layer changes for a user: each flushes, the handler commits.
+
+
+def lock_own_job(db: Session, user: User, job_id: int) -> DownloadJob | None:
+    """One of the user's jobs, re-read under a row lock, for a transition that moves its asset's ref
+    count (resume, cancel, delete, redownload) or writes the status those decide on (pause). Two
+    overlapping requests for one job run one after the other and the second decides on the first's
+    committed result — never both releasing the same hold, and never a pause turning a job a cancel
+    just released back into a holder; either can free a file another job still holds. None: not
+    theirs, or gone.
+
+    The asset is locked BEFORE the job: the order a finishing worker child takes them in
+    (`finalize_if_running`), so the two can never deadlock."""
+    while True:
+        owned = owned_row(db, DownloadJob, job_id, user)
+        if owned is None:
+            return None
+        asset_id = owned.asset_id
+        if asset_id is not None:
+            db.execute(select(MediaAsset.id).where(MediaAsset.id == asset_id).with_for_update())
+        job = db.execute(
+            select(DownloadJob)
+            .where(DownloadJob.id == job_id)
+            .with_for_update()
+            .execution_options(populate_existing=True)
+        ).scalar_one_or_none()
+        if job is None:
+            return None
+        if job.asset_id == asset_id:
+            return job
+        db.rollback()  # another request moved the job to another asset meanwhile: lock that one
+
+
+def hold_for_share(db: Session, user: User, job_id: int) -> DownloadJob | None:
+    """One of the user's files, held (FOR SHARE) for the rest of the transaction: a delete of it at
+    the same moment (the user's, or the retention sweep's) waits and takes the new link with it,
+    rather than failing the link's foreign key — or, done first, leaves None."""
+    if owned_row(db, DownloadJob, job_id, user) is None:
+        return None
+    return db.execute(
+        select(DownloadJob)
+        .where(DownloadJob.id == job_id)
+        .with_for_update(read=True)
+        .execution_options(populate_existing=True)
+    ).scalar_one_or_none()
+
+
+def create_profile(db: Session, user: User, name: str, spec: dict) -> DownloadProfile:
+    """A format profile of the user's own, after their others."""
+    maxpos = db.execute(
+        select(func.coalesce(func.max(DownloadProfile.sort_order), 100)).where(
+            owned_by(DownloadProfile.user_id, user)
+        )
+    ).scalar_one()
+    p = DownloadProfile(
+        user_id=user.id, name=name[:80], spec=spec, is_builtin=False, sort_order=maxpos + 1
+    )
+    db.add(p)
+    db.flush()
+    return p
+
+
+def update_profile(profile: DownloadProfile, *, name: str | None, spec: dict | None) -> None:
+    """Rename a profile and/or replace its spec; None leaves that one as it is."""
+    if name is not None:
+        profile.name = name[:80]
+    if spec is not None:
+        profile.spec = spec
+
+
+def delete_profile(db: Session, profile: DownloadProfile) -> None:
+    db.delete(profile)
+    db.flush()
+
+
+def set_position(db: Session, user: User, job_id: int, position: int) -> None:
+    """The library player's resume point in a file the user can play; 0 clears it.
+
+    A clear is a statement, not a loaded row's delete: the periodic checkpoint and the pagehide beacon
+    of one stop may both clear it, and the second then deletes nothing rather than failing. A set is
+    an upsert, one row per (user, job) even when two checkpoints race, not a read-then-insert that
+    the second would fail on the unique key."""
+    if not position:
+        db.execute(
+            delete_rows(LibraryState).where(
+                owned_by(LibraryState.user_id, user), LibraryState.job_id == job_id
+            )
+        )
+        return
+    now = datetime.now(timezone.utc)
+    db.execute(
+        pg_insert(LibraryState)
+        .values(user_id=user.id, job_id=job_id, position_seconds=position, progress_updated_at=now)
+        .on_conflict_do_update(
+            constraint="uq_library_user_job",
+            set_={"position_seconds": position, "progress_updated_at": now},
+        )
+    )
+
+
+def share(db: Session, job: DownloadJob, sender: User, recipient: User) -> bool:
+    """Hand a file to another user, with a notice for them; False when it was already theirs."""
+    from app.notifications import create_notification
+
+    exists = db.execute(
+        select(DownloadShare.id).where(
+            DownloadShare.job_id == job.id, owned_by(DownloadShare.to_user_id, recipient)
+        )
+    ).first()
+    if exists is not None:
+        return False
+    db.add(DownloadShare(job_id=job.id, from_user_id=sender.id, to_user_id=recipient.id))
+    create_notification(
+        db,
+        user_id=recipient.id,
+        type="download_shared",
+        title=(job.display_name or "A download") + " was shared with you",
+        # The client names the notice in the reader's language from `data`; the title is the
+        # English fallback only.
+        data={
+            "kind": "download_shared",
+            "from": sender.email,
+            "job_id": job.id,
+            "title": job.display_name,
+        },
+        commit=False,
+    )
+    db.flush()
+    return True
+
+
+def unshare(db: Session, job_id: int, recipient: User) -> bool:
+    """Take a file back from one user (its owner's act, or the recipient's own dismissal); False
+    when it was not theirs."""
+    row = db.execute(
+        select(DownloadShare).where(
+            DownloadShare.job_id == job_id, owned_by(DownloadShare.to_user_id, recipient)
+        )
+    ).scalar_one_or_none()
+    if row is None:
+        return False
+    db.delete(row)
+    db.flush()
+    return True
+
+
+# The display fields a user may set on their file; the file on disk keeps its own name.
+DISPLAY_FIELDS = frozenset(
+    {"display_name", "display_uploader", "display_uploader_url", "extra_links"}
+)
+
+
+def set_display(job: DownloadJob, **fields: object) -> None:
+    """Set a file's display fields (each already cleaned by the caller); a field not given is left."""
+    unknown = set(fields) - DISPLAY_FIELDS
+    if unknown:
+        raise ValueError(f"not a display field: {sorted(unknown)}")
+    for name, value in fields.items():
+        setattr(job, name, value)
+
+
+def touch(asset: MediaAsset) -> None:
+    """The file was just served: the cache's idle clock starts again."""
+    asset.last_access_at = datetime.now(timezone.utc)
+
+
+def store_sha256(db: Session, asset_id: int, digest: str) -> None:
+    """Keep a file's hash once it is known — only where none is, so a concurrent hash of the same
+    file never overwrites the first."""
+    db.execute(
+        update(MediaAsset)
+        .where(MediaAsset.id == asset_id, MediaAsset.sha256.is_(None))
+        .values(sha256=digest)
+    )

backend/app/downloads/storage.py

@@ -294,9 +294,13 @@ def served_media_type(media_kind: str | None, container: str | None) -> str | No
 def download_filename(display_name: str, container: str | None, path: Path) -> str:
     """The Content-Disposition filename for a served download: the cleaned display name carrying a single
     correct extension — the asset container, else the file's own suffix — without doubling it when the
-    name already ends in that extension. Shared by the private download and the public watch endpoints."""
+    name already ends in that extension. A file with no extension at all is served without one —
+    never as `name.` (Windows drops a trailing dot, so the saved file would not be the one named).
+    Shared by the private download and the public watch endpoints."""
     ext = container or path.suffix.lstrip(".")
     base = display_filename(display_name)
+    if not ext:
+        return base
     if base.lower().endswith(f".{ext.lower()}"):
         base = base[: -(len(ext) + 1)]
     return f"{base}.{ext}"

backend/app/downloads/upload.py

@@ -37,7 +37,8 @@ from app.config import settings
 from app.downloads import edit as editmod
 from app.downloads import folders, mediakind, playable, probe, quota, service, storage
 from app.downloads.status import JobKind, JobStatus
-from app.models import DownloadJob, MediaAsset, display_text
+from app.models import display_text
+from app.downloads.models import DownloadJob, MediaAsset
 
 
 def _title_from_filename(filename: str) -> str:
@@ -57,12 +58,14 @@ def ingest_upload(
     display_name: str | None = None,
     sha256: str | None = None,
     folder_id: int | None = None,
+    via: str | None = None,
 ) -> DownloadJob:
     """Ingest an already-uploaded file at `src_path` into `user_id`'s library and return its job —
     `done`, or `queued` for the worker's phone-safe conversion (see the module docstring). `src_path`
     is MOVED into place on success (the caller stages it to a temp file first). `sha256` is the
     hash of those bytes, when the caller took it on the way in; `folder_id` the folder it asked for
-    (`folders.upload_target`), else its Uploads folder.
+    (`folders.upload_target`), else its Uploads folder; `via` what the audit row names as the doer
+    when it was not the user themselves (an API token).
 
     Raises folders.FolderError when that folder is not one an upload may land in (or is gone).
 
@@ -145,7 +148,14 @@ def ingest_upload(
             folders.stamp_done(db, job)
         service.incr_ref_count(db, asset.id)
         db.flush()
-        audit.record(db, user_id, AuditAction.FILE_UPLOAD, target_type="file", target_id=job.id)
+        audit.record(
+            db,
+            user_id,
+            AuditAction.FILE_UPLOAD,
+            target_type="file",
+            target_id=job.id,
+            summary=via,
+        )
         db.commit()
     except Exception:
         db.rollback()

backend/app/downloads/urlguard.py

@@ -10,22 +10,12 @@ What this closes: a URL whose host resolves to a loopback / private / link-local
 is refused, at enqueue AND again in the worker before extraction. Both, because DNS can answer
 differently between the two moments, and the worker is where the socket is actually opened.
 
-What it does NOT close, stated plainly rather than implied, because both need the same fix — gating
-the connection itself with a custom socket factory inside yt-dlp, a much larger change than this:
-
-* DNS rebinding: a host that passes the worker's check and then answers yt-dlp's own resolution
-  with an internal address.
-* HTTP redirects: a public host that answers 3xx with an internal ``Location``. yt-dlp follows it,
-  and only the URL the user TYPED was ever judged here. An attacker-controlled host or any public
-  open redirect reaches it, so this is the cheaper of the two. It does NOT usually yield the
-  response body — a non-media body fails the mp4 remux and stores nothing — but it does hand back
-  the readout above: the redaction in ``errors.user_error_text`` fires only for the ``network``
-  class, so a dead internal port (refused → ``network`` → redacted) is distinguishable from a live
-  one (an HTTP answer that is not media → ``unsupported_host``/``format_unavailable``, raw text
-  kept). Tracked as SEC-17.
-
-The two checks here stop the straightforward case (typing ``http://127.0.0.1:6379`` or an internal
-hostname into the box), which is the reachable one.
+What this alone cannot see is every request yt-dlp makes AFTER the typed URL: a public host
+answering 3xx with an internal ``Location``, and DNS rebinding (a name that passes this check and
+then resolves inside for yt-dlp's own lookup). Those are closed one layer down, by `egress`: a guarded
+download's every connection goes through a gate that judges the address it actually dials, with
+this module's `is_forbidden`. The checks here stay as the early, structured refusal — a user typing
+``http://127.0.0.1:6379`` hears why at once instead of from a failed job.
 
 Not used for YouTube jobs: those resolve to a fixed public host from an 11-char id, so there is no
 user-controlled destination to guard.
@@ -33,27 +23,36 @@ user-controlled destination to guard.
 
 import ipaddress
 import socket
+from typing import Literal
 from urllib.parse import urlparse
 
 # Reason codes travel to the client inside the structured 422 detail, so they are part of the API
-# vocabulary (frontend `localizeDetail`) — keep them stable.
-REASON_SCHEME = "scheme"  # not http(s)
-REASON_NO_HOST = "no_host"  # URL carries no hostname
-REASON_MALFORMED = "malformed"  # unparseable authority or port; urlparse raises late, so guard it
-REASON_UNRESOLVABLE = "unresolvable"  # DNS said nothing (honest: not the same as "blocked")
-REASON_PRIVATE = "private"  # resolves into a network the server must not fetch from
+# vocabulary — keep them stable. The UI names each one (`downloads.errors.urlBlocked.<reason>`, held by
+# the frontend's backendIds contract, which reads this Literal).
+UrlBlockedReason = Literal["scheme", "no_host", "malformed", "unresolvable", "private"]
+REASON_SCHEME: UrlBlockedReason = "scheme"  # not http(s)
+REASON_NO_HOST: UrlBlockedReason = "no_host"  # URL carries no hostname
+REASON_MALFORMED: UrlBlockedReason = (
+    "malformed"  # unparseable authority or port; urlparse raises late
+)
+REASON_UNRESOLVABLE: UrlBlockedReason = (
+    "unresolvable"  # DNS said nothing (not the same as "blocked")
+)
+REASON_PRIVATE: UrlBlockedReason = (
+    "private"  # resolves into a network the server must not fetch from
+)
 
 
 class BlockedUrl(Exception):
     """A user-supplied URL that must not be fetched server-side. ``reason`` is one of the
     REASON_* codes above."""
 
-    def __init__(self, reason: str):
+    def __init__(self, reason: UrlBlockedReason):
         super().__init__(reason)
-        self.reason = reason
+        self.reason: UrlBlockedReason = reason
 
 
-def _is_forbidden(ip: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
+def is_forbidden(ip: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
     """Every address family we must not open a connection to. Checked explicitly rather than
     leaning on ``is_private`` alone, whose coverage differs between IPv4 and IPv6."""
     # ::ffff:127.0.0.1 is loopback wearing an IPv6 hat — unwrap before judging it.
@@ -104,5 +103,5 @@ def check_url(raw: str) -> None:
             ip = ipaddress.ip_address(addr)
         except ValueError:  # getaddrinfo returned something unparseable; treat as unusable
             raise BlockedUrl(REASON_PRIVATE) from None
-        if _is_forbidden(ip):
+        if is_forbidden(ip):
             raise BlockedUrl(REASON_PRIVATE)

backend/app/draw.py

@@ -0,0 +1,118 @@
+"""A weighted draw that stays a surprise: what "Surprise me" deals from.
+
+A title's chance to come first is its weight over the pool's (Efraimidis–Spirakis: order by
+-ln(1-u)/w, u uniform per row). The weight is 1 + the title's affinity to what the user watches
+(`plex.affinity`, `discover.queries`), so a title nobody leans towards still comes up — only less
+often. Two things keep it a surprise rather than a ranking:
+
+- **u is the seed's md5 of the row's key**, the Home surprise's seeded order. With every weight 1
+  the order is exactly that md5 order, so a user with no history sees the unweighted draw, and one
+  seed still moves both sources of the Home hero together. A draw that is not seeded (the Plex
+  Discover rail) takes u from random().
+- **One slot of three is blind** (`blend`): dealt from the unweighted order, so the same few
+  favourites cannot fill the rail.
+
+The YouTube surprise draws over a window of the feed scope rather than all of it (`yt_surprise`):
+the window is read in SQL, and `shuffled` / `leaning` order it here with the same u.
+
+Both affinities weigh a watch by `recency` and score against the strongest with `scaled`, so the two
+sources lean by one rule.
+"""
+
+import hashlib
+import math
+from collections.abc import Callable, Hashable
+from datetime import datetime
+from typing import Any
+
+from sqlalchemy import BigInteger, Float, cast, func, literal
+from sqlalchemy.dialects.postgresql import BIT
+from sqlalchemy.orm import InstrumentedAttribute
+from sqlalchemy.sql.elements import ColumnElement
+
+# 2**32: the first 8 hex digits of an md5 as a fraction of one.
+_SPAN = 4294967296.0
+
+# A watch with no date (an old import) counts as one this many half-lives ago.
+UNDATED_HALF_LIVES = 2
+
+
+def recency(watched_at: datetime | None, now: datetime, half_life_days: float) -> float:
+    """A watch's weight in an affinity: 1 now, half as much every `half_life_days`."""
+    if watched_at is None:
+        return 0.5**UNDATED_HALF_LIVES
+    return 0.5 ** (max(0.0, (now - watched_at).total_seconds() / 86400) / half_life_days)
+
+
+def scaled(scores: dict[str, float]) -> dict[str, float]:
+    """Each score over the strongest, so the strongest is 1; empty when nothing scored."""
+    top = max(scores.values(), default=0.0)
+    return {k: v / top for k, v in scores.items()} if top else {}
+
+
+def seeded_u(
+    key: ColumnElement[Any] | InstrumentedAttribute[Any], seed: int
+) -> ColumnElement[float]:
+    """A uniform u in [0, 1) per row, from md5(key || seed) — the order `md5(...)` ascending gives."""
+    digest = func.md5(func.concat(key, str(seed)))
+    head = cast(cast(literal("x") + func.substr(digest, 1, 8), BIT(32)), BigInteger)
+    return cast(head, Float) / _SPAN
+
+
+def weighted_key(u: ColumnElement[float], weight: ColumnElement[float]) -> ColumnElement[float]:
+    """The row's place in a weighted draw, ascending: -ln(1-u)/w. Monotonic in u when every w is 1,
+    so the unweighted order is unchanged."""
+    return -func.ln(1.0 - u) / weight
+
+
+def random_u() -> ColumnElement[float]:
+    return func.random()
+
+
+def u_of(key: str, seed: int) -> float:
+    """`seeded_u` in Python: the same u for the same key and seed."""
+    return int(hashlib.md5(f"{key}{seed}".encode()).hexdigest()[:8], 16) / _SPAN
+
+
+def shuffled(items: list, seed: int, key: Callable[[Any], str]) -> list:
+    """`items` in the seed's md5 order — the order `ORDER BY md5(key || seed)` gives."""
+    return sorted(items, key=lambda it: hashlib.md5(f"{key(it)}{seed}".encode()).hexdigest())
+
+
+def leaning(
+    items: list, seed: int, key: Callable[[Any], str], weight: Callable[[Any], float]
+) -> list:
+    """`items` in the weighted draw's order (`weighted_key` in Python), ties in the md5 order."""
+    order = {key(it): i for i, it in enumerate(shuffled(items, seed, key))}
+    return sorted(
+        items,
+        key=lambda it: (-math.log(1.0 - u_of(key(it), seed)) / weight(it), order[key(it)]),
+    )
+
+
+def blend(
+    weighted: list, blind: list, offset: int, n: int, key: Callable[[Any], Hashable] = lambda x: x
+) -> list:
+    """`n` picks: slot i comes from `blind` when (i + offset) % 3 == 0, else from `weighted`, each
+    skipping what an earlier slot took (`key` tells two picks apart). A side that runs out leaves the
+    rest to the other."""
+    out: list = []
+    taken: set = set()
+    sources = (iter(weighted), iter(blind))
+
+    def nxt(source):
+        for item in source:
+            if key(item) not in taken:
+                return item
+        return None
+
+    for i in range(n):
+        first, second = (sources[1], sources[0]) if (i + offset) % 3 == 0 else sources
+        item = nxt(first)
+        if item is None:
+            item = nxt(second)
+        if item is None:
+            break
+        taken.add(key(item))
+        out.append(item)
+    return out

backend/app/feedquery.py

@@ -1,10 +1,11 @@
-"""Feed read-model: the pure query-building and row-serialization layer behind the feed.
+"""Feed read-model: Reader's query-building, paging and row serialization behind the feed.
 
-Extracted from `routes/feed.py` so the read-model has one home instead of living inside the
-router module — the feed route handlers and the playlist-detail query share these helpers (the
-projection, the filter builder, and keyset paging). Nothing here defines a route or reaches for
-request state; it's all `(db, user, params) -> query`, and a malformed cursor raises a plain
-`ValueError` for the route handler to translate (the tsquery sanitizer lives in `app.utils`).
+One home for the read-model rather than the router module: the feed routes, the playlist-detail
+query and Home share these helpers (the projection, the filter builder, and keyset paging), and the
+feed's routes only validate and answer — a page, its counts and its facets are built here. Nothing
+here defines a route or reaches for request state; it's all `(db, user, params) -> query`, and a
+malformed cursor raises `BadCursor` for the route handler to translate (the tsquery sanitizer lives
+in `app.utils`).
 """
 
 import base64
@@ -12,26 +13,35 @@ import binascii
 import json
 from datetime import date, datetime, timedelta, timezone
 
-from sqlalchemy import BigInteger, Select, and_, case, cast, false, func, or_, select
+from sqlalchemy import (
+    BigInteger,
+    Select,
+    and_,
+    case,
+    cast,
+    false,
+    func,
+    literal,
+    or_,
+    select,
+    tuple_,
+)
 from sqlalchemy.orm import Session, aliased
 
 from app.collation import collated
-from app.models import (
+from app.models import User
+from app.catalog.models import (
     LIVE_OR_UPCOMING,
     BlockedChannel,
     Channel,
-    ChannelTag,
     ExploredChannel,
-    Playlist,
-    PlaylistItem,
     SearchFind,
     Subscription,
-    Tag,
-    User,
     Video,
-    VideoState,
 )
+from app.reader.models import ChannelTag, Playlist, PlaylistItem, Tag, VideoState
 from app.utils import iso, like_escape, to_tsquery_str
+from app.userscope import owned_by
 
 
 def _channel_url(channel_id: str, handle: str | None) -> str:
@@ -48,7 +58,7 @@ def _saved_expr(user: User):
         .select_from(PlaylistItem)
         .join(Playlist, Playlist.id == PlaylistItem.playlist_id)
         .where(
-            Playlist.user_id == user.id,
+            owned_by(Playlist.user_id, user.id),
             Playlist.kind == "watch_later",
             PlaylistItem.video_id == Video.id,
         )
@@ -70,7 +80,7 @@ def feed_columns(user: User, state) -> list:
         Channel.handle.label("channel_handle"),
         Video.published_at,
         Video.thumbnail_url,
-        Video.duration_seconds,
+        Video.length_seconds.label("length_seconds"),
         Video.view_count,
         Video.is_short,
         Video.live_status,
@@ -90,7 +100,7 @@ def serialize(row) -> dict:
         "channel_url": _channel_url(row.channel_id, row.channel_handle),
         "published_at": iso(row.published_at),
         "thumbnail_url": row.thumbnail_url,
-        "duration_seconds": row.duration_seconds,
+        "duration_seconds": row.length_seconds,
         "view_count": row.view_count,
         "is_short": row.is_short,
         "live_status": row.live_status,
@@ -103,7 +113,7 @@ def serialize(row) -> dict:
 
 def _tag_visible(user: User):
     """A ChannelTag row this user may see: a shared system tag (user_id NULL) or their own."""
-    return or_(ChannelTag.user_id.is_(None), ChannelTag.user_id == user.id)
+    return or_(ChannelTag.user_id.is_(None), owned_by(ChannelTag.user_id, user.id))
 
 
 # The pseudo-category of a selected tag id no Tag row answers (a saved view or the feed state outlives
@@ -122,7 +132,7 @@ def tags_by_category(db: Session, user: User, tag_ids: list[int]) -> dict[str, l
     if not tag_ids:
         return {}  # skip the pointless `WHERE id IN ()` round-trip on the no-tag-filter hot path
     out: dict[str, list[int]] = {}
-    visible = or_(Tag.user_id.is_(None), Tag.user_id == user.id)
+    visible = or_(Tag.user_id.is_(None), owned_by(Tag.user_id, user.id))
     for tid, cat in db.execute(
         select(Tag.id, Tag.category).where(Tag.id.in_(tag_ids), visible)
     ).all():
@@ -142,7 +152,7 @@ def not_blocked_cond(user: User):
     position — and the one query that copied only PART of this predicate set is exactly how that
     video came back as the landing page's suggestion after the user had said "never again"."""
     return Video.channel_id.notin_(
-        select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)
+        select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user.id))
     )
 
 
@@ -187,7 +197,7 @@ def filtered_query(
             Subscription,
             and_(
                 Subscription.channel_id == Video.channel_id,
-                Subscription.user_id == user.id,
+                owned_by(Subscription.user_id, user.id),
             ),
         )
     else:
@@ -199,12 +209,12 @@ def filtered_query(
             Subscription,
             and_(
                 Subscription.channel_id == Video.channel_id,
-                Subscription.user_id == user.id,
+                owned_by(Subscription.user_id, user.id),
                 Subscription.hidden.is_(False),
             ),
         ).outerjoin(
             SearchFind,
-            and_(SearchFind.video_id == Video.id, SearchFind.user_id == user.id),
+            and_(SearchFind.video_id == Video.id, owned_by(SearchFind.user_id, user.id)),
         )
 
     # The state-gated views (watched/hidden/in_progress) only ever return videos that HAVE a
@@ -214,7 +224,7 @@ def filtered_query(
     # exactly. 'unwatched'/'all' include row-less 'new' videos, so they keep the outer join.
     state_required = show in ("watched", "hidden", "in_progress")
     join_state = query.join if state_required else query.outerjoin
-    query = join_state(state, and_(state.video_id == Video.id, state.user_id == user.id))
+    query = join_state(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
 
     # Hide videos the maintenance job has flagged as currently unplayable (deleted/private/
     # abandoned). They're either deleted after a grace period or unhidden if they recover;
@@ -227,8 +237,8 @@ def filtered_query(
     # catalog. Applied in BOTH scopes; the channel page reaches its ephemeral videos by
     # asking for scope=all + channel_ids (the channel is in this accessible set).
     accessible_explore = (
-        select(Subscription.channel_id).where(Subscription.user_id == user.id)
-    ).union(select(ExploredChannel.channel_id).where(ExploredChannel.user_id == user.id))
+        select(Subscription.channel_id).where(owned_by(Subscription.user_id, user.id))
+    ).union(select(ExploredChannel.channel_id).where(owned_by(ExploredChannel.user_id, user.id)))
     query = query.where(
         or_(
             Video.via_explore.is_(False),
@@ -262,9 +272,9 @@ def filtered_query(
         # OR across the picked channels; every other filter still applies on top.
         query = query.where(Video.channel_id.in_(channel_ids))
     if min_duration is not None:
-        query = query.where(Video.duration_seconds >= min_duration)
+        query = query.where(Video.length_seconds >= min_duration)
     if max_duration is not None:
-        query = query.where(Video.duration_seconds <= max_duration)
+        query = query.where(Video.length_seconds <= max_duration)
     if max_age_days is not None:
         cutoff = datetime.now(timezone.utc).timestamp() - max_age_days * 86400
         query = query.where(Video.published_at >= datetime.fromtimestamp(cutoff, tz=timezone.utc))
@@ -346,7 +356,7 @@ def filtered_query(
     if untagged:
         query = query.where(
             Video.channel_id.notin_(
-                select(ChannelTag.channel_id).where(ChannelTag.user_id == user.id)
+                select(ChannelTag.channel_id).where(owned_by(ChannelTag.user_id, user.id))
             )
         )
 
@@ -505,10 +515,10 @@ def sort_keys(sort: str, seed: int, rank_expr=None, lang: str | None = None) ->
         "views": [{"expr": Video.view_count, "asc": False, "nullable": True, "kind": "num"}],
         "views_asc": [{"expr": Video.view_count, "asc": True, "nullable": True, "kind": "num"}],
         "duration_desc": [
-            {"expr": Video.duration_seconds, "asc": False, "nullable": True, "kind": "num"}
+            {"expr": Video.length_seconds, "asc": False, "nullable": True, "kind": "num"}
         ],
         "duration_asc": [
-            {"expr": Video.duration_seconds, "asc": True, "nullable": True, "kind": "num"}
+            {"expr": Video.length_seconds, "asc": True, "nullable": True, "kind": "num"}
         ],
         "title": [{"expr": title, "asc": True, "nullable": True, "kind": "str"}],
         "title_desc": [{"expr": title, "asc": False, "nullable": True, "kind": "str"}],
@@ -538,15 +548,17 @@ def sort_keys(sort: str, seed: int, rank_expr=None, lang: str | None = None) ->
             },
             pub,
         ],
-        # Deterministic per-seed shuffle: the same seed yields the same total order, so
-        # the cursor stays valid across pages of one shuffled run.
+        # The ring of stored random keys from the seed's point (`shuffle_walk`): the keys at or after
+        # it, then the ones before. The same seed yields the same total order, so the cursor stays
+        # valid across pages; `feed_page` reads it as two index ranges, never as this sort.
         "shuffle": [
             {
-                "expr": func.md5(func.concat(Video.id, str(seed))),
+                "expr": case((Video.shuffle_key < shuffle_start(seed), 1), else_=0),
                 "asc": True,
                 "nullable": False,
-                "kind": "str",
-            }
+                "kind": "num",
+            },
+            {"expr": Video.shuffle_key, "asc": True, "nullable": False, "kind": "num"},
         ],
     }
     # Relevance (search-result ranking) only exists when there's a query to rank against;
@@ -696,3 +708,143 @@ def decode_cursor(keys: list[dict], cursor: str) -> tuple[list, str, datetime |
         # Framework-free: the route handler translates this into an HTTP 400 so the
         # read-model stays web-layer-agnostic (see routes/feed.get_feed).
         raise ValueError("Invalid feed cursor") from exc
+
+
+class BadCursor(ValueError):
+    """A `cursor` this module did not write (or one from another sort): the caller's bad input."""
+
+
+# The golden ratio's fraction: consecutive seeds land far apart on the ring, never next to each other.
+_GOLDEN = 0.6180339887498949
+
+
+def shuffle_start(seed: int) -> float:
+    """The point in [0, 1) where a seed's shuffle starts on the ring of `Video.shuffle_key`.
+
+    Folded to 32 bits first: an int past a float's range overflowed the product, and one past 2**53
+    left the product no fraction to land on. A seed the client draws (under a billion) is unchanged."""
+    return ((seed % 2**32) * _GOLDEN) % 1.0
+
+
+def shuffle_walk(
+    db: Session, query: Select, seed: int, n: int, after: tuple[int, float, str] | None = None
+) -> list:
+    """Up to `n` rows of `query` in the seed's ring order: shuffle_key from the seed's point upwards,
+    then from 0 up to that point. `after` is a cursor's (phase, key, id): the walk resumes past it.
+
+    Each arc is an ascending range read along ix_videos_shuffle_key with the scope's filters checked
+    row by row, so a draw costs the rows it reads, not a sort of the scope — the md5(id || seed) order
+    it replaced read and sorted every row the filters let through (~350 ms on prod for ~213k)."""
+    start = shuffle_start(seed)
+    phase_from, key, last_id = after if after is not None else (0, None, None)
+    rows: list = []
+    for phase in (0, 1):
+        if phase < phase_from or len(rows) >= n:
+            continue
+        arc = Video.shuffle_key >= start if phase == 0 else Video.shuffle_key < start
+        q = query.where(arc)
+        if key is not None and phase == phase_from:
+            q = q.where(
+                tuple_(Video.shuffle_key, Video.id) > tuple_(literal(key), literal(last_id))
+            )
+        rows += db.execute(q.order_by(Video.shuffle_key, Video.id).limit(n - len(rows))).all()
+    return rows
+
+
+def feed_page(
+    db: Session,
+    user: User,
+    params: dict,
+    *,
+    sort: str,
+    seed: int,
+    limit: int,
+    cursor: str | None,
+    lang: str,
+) -> dict:
+    """One page of the feed, and the cursor for the next. A malformed cursor raises `BadCursor`."""
+    query, rank_expr = filtered_query(db, user, **params)
+    keys = sort_keys(sort, seed, rank_expr, lang)
+
+    vals, cursor_id, window = None, None, None
+    if cursor:
+        try:
+            vals, cursor_id, window = decode_cursor(keys, cursor)
+        except ValueError as e:
+            raise BadCursor(str(e)) from e
+    ranked = sort == "relevance" and rank_expr is not None
+    if ranked and not cursor:
+        window = rank_window_start(db, query)
+    if ranked and window is not None:
+        keys = sort_keys(sort, seed, windowed_rank(rank_expr, window), lang)
+
+    # Expose each key column on the row so we can build the next cursor from the last item.
+    query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
+    if vals is not None:
+        for k, v in zip(keys, vals, strict=True):
+            k["value"] = v
+    if ranked and window is not None:
+        rows = windowed_relevance_rows(db, query, keys, window, cursor_id, limit + 1)
+    elif sort == "shuffle":
+        after = None
+        if vals is not None and cursor_id is not None:
+            try:
+                after = (int(vals[0]), float(vals[1]), str(cursor_id))
+            except (TypeError, ValueError, OverflowError) as e:  # Infinity, a 400-digit key
+                raise BadCursor("Invalid feed cursor") from e
+        rows = shuffle_walk(db, query, seed, limit + 1, after)
+    else:
+        query = query.order_by(*order_by(keys))
+        if cursor_id is not None:
+            query = query.where(keyset_predicate(keys, cursor_id))
+        rows = db.execute(query.limit(limit + 1)).all()
+    has_more = len(rows) > limit
+    page = rows[:limit]
+    return {
+        "items": [serialize(r) for r in page],
+        "next_cursor": (
+            encode_cursor(keys, page[-1], window if ranked else None)
+            if (has_more and page)
+            else None
+        ),
+        "has_more": has_more,
+        "limit": limit,
+    }
+
+
+def feed_count(db: Session, user: User, **params) -> int:
+    """How many videos the filters match — the reference oracle `feed_summary`'s count is tested
+    against."""
+    query, _rank = filtered_query(db, user, **params)
+    return db.scalar(select(func.count()).select_from(query.subquery())) or 0
+
+
+def facet_counts(db: Session, user: User, **params) -> dict[int, int]:
+    """Per-tag channel counts for the current filter context, one pass per category — the reference
+    oracle `feed_summary`'s facets are tested against.
+
+    Each category is counted with every other filter applied but its own selections ignored
+    (drill-down faceting: selecting one topic doesn't zero out the rest). A conjunctive (AND)
+    category keeps them applied, so each remaining chip narrows to channels that ALSO have every
+    selected tag, and tags that can't co-occur drop to zero."""
+    visible = _tag_visible(user)
+    counts: dict[int, int] = {}
+    for category in ("language", "topic", "other"):
+        conjunctive = params.get("tag_modes", {}).get(category) == "and"
+        base, _rank = filtered_query(
+            db,
+            user,
+            **{**params, "exclude_tag_category": None if conjunctive else category},
+        )
+        channels = base.with_only_columns(Video.channel_id).distinct().subquery()
+        rows = db.execute(
+            select(ChannelTag.tag_id, func.count(func.distinct(ChannelTag.channel_id)))
+            .select_from(channels)
+            .join(ChannelTag, ChannelTag.channel_id == channels.c.channel_id)
+            .join(Tag, and_(Tag.id == ChannelTag.tag_id, Tag.category == category))
+            .where(visible)
+            .group_by(ChannelTag.tag_id)
+        ).all()
+        for tag_id, count in rows:
+            counts[tag_id] = count
+    return counts

backend/app/jobruns.py

@@ -0,0 +1,141 @@
+"""The durable history of background runs — the `job_runs` table.
+
+A run is written twice: a `running` row when it starts, its outcome when it ends. Each write is its
+OWN short session, committed at once, for the reason `quota.record_usage` gives: the run's own work
+may roll back (its `except: db.rollback()`), and the history of what happened must not go with it —
+nor may this bookkeeping commit the job's half-done session.
+
+Only the API process runs jobs (the worker never starts the scheduler), and one API process serves a
+database (`.env.example`, the Scheduler section: the single-run guard is in its memory too), so at its
+startup nothing can still be running: `interrupt_leftovers` closes what a stopped process left open.
+"""
+
+from datetime import datetime, timedelta, timezone
+
+from sqlalchemy import delete, func, select, union_all, update
+from sqlalchemy.engine import CursorResult
+from sqlalchemy.orm import Session
+
+from app.db import SessionLocal
+from app.models import JobRun, QuotaEvent
+from app.runstatus import RunStatus, RunTrigger
+
+
+def start(job: str, trigger: RunTrigger, actor_id: int | None, target: str | None = None) -> int:
+    """Open a run's row; returns its id."""
+    with SessionLocal() as s:
+        row = JobRun(
+            job=job,
+            trigger=trigger,
+            actor_id=actor_id,
+            target=target,
+            status=RunStatus.RUNNING,
+            started_at=datetime.now(timezone.utc),
+        )
+        s.add(row)
+        s.commit()
+        return row.id
+
+
+def finish(
+    run_id: int,
+    status: RunStatus,
+    *,
+    summary: str | None = None,
+    result: dict | None = None,
+    error: str | None = None,
+    failure: dict | None = None,
+) -> None:
+    """Close a run with its outcome and what it spent. Conditional on the row still running: a row
+    the startup sweep already called interrupted is not rewritten by a thread that outlived it."""
+    with SessionLocal() as s:
+        units = s.scalar(
+            select(func.coalesce(func.sum(QuotaEvent.units), 0)).where(
+                QuotaEvent.job_run_id == run_id
+            )
+        )
+        s.execute(
+            update(JobRun)
+            .where(JobRun.id == run_id, JobRun.status == RunStatus.RUNNING)
+            .values(
+                status=status,
+                finished_at=datetime.now(timezone.utc),
+                summary=summary,
+                result=result,
+                error=error,
+                failure=failure,
+                quota_units=units or 0,
+            )
+        )
+        s.commit()
+
+
+def interrupt_leftovers() -> int:
+    """Every row a previous process left `running` → `interrupted`, with what it spent until then
+    (its quota rows outlive it, as `finish` sums them). Called once at startup, before the scheduler
+    can open a new run. Returns how many."""
+    spent = (
+        select(func.coalesce(func.sum(QuotaEvent.units), 0))
+        .where(QuotaEvent.job_run_id == JobRun.id)
+        .scalar_subquery()
+    )
+    with SessionLocal() as s:
+        res = s.execute(
+            update(JobRun)
+            .where(JobRun.status == RunStatus.RUNNING)
+            .values(
+                status=RunStatus.INTERRUPTED,
+                finished_at=datetime.now(timezone.utc),
+                quota_units=spent,
+            )
+        )
+        s.commit()
+        return res.rowcount if isinstance(res, CursorResult) else 0
+
+
+def recent_runs(db: Session, jobs: list[str], per_job: int) -> dict[str, list[JobRun]]:
+    """Each of `jobs`' last `per_job` runs, newest first, by job id — one query, each job's part a
+    short walk of `ix_job_runs_job_started`, so a poll costs the same at any history length (a window
+    over the whole table grew with every run kept)."""
+    if not jobs:
+        return {}
+    newest = union_all(
+        *(
+            select(JobRun.id)
+            .where(JobRun.job == job)
+            .order_by(JobRun.started_at.desc(), JobRun.id.desc())
+            .limit(per_job)
+            for job in jobs
+        )
+    ).subquery()
+    rows = db.execute(
+        select(JobRun)
+        .where(JobRun.id.in_(select(newest.c.id)))
+        .order_by(JobRun.job, JobRun.started_at.desc(), JobRun.id.desc())
+    ).scalars()
+    out: dict[str, list[JobRun]] = {}
+    for r in rows:
+        out.setdefault(r.job, []).append(r)
+    return out
+
+
+def last_run(db: Session, job: str, target: str) -> JobRun | None:
+    """The newest run of an ad-hoc `job` on `target` (a user's own import), or None."""
+    return db.execute(
+        select(JobRun)
+        .where(JobRun.job == job, JobRun.target == target)
+        .order_by(JobRun.started_at.desc(), JobRun.id.desc())
+        .limit(1)
+    ).scalar_one_or_none()
+
+
+def gc(db: Session, retention_days: int) -> int:
+    """Delete finished runs older than `retention_days` (0 = keep forever). A run still going is
+    never reaped. The caller commits."""
+    if retention_days <= 0:
+        return 0
+    cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
+    res = db.execute(
+        delete(JobRun).where(JobRun.started_at < cutoff, JobRun.status != RunStatus.RUNNING)
+    )
+    return res.rowcount if isinstance(res, CursorResult) else 0

backend/app/logs.py

@@ -10,8 +10,8 @@ the logs were both unjoinable and leaky. `configure_logging()` replaces all four
 Event shape — the OpenTelemetry log data model, without the SDK; one JSON object per line:
     ts · level · sev · source · logger · event · msg · attrs
 plus whatever correlation the context carries (`request_id`, `user_id`, `debug_session_id`,
-`job_id`) and the app version. That shape is what the admin log console filters on and what a Gitea
-issue attachment will carry, so it is deliberately flat and stable.
+`job_id`, `run_id`) and the app version. That shape is what the admin log console filters on and
+what a Gitea issue attachment will carry, so it is deliberately flat and stable.
 
 Call sites stay ordinary stdlib logging:
 
@@ -81,15 +81,28 @@ _debug_session: contextvars.ContextVar[str | None] = contextvars.ContextVar(
     "log_debug_session", default=None
 )
 _job_id: contextvars.ContextVar[int | None] = contextvars.ContextVar("log_job_id", default=None)
+# The background run (app/jobruns.py) this context is doing; quota's spend is tagged with it too.
+_run_id: contextvars.ContextVar[int | None] = contextvars.ContextVar("log_run_id", default=None)
 
 _CONTEXT_VARS = {
     "request_id": _request_id,
     "user_id": _user_id,
     "debug_session_id": _debug_session,
     "job_id": _job_id,
+    "run_id": _run_id,
 }
 
 
+def current_request_id() -> str | None:
+    """The HTTP request this context is serving, if any — the request middleware sets it."""
+    return _request_id.get()
+
+
+def current_run_id() -> int | None:
+    """The background run this context is doing, if any — `bind(run_id=…)` sets it."""
+    return _run_id.get()
+
+
 class _RequestScope:
     """What the auth dependency learns about a request, for every line logged after it.
 

backend/app/logsink.py

@@ -362,6 +362,7 @@ def serialize(row: LogEvent) -> dict:
         "request_id": row.request_id,
         "user_id": row.user_id,
         "job_id": row.job_id,
+        "run_id": row.run_id,
         "debug_session_id": row.debug_session_id,
         "version": row.version,
     }

backend/app/main.py

@@ -28,6 +28,7 @@ log = logs.get_logger(__name__)
 from app import bodylimit, collation, logsink, state, sysconfig  # noqa: E402 — logging is configured first
 from app.config import settings  # noqa: E402 — logging is configured first
 from app.db import SessionLocal, engine  # noqa: E402 — logging is configured first
+import app.tables  # noqa: E402 — every table, before the first query
 from app.downloads import service as download_service  # noqa: E402 — logging is configured first
 from app.downloads import storage  # noqa: E402 — logging is configured first
 from app.plex import stream as plex_stream  # noqa: E402 — logging is configured first
@@ -52,12 +53,14 @@ from app.routes import (  # noqa: E402 — logging is configured first
     plex as plex_routes,
     public as public_routes,
     quota,
+    runs as runs_routes,
     saved_views,
     scheduler as scheduler_routes,
     search as search_routes,
     setup as setup_routes,
     sync,
     tags,
+    tokens as tokens_routes,
     version,
 )
 from app.scheduler import shutdown_scheduler, start_scheduler  # noqa: E402 — logging is configured first
@@ -148,7 +151,7 @@ async def _reap_hls_sessions() -> None:
     somebody pressed play again or the process restarted.
 
     A FINISHED session is reaped on the short rule here, not kept for the full idle window: an open
-    player checkpoints every 10 s and that route touches the session, so a paused viewer is no longer
+    player keeps checkpointing and that route touches the session, so a paused viewer is no longer
     invisible to this pass (see `reap_idle`). Keeping every finished session for ten minutes instead
     held one whole segmented copy of a film per item on the download volume."""
     while True:
@@ -337,12 +340,14 @@ app.include_router(plex_routes.router)
 app.include_router(downloads.router)
 app.include_router(downloads.admin_router)
 app.include_router(files_routes.router)
+app.include_router(tokens_routes.router)
 app.include_router(public_routes.router)
 app.include_router(admin.router)
 app.include_router(config_routes.router)
 app.include_router(scheduler_routes.router)
 app.include_router(audit_routes.router)
 app.include_router(admin_logs_routes.router)
+app.include_router(runs_routes.router)
 app.include_router(quota.router)
 app.include_router(version.router)
 app.include_router(setup_routes.router)
@@ -363,6 +368,10 @@ INDEX_HTML = STATIC_DIR / "index.html"
 # A public share link's own page: the player alone, without the app's bundle (frontend/watch.html).
 WATCH_HTML = STATIC_DIR / "watch.html"
 INDEX_HEADERS = {"Cache-Control": "no-cache"}
+# The paths a share link opens on, by what it opens (`links.share_path`). Any of them opens any link.
+# A new one also goes in frontend/public/robots.txt, the Vite dev entry and WatchPage's token parsing
+# (watchEntry.contract.test.ts).
+SHARE_PREFIXES = ("watch/", "file/", "folder/")
 app.mount(
     "/assets",
     StaticFiles(directory=STATIC_DIR / "assets", check_dir=False),
@@ -376,8 +385,10 @@ async def index() -> FileResponse:
 
 
 @app.get("/watch/{token}")
-async def watch_page(token: str):
-    """Serve the public player page for a share link, with per-video Open Graph tags injected so the
+@app.get("/file/{token}")
+@app.get("/folder/{token}")
+async def share_page(token: str):
+    """Serve the public page of a share link, with per-link Open Graph tags injected so the
     link unfurls richly (title/channel/thumbnail) in Messenger/social crawlers. A real browser
     ignores the extra tags and boots WatchPage; invalid/expired/password links fall back to the
     generic card."""
@@ -404,8 +415,8 @@ async def spa_fallback(full_path: str) -> FileResponse:
             # Real SPA-root assets (welcome images, favicon, robots.txt) rarely change and have
             # stable names, so a moderate cache is safe and satisfies the cache-lifetime audit.
             return FileResponse(candidate, headers={"Cache-Control": "public, max-age=604800"})
-    # A /watch/ path the route above does not match (a trailing segment, no token) is still the
+    # A share path the route above does not match (a trailing segment, no token) is still the
     # player's: WatchPage reads the token off the path and says when a link is gone.
-    if full_path.startswith("watch/"):
+    if full_path.startswith(SHARE_PREFIXES):
         return FileResponse(WATCH_HTML, headers=INDEX_HEADERS)
     return FileResponse(INDEX_HTML, headers=INDEX_HEADERS)

backend/app/notifications.py

@@ -9,6 +9,7 @@ table can't grow without bound for a heavy user.
 from sqlalchemy import delete, select
 from sqlalchemy.orm import Session
 
+from app.userscope import owned_by
 from app.models import Notification
 
 # Keep at most this many READ notifications per user; older read ones are pruned. Unread
@@ -48,11 +49,13 @@ def create_notification(
 
 def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
     """Delete a user's oldest READ notifications beyond `cap`. Returns the number removed.
-    Idempotent and cheap; safe to call after marking things read."""
+    Idempotent and cheap; safe to call after marking things read. Flushes first, so a notice the
+    caller has just marked read counts; the caller commits."""
+    db.flush()
     ids = (
         db.execute(
             select(Notification.id)
-            .where(Notification.user_id == user_id, Notification.read.is_(True))
+            .where(owned_by(Notification.user_id, user_id), Notification.read.is_(True))
             .order_by(Notification.created_at.desc())
             .offset(cap)
         )
@@ -63,5 +66,4 @@ def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
         return 0
     # One bulk DELETE rather than a get+delete per row (the ids are already in hand).
     db.execute(delete(Notification).where(Notification.id.in_(ids)))
-    db.commit()
     return len(ids)

backend/app/plex/affinity.py

@@ -0,0 +1,110 @@
+"""What a user's Plex history leans towards, as a weight for a surprise draw (`app.draw`).
+
+Two signals, both already in the database (no TMDB call):
+- **genres** of the titles the user watched, the recent ones weighing more (a half-life of
+  HALF_LIFE_DAYS), each scored 0..1 against the strongest;
+- the **TMDB recommendations** stored on those titles (`recommendations`), each scored by the most
+  recent title that recommends it.
+
+A title counts once however many of its episodes were watched — a binge of one show would otherwise
+outvote every film — and only the RECENT_TITLES latest titles count. A candidate's weight is
+1 + GENRE_WEIGHT × (the mean score of its genres) + REC_WEIGHT × (its recommendation score), so a
+title with no affinity keeps weight 1 and still comes up.
+"""
+
+import json
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+
+from sqlalchemy import Float, cast, func, literal, literal_column, select
+from sqlalchemy.dialects.postgresql import JSONB
+from sqlalchemy.orm import Session
+from sqlalchemy.sql.elements import ColumnElement
+
+from app import draw
+from app.plex.models import PlexItem, PlexShow, PlexState
+from app.userscope import owned_by
+
+HALF_LIFE_DAYS = 180
+RECENT_TITLES = 200
+GENRE_WEIGHT = 2.0
+REC_WEIGHT = 3.0
+
+
+@dataclass(frozen=True)
+class Affinity:
+    genres: dict[str, float] = field(default_factory=dict)
+    movie_recs: dict[str, float] = field(default_factory=dict)
+    show_recs: dict[str, float] = field(default_factory=dict)
+
+    def __bool__(self) -> bool:
+        return bool(self.genres or self.movie_recs or self.show_recs)
+
+    def weight(self, model: type[PlexItem] | type[PlexShow]) -> ColumnElement[float]:
+        """The draw weight of a `model` row (a movie or a show)."""
+        scores = cast(literal(json.dumps(self.genres)), JSONB)
+        recs = self.show_recs if model is PlexShow else self.movie_recs
+        genre = func.jsonb_array_elements_text(model.genres).table_valued("value").alias("g")
+        genre_aff = (
+            select(
+                func.coalesce(
+                    func.avg(func.coalesce(cast(scores.op("->>")(genre.c.value), Float), 0.0)),
+                    0.0,
+                )
+            )
+            .select_from(genre)
+            .scalar_subquery()
+        )
+        rec_aff = func.coalesce(
+            cast(cast(literal(json.dumps(recs)), JSONB).op("->>")(model.guid), Float), 0.0
+        )
+        return literal_column("1.0") + GENRE_WEIGHT * genre_aff + REC_WEIGHT * rec_aff
+
+
+def for_user(db: Session, uid: int) -> Affinity:
+    """The user's affinity from their latest watched titles; empty (`not aff`) with no history."""
+    # One key per title: a show for an episode, the item for a film. concat() of a NULL show_id is
+    # just "s", which nullif turns back into NULL so the film's key takes over.
+    title = func.coalesce(
+        func.nullif(func.concat("s", PlexItem.show_id), "s"), func.concat("m", PlexItem.id)
+    ).label("title")
+    latest = (
+        select(title, PlexState.watched_at, PlexItem.id.label("item_id"), PlexItem.show_id)
+        .join(PlexItem, PlexItem.id == PlexState.item_id)
+        .where(owned_by(PlexState.user_id, uid), PlexState.status == "watched")
+        .distinct(title)
+        .order_by(title, PlexState.watched_at.desc().nullslast())
+        .subquery()
+    )
+    rows = db.execute(
+        select(
+            latest.c.watched_at,
+            PlexItem.genres,
+            PlexItem.recommendations,
+            PlexShow.genres,
+            PlexShow.recommendations,
+        )
+        .select_from(latest)
+        .join(PlexItem, PlexItem.id == latest.c.item_id)
+        .outerjoin(PlexShow, PlexShow.id == latest.c.show_id)
+        .order_by(latest.c.watched_at.desc().nullslast())
+        .limit(RECENT_TITLES)
+    ).all()
+    if not rows:
+        return Affinity()
+
+    now = datetime.now(timezone.utc)
+    genres: dict[str, float] = {}
+    movie_recs: dict[str, float] = {}
+    show_recs: dict[str, float] = {}
+    for watched_at, item_genres, item_recs, show_genres, show_recs_json in rows:
+        r = draw.recency(watched_at, now, HALF_LIFE_DAYS)
+        for g in (show_genres if show_genres is not None else item_genres) or []:
+            genres[g] = genres.get(g, 0.0) + r
+        for rec in (show_recs_json if show_recs_json is not None else item_recs) or []:
+            if not rec.get("tmdb_id"):
+                continue
+            bucket = show_recs if rec.get("media_type") == "tv" else movie_recs
+            guid = f"tmdb:{rec['tmdb_id']}"
+            bucket[guid] = max(bucket.get(guid, 0.0), r)
+    return Affinity(draw.scaled(genres), draw.scaled(movie_recs), draw.scaled(show_recs))

backend/app/plex/cards.py

@@ -5,7 +5,7 @@ router. Pure functions — no DB, no request state."""
 
 from typing import NamedTuple
 
-from app.models import PlexItem, PlexShow, PlexState
+from app.plex.models import PlexItem, PlexShow, PlexState
 
 
 class ShowState(NamedTuple):

backend/app/plex/collections.py

@@ -0,0 +1,122 @@
+"""What a collection is to the app: where it comes from, whether it may be edited, its card, its
+effective content rating, and how its members order — shared by the collection pages, the info
+page's collection strips and the admin editor."""
+
+from collections.abc import Callable
+from datetime import date, datetime
+from typing import Any, NamedTuple
+
+from app.plex.models import PlexCollection
+
+# The most-restrictive → least ordering used to pick a collection's effective content rating from its
+# members (Plex shows one cert on the collection; we don't store it, so derive it).
+_CERT_RANK = {
+    "G": 1,
+    "TV-Y": 1,
+    "TV-Y7": 1,
+    "TV-G": 1,
+    "PG": 2,
+    "TV-PG": 2,
+    "PG-13": 3,
+    "TV-14": 3,
+    "R": 4,
+    "TV-MA": 4,
+    "NC-17": 5,
+}
+
+
+def collection_cert(certs: list[str | None]) -> str | None:
+    ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
+    return max(ranked, key=lambda x: x[1])[0] if ranked else None
+
+
+_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
+
+
+def collection_source(col: PlexCollection) -> str:
+    first = (col.title or "").strip().split(" ", 1)[0].lower()
+    if first in _STRIP_PROVIDERS:
+        return first
+    if col.smart:
+        return "smart"
+    return "collection"
+
+
+def collection_editable(c: PlexCollection) -> bool:
+    """Effective editability: the admin marked it editable AND it's a plain manual collection — never
+    smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
+    return bool(c.editable) and not c.smart and collection_source(c) == "collection"
+
+
+def collection_card(c: PlexCollection) -> dict:
+    return {
+        "id": c.rating_key,
+        "title": c.title,
+        "summary": c.summary,
+        "thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
+        "child_count": c.child_count,
+        "smart": c.smart,
+        "source": collection_source(c),
+        "editable": c.editable,
+        "can_edit": collection_editable(
+            c
+        ),  # whether the edit controls should be offered (admin UI)
+    }
+
+
+class Member(NamedTuple):
+    """One collection member flattened to everything the Sort control can order by. The page's two
+    legs are different tables (movies and shows), so there is no single query to ORDER BY — the
+    ordering is a pure list sort over these, which is also what makes it unit-testable."""
+
+    kind: str  # "movie" | "show"
+    obj: Any  # PlexItem | PlexShow, carried through to the card serializer
+    title: str  # lower-cased; never NULL (the column is not nullable)
+    year: int | None
+    rating: float | None
+    added: datetime | None
+    release: date | None
+    duration: int | None  # movies only — a show has no single runtime
+    last_watched: datetime | None
+
+
+# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
+# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
+# that silently does nothing here.
+MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
+    "added": lambda m: m.added,
+    "lastwatched": lambda m: m.last_watched,
+    "release": lambda m: m.release,
+    "year": lambda m: m.year,
+    "rating": lambda m: m.rating,
+    "duration": lambda m: m.duration,
+    "title": lambda m: m.title,
+}
+
+# What the page ordered by before it had a control, and still its default: a film series wants to be
+# read in release order, not by whenever Plex happened to ingest it.
+DEFAULT_MEMBER_SORT = "year"
+
+
+def sort_members(
+    members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
+) -> list[Member]:
+    """Order a collection's members by one of `PLEX_SORT_KEYS`.
+
+    Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
+    members with no value for the key sink to the END in BOTH directions (matching the browse grid's
+    `nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
+    title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
+    it. An unknown key falls back to the page's default instead of returning an arbitrary order.
+
+    `alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
+    compare as Python strings, which puts every accented initial after "z"."""
+    by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
+    key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
+    if key is MEMBER_SORT_KEYS["title"]:
+        key = by_title
+    ordered = sorted(members, key=by_title)  # stable secondary, applied first
+    have = [m for m in ordered if key(m) is not None]
+    missing = [m for m in ordered if key(m) is None]
+    have.sort(key=key, reverse=sort_dir != "asc")
+    return have + missing

backend/app/plex/detail.py

@@ -1,10 +1,11 @@
 """Parse the "info page" rich metadata out of a full Plex metadata dict, and persist the subset that
 the detail endpoints serve from the DB row.
 
-Shared by the live info-page path (routes.plex.browse) and the background rich re-sync (plex.sync) so
-the two never diverge — the same one-parser rule that keeps `people.py`'s cast handling consistent.
-`rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY fields onto a movie/show row
-(genres/directors/studio/content_rating already land from the cheap listing at `_apply_facet_fields`)."""
+Shared by the live info-page path (`plex.service.enrich_movie` / `enrich_show`) and the background
+rich re-sync (plex.sync) so the two never diverge — the same one-parser rule that keeps `people.py`'s
+cast handling consistent. `rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY
+fields onto a movie/show row (genres/directors/studio/content_rating already land from the cheap
+listing at `_apply_facet_fields`)."""
 
 from app.plex import people
 from app.plex.plaintext import plain_text

backend/app/plex/discover.py

@@ -0,0 +1,544 @@
+"""Plex Discover's reads: the rail-surface read-model.
+
+A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
+Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
+separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
+recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
+every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
+their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
+card paths); the HU overlay coalesces only in the main grid.
+
+The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
+Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
+grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
+the client hides). Genre-language matches the grid; duration is movie-only.
+"""
+
+from datetime import datetime
+
+from sqlalchemy import and_, func, or_, select
+from sqlalchemy.orm import Session
+
+from app.collation import sorted_words
+from app.discover.queries import RAIL_LIMIT
+from app.models import User
+from app import draw
+from app.plex import affinity, rollup
+from app.plex.cards import SHOW_NEW, display_title, leaf_card, movie_card, show_card
+from app.plex.models import PlexItem, PlexShow, PlexState
+from app.plex.query import (
+    LibraryFilters,
+    apply_meta_filters,
+    apply_show_state_filter,
+    content_libs,
+    genres_col,
+    meta_filter_conds,
+    top_art_keys,
+)
+from app.plex.tmdb import resolve_recs
+from app.userscope import owned_by
+
+GENRE_TILES = 12
+SURPRISE_PER_KIND = 3  # Surprise-me: at most this many movies + this many shows (6 total)
+
+
+def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
+    """The movie/show library ids visible for this scope (empty list = out of scope)."""
+    movie_ids, show_ids = content_libs(db)
+    return (
+        movie_ids if scope in ("movie", "both") else [],
+        show_ids if scope in ("show", "both") else [],
+    )
+
+
+def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
+    """Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
+    q = apply_meta_filters(q, PlexItem, p, lang)
+    if f.duration_min is not None:
+        q = q.where(PlexItem.duration_s >= f.duration_min)
+    if f.duration_max is not None:
+        q = q.where(PlexItem.duration_s <= f.duration_max)
+    return q
+
+
+def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
+    sts = rollup.item_state_map(db, uid, [it.id for it in items])
+    return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
+
+
+def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
+    statuses = rollup.state_map(db, uid, [s.id for s in shows])
+    return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
+
+
+def _recently_added(
+    db: Session,
+    uid: int,
+    movie_ids: list[int],
+    show_ids: list[int],
+    p: dict,
+    lang: str,
+    f: LibraryFilters,
+) -> list[dict]:
+    """Newest movies + shows, interleaved by `added_at` desc into one rail."""
+    movies = (
+        db.execute(
+            _movie_meta(
+                select(PlexItem).where(
+                    PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
+                ),
+                p,
+                lang,
+                f,
+            )
+            .order_by(PlexItem.added_at.desc().nullslast())
+            .limit(RAIL_LIMIT)
+        )
+        .scalars()
+        .all()
+        if movie_ids
+        else []
+    )
+    shows = (
+        db.execute(
+            apply_meta_filters(
+                select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
+            )
+            .order_by(PlexShow.added_at.desc().nullslast())
+            .limit(RAIL_LIMIT)
+        )
+        .scalars()
+        .all()
+        if show_ids
+        else []
+    )
+    # Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
+    # per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
+    # review found this one still on the older `(is not None, value)` form, i.e. one rule in two
+    # spellings, with the two comments explaining it differently.
+    tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
+    tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
+    top = tagged[:RAIL_LIMIT]
+    # Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
+    # share one), so the merge doesn't depend on object identity staying alive.
+    top_movies = [o for _, kind, o in top if kind == "movie"]
+    top_shows = [o for _, kind, o in top if kind == "show"]
+    mcards = {
+        m.rating_key: c
+        for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
+    }
+    scards = {
+        s.rating_key: c
+        for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
+    }
+    return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
+
+
+def _recently_watched(
+    db: Session,
+    uid: int,
+    movie_ids: list[int],
+    show_ids: list[int],
+    p: dict,
+    lang: str,
+    f: LibraryFilters,
+) -> list[dict]:
+    """Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
+    card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
+    sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
+    rows: list[tuple[datetime | None, dict]] = []  # (watched_at, card)
+    if movie_ids:
+        mq = (
+            select(PlexItem, PlexState)
+            .join(
+                PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
+            )
+            .where(
+                PlexItem.kind == "movie",
+                PlexItem.library_id.in_(movie_ids),
+                PlexState.status == "watched",
+            )
+        )
+        for it, st in db.execute(
+            _movie_meta(mq, p, lang, f)
+            .order_by(PlexState.watched_at.desc().nullslast())
+            .limit(RAIL_LIMIT)
+        ).all():
+            rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
+    if show_ids:
+        eq = (
+            select(PlexItem, PlexState, PlexShow)
+            .join(
+                PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
+            )
+            .join(PlexShow, PlexShow.id == PlexItem.show_id)
+            .where(
+                PlexItem.kind == "episode",
+                PlexItem.library_id.in_(show_ids),
+                PlexState.status == "watched",
+            )
+            .order_by(PlexState.watched_at.desc().nullslast())
+            .limit(RAIL_LIMIT)
+        )
+        for ep, st, sh in db.execute(eq).all():
+            card = leaf_card(
+                ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
+            )
+            rows.append((st.watched_at, card))
+    rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
+    return [card for _, card in rows[:RAIL_LIMIT]]
+
+
+def _continue_watching(
+    db: Session,
+    uid: int,
+    movie_ids: list[int],
+    show_ids: list[int],
+    p: dict,
+    lang: str,
+    f: LibraryFilters,
+) -> list[dict]:
+    """Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
+    A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
+    rows: list[tuple[datetime | None, dict]] = []  # (resumed_at, card)
+
+    if movie_ids:
+        mq = (
+            select(PlexItem, PlexState)
+            .join(
+                PlexState,
+                and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
+            )
+            .where(
+                PlexItem.kind == "movie",
+                PlexItem.library_id.in_(movie_ids),
+                PlexState.position_seconds > 0,
+                # A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
+                # `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
+                or_(
+                    PlexState.status.is_(None),
+                    PlexState.status.notin_(["watched", "hidden"]),
+                ),
+            )
+        )
+        mrows = db.execute(
+            _movie_meta(mq, p, lang, f)
+            .order_by(PlexState.progress_updated_at.desc().nullslast())
+            .limit(RAIL_LIMIT)
+        ).all()
+        for it, st in mrows:
+            rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
+
+    if show_ids:
+        # Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
+        # its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
+        # `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
+        prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
+        sq = apply_show_state_filter(
+            select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
+                prog, prog.c.show_id == PlexShow.id
+            ),
+            prog.c.total,
+            prog.c.watched,
+            prog.c.inprog,
+            "in_progress",
+        ).where(PlexShow.library_id.in_(show_ids))
+        srows = db.execute(
+            apply_meta_filters(sq, PlexShow, p, lang)
+            .order_by(prog.c.resumed_at.desc().nullslast())
+            .limit(RAIL_LIMIT)
+        ).all()
+        for sh, resumed_at, total, watched, inprog in srows:
+            state = rollup.show_state(total, watched, inprog)
+            rows.append((resumed_at, show_card(sh, state, lang=lang)))
+
+    rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
+    return [card for _, card in rows[:RAIL_LIMIT]]
+
+
+def _genre_tiles(
+    db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
+) -> list[dict]:
+    """The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
+    The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
+    The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
+    counts: dict[str, int] = {}
+    for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
+        if not ids:
+            continue
+        gcol = genres_col(model, lang)
+        base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
+            model.library_id.in_(ids),
+            func.jsonb_typeof(gcol) == "array",
+            *meta_filter_conds(model, p, lang),
+        )
+        if model is PlexItem:
+            base = base.where(model.kind == "movie")
+        expanded = base.subquery()
+        for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
+            counts[g] = counts.get(g, 0) + c
+    alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
+    top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
+    # A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
+    art = top_art_keys(
+        db,
+        PlexItem,
+        movie_ids,
+        genres_col(PlexItem, lang),
+        [g for g, _ in top],
+        extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
+    )
+    return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
+
+
+def _because_you_watched(
+    db: Session,
+    uid: int,
+    movie_ids: list[int],
+    show_ids: list[int],
+    p: dict,
+    lang: str,
+    f: LibraryFilters,
+) -> dict | None:
+    """Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
+    recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
+    None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
+    The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
+    if not movie_ids:
+        return None
+    anchor = (
+        db.execute(
+            select(PlexItem)
+            .join(
+                PlexState,
+                and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
+            )
+            .where(
+                PlexItem.kind == "movie",
+                PlexItem.library_id.in_(movie_ids),
+                PlexState.status == "watched",
+                PlexItem.recommendations.isnot(None),
+            )
+            .order_by(PlexState.watched_at.desc().nullslast())
+            .limit(1)
+        )
+        .scalars()
+        .first()
+    )
+    if anchor is None:
+        return None
+    resolved = resolve_recs(db, anchor.recommendations, lang=lang)
+    # Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
+    rks = [
+        r["rating_key"]
+        for r in resolved
+        if r.get("in_library") and r.get("rating_key") != anchor.rating_key
+    ]
+    if not rks:
+        return None
+    movies = {
+        m.rating_key: m
+        for m in db.execute(
+            _movie_meta(
+                select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
+                p,
+                lang,
+                f,
+            )
+        )
+        .scalars()
+        .all()
+    }
+    shows = {
+        s.rating_key: s
+        for s in db.execute(
+            apply_meta_filters(
+                select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
+            )
+        )
+        .scalars()
+        .all()
+    }
+    ordered_movies = [movies[rk] for rk in rks if rk in movies]
+    ordered_shows = [shows[rk] for rk in rks if rk in shows]
+    # rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
+    mcards = {
+        m.rating_key: c
+        for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
+    }
+    scards = {
+        s.rating_key: c
+        for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
+    }
+    items: list[dict] = []
+    for rk in rks:
+        if rk in mcards:
+            items.append(mcards[rk])
+        elif rk in scards:
+            items.append(scards[rk])
+    if not items:
+        return None
+    return {
+        "anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
+        "items": items,
+    }
+
+
+def discover_page(db: Session, user: User, scope: str, lang: str, f: LibraryFilters) -> dict:
+    """The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
+    sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
+    is null when there's no anchor. `lang` is already normalised."""
+    p = f.as_dict()
+    movie_ids, show_ids = _scope_libs(db, scope)
+    return {
+        "scope": scope,
+        "continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
+        "recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
+        "recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
+        "genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
+        "because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
+    }
+
+
+def _draw(db: Session, stmt, key_col, per: int, exclude: list[str], weight=None) -> list:
+    """`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
+    reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
+    short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail.
+    With a `weight` (`plex.affinity`) the draw leans on it, every third slot left blind (`app.draw`)."""
+
+    def pick(q, n: int) -> list:
+        blind = db.execute(q.order_by(func.random()).limit(n)).scalars().all()
+        if weight is None:
+            return list(blind)
+        key = draw.weighted_key(draw.random_u(), weight)
+        leaning = db.execute(q.order_by(key).limit(n)).scalars().all()
+        return draw.blend(list(leaning), list(blind), 1, n, key=lambda r: r.rating_key)
+
+    fresh = pick(stmt.where(key_col.not_in(exclude)), per) if exclude else []
+    if len(fresh) == per:
+        return fresh
+    taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
+    return [*fresh, *pick(stmt.where(*taken), per - len(fresh))]
+
+
+def _kept(
+    db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
+) -> list[dict]:
+    """The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
+    stays in it, marked watched, and one gone from the scope's libraries drops out."""
+    movies = (
+        db.execute(
+            select(PlexItem).where(
+                PlexItem.kind == "movie",
+                PlexItem.library_id.in_(movie_ids),
+                PlexItem.rating_key.in_(keep),
+            )
+        )
+        .scalars()
+        .all()
+        if movie_ids
+        else []
+    )
+    shows = (
+        db.execute(
+            select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
+        )
+        .scalars()
+        .all()
+        if show_ids
+        else []
+    )
+    cards = {
+        c["id"]: c
+        for c in [
+            *_movie_cards(db, uid, list(movies), lang),
+            *_show_cards(db, uid, list(shows), lang),
+        ]
+    }
+    return [cards[k] for k in keep if k in cards]
+
+
+def surprise(
+    db: Session,
+    user: User,
+    scope: str,
+    lang: str,
+    limit: int,
+    exclude: list[str],
+    keep: list[str],
+    f: LibraryFilters,
+) -> dict:
+    """A fresh random sample for the reshuffling Surprise-me rail (a random draw each call), narrowed by
+    the active filters. Only titles the user has NOT seen — a watched movie / a fully-watched show is
+    excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
+    SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
+    one type."""
+    p = f.as_dict()
+    movie_ids, show_ids = _scope_libs(db, scope)
+    if keep:
+        return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
+    per = min(limit, SURPRISE_PER_KIND)
+    # What the user's history leans towards; with none, every title weighs the same (a plain draw).
+    leaning = affinity.for_user(db, user.id)
+    movies = []
+    if movie_ids:
+        # Exclude movies this user has watched.
+        watched_movie = (
+            select(PlexState.id)
+            .where(
+                PlexState.item_id == PlexItem.id,
+                owned_by(PlexState.user_id, user),
+                PlexState.status == "watched",
+            )
+            .exists()
+        )
+        movies = _draw(
+            db,
+            _movie_meta(
+                select(PlexItem).where(
+                    PlexItem.kind == "movie",
+                    PlexItem.library_id.in_(movie_ids),
+                    ~watched_movie,
+                ),
+                p,
+                lang,
+                f,
+            ),
+            PlexItem.rating_key,
+            per,
+            exclude,
+            leaning.weight(PlexItem) if leaning else None,
+        )
+    shows = []
+    if show_ids:
+        # Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
+        # as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
+        prog = rollup.agg_subquery(db, user.id).subquery()
+        not_fully_watched = or_(
+            prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
+        )
+        shows = _draw(
+            db,
+            apply_meta_filters(
+                select(PlexShow)
+                .outerjoin(prog, prog.c.show_id == PlexShow.id)
+                .where(PlexShow.library_id.in_(show_ids), not_fully_watched),
+                PlexShow,
+                p,
+                lang,
+            ),
+            PlexShow.rating_key,
+            per,
+            exclude,
+            leaning.weight(PlexShow) if leaning else None,
+        )
+    mcards = _movie_cards(db, user.id, movies, lang)
+    scards = _show_cards(db, user.id, shows, lang)
+    # Interleave movie/show cards, then trim to `limit`.
+    items: list[dict] = []
+    for i in range(max(len(mcards), len(scards))):
+        if i < len(mcards):
+            items.append(mcards[i])
+        if i < len(scards):
+            items.append(scards[i])
+    return {"scope": scope, "items": items[:limit]}

backend/app/plex/imgcache.py

@@ -15,7 +15,7 @@ from fastapi import Response
 from sqlalchemy.orm import Session
 
 from app import logs
-from app.models import PlexCollection, PlexItem, PlexSeason, PlexShow
+from app.plex.models import PlexCollection, PlexItem, PlexSeason, PlexShow
 from app.plex.cache import CACHE_DIR
 from app.plex.lookups import by_rating_key
 

backend/app/plex/library.py

@@ -0,0 +1,473 @@
+"""The Plex browse grid's reads: the unified movies + shows feed, its search, and its facets.
+
+A user's watch state is joined through `userscope`; the feed and its facets share every filter
+(`plex.query`), so a facet count is the size of what the grid would show.
+"""
+
+from sqlalchemy import Boolean, Integer, String, and_, cast, func, literal, null, or_, select
+from sqlalchemy.orm import Session, aliased
+
+from app.collation import collated, sorted_words
+from app.models import User
+from app.plex import rollup
+from app.plex.cards import norm_lang
+from app.plex.models import PlexItem, PlexShow, PlexState
+from app.plex.query import (
+    FIELD_COLUMNS,
+    PLAYBACK_CLASSES,
+    TS_CONFIG,
+    LibraryFilters,
+    apply_meta_filters,
+    apply_movie_state_filter,
+    apply_show_state_filter,
+    content_libs,
+    field_recheck,
+    genres_col,
+    meta_filter_conds,
+    movie_status_case,
+    playable_conds,
+    playback_class_expr,
+    show_status_case,
+    split_csv,
+    summary_col,
+    title_col,
+    vector_col,
+    vector_match,
+)
+from app.userscope import owned_by
+from app.utils import to_tsquery_str
+
+
+def sort_column(u, sort: str, lang: str = "en"):
+    """The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
+    in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
+    offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
+    An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
+    in the reader's alphabet (`collation`)."""
+    return {
+        "added": u.c.added_at,
+        "title": collated(u.c.title, lang),
+        "year": u.c.year,
+        "rating": u.c.rating,
+        "duration": u.c.duration_s,
+        "release": u.c.rel,
+        "lastwatched": u.c.last_watched,
+    }.get(sort, u.c.added_at)
+
+
+def library_page(
+    db: Session,
+    user: User,
+    *,
+    scope: str,
+    q: str | None,
+    field: str,
+    sort: str,
+    sort_dir: str,
+    show: str,
+    lang: str,
+    offset: int,
+    limit: int,
+    f: LibraryFilters,
+) -> tuple[int, list, list]:
+    """One page of the unified cross-library browse: (total, rows, episodes). `rows` mixes movies
+    and shows (`kind` says which), scoped to movie|show|both, with the shared filters/search/watch
+    state (a show's state is the aggregate of its episodes). On a search that also matches episodes
+    (and shows are in scope), `episodes` holds the matching ones as (item, state, show) — the
+    grouped 'Episodes' section. `lang` is already normalised."""
+    p = f.as_dict()
+    movie_lib_ids, show_lib_ids = content_libs(db)
+    want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
+    want_shows = scope in ("show", "both") and bool(show_lib_ids)
+
+    ts = to_tsquery_str(q) if q else None
+    tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
+    if field not in FIELD_COLUMNS and field != "all":
+        field = "all"
+    # `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
+    # the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
+    grid_field = field != "episode"
+    recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
+
+    selects = []
+    if want_movies and grid_field:
+        st = aliased(PlexState)
+        m_status = movie_status_case(st)
+        mq = (
+            select(
+                PlexItem.rating_key.label("rk"),
+                literal("movie").label("kind"),
+                title_col(PlexItem, lang).label("title"),
+                PlexItem.year.label("year"),
+                PlexItem.rating.label("rating"),
+                # Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
+                func.left(summary_col(PlexItem, lang), 300).label("summary"),
+                PlexItem.added_at.label("added_at"),
+                PlexItem.originally_available_at.label("rel"),
+                PlexItem.duration_s.label("duration_s"),
+                PlexItem.playable.label("playable"),
+                PlexItem.transcode_hw.label("transcode_hw"),
+                cast(null(), Integer).label("season_count"),
+                func.coalesce(st.position_seconds, 0).label("position_seconds"),
+                m_status.label("status"),
+                literal(False).label("resumable"),
+                # "last watched" sort key: when the movie was last resumed, else when it was marked
+                # watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
+                func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
+                (
+                    func.ts_rank(vector_col(PlexItem, lang), tsq)
+                    if tsq is not None
+                    else literal(0.0)
+                ).label("rank"),
+            )
+            .outerjoin(st, and_(st.item_id == PlexItem.id, owned_by(st.user_id, user)))
+            .where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
+        )
+        mq = apply_movie_state_filter(mq, st, show)
+        mq = apply_meta_filters(mq, PlexItem, p, lang)
+        if f.duration_min is not None:
+            mq = mq.where(PlexItem.duration_s >= f.duration_min)
+        if f.duration_max is not None:
+            mq = mq.where(PlexItem.duration_s <= f.duration_max)
+        # Playback-class filter (movie-only — a show is an aggregate with no single class).
+        mq = mq.where(*playable_conds(split_csv(f.playable)))
+        if tsq is not None:
+            mq = mq.where(vector_match(PlexItem, lang, tsq))
+        if recheck is not None and (narrow := recheck(PlexItem, field, tsq, lang)) is not None:
+            mq = mq.where(narrow)
+        selects.append(mq)
+    if want_shows and grid_field:
+        agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
+        tot = func.coalesce(agg.c.total, 0)
+        wat = func.coalesce(agg.c.watched, 0)
+        inp = func.coalesce(agg.c.inprog, 0)
+        s_status = show_status_case(tot, wat, inp)
+        sq = (
+            select(
+                PlexShow.rating_key.label("rk"),
+                literal("show").label("kind"),
+                title_col(PlexShow, lang).label("title"),
+                PlexShow.year.label("year"),
+                PlexShow.rating.label("rating"),
+                func.left(summary_col(PlexShow, lang), 300).label("summary"),
+                PlexShow.added_at.label("added_at"),
+                PlexShow.originally_available_at.label("rel"),
+                cast(null(), Integer).label("duration_s"),
+                cast(null(), String).label("playable"),
+                cast(null(), Boolean).label("transcode_hw"),
+                PlexShow.child_count.label("season_count"),
+                literal(0).label("position_seconds"),
+                s_status.label("status"),
+                # An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
+                (inp > 0).label("resumable"),
+                # Show leg's "last watched": max(resume/watched time) across the show's episodes (see
+                # rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
+                agg.c.resumed_at.label("last_watched"),
+                (
+                    func.ts_rank(vector_col(PlexShow, lang), tsq)
+                    if tsq is not None
+                    else literal(0.0)
+                ).label("rank"),
+            )
+            .outerjoin(agg, agg.c.show_id == PlexShow.id)
+            .where(PlexShow.library_id.in_(show_lib_ids))
+        )
+        sq = apply_show_state_filter(sq, tot, wat, inp, show)
+        sq = apply_meta_filters(sq, PlexShow, p, lang)
+        if tsq is not None:
+            sq = sq.where(vector_match(PlexShow, lang, tsq))
+        if recheck is not None and (narrow := recheck(PlexShow, field, tsq, lang)) is not None:
+            sq = sq.where(narrow)
+        selects.append(sq)
+
+    total = 0
+    rows: list = []
+    if selects:
+        union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
+        u = union.subquery()
+        total = db.scalar(select(func.count()).select_from(u)) or 0
+
+        if tsq is not None:
+            order = [u.c.rank.desc()]
+        else:
+            col = sort_column(u, sort, lang)
+            order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
+        order.append(u.c.rk.desc())
+        rows = list(db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all())
+
+    # Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
+    # and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
+    # target the movie/show grid, so they suppress episodes.
+    eps: list = []
+    if tsq is not None and want_shows and field in ("all", "episode"):
+        ep_st = aliased(PlexState)
+        # In HU an episode is found by its Hungarian title too (the overlay fills it) — and
+        # still by its English one, like `vector_match` does for the grid.
+        if field == "episode":
+            ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
+            ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
+        else:
+            ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
+        hit = ep_vec.op("@@")(tsq)
+        if lang == "hu":
+            hit = or_(ep_vec_hu.op("@@")(tsq), hit)
+            ep_vec = ep_vec_hu
+        eps = list(
+            db.execute(
+                select(PlexItem, ep_st, PlexShow)
+                .outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, owned_by(ep_st.user_id, user)))
+                .outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
+                .where(
+                    PlexItem.kind == "episode",
+                    PlexItem.library_id.in_(show_lib_ids),
+                    hit,
+                )
+                .order_by(func.ts_rank(ep_vec, tsq).desc())
+                .limit(24)
+            ).all()
+        )
+
+    return total, rows, eps
+
+
+def facets(db: Session, user: User, *, scope: str, show: str, lang: str, f: LibraryFilters) -> dict:
+    """Available filter values for the unified library (scope movie|show|both), ADAPTED to the active
+    filters (faceted search): each group's values/bounds are computed with all the OTHER active
+    filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
+    a multi-select group still lets you add more. Merged across the scope's libraries; duration is
+    movie-only. (Search text is not factored in — facets track the sidebar filters.) `lang` is
+    normalised here, after the empty answer."""
+    empty = {
+        "genres": [],
+        "content_ratings": [],
+        "studios": [],
+        "playable": [],
+        "year_min": None,
+        "year_max": None,
+        "rating_max": None,
+        "duration_min": None,
+        "duration_max": None,
+    }
+    all_movie_ids, all_show_ids = content_libs(db)
+    movie_ids = all_movie_ids if scope in ("movie", "both") else []
+    show_ids = all_show_ids if scope in ("show", "both") else []
+    if not movie_ids and not show_ids:
+        return empty
+    lang = norm_lang(lang)
+    p = f.as_dict()
+
+    uid = user.id
+
+    # The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
+    # full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
+    # (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
+    # The same filter on the same rows, so the counts do not change
+    # (test_plex_facets_grid_parity).
+    shows_in_scope: list[int] | None = None
+    if show_ids and show in ("watched", "in_progress", "unwatched"):
+        agg = rollup.agg_subquery(db, uid).subquery()
+        in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
+        in_scope = apply_show_state_filter(
+            in_scope,
+            func.coalesce(agg.c.total, 0),
+            func.coalesce(agg.c.watched, 0),
+            func.coalesce(agg.c.inprog, 0),
+            show,
+        )
+        shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
+
+    def wsq(query, model, ids, pp, kind_movie, with_dur):
+        """Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
+        facet aggregation query, so the facets reflect exactly the visible result set."""
+        query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
+        if kind_movie:
+            query = query.where(model.kind == "movie")
+            if with_dur and pp.get("duration_min") is not None:
+                query = query.where(model.duration_s >= pp["duration_min"])
+            if with_dur and pp.get("duration_max") is not None:
+                query = query.where(model.duration_s <= pp["duration_max"])
+            # Playback-class filter (movie-only) — applied for the OTHER groups so an active class
+            # narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
+            query = query.where(*playable_conds(split_csv(pp.get("playable"))))
+            # Join watch-state and reuse the grid's exact movie state filter (unified_library) —
+            # hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
+            st = aliased(PlexState)
+            query = query.outerjoin(st, and_(st.item_id == model.id, owned_by(st.user_id, uid)))
+            query = apply_movie_state_filter(query, st, show)
+        elif shows_in_scope is not None:
+            query = query.where(model.id.in_(shows_in_scope))
+        return query
+
+    genre_counts: dict[str, int] = {}
+    cr_counts: dict[str, int] = {}
+    studio_counts: dict[str, int] = {}
+    playable_counts: dict[str, int] = {}
+    years: list[int] = []
+    ratings: list[float] = []
+    durs: list[int] = []
+
+    def per_table(model, ids, kind_movie: bool) -> None:
+        if not ids:
+            return
+        # Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
+        # OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
+        # offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
+        # the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
+        # every matching title carries them, so they stay togglable.)
+        genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
+        # Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
+        # `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
+        # jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
+        # so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
+        # whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
+        # In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
+        gcol = genres_col(model, lang)
+        expanded = (
+            wsq(
+                select(func.jsonb_array_elements_text(gcol).label("g")),
+                model,
+                ids,
+                genre_pp,
+                kind_movie,
+                True,
+            )
+            .where(func.jsonb_typeof(gcol) == "array")
+            .subquery()
+        )
+        for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
+            genre_counts[g] = genre_counts.get(g, 0) + c
+        # Content ratings — exclude its own.
+        crq = (
+            wsq(
+                select(model.content_rating, func.count()),
+                model,
+                ids,
+                {**p, "content_ratings": None},
+                kind_movie,
+                True,
+            )
+            .where(model.content_rating.isnot(None))
+            .group_by(model.content_rating)
+        )
+        for cr, c in db.execute(crq).all():
+            cr_counts[cr] = cr_counts.get(cr, 0) + c
+        # Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
+        stq = (
+            wsq(
+                select(model.studio, func.count()),
+                model,
+                ids,
+                {**p, "studios": None},
+                kind_movie,
+                True,
+            )
+            .where(model.studio.isnot(None))
+            .group_by(model.studio)
+        )
+        for s, c in db.execute(stq).all():
+            studio_counts[s] = studio_counts.get(s, 0) + c
+        # Playback class (movie-only) — exclude its own so each class stays an addable OR option.
+        if kind_movie:
+            pcls = playback_class_expr()
+            pcq = (
+                wsq(
+                    select(pcls.label("cls"), func.count()),
+                    model,
+                    ids,
+                    {**p, "playable": None},
+                    kind_movie,
+                    True,
+                )
+                .where(pcls.isnot(None))
+                .group_by(pcls)
+            )
+            for cls, c in db.execute(pcq).all():
+                playable_counts[cls] = playable_counts.get(cls, 0) + c
+        # Year bounds — exclude the year filter.
+        by = db.execute(
+            wsq(
+                select(func.min(model.year), func.max(model.year)),
+                model,
+                ids,
+                {**p, "year_min": None, "year_max": None},
+                kind_movie,
+                True,
+            )
+        ).one()
+        if by[0] is not None:
+            years.append(by[0])
+        if by[1] is not None:
+            years.append(by[1])
+        # Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
+        # rating filter and the UI badge use (IMDb preferred, audience fallback).
+        br = db.scalar(
+            wsq(
+                select(func.max(func.coalesce(model.imdb_rating, model.rating))),
+                model,
+                ids,
+                {**p, "rating_min": None},
+                kind_movie,
+                True,
+            )
+        )
+        if br is not None:
+            ratings.append(float(br))
+        # Duration bounds (movies) — exclude the duration filter.
+        if kind_movie:
+            bd = db.execute(
+                wsq(
+                    select(func.min(model.duration_s), func.max(model.duration_s)),
+                    model,
+                    ids,
+                    p,
+                    kind_movie,
+                    False,
+                )
+            ).one()
+            if bd[0] is not None:
+                durs.append(bd[0])
+            if bd[1] is not None:
+                durs.append(bd[1])
+
+    per_table(PlexItem, movie_ids, True)
+    per_table(PlexShow, show_ids, False)
+    # Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
+    # be absent): the sidebar renders genres solely from this list and hides the whole genre section —
+    # chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
+    # undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
+    for g in split_csv(f.genres):
+        genre_counts.setdefault(g, 0)
+    # Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
+    for s in split_csv(f.studios):
+        studio_counts.setdefault(s, 0)
+    # Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
+    # section when its facet list is empty, which would trap a zero-result selection with no way to undo).
+    for c in split_csv(f.playable):
+        if c in PLAYBACK_CLASSES:
+            playable_counts.setdefault(c, 0)
+    # Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
+    alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
+    return {
+        "genres": [
+            {"value": g, "count": c}
+            for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
+        ],
+        "content_ratings": [
+            {"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
+        ],
+        "studios": [
+            {"value": s, "count": c}
+            for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
+        ],
+        # Fixed good→heavy order (not by count), so the class list reads as a severity scale.
+        "playable": [
+            {"value": c, "count": playable_counts[c]}
+            for c in PLAYBACK_CLASSES
+            if c in playable_counts
+        ],
+        "year_min": min(years) if years else None,
+        "year_max": max(years) if years else None,
+        "rating_max": max(ratings) if ratings else None,
+        "duration_min": min(durs) if durs else None,
+        "duration_max": max(durs) if durs else None,
+    }

backend/app/plex/models.py

@@ -0,0 +1,464 @@
+"""Plex's tables: a self-contained catalog mirrored from a local Plex server — libraries, shows,
+seasons, items, people, collections, the server's playlists — and each user's private part of it:
+watch state, the account link, their own playlists. Only this unit writes them; another unit reads
+them along a declared edge, or asks this unit's service."""
+
+from datetime import datetime
+
+from sqlalchemy import (
+    BigInteger,
+    Boolean,
+    Computed,
+    Date,
+    DateTime,
+    Float,
+    ForeignKey,
+    Index,
+    Integer,
+    JSON,
+    String,
+    Text,
+    UniqueConstraint,
+)
+from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
+from sqlalchemy.orm import Mapped, mapped_column
+
+from app.db import Base
+from app.models import TimestampMixin, UpdatedAtMixin
+
+
+# --- Plex integration (optional) -------------------------------------------------------------
+# A dedicated, self-contained catalog mirrored from a locally-reachable Plex server. Playback is
+# from the LOCAL physical file (Plex is metadata only); per-user watch state lives in plex_states
+# so it works for users without a Plex account. Mirrors the app's "shared catalog, private
+# per-user state" model. The weighted FTS search_vector uses the same unaccent_simple config as
+# `videos` (migrations 0031/0032).
+
+
+class PlexLibrary(Base, TimestampMixin, UpdatedAtMixin):
+    """A mirrored Plex library section. Only `enabled` sections are exposed in the feed.
+    `plex_key` is the section id on the Plex server."""
+
+    __tablename__ = "plex_libraries"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    plex_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
+    title: Mapped[str] = mapped_column(String(255))
+    kind: Mapped[str] = mapped_column(String(16))  # "movie" | "show"
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
+    synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # HU-overlay support. `metadata_lang` tags the section's metadata language. An OVERLAY
+    # library (overlay_of set) is the Hungarian-metadata twin of another section: it is never browsable
+    # on its own — its titles' HU fields are written onto the matched EN rows (paired by `guid`).
+    metadata_lang: Mapped[str] = mapped_column(String(8), default="en", server_default="en")
+    overlay_of: Mapped[int | None] = mapped_column(
+        ForeignKey("plex_libraries.id", ondelete="SET NULL"), index=True
+    )
+
+
+class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
+    """A TV show (grandparent of episodes) — holds poster + summary for the drill-down page, plus
+    the same filterable metadata as movies (genres/rating/content_rating/studio/people) so the TV
+    grid can be filtered/sorted like the movie grid. All mirrored cheaply from the show listing."""
+
+    __tablename__ = "plex_shows"
+    __table_args__ = (
+        Index("ix_plex_shows_genres_gin", "genres", postgresql_using="gin"),
+        Index("ix_plex_shows_directors_gin", "directors", postgresql_using="gin"),
+        Index("ix_plex_shows_cast_gin", "cast_names", postgresql_using="gin"),
+        Index("ix_plex_shows_genres_hu_gin", "genres_hu", postgresql_using="gin"),
+        # ix_plex_shows_collections_gin already created with collections (migration 0047).
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
+    library_id: Mapped[int] = mapped_column(
+        ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
+    )
+    # Canonical external id (e.g. "tmdb:12345") — the key that pairs this EN row with its HU-overlay
+    # twin during sync. Indexed for the guid->row overlay match.
+    guid: Mapped[str | None] = mapped_column(String(64), index=True)
+    title: Mapped[str] = mapped_column(String(512))
+    summary: Mapped[str | None] = mapped_column(Text)
+    year: Mapped[int | None] = mapped_column(Integer)
+    thumb_key: Mapped[str | None] = mapped_column(
+        String(512)
+    )  # Plex thumb path (proxied, not stored)
+    art_key: Mapped[str | None] = mapped_column(String(512))
+    child_count: Mapped[int | None] = mapped_column(Integer)  # seasons
+    collection_keys: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # Plex collections this show is in
+    added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
+    # Filterable / orderable metadata mirrored from the show section listing (parallels PlexItem).
+    rating: Mapped[float | None] = mapped_column(Float, index=True)  # Plex audienceRating (~IMDb)
+    content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
+    studio: Mapped[str | None] = mapped_column(String(255), index=True)  # network/studio
+    originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)  # first air date
+    # none_as_null: an empty tag list must persist as SQL NULL, not JSON `null`. Without it a Python
+    # None round-trips as a JSON scalar that jsonb_array_elements_text() can't unnest ("cannot extract
+    # elements from a scalar") — the trap a metadata-poor library sprang on /facets. Same on plex_items.
+    genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    people_text: Mapped[str | None] = mapped_column(
+        Text
+    )  # cast+directors, folded into search_vector
+    # Set once the show's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
+    people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
+    # the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
+    imdb_rating: Mapped[float | None] = mapped_column(
+        Float
+    )  # IMDb-specific score (vs `rating` = audienceRating)
+    imdb_id: Mapped[str | None] = mapped_column(String(16))  # tt… → the clickable IMDb link
+    tagline: Mapped[str | None] = mapped_column(Text)
+    full_cast: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # ordered [{name, role, thumb}]
+    # --- TMDB enrichment: see the identical block on plex_items for the full rationale. Additive;
+    #     served from the row; NULL until enriched. (`budget`/`revenue` stay NULL for shows — TV has none.) ---
+    tmdb_id: Mapped[int | None] = mapped_column(Integer)
+    tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    tmdb_rating: Mapped[float | None] = mapped_column(Float)  # TMDb vote_average (0–10)
+    tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
+    budget: Mapped[int | None] = mapped_column(BigInteger)
+    revenue: Mapped[int | None] = mapped_column(BigInteger)
+    keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))  # [str]
+    crew: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # [{name, job, tmdb_id, thumb}]
+    studios: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # TMDB production companies [str]
+    production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))  # [str]
+    recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    backdrops: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    # Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched,
+    # so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
+    title_hu: Mapped[str | None] = mapped_column(String(512))
+    tagline_hu: Mapped[str | None] = mapped_column(Text)  # the HU overlay's tagline
+    summary_hu: Mapped[str | None] = mapped_column(Text)
+    genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    search_vector: Mapped[object | None] = mapped_column(
+        TSVECTOR,
+        Computed(
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
+            "setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
+            persisted=True,
+        ),
+    )
+    search_vector_hu: Mapped[object | None] = mapped_column(
+        TSVECTOR,
+        Computed(
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
+            "setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
+            persisted=True,
+        ),
+    )
+
+
+class PlexSeason(Base, TimestampMixin, UpdatedAtMixin):
+    """A season under a show — used to order episodes for prev/next stepping."""
+
+    __tablename__ = "plex_seasons"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
+    show_id: Mapped[int] = mapped_column(
+        ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
+    )
+    title: Mapped[str | None] = mapped_column(String(255))
+    season_number: Mapped[int | None] = mapped_column(Integer, index=True)
+    thumb_key: Mapped[str | None] = mapped_column(String(512))
+
+
+class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
+    """A playable LEAF — a movie or a single episode. This is what the feed lists and the player
+    plays. Movies carry `library_id` and no parents; episodes carry show/season refs +
+    season/episode numbers. `file_path` is the Plex-side absolute path of the physical media,
+    mapped through `plex_path_map` to a local mount at playback time. `playable` is the
+    ffprobe-derived browser-compatibility class (direct | remux | transcode)."""
+
+    __tablename__ = "plex_items"
+    __table_args__ = (
+        Index("ix_plex_items_show_order", "show_id", "season_number", "episode_number"),
+        Index("ix_plex_items_genres_gin", "genres", postgresql_using="gin"),
+        Index("ix_plex_items_directors_gin", "directors", postgresql_using="gin"),
+        Index("ix_plex_items_cast_gin", "cast_names", postgresql_using="gin"),
+        Index("ix_plex_items_collections_gin", "collection_keys", postgresql_using="gin"),
+        Index("ix_plex_items_genres_hu_gin", "genres_hu", postgresql_using="gin"),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
+    library_id: Mapped[int] = mapped_column(
+        ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
+    )
+    kind: Mapped[str] = mapped_column(String(16), index=True)  # "movie" | "episode"
+    # Canonical external id (e.g. "tmdb:12345") — pairs a MOVIE row with its HU-overlay twin during
+    # sync; indexed for the guid->row match. Episodes carry it too but it's unused for them.
+    guid: Mapped[str | None] = mapped_column(String(64), index=True)
+    show_id: Mapped[int | None] = mapped_column(
+        ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
+    )
+    season_id: Mapped[int | None] = mapped_column(
+        ForeignKey("plex_seasons.id", ondelete="CASCADE"), index=True
+    )
+    season_number: Mapped[int | None] = mapped_column(Integer)
+    episode_number: Mapped[int | None] = mapped_column(Integer)
+    title: Mapped[str] = mapped_column(String(512))
+    summary: Mapped[str | None] = mapped_column(Text)
+    year: Mapped[int | None] = mapped_column(Integer)
+    duration_s: Mapped[int | None] = mapped_column(Integer)
+    thumb_key: Mapped[str | None] = mapped_column(String(512))
+    art_key: Mapped[str | None] = mapped_column(String(512))
+    cast: Mapped[list | None] = mapped_column(JSON)  # [role names]
+    # Physical media (from the Plex Part).
+    file_path: Mapped[str | None] = mapped_column(Text)  # Plex-side absolute path
+    part_key: Mapped[str | None] = mapped_column(String(255))  # /library/parts/... (fallback fetch)
+    container: Mapped[str | None] = mapped_column(String(16))
+    codec_video: Mapped[str | None] = mapped_column(String(32))
+    codec_audio: Mapped[str | None] = mapped_column(String(32))
+    size_bytes: Mapped[int | None] = mapped_column(BigInteger)
+    # Browser-playability class from ffprobe: "direct" | "remux" | "transcode" (None = unprobed).
+    playable: Mapped[str | None] = mapped_column(String(12), index=True)
+    probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # Transcode HW-capability facet (the playback-class filter). NULL for non-transcode items; for a
+    # `playable="transcode"` item: True = the prod iGPU ENCODES it (a HW-decodable 8-bit SDR source, a
+    # true-HDR one through the OpenCL tonemap, or an 8-bit source in a codec the CPU decodes for it —
+    # VC-1, XviD, DivX), False = software libx264 (AV1, or a 10-bit SDR source — see plex/stream.py).
+    # Derived by `plex.sync.transcode_hw_capable`: set codec-only at sync (bit depth unknown from the
+    # cheap listing), then refined by the bit-depth backfill (`_probe_media_batch`), which downgrades a
+    # 10-bit SDR HEVC to False. The filterable twin of the rule `_select_encode` applies at play time;
+    # the two share their codec sets, not their code, so a change to one tree is a change to both.
+    transcode_hw: Mapped[bool | None] = mapped_column(Boolean, index=True)
+    # Video bit depth (8/10/12…) from the Plex media Stream, filled by the bit-depth backfill. NULL =
+    # not yet probed. Surfaced on the info page (e.g. "HEVC · 10-bit") and feeds `transcode_hw` (a
+    # 10-bit SDR source drops to software; an HDR one keeps the GPU through the tonemap).
+    video_bit_depth: Mapped[int | None] = mapped_column(Integer)
+    # Watermark for the bit-depth backfill: when this item's media Stream was last probed for
+    # bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
+    # probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
+    media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
+    # Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
+    # `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
+    # filters (GIN-indexed) and double as content signals for a future watch-habit recommender.
+    rating: Mapped[float | None] = mapped_column(Float, index=True)
+    content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
+    studio: Mapped[str | None] = mapped_column(String(255), index=True)
+    originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)
+    genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    # rating_keys of the Plex collections this movie belongs to (GIN — filter + card-strip siblings).
+    collection_keys: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    # Cast + director names, space-joined — folded into search_vector (weight B) so the search box
+    # finds titles by a person's name (and ranks them above summary-only mentions).
+    people_text: Mapped[str | None] = mapped_column(Text)
+    # Set once the movie's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
+    people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
+    # the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
+    imdb_rating: Mapped[float | None] = mapped_column(
+        Float
+    )  # IMDb-specific score (vs `rating` = audienceRating)
+    imdb_id: Mapped[str | None] = mapped_column(String(16))  # tt… → the clickable IMDb link
+    tagline: Mapped[str | None] = mapped_column(Text)
+    full_cast: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # ordered [{name, role, thumb}]
+    # --- TMDB enrichment: matched by the canonical `guid` ("tmdb:12345"), filled by the TMDB
+    #     backfill job, SERVED FROM THIS ROW (never a live TMDB call). Additive — imdb_rating/rating
+    #     above stay Plex-sourced; `tmdb_rating` is the distinct TMDb community score. NULL until enriched;
+    #     `tmdb_enriched_at` is the "tried" watermark (stamped on a definitive no-match too, so an
+    #     unresolvable title isn't re-fetched every run; left NULL only on a transient fetch error). ---
+    tmdb_id: Mapped[int | None] = mapped_column(Integer)
+    tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    tmdb_rating: Mapped[float | None] = mapped_column(Float)  # TMDb vote_average (0–10)
+    tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
+    budget: Mapped[int | None] = mapped_column(BigInteger)  # USD (movies; TV has none)
+    revenue: Mapped[int | None] = mapped_column(BigInteger)
+    keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))  # [str]
+    crew: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # [{name, job, tmdb_id, thumb}]
+    studios: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # TMDB production companies [str]
+    production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))  # [str]
+    # [{tmdb_id, title, poster, media_type}] — resolved to library rows (by guid) at serve time.
+    recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    backdrops: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # TMDB backdrop file_paths [str] (crossfade pool)
+    # Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched
+    # (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
+    title_hu: Mapped[str | None] = mapped_column(String(512))
+    tagline_hu: Mapped[str | None] = mapped_column(Text)  # the HU overlay's tagline
+    summary_hu: Mapped[str | None] = mapped_column(Text)
+    genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
+    search_vector: Mapped[object | None] = mapped_column(
+        TSVECTOR,
+        Computed(
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
+            "setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
+            persisted=True,
+        ),
+    )
+    search_vector_hu: Mapped[object | None] = mapped_column(
+        TSVECTOR,
+        Computed(
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
+            "setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
+            "setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
+            persisted=True,
+        ),
+    )
+
+
+class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
+    """A small cast/crew headshot cache, keyed by name, so person filter-chips (the cards for the
+    actors currently in the filter) can show a photo + count without a live per-card metadata fetch.
+    Populated opportunistically when an item's RICH metadata is fetched — the info-page view and the
+    periodic rich re-sync — since the cheap section listing that drives the main sync only carries the
+    top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
+    removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
+    metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
+    row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
+    (the headshot FILE lives in the image cache, filled by the people_enrich job)."""
+
+    __tablename__ = "plex_people"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    name: Mapped[str] = mapped_column(String(255), unique=True, index=True)
+    photo_thumb: Mapped[str | None] = mapped_column(Text)  # proxied /person-image URL, or None
+    # TMDB person id, linked by name from a title's TMDB credits (the /person
+    # biography fetch builds on it). NULL until a TMDB-enriched title names this person.
+    tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
+    # --- TMDB person enrichment: fetched from /person/{tmdb_person_id} by the people_enrich job,
+    #     or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
+    #     no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
+    #     media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
+    biography: Mapped[str | None] = mapped_column(Text)
+    # TMDB's Hungarian biography, from the same request's `translations`; a Hungarian reader is
+    # served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
+    # keywords have no Hungarian on TMDB, so they stay in the instance language.
+    biography_hu: Mapped[str | None] = mapped_column(Text)
+    birthday: Mapped[str | None] = mapped_column(String(10))  # "YYYY-MM-DD", served as-is
+    deathday: Mapped[str | None] = mapped_column(String(10))
+    place_of_birth: Mapped[str | None] = mapped_column(Text)
+    known_for_department: Mapped[str | None] = mapped_column(String(64))
+    also_known_as: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))  # [str]
+    known_for: Mapped[list | None] = mapped_column(
+        JSONB(none_as_null=True)
+    )  # [{tmdb_id, media_type, title}]
+    tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+
+class PlexCollection(Base, TimestampMixin, UpdatedAtMixin):
+    """A mirrored Plex collection (a grouping of movies or shows — franchises like Avatar, curated
+    sets like MCU, or auto/community lists like IMDb Top 250). Mirrored in full by the background
+    sync so reads never hit Plex's slow collection queries. Membership is stored as `collection_keys`
+    on the member movies (plex_items) / shows (plex_shows). `smart` = Plex's query-based flag;
+    `editable` = we may write changes back to Plex (default False = read-only; set for curated ones)."""
+
+    __tablename__ = "plex_collections"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
+    library_id: Mapped[int] = mapped_column(
+        ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
+    )
+    title: Mapped[str] = mapped_column(String(512))
+    summary: Mapped[str | None] = mapped_column(Text)
+    thumb_key: Mapped[str | None] = mapped_column(String(512))
+    child_count: Mapped[int | None] = mapped_column(Integer)
+    smart: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    editable: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+
+class PlexState(Base, UpdatedAtMixin):
+    """Per-user watch state for a Plex item — mirrors VideoState. Lives in Siftlode's own DB so
+    it works for users WITHOUT a Plex account (the whole point of the access-layer design).
+    `synced_to_plex` marks whether this state was pushed back to a linked Plex account."""
+
+    __tablename__ = "plex_states"
+    __table_args__ = (UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    item_id: Mapped[int] = mapped_column(
+        ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
+    )
+    status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
+    position_seconds: Mapped[int] = mapped_column(
+        Integer, default=0, server_default="0", nullable=False
+    )
+    watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    synced_to_plex: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+
+
+class PlexLink(Base, UpdatedAtMixin):
+    """Links a Siftlode user to a Plex account for two-way watch-state sync (one row per user).
+
+    For the owner (the MVP scope) the row uses the server's admin token — `uses_admin=True`,
+    `plex_token_enc=None` — so no separate Plex login is needed; the admin token already reads/writes
+    the owner account's watch state. `plex_account_id` is that Plex account's numeric id, used to
+    filter the shared watch-history feed to this user (Phase C). Reserved for later: a friend can link
+    their OWN Plex account via PIN OAuth, storing a personal `plex_token_enc` (uses_admin=False) — the
+    sync loop then iterates link rows uniformly.
+
+    `initial_import_done` guards the one-time "Plex is master" import; `last_watch_sync_at` is the
+    high-water mark for the incremental Plex→Siftlode pass."""
+
+    __tablename__ = "plex_link"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(
+        ForeignKey("users.id", ondelete="CASCADE"), unique=True, index=True
+    )
+    uses_admin: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
+    plex_token_enc: Mapped[str | None] = mapped_column(Text)  # P5b friend tokens (encrypted)
+    plex_account_id: Mapped[int | None] = mapped_column(Integer)
+    plex_username: Mapped[str | None] = mapped_column(String(255))
+    sync_enabled: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    initial_import_done: Mapped[bool] = mapped_column(
+        Boolean, default=False, server_default="false"
+    )
+    last_watch_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    linked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+
+class PlexPlaylist(Base, TimestampMixin, UpdatedAtMixin):
+    """A per-user ORDERED watch-list of Plex items — Siftlode-native (lives in our own DB, works for
+    users WITHOUT a Plex account, like plex_states). Unlike collections (shared, library-level), a
+    playlist is owned by one user and its items are ordered. Optional Plex-direction sync is a later
+    phase (`plex_rating_key` set once pushed back to the owner's Plex account)."""
+
+    __tablename__ = "plex_playlists"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+    title: Mapped[str] = mapped_column(String(512))
+    plex_rating_key: Mapped[str | None] = mapped_column(String(32))
+
+
+class PlexPlaylistItem(Base):
+    """One ordered entry in a playlist. `position` (0-based) orders playback."""
+
+    __tablename__ = "plex_playlist_items"
+    __table_args__ = (UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    playlist_id: Mapped[int] = mapped_column(ForeignKey("plex_playlists.id", ondelete="CASCADE"))
+    item_id: Mapped[int] = mapped_column(
+        ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
+    )
+    position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)

backend/app/plex/overlay.py

@@ -13,7 +13,7 @@ from sqlalchemy import select
 from sqlalchemy.orm import Session
 
 from app import logs, sysconfig
-from app.models import PlexItem, PlexLibrary, PlexShow
+from app.plex.models import PlexItem, PlexLibrary, PlexShow
 from app.plex import sync as plex_sync
 from app.plex.client import PlexClient
 from app.plex.genres import hu_genres

backend/app/plex/people.py

@@ -15,7 +15,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.orm import Session
 
 from app import logs, sysconfig
-from app.models import PlexPerson
+from app.plex.models import PlexPerson
 from app.plex import imgcache
 from app.plex.client import PlexClient, PlexError, PlexNotConfigured
 

backend/app/plex/queries.py

@@ -0,0 +1,507 @@
+"""Plex's reads for the HTTP layer: the mirrored rows a page shows, and a user's watch state on them.
+
+A user's own rows (`PlexState`) are read through `userscope`, so "someone else's row" is the same
+answer as a missing one. A rating key with a NUL in it names no row (`lookups.by_rating_key`).
+"""
+
+from typing import Literal
+
+from sqlalchemy import func, or_, select
+from sqlalchemy.orm import Session
+
+from app.collation import collated
+from app.models import User
+from app.plex import rollup
+from app.plex.lookups import by_rating_key
+from app.plex.models import (
+    PlexCollection,
+    PlexItem,
+    PlexLibrary,
+    PlexLink,
+    PlexPerson,
+    PlexPlaylist,
+    PlexPlaylistItem,
+    PlexSeason,
+    PlexShow,
+    PlexState,
+)
+from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
+from app.userscope import owned_by, owned_row
+from app.utils import like_escape
+
+# --- one row ---------------------------------------------------------------------------------
+
+
+def item(db: Session, rating_key: str) -> PlexItem | None:
+    return by_rating_key(db, PlexItem, rating_key)
+
+
+def movie(db: Session, rating_key: str) -> PlexItem | None:
+    """The movie with this key — an episode's key is not one."""
+    it = item(db, rating_key)
+    return it if it is not None and it.kind == "movie" else None
+
+
+def show(db: Session, rating_key: str) -> PlexShow | None:
+    return by_rating_key(db, PlexShow, rating_key)
+
+
+def collection(db: Session, rating_key: str) -> PlexCollection | None:
+    return by_rating_key(db, PlexCollection, rating_key)
+
+
+def season(db: Session, rating_key: str) -> PlexSeason | None:
+    return by_rating_key(db, PlexSeason, rating_key)
+
+
+def season_episodes(db: Session, se: PlexSeason) -> list[PlexItem]:
+    return list(
+        db.execute(
+            select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode")
+        ).scalars()
+    )
+
+
+def show_by_id(db: Session, show_id: int) -> PlexShow | None:
+    return db.get(PlexShow, show_id)
+
+
+def season_by_id(db: Session, season_id: int) -> PlexSeason | None:
+    return db.get(PlexSeason, season_id)
+
+
+def library(db: Session, library_id: int) -> PlexLibrary | None:
+    return db.get(PlexLibrary, library_id)
+
+
+def library_by_key(db: Session, plex_key: str) -> PlexLibrary | None:
+    return (
+        db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(plex_key)))
+        .scalars()
+        .first()
+    )
+
+
+def library_key(db: Session, library_id: int | None) -> str | None:
+    """The Plex section key for a library id — the info/show page passes it to the admin collection
+    editor (in the unified cross-library view there's no single library prop otherwise)."""
+    if library_id is None:
+        return None
+    lib = db.get(PlexLibrary, library_id)
+    return lib.plex_key if lib else None
+
+
+def person(db: Session, name: str) -> PlexPerson | None:
+    return db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
+
+
+# --- a user's watch state --------------------------------------------------------------------
+
+
+def state(db: Session, user: User | int, item_id: int) -> PlexState | None:
+    return (
+        db.execute(
+            select(PlexState).where(owned_by(PlexState.user_id, user), PlexState.item_id == item_id)
+        )
+        .scalars()
+        .first()
+    )
+
+
+# --- a show ----------------------------------------------------------------------------------
+
+
+def seasons(db: Session, sh: PlexShow) -> list[PlexSeason]:
+    return list(
+        db.execute(
+            select(PlexSeason)
+            .where(PlexSeason.show_id == sh.id)
+            .order_by(PlexSeason.season_number.asc().nullsfirst())
+        ).scalars()
+    )
+
+
+def episodes(db: Session, sh: PlexShow) -> list[PlexItem]:
+    """A show's episodes in season/episode order."""
+    return list(
+        db.execute(
+            select(PlexItem)
+            .where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
+            .order_by(
+                PlexItem.season_number.asc().nullsfirst(),
+                PlexItem.episode_number.asc().nullsfirst(),
+            )
+        ).scalars()
+    )
+
+
+def episode_keys(db: Session, show_id: int) -> list[str]:
+    """The rating keys of a show's episodes, in the order `episodes` gives."""
+    return list(
+        db.execute(
+            select(PlexItem.rating_key)
+            .where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
+            .order_by(
+                PlexItem.season_number.asc().nullsfirst(),
+                PlexItem.episode_number.asc().nullsfirst(),
+            )
+        ).scalars()
+    )
+
+
+def shows_by_keys(db: Session, rating_keys: list[str], exclude: int) -> dict[str, PlexShow]:
+    """The mirrored shows among these keys, but `exclude`, by key."""
+    return {
+        s.rating_key: s
+        for s in db.execute(
+            select(PlexShow).where(PlexShow.rating_key.in_(rating_keys), PlexShow.id != exclude)
+        ).scalars()
+    }
+
+
+# --- libraries -------------------------------------------------------------------------------
+
+
+def browsable_libraries(db: Session, lang: str) -> list[tuple[PlexLibrary, int]]:
+    """The mirrored libraries a user can browse, each with its title count (movies, or shows). An
+    HU overlay library is left out: it is never browsed, it only feeds its EN twin's Hungarian
+    metadata."""
+    libs = db.execute(
+        select(PlexLibrary)
+        .where(PlexLibrary.overlay_of.is_(None))
+        .order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang))
+    ).scalars()
+    out = []
+    for lib in libs:
+        if lib.kind == "movie":
+            count = db.scalar(
+                select(func.count(PlexItem.id)).where(
+                    PlexItem.library_id == lib.id, PlexItem.kind == "movie"
+                )
+            )
+        else:
+            count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
+        out.append((lib, count or 0))
+    return out
+
+
+# --- people ----------------------------------------------------------------------------------
+
+
+def person_photos(db: Session, names: list[str]) -> dict[str, str | None]:
+    return {
+        p.name: p.photo_thumb
+        for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(names))).scalars()
+    }
+
+
+def cast_count(db: Session, name: str, movie_lib_ids: list[int], show_lib_ids: list[int]) -> int:
+    """How many movies (in `movie_lib_ids`) and shows (in `show_lib_ids`) name this person in their
+    cast. An empty id list leaves its side out."""
+    count = 0
+    if movie_lib_ids:
+        count += (
+            db.scalar(
+                select(func.count(PlexItem.id)).where(
+                    PlexItem.kind == "movie",
+                    PlexItem.library_id.in_(movie_lib_ids),
+                    PlexItem.cast_names.contains([name]),
+                )
+            )
+            or 0
+        )
+    if show_lib_ids:
+        count += (
+            db.scalar(
+                select(func.count(PlexShow.id)).where(
+                    PlexShow.library_id.in_(show_lib_ids),
+                    PlexShow.cast_names.contains([name]),
+                )
+            )
+            or 0
+        )
+    return count
+
+
+def filmography(
+    db: Session,
+    kind: Literal["movie", "show"],
+    name: str,
+    lib_ids: list[int],
+    lang: str,
+    limit: int,
+) -> list:
+    """The movies or shows in these libraries whose cast or directors name this person, newest
+    first."""
+    model: type[PlexItem] | type[PlexShow] = PlexItem if kind == "movie" else PlexShow
+    query = select(model).where(
+        model.library_id.in_(lib_ids),
+        or_(model.cast_names.contains([name]), model.directors.contains([name])),
+    )
+    if model is PlexItem:
+        query = query.where(PlexItem.kind == "movie")
+    return list(
+        db.execute(
+            query.order_by(
+                model.year.desc().nullslast(), collated(title_col(model, lang), lang)
+            ).limit(limit)
+        ).scalars()
+    )
+
+
+# --- collections -----------------------------------------------------------------------------
+
+
+def collections_page(
+    db: Session, library: str | None, q: str | None, keys: list[str] | None, lang: str
+) -> list[PlexCollection]:
+    """Collections, most-populated first: one library's (by its plex key), else every enabled
+    library's; name-filtered by `q`; narrowed to `keys` when given (an empty list asks for none)."""
+    if library is not None:
+        lib = library_by_key(db, library)
+        if lib is None:
+            return []
+        query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
+    else:
+        enabled_ids = list(
+            db.execute(
+                select(PlexLibrary.id).where(
+                    PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
+                )
+            ).scalars()
+        )
+        if not enabled_ids:
+            return []
+        query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
+    term = (q or "").strip()
+    if term:
+        query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
+    if keys is not None:
+        # An explicit but empty `keys=` asks for nothing, which is not the same as no `keys` (every
+        # collection): answered here without a query. The empty IN () would also match nothing.
+        if not keys:
+            return []
+        query = query.where(PlexCollection.rating_key.in_(keys))
+    return list(
+        db.execute(
+            query.order_by(
+                PlexCollection.child_count.desc().nullslast(), collated(PlexCollection.title, lang)
+            )
+        ).scalars()
+    )
+
+
+def library_collections(db: Session, library_id: int, lang: str) -> list[PlexCollection]:
+    return list(
+        db.execute(
+            select(PlexCollection)
+            .where(PlexCollection.library_id == library_id)
+            .order_by(collated(PlexCollection.title, lang))
+        ).scalars()
+    )
+
+
+def collections_by_keys(db: Session, keys: list[str]) -> list[PlexCollection]:
+    return list(
+        db.execute(select(PlexCollection).where(PlexCollection.rating_key.in_(keys))).scalars()
+    )
+
+
+def collection_named(db: Session, library_id: int, title: str) -> PlexCollection | None:
+    """A collection of this library with this name, case aside."""
+    return (
+        db.execute(
+            select(PlexCollection).where(
+                PlexCollection.library_id == library_id,
+                func.lower(PlexCollection.title) == title.lower(),
+            )
+        )
+        .scalars()
+        .first()
+    )
+
+
+def collection_members(
+    db: Session, rating_key: str, lang: str
+) -> tuple[list[PlexItem], list[PlexShow]]:
+    """A collection's movies and shows, each by year then title."""
+    movies = db.execute(
+        select(PlexItem)
+        .where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
+        .order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
+    ).scalars()
+    shows = db.execute(
+        select(PlexShow)
+        .where(PlexShow.collection_keys.contains([rating_key]))
+        .order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
+    ).scalars()
+    return list(movies), list(shows)
+
+
+def library_movies_in(
+    db: Session, library_id: int, rating_key: str, lang: str, limit: int
+) -> list[PlexItem]:
+    """The movies of one library in this collection, by year then title."""
+    return list(
+        db.execute(
+            select(PlexItem)
+            .where(
+                PlexItem.library_id == library_id,
+                PlexItem.kind == "movie",
+                PlexItem.collection_keys.contains([rating_key]),
+            )
+            .order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
+            .limit(limit)
+        ).scalars()
+    )
+
+
+def show_rollups(db: Session, user_id: int, show_ids: list[int]) -> list:
+    """The user's episode rollup (total, watched, in progress, last resumed) of each of these shows
+    that has episodes."""
+    if not show_ids:
+        return []
+    return list(
+        db.execute(
+            rollup.agg_subquery(db, user_id, include_resumed=True).where(
+                PlexItem.show_id.in_(show_ids)
+            )
+        ).all()
+    )
+
+
+def collection_tiles(
+    db: Session, keys: list[str]
+) -> tuple[dict[str, list[str]], dict[str, list[str]], dict[str, int]]:
+    """What each collection's tile shows: the top two movie members' keys, the top two show members'
+    keys (a collection lives in one library, so exactly one side has any), and its movies' total
+    runtime (a show has no single duration)."""
+    movie_ids, show_ids = content_libs(db)
+    art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
+    art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
+    runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
+    return art_m, art_s, runtime
+
+
+# --- the admin page --------------------------------------------------------------------------
+
+
+def sample_file_path(db: Session) -> str | None:
+    """Any mirrored item's Plex-side file path — a sample to test the mount and path map against."""
+    return db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
+
+
+def watch_link(db: Session, user: User | int) -> PlexLink | None:
+    """The user's Plex watch-sync link, if they have one."""
+    return db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user))).scalars().first()
+
+
+# --- a user's playlists ----------------------------------------------------------------------
+
+
+def own_playlist(
+    db: Session, user: User, playlist_id: int, *, lock: bool = False
+) -> PlexPlaylist | None:
+    """The user's playlist with this id; someone else's is the same answer as a missing one. `lock`
+    takes it FOR UPDATE: every add, remove and reorder does, so two of them never read the same max
+    position, and the list row is always locked before its items."""
+    return owned_row(db, PlexPlaylist, playlist_id, user, lock=lock)
+
+
+def playlists_of(db: Session, user: User) -> list[PlexPlaylist]:
+    """The user's playlists, newest-touched first."""
+    return list(
+        db.execute(
+            select(PlexPlaylist)
+            .where(owned_by(PlexPlaylist.user_id, user))
+            .order_by(PlexPlaylist.updated_at.desc())
+        ).scalars()
+    )
+
+
+def playlist_count(db: Session, playlist_id: int) -> int:
+    return (
+        db.scalar(
+            select(func.count(PlexPlaylistItem.id)).where(
+                PlexPlaylistItem.playlist_id == playlist_id
+            )
+        )
+        or 0
+    )
+
+
+def _playlist_rows(playlist_id: int):
+    return (
+        select(PlexPlaylistItem, PlexItem)
+        .join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
+        .where(PlexPlaylistItem.playlist_id == playlist_id)
+        .order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
+    )
+
+
+def playlist_cover(db: Session, playlist_id: int) -> str | None:
+    """The rating key of a playlist's first item — its cover."""
+    first = db.execute(_playlist_rows(playlist_id)).first()
+    return first[1].rating_key if first else None
+
+
+def playlist_items(db: Session, playlist_id: int) -> list[PlexItem]:
+    """A playlist's items in play order."""
+    return [it for _row, it in db.execute(_playlist_rows(playlist_id)).all()]
+
+
+def playlist_holds(db: Session, playlist_id: int, item_ids: set[int]) -> int:
+    """How many of these items the playlist holds."""
+    if not item_ids:
+        return 0
+    return (
+        db.scalar(
+            select(func.count(PlexPlaylistItem.id)).where(
+                PlexPlaylistItem.playlist_id == playlist_id,
+                PlexPlaylistItem.item_id.in_(item_ids),
+            )
+        )
+        or 0
+    )
+
+
+def item_id(db: Session, rating_key: str) -> int | None:
+    """The id of the mirrored item with this key. Unlike `item`, a NUL in the key reaches the query,
+    which the app answers as a 422 (`main._nul_in_text`), as for every other compared string."""
+    return db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(rating_key)))
+
+
+def item_ids(db: Session, rating_keys: list[str]) -> set[int]:
+    """The ids of the mirrored items with these keys."""
+    if not rating_keys:
+        return set()
+    return set(
+        db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rating_keys))).scalars()
+    )
+
+
+def items_by_keys(db: Session, rating_keys: list) -> list[PlexItem]:
+    """The mirrored items with these keys, in the given order; an unknown key and a repeat drop out."""
+    order = [str(k) for k in rating_keys if str(k)]
+    if not order:
+        return []
+    by_rk = {
+        it.rating_key: it
+        for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
+    }
+    seen: set[str] = set()
+    out: list[PlexItem] = []
+    for rk in order:
+        it = by_rk.get(rk)
+        if it is not None and rk not in seen:
+            seen.add(rk)
+            out.append(it)
+    return out
+
+
+def shows_by_ids(db: Session, show_ids: set[int]) -> dict[int, PlexShow]:
+    if not show_ids:
+        return {}
+    return {
+        sh.id: sh for sh in db.execute(select(PlexShow).where(PlexShow.id.in_(show_ids))).scalars()
+    }

backend/app/plex/query.py

@@ -10,7 +10,7 @@ from sqlalchemy import and_, case, func, or_, select, type_coerce
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.orm import Session
 
-from app.models import PlexItem, PlexLibrary
+from app.plex.models import PlexItem, PlexLibrary
 from app.plex.genres import hu_genre
 
 TS_CONFIG = "public.unaccent_simple"
@@ -236,7 +236,7 @@ def apply_meta_filters(q, model, p: dict, lang: str = "en"):
 # The 4 classes the feed filter + facet expose, derived from the stored `playable` + `transcode_hw`.
 # ONE definition (this SQL CASE) so the filter WHERE and the facet counts can't drift: `playable_conds`
 # filters on it, `facets` groups by it. transcode_hw IS NOT True groups NULL (unprobed) with software.
-# Public — it is the shared source of truth imported by routes.plex.browse (facet order + zero-fill).
+# Public — it is the shared source of truth imported by plex.library (facet order + zero-fill).
 PLAYBACK_CLASSES = ("direct", "remux", "transcode_hw", "transcode_sw")
 
 

backend/app/plex/rollup.py

@@ -8,7 +8,9 @@ from datetime import datetime, timezone
 from sqlalchemy import and_, case, func, or_, select
 from sqlalchemy.orm import Session, aliased
 
-from app.models import PlexItem, PlexState
+from app.models import User
+from app.userscope import owned_by
+from app.plex.models import PlexItem, PlexState
 from app.plex.cards import ShowState
 
 # Sort sentinel for a rail row whose timestamp is missing, so `key=lambda r: r[0] or NO_TS` reads the
@@ -92,7 +94,7 @@ def agg_subquery(db: Session, user_id: int, include_resumed: bool = False):
         )
     return (
         select(*cols)
-        .outerjoin(ps, and_(ps.item_id == PlexItem.id, ps.user_id == user_id))
+        .outerjoin(ps, and_(ps.item_id == PlexItem.id, owned_by(ps.user_id, user_id)))
         .where(PlexItem.kind == "episode", PlexItem.show_id.isnot(None))
         .group_by(PlexItem.show_id)
     )
@@ -105,7 +107,7 @@ def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, ShowS
     return {r.show_id: show_state(r.total, r.watched, r.inprog) for r in rows}
 
 
-def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int, PlexState]:
+def item_state_map(db: Session, user_id: User | int, item_ids: list[int]) -> dict[int, PlexState]:
     """A user's PlexState rows for a set of leaf items (movies or episodes), keyed by item_id — the
     leaf analogue of `state_map` (which rolls a show's episodes up to a `ShowState`). One batched
     fetch, so the caller serializes cards without a per-card query. Empty in → empty dict (no `IN ()`)."""
@@ -114,6 +116,8 @@ def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int,
     return {
         s.item_id: s
         for s in db.execute(
-            select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id.in_(item_ids))
+            select(PlexState).where(
+                owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(item_ids)
+            )
         ).scalars()
     }

backend/app/plex/service.py

@@ -0,0 +1,434 @@
+"""Plex's writes for the HTTP layer, one function per use case. None of them commits: the handler
+does, so a page that fails to save an enrichment still renders, and a Plex write and its local
+mirror land together.
+
+The rest of the mirror's writes stay with the sync (`plex.sync`): a collection's targeted re-sync
+after a write-back, its local cleanup after a delete.
+"""
+
+from datetime import datetime, timedelta, timezone
+
+from sqlalchemy import CursorResult, delete, func, or_, select, update
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+from sqlalchemy.orm import Session
+
+from app import logs
+from app.db import SessionLocal
+from app.plex import detail as plex_detail
+from app.plex import people as plex_people
+from app.plex import tmdb as plex_tmdb
+from app.plex.lookups import by_rating_key
+from app.models import User
+from app.plex import queries, rollup
+from app.plex.models import (
+    PlexCollection,
+    PlexItem,
+    PlexLibrary,
+    PlexLink,
+    PlexPerson,
+    PlexPlaylist,
+    PlexPlaylistItem,
+    PlexShow,
+    PlexState,
+)
+from app.userscope import owned_by
+
+log = logs.get_logger(__name__)
+
+
+def enrich_show(db: Session, sh: PlexShow, meta: dict) -> None:
+    """Fill a not-yet-enriched show from its live metadata: the full cast (filterable), IMDb and
+    tagline, stamped so later views serve from the row."""
+    plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
+    plex_detail.apply_rich_detail(sh, meta)
+    sh.people_enriched_at = datetime.now(timezone.utc)
+
+
+def enrich_movie(db: Session, it: PlexItem, meta: dict) -> None:
+    """The same for a movie. The listing sync stored only its top ~3 cast; the live metadata
+    carries all of them."""
+    cast = plex_people.cast_from_meta(meta)
+    if cast:
+        plex_people.enrich_row_people(db, it, cast)
+    plex_detail.apply_rich_detail(it, meta)
+    it.people_enriched_at = datetime.now(timezone.utc)
+
+
+def set_collection_editable(col: PlexCollection, editable: bool) -> None:
+    col.editable = editable
+
+
+def keep_created_collection(
+    db: Session, lib: PlexLibrary, rating_key: str, title: str, seed: str
+) -> None:
+    """A collection Plex made, whose follow-up read failed: its row is kept, editable. The full sync
+    preserves `editable` but creates a missing row read-only, so the collection would drop out of
+    every "add to" list; and the duplicate-name check reads these rows, so a retry of the same name
+    would make a second one on Plex. Plex made it with exactly the seed in it, so the seed's row says
+    so too — otherwise the title showed unticked in the collection it was just added to, and the
+    collection empty, until that sync."""
+    db.add(
+        PlexCollection(
+            rating_key=rating_key, library_id=lib.id, title=title, editable=True, child_count=1
+        )
+    )
+    Model = PlexItem if lib.kind == "movie" else PlexShow
+    if row := by_rating_key(db, Model, seed):
+        row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), rating_key]))
+    db.flush()
+
+
+def enrich_person_bg(person_id: int) -> None:
+    """Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
+    AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
+    "no live TMDB call at serve time" invariant). The request's session is already closed by then, so
+    open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
+    next open, and the bio lands from the row by then."""
+    db = SessionLocal()
+    try:
+        person = db.get(PlexPerson, person_id)
+        if person is not None:
+            plex_tmdb.enrich_person(db, person)
+    except Exception:  # noqa: BLE001 — background best-effort; never surfaces to a user
+        db.rollback()
+        log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
+    finally:
+        db.close()
+
+
+def set_watch_link(db: Session, user: User, enabled: bool) -> PlexLink:
+    """Turn the user's two-way watch sync on or off, creating their link on first use. The link uses
+    the server admin token (`uses_admin`), so it needs no Plex login of its own."""
+    link = queries.watch_link(db, user)
+    if link is None:
+        link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
+        db.add(link)
+    link.sync_enabled = enabled
+    db.flush()
+    return link
+
+
+# --- a user's playlists ----------------------------------------------------------------------
+
+
+def create_playlist(db: Session, user: User, title: str, seed: PlexItem | None) -> PlexPlaylist:
+    """A new playlist of the user's, holding `seed` when one is given."""
+    pl = PlexPlaylist(user_id=user.id, title=title)
+    db.add(pl)
+    db.flush()
+    if seed is not None:
+        db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=seed.id, position=0))
+    db.flush()
+    return pl
+
+
+def rename_playlist(pl: PlexPlaylist, title: str) -> None:
+    pl.title = title
+
+
+def delete_playlist(db: Session, pl: PlexPlaylist) -> None:
+    db.delete(pl)
+    db.flush()
+
+
+def _touch(pl: PlexPlaylist) -> None:
+    pl.updated_at = datetime.now(timezone.utc)
+
+
+def _last_position(db: Session, playlist_id: int) -> int:
+    return db.execute(
+        select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
+            PlexPlaylistItem.playlist_id == playlist_id
+        )
+    ).scalar_one()
+
+
+def add_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
+    """Append `it` to the (locked) playlist; an item already in it stays where it is."""
+    exists = db.execute(
+        select(PlexPlaylistItem.id).where(
+            PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
+        )
+    ).first()
+    if exists is None:
+        pos = _last_position(db, pl.id) + 1
+        db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
+        _touch(pl)
+    db.flush()
+
+
+def remove_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
+    db.execute(
+        delete(PlexPlaylistItem).where(
+            PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
+        )
+    )
+    _touch(pl)
+    db.flush()
+
+
+def add_items(db: Session, pl: PlexPlaylist, items: list[PlexItem]) -> int:
+    """Append many items in their given order (a whole season or show); those already in the list
+    are skipped. Says how many were added."""
+    existing = set(
+        db.execute(
+            select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pl.id)
+        ).scalars()
+    )
+    pos = _last_position(db, pl.id)
+    added = 0
+    for it in items:
+        if it.id in existing:
+            continue
+        pos += 1
+        db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
+        existing.add(it.id)
+        added += 1
+    if added:
+        _touch(pl)
+    db.flush()
+    return added
+
+
+def insert_items_at(db: Session, pl: PlexPlaylist, placed: list[tuple[int, PlexItem]]) -> int:
+    """Insert new rows at their indices (ascending, so each index is where the row stood when the
+    rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
+    already in the list does not move: the add stays idempotent."""
+    rows = list(
+        db.execute(
+            select(PlexPlaylistItem)
+            .where(PlexPlaylistItem.playlist_id == pl.id)
+            .order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
+        ).scalars()
+    )
+    present = {r.item_id for r in rows}
+    added = 0
+    for at, it in sorted(placed, key=lambda p: p[0]):
+        if it.id in present:
+            continue
+        row = PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0)
+        db.add(row)
+        rows.insert(min(at, len(rows)), row)
+        present.add(it.id)
+        added += 1
+    for i, r in enumerate(rows):
+        r.position = i
+    if added:
+        _touch(pl)
+    db.flush()
+    return added
+
+
+def remove_items(db: Session, pl: PlexPlaylist, item_ids: list[int]) -> int:
+    """Remove these items from the playlist; says how many were there."""
+    if not item_ids:
+        return 0
+    result = db.execute(
+        delete(PlexPlaylistItem).where(
+            PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id.in_(item_ids)
+        ),
+        execution_options={"synchronize_session": False},
+    )
+    removed = result.rowcount if isinstance(result, CursorResult) else 0
+    if removed:
+        _touch(pl)
+    db.flush()
+    return removed
+
+
+def reorder_playlist(db: Session, pl: PlexPlaylist, rating_keys: list[str]) -> None:
+    """Set the play order from an explicit list of item keys (0-based positions)."""
+    id_by_rk = {
+        it.rating_key: it.id
+        for it in db.execute(
+            select(PlexItem).where(PlexItem.rating_key.in_(rating_keys or [""]))
+        ).scalars()
+    }
+    rows = {
+        r.item_id: r
+        for r in db.execute(
+            select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pl.id)
+        ).scalars()
+    }
+    pos = 0
+    seen: set[int] = set()
+    for rk in rating_keys:
+        row = rows.get(id_by_rk.get(rk, -1))
+        if row is not None and row.item_id not in seen:
+            row.position = pos
+            pos += 1
+            seen.add(row.item_id)
+    # Rows the payload omitted (added in another tab meanwhile) keep their relative order after
+    # the given ones, so no two rows share a position (the YouTube reorder does the same).
+    for row in sorted(
+        (r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
+    ):
+        row.position = pos
+        pos += 1
+    _touch(pl)
+    db.flush()
+
+
+# --- a user's watch state --------------------------------------------------------------------
+
+# How long after a watched mark a mid-item position is read as the finishing player's late tick
+# (`checkpoint`): the tick is every 10 s, plus the request's own latency and a retry.
+_LATE_CHECKPOINT_S = 30
+
+
+def mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
+    """Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
+    checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
+    read-then-write let both see "not watched" and both scrobble. The transition is decided by the
+    write itself: an insert that did not conflict, or an update conditional on the old status —
+    under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
+    values = {
+        "status": "watched",
+        "watched_at": now,
+        "position_seconds": 0,
+        "progress_updated_at": now,
+        "synced_to_plex": False,
+    }
+    inserted = db.execute(
+        pg_insert(PlexState)
+        .values(user_id=user_id, item_id=item_id, **values)
+        .on_conflict_do_nothing(constraint="uq_plex_user_item")
+        .returning(PlexState.id)
+    ).first()
+    if inserted is not None:
+        return True
+    mine = owned_by(PlexState.user_id, user_id) & (PlexState.item_id == item_id)
+    result = db.execute(update(PlexState).where(mine, PlexState.status != "watched").values(values))
+    if isinstance(result, CursorResult) and result.rowcount:
+        return True
+    # Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
+    db.execute(update(PlexState).where(mine).values(values))
+    return False
+
+
+def state_row(
+    db: Session,
+    user: User | int,
+    item_id: int,
+    st: PlexState | None = None,
+    *,
+    synced: bool = False,
+) -> PlexState:
+    """The user's state row on this item, made if the read before found none. Made by the insert
+    itself (ON CONFLICT DO NOTHING) and then read: the first checkpoint of an item and the `final`
+    flush beside it routinely both read "none", and a plain add let the second one fail on
+    `uq_plex_user_item` — a 500, and the push it was carrying lost. Every first row goes through
+    here: a checkpoint, a status, a season marked at once, a pull from Plex. `synced` is what a row
+    made now says about Plex; a row another writer made first keeps its own."""
+    if st is not None:
+        return st
+    db.execute(
+        pg_insert(PlexState)
+        .values(
+            user_id=user if isinstance(user, int) else user.id,
+            item_id=item_id,
+            synced_to_plex=synced,
+        )
+        .on_conflict_do_nothing(constraint="uq_plex_user_item")
+    )
+    made = queries.state(db, user, item_id)
+    assert made is not None  # inserted now, or committed by the request that won
+    return made
+
+
+def clear_progress(db: Session, st: PlexState | None, now: datetime) -> None:
+    """A checkpoint too early to keep: a state that held nothing else goes, any other forgets its
+    position."""
+    if st is not None and st.status == "new":
+        db.delete(st)
+    elif st is not None:
+        st.position_seconds = 0
+        st.progress_updated_at = now
+    db.flush()
+
+
+def checkpoint(
+    db: Session,
+    user: User,
+    item_id: int,
+    st: PlexState | None,
+    position: int,
+    now: datetime,
+    *,
+    dirty: bool,
+) -> bool:
+    """Record a resume position, and say whether it was recorded. `dirty` marks it not yet mirrored
+    to Plex — set before a push, so the reconcile re-pushes it if the push fails and the push can
+    settle it once Plex confirms.
+
+    Not onto a row marked watched in the last `_LATE_CHECKPOINT_S`: that position is the player's
+    tick sent just before its own finish (or a mark from another tab), and writing it made a watched
+    item in progress again — on Plex too, through the resume push it carried. Decided by the write
+    itself, since the mark can commit between any read here and the write. A rewatch is checkpointed
+    as before once the mark is that far behind it."""
+    st = state_row(db, user, item_id, st)
+    values: dict[str, object] = {"position_seconds": position, "progress_updated_at": now}
+    if dirty:
+        values["synced_to_plex"] = False
+    not_just_finished = or_(
+        PlexState.status != "watched",
+        PlexState.watched_at.is_(None),
+        PlexState.watched_at <= now - timedelta(seconds=_LATE_CHECKPOINT_S),
+    )
+    result = db.execute(
+        update(PlexState)
+        .where(PlexState.id == st.id, not_just_finished)
+        .values(values)
+        .execution_options(synchronize_session="fetch")
+    )
+    db.flush()
+    return isinstance(result, CursorResult) and result.rowcount > 0
+
+
+def forget_state(db: Session, st: PlexState) -> None:
+    db.delete(st)
+    db.flush()
+
+
+def set_status(
+    db: Session, user: User, item_id: int, st: PlexState | None, status: str
+) -> PlexState:
+    """Set a watched or hidden status; watched also clears the resume point."""
+    st = state_row(db, user, item_id, st)
+    st.status = status
+    if status == "watched":
+        st.watched_at = datetime.now(timezone.utc)
+        st.position_seconds = 0
+    st.synced_to_plex = False
+    db.flush()
+    return st
+
+
+def bulk_state(
+    db: Session, user_id: int, eps: list[PlexItem], watched: bool
+) -> list[tuple[int, str, str]]:
+    """Mark every episode in `eps` watched (or unwatched) for a user and say what changed, as the
+    (item id, rating key, action) pushes a linked Plex account needs. Unwatching keeps the user's
+    private "hidden" marks."""
+    now = datetime.now(timezone.utc)
+    existing = rollup.item_state_map(db, user_id, [e.id for e in eps])
+    to_push: list[tuple[int, str, str]] = []
+    for e in eps:
+        st = existing.get(e.id)
+        prev = st.status if st is not None else "new"
+        if watched:
+            st = state_row(db, user_id, e.id, st)
+            prev = st.status  # a first checkpoint beside this may have made it meanwhile
+            st.status = "watched"
+            st.watched_at = now
+            st.position_seconds = 0
+            st.synced_to_plex = False
+            if prev != "watched":
+                to_push.append((e.id, e.rating_key, "watched"))
+        else:  # → new: drop any state (but keep the user's private "hidden" marks)
+            if st is not None and prev != "hidden":
+                db.delete(st)
+                if prev != "new":
+                    to_push.append((e.id, e.rating_key, "unwatched"))
+    db.flush()
+    return to_push

backend/app/plex/stream.py

@@ -4,9 +4,10 @@ Playback is from the LOCAL physical file. Browser-compatible files (playable="di
 raw with HTTP range requests. Everything else that is h264 video (playable="remux") is remuxed on
 the fly to HLS with **video stream-copy** (cheap, I/O-bound — no video re-encode) and audio to AAC
 when needed. Non-h264 video (playable="transcode": HEVC/VP9/AV1/MPEG-2/XviD…) needs a full re-encode
-to H.264: on the prod host that runs on the Intel iGPU via VAAPI (`h264_vaapi`, near-zero CPU), with
-software libx264 as the automatic fallback where the render node is absent or the source codec isn't
-hardware-decodable. Same HLS session/seek machinery either way — only the video encoder differs.
+to H.264: on the prod host that is encoded on the Intel iGPU via VAAPI (`h264_vaapi`) — decoded there
+too when it can, else decoded on the CPU and uploaded (VC-1, XviD, DivX) — with software libx264 as
+the fallback where the render node is absent, for a source neither GPU route takes (AV1, 10-bit SDR),
+and when a GPU session produces nothing. Same HLS session/seek machinery either way.
 
 Seek model (Jellyfin-style): one ffmpeg session per item, started at a given offset via `-ss`
 (fast keyframe seek). A seek beyond the generated region restarts the session at the new offset,
@@ -33,9 +34,9 @@ from sqlalchemy.orm import Session as DbSession
 from app import logs, sysconfig
 from app.config import settings
 from app.downloads.edit import encode_thread_cap
-from app.models import PlexItem
+from app.plex.models import PlexItem
 from app.plex import paths
-from app.plex.sync import HDR_TRANSFERS, HW_DECODABLE_CODECS
+from app.plex.sync import CPU_DECODE_CODECS, HDR_TRANSFERS, HW_DECODABLE_CODECS
 
 log = logs.get_logger(__name__)
 _HLS_ROOT = (
@@ -66,7 +67,9 @@ _SESSION_IDLE_S = 600  # reap a session with no access for this long
 # restart leaves them behind like any other. `sweep_orphans` deletes ONLY names
 # matching one of those two shapes: `_HLS_ROOT` is admin-configurable (PLEX_HLS_DIR), so it may
 # legitimately point at a shared scratch path we do not own, and a blanket "delete every directory
-# under the root" would take the neighbours with it.
+# under the root" would take the neighbours with it. A torn-down session's directory is renamed to
+# its own name + `_TOMB` + 8 hex digits before it is deleted (see `_detach`), so a crash between the
+# two leaves a name this sweep still recognises.
 #
 # The KEY is anchored to digits (a Plex rating_key is a number) — that is what keeps foreign names
 # out. A `.+` there let `snapshot_2024_backup_+1.00` through, and pairing `.+` with an optional tail
@@ -74,8 +77,10 @@ _SESSION_IDLE_S = 600  # reap a session with no access for this long
 # non-numeric rating_key would merely leave its directory unswept, which is the safe direction to
 # fail. Covered by test_plex_stream.py — this regex is the only thing standing between the sweep
 # and someone else's data.
+_TOMB = ".dead-"
 _SESSION_DIR_RE = re.compile(
-    r"^\d+_\d+(_[^_]+_[+-]\d+\.\d{2})?$|^\d+_u\d+_\d+_[^_]+_[+-]\d+\.\d{2}$"
+    r"^\d+_\d+(_[^_]+_[+-]\d+\.\d{2})?$"
+    rf"|^\d+_u\d+_\d+_[^_]+_[+-]\d+\.\d{{2}}({re.escape(_TOMB)}[0-9a-f]{{8}})?$"
 )
 # When this process started. `sweep_orphans` only deletes directories older than this: everything
 # newer belongs to a session THIS run created, and since `start_session` reuses a directory path
@@ -93,7 +98,7 @@ _MTIME_GRACE_S = 2.0
 
 _lock = threading.Lock()
 # Keyed by VIEWER and item, (user_id, rating_key). Keyed by the item alone, a second account opening
-# the same film `_kill`ed the first one's session: the displaced viewer got a 404, reclaimed at its
+# the same film tore down the first one's session: the displaced viewer got a 404, reclaimed at its
 # playhead, and so evicted the other in turn — two people could not watch one film at once. (One
 # account on two devices still shares a session; that case ends in the player's bounded reclaim.)
 _sessions: dict[tuple[int, str], "HlsSession"] = {}
@@ -116,12 +121,13 @@ class HlsSession:
         hw: bool = False,
         tonemap: bool = False,
         user_id: int = 0,
+        cpu_decode: bool = False,
     ):
         # Unguessable per-session id, carried in the URLs the client is handed. A seek-restart
         # replaces the viewer's registered session for the item — and the old player's next request
         # for `seg_12.ts`, resolved by (viewer, rating_key) alone, was answered 200 with the NEW session's twelfth segment, produced from a different offset
         # under `-copyts`. Wrong bytes, not an error, so the client's reclaim never fired on it.
-        # A request naming a session that is no longer registered is a 404,
+        # A request naming a session that is no longer registered is a 410,
         # which the player recovers from by restarting at the playhead.
         self.id = secrets.token_urlsafe(9)
         self.key = key  # the item's rating_key (what the logs name)
@@ -131,6 +137,7 @@ class HlsSession:
         self.transcode = transcode  # video re-encoded (non-h264 source), not stream-copied
         self.hw = hw  # transcode ran on the VAAPI hardware encoder (else libx264 software)
         self.tonemap = tonemap  # HW HDR/10-bit path: OpenCL tonemap between VAAPI decode and encode
+        self.cpu_decode = cpu_decode  # HW encode of frames the CPU decoded (a codec the GPU can't)
         self.start_s = start_s  # the REQUESTED seek offset
         # The REAL media start: with `-ss X -c:v copy` ffmpeg can only cut at the nearest earlier
         # keyframe K<=X, and hls.js zero-bases the timeline to K (so video.currentTime=0 is K, not X).
@@ -184,8 +191,11 @@ _HLS_TAIL = [
 #   * VAAPI hardware (`h264_vaapi`) — the prod default once the iGPU is passed in. A full GPU pipeline
 #     (decode + scale + encode all on-device) offloads the CPU almost entirely, so several concurrent
 #     1080p transcodes are fine and the API stays responsive.
+#   * CPU decode → VAAPI encode (`format=nv12,hwupload` + `h264_vaapi`) — a codec the GPU cannot decode
+#     but whose 8-bit picture it encodes (`CPU_DECODE_CODECS`: VC-1, XviD, DivX). The encode is the
+#     expensive half, so this still takes most of the load off the CPU.
 #   * libx264 software — the automatic fallback where `/dev/dri` is absent (dev box, or prod before the
-#     iGPU enablement) or the source codec isn't hardware-decodable (AV1/XviD on Gen 9.5). CPU-bound, so
+#     iGPU enablement) or the source fits neither GPU route (AV1, a 10-bit SDR source). CPU-bound, so
 #     it reserves a core via `-threads` (see encode_thread_cap).
 # Everything else — the `-copyts`/`_HLS_TAIL`/segment/seek-restart model and the AAC audio path — is
 # unchanged, so hls.js, subtitles, multi-audio and seeking keep working exactly as for remux.
@@ -194,10 +204,10 @@ _VAAPI_DEVICE = "/dev/dri/renderD128"
 # here; its presence is the precondition for the HW HDR→SDR tonemap graph (see _opencl_present).
 _OPENCL_ICD_DIR = Path("/etc/OpenCL/vendors")
 # Codecs the prod iGPU (UHD 630 / Quick Sync Gen 9.5) can hardware-DECODE — the precondition for the
-# full-GPU VAAPI pipeline. A source outside it (AV1, MPEG-4 ASP/XviD…) falls back to software even with
-# the render node present, because the GPU can't decode it to feed its own encoder. THE single source is
-# plex.sync, so the runtime encoder choice here and the catalog `transcode_hw` facet can't disagree by
-# codec; aliased to the old name to keep the call sites below unchanged.
+# full-GPU VAAPI pipeline. A source outside it is decoded on the CPU when its codec is in
+# `CPU_DECODE_CODECS` (the GPU still encodes), else transcoded in software. THE single source of both
+# sets is plex.sync, so the runtime encoder choice here and the catalog `transcode_hw` facet can't
+# disagree by codec; aliased to the old name to keep the call sites below unchanged.
 _HW_DECODABLE = HW_DECODABLE_CODECS
 # Color-transfer values that mark a true-HDR source (PQ / HLG). A HW-decodable HDR source can't take the
 # plain scale_vaapi=format=nv12 path (Gen 9.5 VPP refuses it) — it needs the OpenCL tonemap graph
@@ -298,16 +308,22 @@ def _opencl_present() -> bool:
         return False
 
 
-def _hwaccel_in_opts(hw: bool, tonemap: bool = False) -> list[str]:
+def _hwaccel_in_opts(hw: bool, tonemap: bool = False, cpu_decode: bool = False) -> list[str]:
     """VAAPI decode options, placed before `-i` so the source is decoded (and kept) on the GPU.
     `-hwaccel_output_format vaapi` keeps frames as GPU surfaces all the way to `h264_vaapi` — no
     round-trip to system memory. Empty for the software path.
 
     The tonemap variant additionally derives an OpenCL device FROM the VAAPI one (`opencl=ocl@va`) so
     tonemap_opencl can map the decoded surfaces zero-copy (see _venc_opts); it needs named devices, so it
-    uses `-init_hw_device` + `-hwaccel_device` rather than the plain path's implicit `-vaapi_device`."""
+    uses `-init_hw_device` + `-hwaccel_device` rather than the plain path's implicit `-vaapi_device`.
+
+    The CPU-decode variant opens the VAAPI device for the FILTER graph only (`hwupload` needs one) and
+    asks for no `-hwaccel`: the codec is one the GPU refuses to decode. This exact form is the one
+    measured on prod with the HLS muxer and `-copyts` (and to the end of the file)."""
     if not hw:
         return []
+    if cpu_decode:
+        return ["-init_hw_device", f"vaapi=va:{_VAAPI_DEVICE}", "-filter_hw_device", "va"]
     if tonemap:
         return [
             "-init_hw_device",
@@ -355,6 +371,7 @@ def _venc_opts(
     preset: str = "veryfast",
     crf: int = 20,
     max_height: int = 0,
+    cpu_decode: bool = False,
 ) -> list[str]:
     """The video-encoder args that replace `-c:v copy` for a transcode session (returns the copy args
     when not transcoding). A re-encode must MINT its own keyframes at the segment boundaries — the copy
@@ -381,6 +398,10 @@ def _venc_opts(
             "h264_vaapi",
             *force_kf,
         ]
+    if hw and cpu_decode:
+        # The CPU decoded the frames: convert them to the encoder's 8-bit nv12 and upload them to the
+        # device `-filter_hw_device` named (see _hwaccel_in_opts).
+        return ["-vf", "format=nv12,hwupload", "-c:v", "h264_vaapi", *force_kf]
     if hw:
         # scale_vaapi with format=nv12 downconverts a 10-bit (p010) HEVC source to 8-bit nv12 so
         # h264_vaapi (8-bit High) can encode it; for an 8-bit source it's a cheap format pass-through.
@@ -418,13 +439,18 @@ def _render_node_present() -> bool:
     return Path(_VAAPI_DEVICE).exists()
 
 
-def _select_encode(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> tuple[bool, bool]:
-    """Pick the transcode video pipeline for this session — returns `(hw, tonemap)`:
-      * `(False, False)` → software libx264 (the fallback: no render node, non-decodable source, or a
-        10-bit/HDR source with no OpenCL runtime to tonemap on the GPU).
-      * `(True, False)`  → full-GPU VAAPI (`scale_vaapi=format=nv12` + `h264_vaapi`) — an 8-bit SDR source.
-      * `(True, True)`   → VAAPI decode → OpenCL tonemap → VAAPI encode — a true-HDR (PQ/HLG) source, when
-        the OpenCL runtime is present (Gen 9.5 VPP can't tonemap natively).
+def _select_encode(
+    db: DbSession, item: PlexItem, transcode: bool, src: Path
+) -> tuple[bool, bool, bool]:
+    """Pick the transcode video pipeline for this session — returns `(hw, tonemap, cpu_decode)`:
+      * `(False, False, False)` → software libx264 (the fallback: no render node, a source neither GPU
+        route takes, or a 10-bit/HDR source with no OpenCL runtime to tonemap on the GPU).
+      * `(True, False, False)`  → full-GPU VAAPI (`scale_vaapi=format=nv12` + `h264_vaapi`) — an 8-bit SDR
+        source the GPU decodes.
+      * `(True, True, False)`   → VAAPI decode → OpenCL tonemap → VAAPI encode — a true-HDR (PQ/HLG)
+        source, when the OpenCL runtime is present (Gen 9.5 VPP can't tonemap natively).
+      * `(True, False, True)`   → CPU decode → `hwupload` → VAAPI encode — an 8-bit SDR source in a codec
+        the GPU cannot decode (`CPU_DECODE_CODECS`).
 
     A 10-bit *SDR* source is deliberately NOT a `(True, True)` case: it needs a plain 10→8-bit downconvert,
     not the HDR tone curve (which would visibly darken/flatten an SDR picture), so it drops to software.
@@ -434,31 +460,40 @@ def _select_encode(db: DbSession, item: PlexItem, transcode: bool, src: Path) ->
     backstops a session that produces nothing — but still routing a true-HDR source through the tonemap
     graph so it doesn't hit the scale_vaapi wall); "auto" (default) → HW only when the render node is
     present AND the iGPU can hardware-decode the source, with tonemap chosen only for a true-HDR (PQ/HLG)
-    source that has an OpenCL runtime, while a 10-bit SDR source drops to software."""
+    source that has an OpenCL runtime, while a 10-bit SDR source drops to software. In both "on" and
+    "auto" a codec the GPU is known not to decode goes the CPU-decode route: asking the GPU to decode it
+    only bought the fallback's wasted second before libx264."""
+    sw = (False, False, False)
     if not transcode:
-        return (False, False)
+        return sw
     mode = sysconfig.get_str(db, "plex_transcode_hw")
     if mode == "off" or not _render_node_present():
-        return (False, False)
+        return sw
+    codec = (item.codec_video or "").lower()
     if mode == "on":
+        if codec in CPU_DECODE_CODECS:
+            return (True, False, True)
         # Trust the admin: HW wherever the render node exists, skipping the decodability gate (the runtime
         # backstop catches a miss). Only a true-HDR (PQ/HLG) source needs the tonemap graph — a 10-bit SDR
         # source takes the plain scale path here, NOT the HDR tone curve. Probe only when OpenCL exists
         # (short-circuit), so "on" stays probe-free where tonemap can't apply anyway.
         tonemap = _opencl_present() and _probe_video_kind(src) == _VK_HDR
-        return (True, tonemap)
+        return (True, tonemap, False)
     # auto
-    if (item.codec_video or "").lower() not in _HW_DECODABLE:
-        return (False, False)
+    if codec in CPU_DECODE_CODECS:
+        # The upload converts to 8-bit nv12: only an 8-bit SDR picture comes through it unchanged.
+        return (True, False, True) if _probe_video_kind(src) == _VK_HWSAFE else sw
+    if codec not in _HW_DECODABLE:
+        return sw
     kind = _probe_video_kind(src)
     if kind == _VK_HWSAFE:
-        return (True, False)  # 8-bit SDR → full-GPU scale path
+        return (True, False, False)  # 8-bit SDR → full-GPU scale path
     if kind == _VK_HDR:
         # PQ/HLG: HW OpenCL tonemap when the runtime is present, else software libx264.
-        return (True, True) if _opencl_present() else (False, False)
+        return (True, True, False) if _opencl_present() else sw
     # _VK_SDR10 — 10-bit SDR: a plain 10→8-bit downconvert, done exactly on the CPU. Never the HDR tone
     # curve (it would alter an SDR picture), and off the unverified scale_vaapi 10-bit path — software.
-    return (False, False)
+    return sw
 
 
 def _want_hw(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> bool:
@@ -467,16 +502,17 @@ def _want_hw(db: DbSession, item: PlexItem, transcode: bool, src: Path) -> bool:
     return _select_encode(db, item, transcode, src)[0]
 
 
-# CPU-nice increment for the SOFTWARE transcode child. The Plex ffmpeg is a Popen in the long-lived API
-# process (which must NOT itself be niced), so — unlike the download worker's _deprioritize (run once in
-# its own main()) — we lower priority per-child via preexec_fn in the fork. The GPU path needs none of
-# this. Reuses the download child's nice default so all co-located re-encoders share one handicap.
+# CPU-nice increment for a transcode child that works the CPU: a software encode, or a CPU decode
+# feeding the GPU encoder. The Plex ffmpeg is a Popen in the long-lived API process (which must NOT
+# itself be niced), so — unlike the download worker's _deprioritize (run once in its own main()) — we
+# lower priority per-child via preexec_fn in the fork. The full-GPU path needs none of this. Reuses
+# the download child's nice default so all co-located re-encoders share one handicap.
 _SW_TRANSCODE_NICE = settings.download_child_nice
 
 
 def _sw_child_preexec() -> None:
-    """Runs in the forked child (POSIX) before exec: renice this software-transcode ffmpeg down so it
-    can't starve the co-located API / Postgres / playback. Best-effort — a failed renice must not fail
+    """Runs in the forked child (POSIX) before exec: renice this CPU-bound transcode ffmpeg down so
+    it can't starve the co-located API / Postgres / playback. Best-effort — a failed renice must not fail
     playback."""
     try:
         os.nice(_SW_TRANSCODE_NICE)
@@ -536,13 +572,15 @@ def _ffmpeg_cmd(
     crf: int = 20,
     max_height: int = 0,
     loudnorm: bool = False,
+    cpu_decode: bool = False,
 ) -> list[str]:
     # For a transcode session the video is re-encoded (VAAPI or libx264) instead of stream-copied; the
     # hwaccel decode options go before -i, the encoder args replace `-c:v copy` below. Everything else is
     # identical to remux. `tonemap` is the HDR/10-bit HW variant (OpenCL tonemap between VAAPI decode and
     # encode) — it needs matching decode-side (named devices) and encode-side (the OpenCL filter graph) opts.
-    venc = _venc_opts(transcode, hw, tonemap, preset, crf, max_height)
-    args = ["ffmpeg", "-nostdin", "-loglevel", "error", *_hwaccel_in_opts(hw, tonemap)]
+    # `cpu_decode` is the HW encode of CPU-decoded frames: a device for the filter graph, no -hwaccel.
+    venc = _venc_opts(transcode, hw, tonemap, preset, crf, max_height, cpu_decode)
+    args = ["ffmpeg", "-nostdin", "-loglevel", "error", *_hwaccel_in_opts(hw, tonemap, cpu_decode)]
     # Keep ORIGINAL timestamps on the output so the first segment carries the true absolute PTS of the
     # keyframe ffmpeg actually seeked to. hls.js still zero-bases playback to that PTS; we read it back
     # (see _probe_first_pts) to learn the real content offset. Without -copyts the segment PTS is
@@ -641,29 +679,82 @@ def _probe_first_pts(seg: Path) -> float | None:
     return None
 
 
-def _kill(s: HlsSession) -> None:
+# Teardown is split in two, because every segment request and checkpoint takes `_lock`: waiting up to
+# 3 s for an ffmpeg to exit and deleting a whole segmented film under it stalled every other
+# viewer's playback for one viewer's seek or one reap pass. `_detach` (under the lock) signals the
+# process and renames its directory aside — so the deletion can never hit a NEW session that reuses
+# the path — and `_finish` (after the lock is released) waits and deletes.
+#
+# What the rename cannot cover is the dying ffmpeg itself: it writes by path, so until it has exited
+# it can still put a segment, or its final ENDLIST playlist, into a new session's directory on the
+# same path. `_dying` remembers it by that path, and `_spawn` waits it out — under the lock, on
+# purpose — before it creates the directory again.
+_dying: dict[Path, subprocess.Popen] = {}
+
+
+def _await_exit(key: str, proc: subprocess.Popen) -> None:
     try:
-        s.proc.terminate()
         try:
-            s.proc.wait(timeout=3)
+            proc.wait(timeout=3)
         # silent-ok: the escalation IS the handling. A transcode that ignores TERM for three
         # seconds gets KILL, which is exactly what this branch does.
         except subprocess.TimeoutExpired:
-            s.proc.kill()
+            proc.kill()
     except Exception as exc:  # noqa: BLE001 — teardown finishes whatever the process does
         # Not nothing: what survives this is an ffmpeg nobody owns any more, holding a VAAPI
         # handle and a core, and the transcode cap that is supposed to bound them counts rows in
         # `_sessions` rather than processes. One line is the only way that ever surfaces.
+        log.info(
+            "could not stop the transcode process",
+            extra=logs.event("plex.stream.kill_failed", key=key, reason=type(exc).__name__),
+        )
+
+
+def _detach(s: HlsSession) -> tuple[HlsSession, Path | None]:
+    """Caller holds _lock and has taken `s` out of `_sessions`. Hand the result to `_finish` once the
+    lock is released."""
+    try:
+        s.proc.terminate()
+    except Exception as exc:  # noqa: BLE001 — teardown finishes whatever the process does
         log.info(
             "could not stop the transcode process",
             extra=logs.event("plex.stream.kill_failed", key=s.key, reason=type(exc).__name__),
         )
-    shutil.rmtree(s.dir, ignore_errors=True)
+    _dying[s.dir] = s.proc
+    tomb = s.dir.with_name(f"{s.dir.name}{_TOMB}{secrets.token_hex(4)}")
+    try:
+        s.dir.rename(tomb)
+    # silent-ok: nothing on disk (ffmpeg died before writing, or a sweep took it) — nothing to delete.
+    except FileNotFoundError:
+        return s, None
+    except OSError as exc:
+        # Cannot be moved aside, so it cannot be deleted outside the lock either: the path may be a
+        # new session's a moment from now. Deleted here instead, the old way.
+        log.info(
+            "a session directory could not be moved aside; deleted under the lock",
+            extra=logs.event("plex.stream.tomb_failed", key=s.key, reason=type(exc).__name__),
+        )
+        shutil.rmtree(s.dir, ignore_errors=True)
+        return s, None
+    return s, tomb
 
 
-def _enforce_caps(user_id: int, max_transcodes: int, max_sessions: int) -> None:
-    """Caller holds _lock. Three bounds, applied after a session is registered, each dropping the
-    least recently accessed sessions over it:
+def _finish(doomed: "list[tuple[HlsSession, Path | None]]") -> None:
+    """The slow half of a teardown, called WITHOUT `_lock`."""
+    for s, tomb in doomed:
+        _await_exit(s.key, s.proc)
+        with _lock:
+            if _dying.get(s.dir) is s.proc:
+                del _dying[s.dir]
+        if tomb is not None:
+            shutil.rmtree(tomb, ignore_errors=True)
+
+
+def _enforce_caps(
+    user_id: int, max_transcodes: int, max_sessions: int
+) -> "list[tuple[HlsSession, Path | None]]":
+    """Caller holds _lock, and `_finish`es what this returns once it has released it. Three bounds,
+    applied after a session is registered, each dropping the least recently accessed sessions over it:
 
     * this account's own sessions (`_MAX_SESSIONS_PER_USER`) — first, so a single account filling the
       box evicts itself before anyone else;
@@ -672,10 +763,12 @@ def _enforce_caps(user_id: int, max_transcodes: int, max_sessions: int) -> None:
       whole segmented copy of the film under `_HLS_ROOT`, written faster than realtime. Counted only
       by the transcode cap, N distinct items were N ffmpegs and N film-sized scratch directories.
 
-    Somebody's playback stops at each drop. The route turns the missing session into a retryable 404
+    Somebody's playback stops at each drop. The route turns the missing session into a 410
     and the player restarts at the playhead, so from the server nothing failed — and "my film keeps
     rebuffering" has no other explanation anywhere, hence the line."""
 
+    doomed: list[tuple[HlsSession, Path | None]] = []
+
     def cull(pool: list[tuple[tuple[int, str], HlsSession]], cap: int, bound: str) -> None:
         if len(pool) <= cap:
             return
@@ -690,12 +783,13 @@ def _enforce_caps(user_id: int, max_transcodes: int, max_sessions: int) -> None:
                     sessions=len(pool),
                 ),
             )
-            _kill(sess)
             _sessions.pop(reg, None)
+            doomed.append(_detach(sess))
 
     cull([kv for kv in _sessions.items() if kv[0][0] == user_id], _MAX_SESSIONS_PER_USER, "user")
     cull([kv for kv in _sessions.items() if kv[1].transcode], max_transcodes, "transcode")
     cull(list(_sessions.items()), max_sessions, "total")
+    return doomed
 
 
 def start_session(
@@ -740,7 +834,11 @@ def start_session(
         return directory / segment_name(i, _VIDEO_VARIANT if variants else None)
 
     def _spawn(
-        hw: bool, tonemap: bool = False, *, replace_only: "HlsSession | None" = None
+        hw: bool,
+        tonemap: bool = False,
+        cpu_decode: bool = False,
+        *,
+        replace_only: "HlsSession | None" = None,
     ) -> "HlsSession | None":
         # `replace_only` makes the supersede identity-aware for the runtime fallback: the whole
         # check-and-replace happens under ONE _lock, so a newer same-key session (a concurrent
@@ -755,72 +853,88 @@ def start_session(
             preset = sysconfig.get_str(db, "plex_transcode_preset")
             crf = sysconfig.get_int(db, "plex_transcode_crf")
             max_height = sysconfig.get_int(db, "plex_transcode_max_height")
-        # Nice the SOFTWARE transcode child (not the HW path, and not remux) so a CPU-bound re-encode
-        # can't starve the co-located API/playback. preexec_fn runs in the forked child (POSIX); on a
-        # platform without fork (the API always runs in a Linux container in prod) it's simply omitted.
-        preexec = _sw_child_preexec if (transcode and not hw and os.name == "posix") else None
-        with _lock:
-            if replace_only is not None and _sessions.get(reg) is not replace_only:
-                return None  # superseded/culled while we waited out of lock — don't clobber the newer one
-            old = _sessions.pop(reg, None)
-            if old is not None:
-                _kill(old)
-            shutil.rmtree(directory, ignore_errors=True)
-            directory.mkdir(parents=True, exist_ok=True)
-            proc = subprocess.Popen(
-                _ffmpeg_cmd(
-                    src,
-                    start_s,
+        # Nice the child that works the CPU — a SOFTWARE transcode, or a CPU decode feeding the GPU
+        # encoder (not the full-GPU path, and not remux) — so it can't starve the co-located
+        # API/playback. preexec_fn runs in the forked child (POSIX); on a platform without fork (the API
+        # always runs in a Linux container in prod) it's simply omitted.
+        cpu_bound = transcode and (not hw or cpu_decode)
+        preexec = _sw_child_preexec if (cpu_bound and os.name == "posix") else None
+        doomed: list[tuple[HlsSession, Path | None]] = []
+        # Finished even when the new session fails to start (no space, no ffmpeg, a DB error): what
+        # was moved aside under the lock is otherwise left for a restart's sweep.
+        try:
+            with _lock:
+                if replace_only is not None and _sessions.get(reg) is not replace_only:
+                    return None  # superseded/culled while we waited out of lock — don't clobber the newer one
+                old = _sessions.pop(reg, None)
+                if old is not None:
+                    doomed.append(_detach(old))
+                # An ffmpeg still exiting from THIS path (the old session, or one stopped a moment ago)
+                # would write into the directory about to be made; see `_dying`.
+                prev = _dying.pop(directory, None)
+                if prev is not None:
+                    _await_exit(key, prev)
+                shutil.rmtree(directory, ignore_errors=True)
+                directory.mkdir(parents=True, exist_ok=True)
+                proc = subprocess.Popen(
+                    _ffmpeg_cmd(
+                        src,
+                        start_s,
+                        directory,
+                        audio_ord,
+                        aoff,
+                        audio_langs if is_multi else None,
+                        transcode=transcode,
+                        hw=hw,
+                        tonemap=tonemap,
+                        preset=preset,
+                        crf=crf,
+                        max_height=max_height,
+                        loudnorm=loudnorm,
+                        cpu_decode=cpu_decode,
+                    ),
+                    stdout=subprocess.DEVNULL,
+                    stderr=subprocess.DEVNULL,
+                    preexec_fn=preexec,
+                )
+                s = HlsSession(
+                    key,
                     directory,
-                    audio_ord,
-                    aoff,
-                    audio_langs if is_multi else None,
+                    proc,
+                    start_s,
+                    variants,
                     transcode=transcode,
                     hw=hw,
                     tonemap=tonemap,
-                    preset=preset,
-                    crf=crf,
-                    max_height=max_height,
-                    loudnorm=loudnorm,
-                ),
-                stdout=subprocess.DEVNULL,
-                stderr=subprocess.DEVNULL,
-                preexec_fn=preexec,
-            )
-            s = HlsSession(
-                key,
-                directory,
-                proc,
-                start_s,
-                variants,
-                transcode=transcode,
-                hw=hw,
-                tonemap=tonemap,
-                user_id=user_id,
-            )
-            _sessions[reg] = s
-            # The one line that says which pipeline a session actually got. Everything downstream
-            # — the cap, the HW→SW fallback, the subtitle offset — is about this choice, and until
-            # now it was only inferable from the ffmpeg command nobody keeps.
-            log.info(
-                "an HLS session started",
-                extra=logs.event(
-                    "plex.stream.session_started",
-                    key=key,
-                    transcode=transcode,
-                    hw=hw,
-                    tonemap=tonemap,
-                    variants=len(variants or ()),
-                    start_s=round(start_s, 1),
-                ),
-            )
-            # Admin-configurable caps (Configuration → Plex); at least 1 each so playback can't be
-            # capped to zero.
-            _enforce_caps(
-                user_id,
-                max(1, sysconfig.get_int(db, "plex_max_transcodes")),
-                max(1, sysconfig.get_int(db, "plex_max_sessions")),
-            )
+                    user_id=user_id,
+                    cpu_decode=cpu_decode,
+                )
+                _sessions[reg] = s
+                # The one line that says which pipeline a session actually got. Everything downstream
+                # — the cap, the HW→SW fallback, the subtitle offset — is about this choice, and until
+                # now it was only inferable from the ffmpeg command nobody keeps.
+                log.info(
+                    "an HLS session started",
+                    extra=logs.event(
+                        "plex.stream.session_started",
+                        key=key,
+                        transcode=transcode,
+                        hw=hw,
+                        tonemap=tonemap,
+                        cpu_decode=cpu_decode,
+                        variants=len(variants or ()),
+                        start_s=round(start_s, 1),
+                    ),
+                )
+                # Admin-configurable caps (Configuration → Plex); at least 1 each so playback can't be
+                # capped to zero.
+                doomed += _enforce_caps(
+                    user_id,
+                    max(1, sysconfig.get_int(db, "plex_max_transcodes")),
+                    max(1, sysconfig.get_int(db, "plex_max_sessions")),
+                )
+        finally:
+            _finish(doomed)
         return s
 
     s = _spawn(*_select_encode(db, item, transcode, src))
@@ -847,7 +961,7 @@ def start_session(
             # stale and stops the session it names (`stop_session`), which stopped the playback that won.
             # Never None either: the POST /session route turns a None return into a "Media file not found
             # on disk" 404 — dishonest for a file that exists. A dead session instead degrades to a
-            # retryable 404 ("No active playback session" / "Not available yet") the client re-requests.
+            # 410 ("session_gone") on its URLs, and the client asks again.
             return s
         s = respawned
         ready = wait_for(_seg(0), timeout=25.0, proc=s.proc)
@@ -866,7 +980,8 @@ def start_session(
             if not wait_for(_seg(i), timeout=_WARMUP_SEG_TIMEOUT_S, proc=s.proc):
                 break
     log.info(
-        "plex hls session start key=%s req=%.1f real=%.3f multi=%s audio=%s transcode=%s hw=%s tonemap=%s",
+        "plex hls session start key=%s req=%.1f real=%.3f multi=%s audio=%s transcode=%s hw=%s"
+        " tonemap=%s cpu_decode=%s",
         key,
         start_s,
         s.media_start_s,
@@ -875,6 +990,7 @@ def start_session(
         transcode,
         s.hw,
         s.tonemap,
+        s.cpu_decode,
     )
     return s
 
@@ -974,6 +1090,11 @@ def artifact_may_arrive(s: "HlsSession", filename: str) -> bool:
     # session, absent for a single-audio one.
     if variant not in (s.variants or (None,)):
         return False
+    # And the name must be the one ffmpeg SPELLS for that sequence: `int()` also reads `seg_007.ts`
+    # and a non-ASCII digit run as a sequence the session may have reached, and neither file is
+    # ever written — a yes there parked the request for the whole wait.
+    if filename != segment_name(seq, variant):
+        return False
     if seq == 0:
         return True
     try:
@@ -1084,9 +1205,13 @@ def sweep_orphans(older_than: float | None = None) -> int:
     return dropped
 
 
-# How long a session whose ffmpeg has FINISHED may go unfetched before a new session request reaps
-# it (see reap_idle).
-_FINISHED_IDLE_S = 30
+# The slowest heartbeat a live player sends: a browser cuts the timers of a hidden page that makes no
+# sound to one a minute (measured on Chromium: a paused and a muted player, about a minute after
+# hiding; a hidden player with sound keeps its 10 s).
+_THROTTLED_HEARTBEAT_S = 60
+# How long a session whose ffmpeg has FINISHED may go without a signal before `reap_idle` takes it:
+# the beat before the reaper looks, one lost on the way, and the time the next one takes to arrive.
+_FINISHED_IDLE_S = 3 * _THROTTLED_HEARTBEAT_S
 
 
 def reap_idle(finished_idle_s: float = _FINISHED_IDLE_S) -> int:
@@ -1095,29 +1220,39 @@ def reap_idle(finished_idle_s: float = _FINISHED_IDLE_S) -> int:
 
     A finished session is also what a PAUSED viewer has: a remux runs far faster than realtime, so
     ffmpeg is done a minute or two in and writes ENDLIST; hls.js then stops asking for the playlist,
-    and with its forward buffer full a paused player fetches nothing at all. With the 30 s rule that
-    killed a one-minute pause, and playback died ~30 s after resuming.
+    and with its forward buffer full a paused player fetches nothing at all.
 
-    What makes the short rule safe again is that fetching is no longer the only signal: a player
-    parked MID-ITEM checkpoints its position every 10 s, and that route calls `touch_session`, so a
-    viewer who is merely paused says so far more often than this rule asks. The heartbeat
+    What makes the short rule safe is that fetching is not the only signal: a player parked MID-ITEM
+    checkpoints its position and that route calls `touch_session`. That beat is every 10 s only while
+    the page is visible or making sound; a hidden, silent one sends it once a minute, which a 30 s
+    rule killed whenever the reaper happened to run more than 30 s after the beat (measured: a paused
+    session gone that way, and one that lived on the same cadence because its phase was luckier).
+    So the rule is sized to the throttled beat (`_THROTTLED_HEARTBEAT_S`). The heartbeat
     stops once the item is finished or before a position is known (`usePlexProgress.flush`) — both
-    states where reaping is correct, and where the player's 404 → reclaim recovers a viewer who
-    scrubs back in anyway. Do not restate this as "every 10 s whether or not it is playing":
+    states where reaping is correct, and where the player's 410 → reclaim recovers a viewer who
+    scrubs back in anyway. Do not restate this as "whether or not it is playing":
     a review found that claim in two comments and not in the code. An earlier blanket
     `reap_abandoned` bought the same safety by keeping every finished session for ten minutes — each
     one a whole segmented copy of the film on a volume shared with the download center."""
     now = time.time()
-    dropped = 0
+    doomed: list[tuple[HlsSession, Path | None]] = []
+    reaped: list[tuple[str, str, int]] = []
     with _lock:
         for reg, s in list(_sessions.items()):
             done = s.proc.poll() is not None
             idle = now - s.last_access
             if idle > _SESSION_IDLE_S or (done and idle > finished_idle_s):
-                _kill(s)
                 _sessions.pop(reg, None)
-                dropped += 1
-    return dropped
+                doomed.append(_detach(s))
+                reaped.append((s.key, "finished_idle" if done else "idle", int(idle)))
+    # The viewer of a session wrongly taken sees only a 410 later, so this line is the one record of why.
+    for key, reason, idle_s in reaped:
+        log.info(
+            "an idle session was reaped",
+            extra=logs.event("plex.stream.session_reaped", key=key, reason=reason, idle_s=idle_s),
+        )
+    _finish(doomed)
+    return len(doomed)
 
 
 def stop_session(user_id: int, key: str, session_id: str) -> bool:
@@ -1132,8 +1267,9 @@ def stop_session(user_id: int, key: str, session_id: str) -> bool:
         s = _sessions.get((user_id, key))
         if s is None or s.id != session_id:
             return False
-        _kill(s)
         _sessions.pop((user_id, key), None)
+        doomed = [_detach(s)]
+    _finish(doomed)
     return True
 
 

backend/app/plex/subtitles.py

@@ -47,7 +47,7 @@ from sqlalchemy.orm import Session
 
 from app import logs
 from app.db import SessionLocal
-from app.models import PlexItem
+from app.plex.models import PlexItem
 from app.plex import paths as plex_paths
 from app.plex.cache import CACHE_DIR
 from app.plex.client import PlexClient, PlexError

backend/app/plex/sync.py

@@ -21,7 +21,7 @@ from sqlalchemy import func, select
 from sqlalchemy.orm import Session
 
 from app import logs, sysconfig
-from app.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
+from app.plex.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
 from app.plex import detail, overlay, people
 from app.plex.client import PlexClient, PlexError, PlexNotConfigured
 from app.plex.genres import hu_genres
@@ -60,14 +60,20 @@ def classify_playable(vcodec: str | None, acodec: str | None, container: str | N
 
 # Video codecs the prod iGPU (UHD 630 / Quick Sync Gen 9.5) can hardware-DECODE — the precondition for
 # the full-GPU VAAPI transcode pipeline. THE single source of this set: plex.stream imports it here so
-# the runtime encoder choice (`_want_hw`) and this catalog facet can never disagree by codec. A source
-# outside it (AV1, MPEG-4 ASP/XviD…) is software-only even on the iGPU (it can't decode it to feed its
-# own encoder).
-HW_DECODABLE_CODECS = frozenset({"h264", "hevc", "h265", "vp9", "vc1", "mpeg2video"})
+# the runtime encoder choice (`_select_encode`) and this catalog facet can never disagree by codec. VC-1
+# is NOT in it: measured on prod, the iHD driver answers "No support for codec vc1 profile 3" (VC-1
+# Advanced, what a Blu-ray remux carries), so every session that asked the GPU to decode one died and
+# fell back to libx264.
+HW_DECODABLE_CODECS = frozenset({"h264", "hevc", "h265", "vp9", "mpeg2video"})
+# Codecs the iGPU cannot decode but whose picture it can still ENCODE: the CPU decodes, the frames go up
+# to the GPU as 8-bit nv12, and h264_vaapi encodes (plex.stream's `cpu_decode` path). Measured on prod:
+# three segments in ~5 s for a 1080p VC-1 remux (libx264: ~17 s) and ~1 s for an SD XviD or DivX
+# episode. Only codecs that are always 8-bit SDR, and only ones measured; any other codec stays software.
+CPU_DECODE_CODECS = frozenset({"vc1", "mpeg4", "msmpeg4v3"})
 # Color-transfer values marking an HDR source (PQ / HLG). THE single source of this set: plex.stream
 # (the tonemap choice) and downloads.clip (a clip's tonemap) import it here, so a transfer added for
-# one is tonemapped by all. A 10-bit/HDR source breaks the VAAPI scale=nv12 path on Gen 9.5 (see
-# plex.stream) → software transcode, so it counts as NOT HW-capable.
+# one is tonemapped by all. An HDR source breaks the plain VAAPI scale=nv12 path on Gen 9.5, so it takes
+# the OpenCL tonemap graph (see plex.stream) — still the GPU encoder.
 HDR_TRANSFERS = frozenset({"smpte2084", "arib-std-b67"})
 
 
@@ -78,18 +84,24 @@ def transcode_hw_capable(
     hdr: bool = False,
 ) -> bool | None:
     """The stored `PlexItem.transcode_hw` facet: for a `playable="transcode"` item, whether the prod
-    iGPU can HARDWARE-transcode it (else software libx264 only); None for a non-transcode item. HW needs
-    a HW-decodable codec AND an 8-bit SDR source — a 10-bit/HDR source (bit_depth>8 or an HDR transfer)
-    breaks the VAAPI nv12 pipeline and falls back to software. `bit_depth`/`hdr` are unknown at sync
-    time (the cheap listing carries no Stream), so they default to 8-bit SDR (codec-only) and the
-    bit-depth backfill refines a 10-bit/HDR HEVC to False later."""
+    iGPU ENCODES it ("Transcoded on the hardware encoder"), else software libx264; None for a
+    non-transcode item. The same answer plex.stream's `_select_encode` gives on prod (render node and
+    OpenCL runtime present, policy "auto"):
+      * a HW-decodable codec: 8-bit SDR → the full-GPU path; HDR → the OpenCL tonemap, also on the GPU;
+        10-bit SDR → software (an exact downconvert, never the HDR tone curve);
+      * a CPU-decode codec (`CPU_DECODE_CODECS`) at 8 bits → CPU decode, GPU encode;
+      * any other codec → software.
+    `bit_depth`/`hdr` are unknown at sync time (the cheap listing carries no Stream), so they default to
+    8-bit SDR and the bit-depth backfill refines a 10-bit SDR HEVC to False later."""
     if playable != "transcode":
         return None
-    if (vcodec or "").lower() not in HW_DECODABLE_CODECS:
-        return False
-    if bit_depth is not None and bit_depth > 8:
-        return False
-    return not hdr
+    codec = (vcodec or "").lower()
+    deep = bit_depth is not None and bit_depth > 8
+    if codec in HW_DECODABLE_CODECS:
+        return hdr or not deep
+    if codec in CPU_DECODE_CODECS:
+        return not deep and not hdr
+    return False
 
 
 def _video_stream_props(meta: dict) -> tuple[int | None, bool]:
@@ -266,8 +278,9 @@ def sync(db: Session) -> dict:
                 # a Plex shape change, a DB error or our own bug, and `%s` told them apart badly.
                 log.exception("the Plex people-enrich batch failed")
             # Probe a bounded batch of HW-decodable transcode items for bit-depth/HDR, so the
-            # playback-class filter's HW/SW split is precise (a 10-bit/HDR HEVC → software) over
-            # successive runs. Only HW-decodable-codec transcodes are probed — the rest are SW by codec.
+            # playback-class filter's HW/SW split is precise (a 10-bit SDR HEVC → software) over
+            # successive runs. Only HW-decodable-codec transcodes are probed — the rest are classed
+            # by codec alone.
             try:
                 stats["media_probed"] = _probe_media_batch(db, plex)
             except Exception:  # best-effort; never fail the sync over a backfill
@@ -294,15 +307,15 @@ def sync(db: Session) -> dict:
 # whole library is covered over successive runs (each fetched item is stamped people_enriched_at once).
 _ENRICH_BATCH = 60
 # Same bounded-per-run pattern for the bit-depth probe. Only HW-decodable-codec transcode items are
-# probed (typically a small minority — the bulk of transcodes are old codecs that are SW by codec), so
+# probed (typically a small minority — the bulk of transcodes are old codecs classed by codec), so
 # a modest batch covers the work-list in a run or two; a stuck item retries next run (watermark unset).
 _PROBE_BATCH = 60
 
 
 def _probe_media_batch(db: Session, plex: PlexClient, limit: int = _PROBE_BATCH) -> int:
     """Rich-fetch up to `limit` not-yet-probed HW-decodable transcode items, read the video Stream's
-    bit-depth + HDR, and store `video_bit_depth` + the refined `transcode_hw` (a 10-bit/HDR HEVC →
-    False/software). Stamps `media_probed_at` on every attempt (success OR a Plex miss with no Stream),
+    bit-depth + HDR, and store `video_bit_depth` + the refined `transcode_hw` (a 10-bit SDR HEVC →
+    False/software; an HDR one stays on the GPU through the tonemap). Stamps `media_probed_at` on every attempt (success OR a Plex miss with no Stream),
     so an item is probed once and not re-fetched every sync; a transient fetch FAILURE leaves the
     watermark unset to retry. Oldest-id first. Returns how many items were probed this run."""
     rows = (

backend/app/plex/tmdb.py

@@ -22,7 +22,7 @@ from sqlalchemy.orm import Session
 
 from app import logs, sysconfig
 from app.config import settings
-from app.models import PlexItem, PlexPerson, PlexShow
+from app.plex.models import PlexItem, PlexPerson, PlexShow
 from app.plex.cards import display_title
 from app.plex.people import people_by_credits, warm_portraits
 
@@ -619,9 +619,9 @@ def enrich_person(db: Session, person: PlexPerson) -> bool:
     if not person.tmdb_person_id or not sysconfig.get_str(db, "tmdb_api_key"):
         return False
     try:
-        # Called from a background task (off the request path — see people._enrich_person_bg), so it no
-        # longer blocks a serve; still cap the outbound TMDB call with a short timeout so a hanging TMDB
-        # doesn't tie up a threadpool worker for the default 30s.
+        # Called from a background task (off the request path — see `plex.service.enrich_person_bg`),
+        # so it no longer blocks a serve; still cap the outbound TMDB call with a short timeout so a
+        # hanging TMDB doesn't tie up a threadpool worker for the default 30s.
         with TmdbClient(db, timeout=8.0) as tmdb:
             _fetch_person_into(tmdb, person)
     except TmdbError as exc:

backend/app/plex/watch_sync.py

@@ -29,8 +29,11 @@ from sqlalchemy.orm import Session
 
 from app import logs, sysconfig
 from app.db import SessionLocal
-from app.models import PlexItem, PlexLibrary, PlexLink, PlexState
+from app.userscope import owned_by
+from app.plex.models import PlexItem, PlexLibrary, PlexLink, PlexState
 from app.plex.client import PlexClient, PlexError, PlexNotConfigured
+from app.plex import queries
+from app.plex.service import state_row
 from app.plex.sync import (
     _TYPE_EPISODE,
     _TYPE_MOVIE,
@@ -196,7 +199,7 @@ def link_for_push(db: Session, user_id: int) -> PlexLink | None:
     """The active owner watch-push link for this user, or None when push is off / not the owner.
     Push is gated on `initial_import_done` so we never push a half-known local state over Plex's
     authoritative one before the one-time "Plex is master" import has reconciled them."""
-    link = db.execute(select(PlexLink).where(PlexLink.user_id == user_id)).scalars().first()
+    link = db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user_id))).scalars().first()
     if link and link.uses_admin and link.sync_enabled and link.initial_import_done:
         return link
     return None
@@ -251,13 +254,7 @@ def push_state_to_plex(
         # Flag the row synced — but an "unwatched" via status=new deletes the row, so there may be
         # nothing to flag; that's fine (unscrobble still happened). Skip the flag if the row was
         # re-edited since we captured `expect_ts` (leave it dirty for its own push / the reconcile).
-        st = (
-            db.execute(
-                select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
-            )
-            .scalars()
-            .first()
-        )
+        st = queries.state(db, user_id, item_id)
         if st is not None and _row_matches(st, expect_ts):
             st.synced_to_plex = True
             db.commit()
@@ -318,7 +315,7 @@ def push_bulk_state_to_plex(user_id: int, changes: list[tuple[int, str, str]]) -
         if synced_ids:
             db.execute(
                 update(PlexState)
-                .where(PlexState.user_id == user_id, PlexState.item_id.in_(synced_ids))
+                .where(owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(synced_ids))
                 .values(synced_to_plex=True)
             )
             db.commit()
@@ -391,13 +388,7 @@ def _pull_apply(
     """Bring one Plex-side positive signal into Siftlode under last-write-wins. Returns
     "watched"/"in_progress" if it changed the row, else None. Never clears a row (un-watch is the
     full reconcile's job) — this only pulls in watched/resume."""
-    st = (
-        db.execute(
-            select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
-        )
-        .scalars()
-        .first()
-    )
+    st = queries.state(db, user_id, item_id)
     if _same_state(st, status, pos):
         if st is not None and not st.synced_to_plex:
             st.synced_to_plex = True  # already matches Plex → just settle the flag, no churn
@@ -418,8 +409,9 @@ def _pull_apply(
         # "almost always already pushed" assumption above excludes.
         return None
     if st is None:
-        st = PlexState(user_id=user_id, item_id=item_id)
-        db.add(st)
+        st = state_row(db, user_id, item_id, synced=True)
+        if not st.synced_to_plex:  # a local change made it first, beside this pull: the guard above
+            return None
     if status == "watched":
         st.status, st.position_seconds, st.watched_at, st.progress_updated_at = (
             "watched",
@@ -449,7 +441,7 @@ def _repush_dirty(db: Session, plex: PlexClient, link: PlexLink) -> int:
     dirty = (
         db.execute(
             select(PlexState).where(
-                PlexState.user_id == link.user_id,
+                owned_by(PlexState.user_id, link.user_id),
                 PlexState.synced_to_plex.is_(False),
                 PlexState.status != "hidden",
             )
@@ -505,7 +497,7 @@ def _reconcile_inprogress_movies(
     and aren't affected). Note: like the rest of this job, `metadata()` reads the TOKEN account (the
     owner) — a HTPC playing under a managed Plex user (accountID>1) is out of scope here as elsewhere."""
     conds = [
-        PlexState.user_id == link.user_id,
+        owned_by(PlexState.user_id, link.user_id),
         PlexState.status.notin_(("watched", "hidden")),
         PlexState.position_seconds > _PROGRESS_MIN_S,
         PlexItem.kind == "movie",
@@ -627,7 +619,7 @@ def _clear_unwatched(db: Session, user_id: int, plex_states: dict, settled: set[
         select(PlexState)
         .join(PlexItem, PlexItem.id == PlexState.item_id)
         .join(PlexLibrary, PlexLibrary.id == PlexItem.library_id)
-        .where(PlexState.user_id == user_id, PlexLibrary.plex_key.in_(settled))
+        .where(owned_by(PlexState.user_id, user_id), PlexLibrary.plex_key.in_(settled))
     )
     for st in rows.scalars().all():
         if st.item_id in plex_states or st.status == "hidden" or not st.synced_to_plex:

backend/app/quota.py

@@ -16,7 +16,8 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.engine import CursorResult
 from sqlalchemy.orm import Session
 
-from app import sysconfig
+from app.userscope import owned_by
+from app import logs, sysconfig
 from app.db import SessionLocal
 from app.models import ApiQuotaUsage, QuotaEvent
 
@@ -43,6 +44,7 @@ class QuotaAction:
     VIDEOS_ENRICH = "videos_enrich"
     VIDEOS_LOOKUP = "videos_lookup"
     VIDEOS_SEARCH = "videos_search"
+    VIDEOS_RELATED = "videos_related"
     CHANNELS_DISCOVER = "channels_discover"
     CHANNELS_EXPLORE = "channels_explore"
     CHANNELS_SUBSCRIBE = "channels_subscribe"
@@ -141,7 +143,7 @@ def backfill_reserve_reached() -> bool:
     condition for every backfill/explore path (recent, deep, immediate full-history, explore, and
     rolling revalidation).
 
-    Reads in its OWN short-lived session (like record_usage / log_action), for two reasons: it
+    Reads in its OWN short-lived session (like record_usage / claim_action), for two reasons: it
     replaces the remaining/reserve comparison that was open-coded at every call site, and — critically
     for the per-page guards in paging loops — a guard must not hold the caller's transaction open
     across the network fetch that follows it. Quota is committed independently of any caller, so a
@@ -150,31 +152,48 @@ def backfill_reserve_reached() -> bool:
         return remaining_today(qs) <= sysconfig.get_int(qs, "backfill_quota_reserve")
 
 
-def log_action(user_id: int, action: str) -> None:
-    """Record a zero-cost action event so per-user daily caps still count it.
+def claim_action(user_id: int, action: str, limit: int) -> int | None:
+    """Take one of this user's `limit` daily `action` slots, or None when they are all taken — the
+    check and the take in one transaction, under a lock per user and action. Checked first and
+    logged after the work, N requests sent together at the last free slot all passed the check
+    (the work in between is seconds of scraping). Returns the event's id, for `refund_action` when
+    the work then produced nothing.
 
-    `record_usage` only logs when units are actually charged; a scrape-based search spends no
-    quota yet must still be rate-limited per user, so it logs its event here directly.
-
-    Writes in its OWN session (like record_usage) — quota accounting is durable independent of the
-    caller's transaction and must never commit the caller's session (that would flush a job's
-    partial state mid-run, defeating its own rollback)."""
+    The ONE way to count a capped action: a claim is a zero-unit event, and `actions_today` counts
+    those alone, so a call the action then pays for (search.list's own `record_usage` event, under
+    the same action) is accounting, never a second slot. Own session, committed at once — quota
+    accounting is durable whatever the caller's transaction does next, and must never commit it."""
     with SessionLocal() as qs:
-        qs.add(QuotaEvent(user_id=user_id, action=action, units=0))
+        qs.execute(select(func.pg_advisory_xact_lock(func.hashtext(action), user_id)))
+        if actions_today(qs, user_id, action) >= limit:
+            return None
+        event = QuotaEvent(user_id=user_id, action=action, units=0)
+        qs.add(event)
+        qs.commit()
+        return event.id
+
+
+def refund_action(event_id: int) -> None:
+    """Give a slot `claim_action` took back: the work it was taken for failed outright, and a
+    request that showed the user nothing must not count against them."""
+    with SessionLocal() as qs:
+        qs.execute(delete(QuotaEvent).where(QuotaEvent.id == event_id))
         qs.commit()
 
 
 def actions_today(db: Session, user_id: int, action: str) -> int:
-    """How many quota events of `action` this user has logged so far in the current Pacific
-    day — for per-user, per-action daily caps (e.g. the live-search limit). Counts events,
-    not units, so it only works for actions charged exactly once per user action."""
+    """How many slots of `action` this user has claimed so far in the current Pacific day — for
+    per-user, per-action daily caps (the live search, the related lists). Counts the zero-unit
+    claim events (`claim_action`) only: `record_usage`'s events for units the action spent are not
+    slots (it logs only when units were charged)."""
     return (
         db.scalar(
             select(func.count())
             .select_from(QuotaEvent)
             .where(
-                QuotaEvent.user_id == user_id,
+                owned_by(QuotaEvent.user_id, user_id),
                 QuotaEvent.action == action,
+                QuotaEvent.units == 0,
                 QuotaEvent.created_at >= pacific_day_start_utc(),
             )
         )
@@ -204,6 +223,14 @@ def record_usage(units: int) -> None:
     )
     with SessionLocal() as qs:
         qs.execute(stmt)
-        # Audit detail: who/what spent it (per-user attribution; NULL actor = system).
-        qs.add(QuotaEvent(user_id=_actor_id.get(), action=_action.get(), units=units))
+        # Audit detail: who/what spent it (per-user attribution; NULL actor = system), and the
+        # background run it belongs to, so the run's row can say what it cost.
+        qs.add(
+            QuotaEvent(
+                user_id=_actor_id.get(),
+                action=_action.get(),
+                units=units,
+                job_run_id=logs.current_run_id(),
+            )
+        )
         qs.commit()

backend/app/ratelimit.py

@@ -42,6 +42,14 @@ class RateLimiter:
         with self._lock:
             return sum(c for t, c in self._hits.get(key, []) if t > cutoff) >= self.max_events
 
+    def room(self, key: str) -> float:
+        """How much cost ``key`` may still spend in the current window, WITHOUT counting anything —
+        for work counted only once it has succeeded: ask it first, ``allow`` what was done."""
+        cutoff = time.monotonic() - self.window
+        with self._lock:
+            used = sum(c for t, c in self._hits.get(key, []) if t > cutoff)
+        return max(0.0, self.max_events - used)
+
     def allow(self, key: str, cost: float = 1) -> bool:
         """Record an attempt costing ``cost`` for ``key`` and return whether it is within the limit.
 

backend/app/reader/models.py

@@ -0,0 +1,298 @@
+"""Reader's tables: what a user does with the shared catalog — where they are in a video
+(`VideoState`), their playlists, their saved views, their labels on channels (`Tag`, `ChannelTag`)
+and the comfort shelf on Home. Only this unit writes them; another unit reads them along a declared
+edge, or asks this unit's service."""
+
+from datetime import datetime
+from enum import StrEnum
+
+from sqlalchemy import (
+    Boolean,
+    CheckConstraint,
+    DateTime,
+    Float,
+    ForeignKey,
+    Index,
+    Integer,
+    JSON,
+    String,
+    Text,
+    UniqueConstraint,
+    func,
+    text,
+)
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from app.db import Base
+from app.models import TimestampMixin, UpdatedAtMixin
+
+
+class Tag(Base, TimestampMixin):
+    """A label. System tags (user_id NULL) are computed automatically and shared by all
+    users; user tags (user_id set) are personal. category groups them in the UI."""
+
+    __tablename__ = "tags"
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int | None] = mapped_column(
+        ForeignKey("users.id", ondelete="CASCADE"), index=True
+    )
+    name: Mapped[str] = mapped_column(String(64))
+    color: Mapped[str | None] = mapped_column(String(16))
+    # language | topic | other
+    category: Mapped[str] = mapped_column(String(16), default="other", server_default="other")
+
+
+class ChannelTag(Base):
+    """Links a tag to a channel. source='auto' rows are managed by the auto-tagger and
+    regenerated freely; source='user' rows are never touched by the system."""
+
+    __tablename__ = "channel_tags"
+    __table_args__ = (UniqueConstraint("channel_id", "tag_id", "user_id", name="uq_channel_tag"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    channel_id: Mapped[str] = mapped_column(
+        ForeignKey("channels.id", ondelete="CASCADE"), index=True
+    )
+    tag_id: Mapped[int] = mapped_column(ForeignKey("tags.id", ondelete="CASCADE"), index=True)
+    # NULL = system/auto link visible to everyone; set = a specific user's link.
+    user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    source: Mapped[str] = mapped_column(String(8), default="auto", server_default="auto")
+    confidence: Mapped[float | None] = mapped_column(Float)
+
+
+class VideoState(Base, UpdatedAtMixin):
+    """Per-user watch state for a video. Rows exist only for non-default states; the
+    absence of a row means 'new/unseen'."""
+
+    __tablename__ = "video_states"
+    __table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_video"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
+    # new | watched | hidden  ("saved" is now Watch-later playlist membership, not a status)
+    status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
+    watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # Resume position (seconds into the video) for the in-app player. Orthogonal to
+    # status: a partially-watched video keeps status "new" but a non-zero position, so
+    # it can render a progress bar and match the "in progress" feed filter. 0 = not
+    # started / finished (trivially-early and near-finished positions are not stored).
+    # int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
+    position_seconds: Mapped[int] = mapped_column(
+        Integer, default=0, server_default="0", nullable=False
+    )
+    progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+
+class VideoUnmark(Base):
+    """When the user last took a video back to "new" here — un-marked it, or reset it. That leaves
+    no `VideoState` (its absence is "new"), so without this row a later import of what they watched
+    on YouTube could not tell a video they un-marked from one never touched, and marked it watched
+    again. Kept by user and video; the video's own deletion takes it."""
+
+    __tablename__ = "video_unmarks"
+    __table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_video_unmark"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
+    at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
+
+
+class Playlist(Base, TimestampMixin, UpdatedAtMixin):
+    """A per-user named, ordered collection of videos from the shared catalog.
+
+    `kind`: 'user' (normal) or 'watch_later' (built-in, undeletable, one per user).
+    `source`: 'local' (created here) or 'youtube' (mirrored from the user's YouTube
+    playlists). `yt_playlist_id` links a mirrored/exported playlist to its YouTube id.
+    `dirty` marks a YouTube-linked playlist with local edits not yet pushed, so the read
+    sync won't clobber them. (source/yt_playlist_id/dirty are forward-looking for the YT
+    sync phases; the foundation phase only uses local playlists.)"""
+
+    __tablename__ = "playlists"
+    __table_args__ = (
+        # One Watch later per user, held by the database: two first saves at once each made one.
+        Index(
+            "uq_playlist_watch_later",
+            "user_id",
+            unique=True,
+            postgresql_where=text("kind = 'watch_later'"),
+        ),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+    name: Mapped[str] = mapped_column(String(255))
+    kind: Mapped[str] = mapped_column(String(16), default="user", server_default="user")
+    source: Mapped[str] = mapped_column(String(16), default="local", server_default="local")
+    yt_playlist_id: Mapped[str | None] = mapped_column(String(64))
+    # True when the current items/name differ from `synced_fingerprint` (unpushed edits).
+    dirty: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+    # SHA-256 of name + ordered video ids as last synced from / pushed to YouTube; the
+    # baseline `dirty` is derived against. NULL = no baseline yet (treated as dirty).
+    synced_fingerprint: Mapped[str | None] = mapped_column(String)
+    # JSON list of the ordered video ids as of the last sync — the 3-way-merge baseline the
+    # two-way YouTube sync classifies each item against (local-add vs YT-add vs real delete).
+    # NULL = no baseline yet; the first merge then does a lossless union (never deletes).
+    synced_video_ids: Mapped[str | None] = mapped_column(Text)
+    position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+
+    items: Mapped[list["PlaylistItem"]] = relationship(
+        back_populates="playlist",
+        cascade="all, delete-orphan",
+        order_by="PlaylistItem.position",
+    )
+
+
+class PlaylistItem(Base):
+    """A video's membership in a playlist, with its order within that playlist."""
+
+    __tablename__ = "playlist_items"
+    __table_args__ = (UniqueConstraint("playlist_id", "video_id", name="uq_playlist_video"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    playlist_id: Mapped[int] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
+    video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
+    position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    added_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
+
+    playlist: Mapped["Playlist"] = relationship(back_populates="items")
+
+
+class SavedView(Base, TimestampMixin, UpdatedAtMixin):
+    """A per-user named snapshot of a module's filter/sort state (a "smart view").
+
+    `source` names the module the view belongs to ("feed" | "plex"); `filters` is that module's
+    serialized filter JSON the frontend applies with one click. `position` orders them in the sidebar
+    (drag-reorder, per source); `is_default` marks the one view applied automatically on load — at most
+    one per (user, source), enforced by the API on set. The name is unique per (user, source)."""
+
+    __tablename__ = "saved_views"
+    __table_args__ = (
+        UniqueConstraint("user_id", "source", "name", name="uq_user_source_saved_view"),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    source: Mapped[str] = mapped_column(String(16), default="feed", server_default="feed")
+    name: Mapped[str] = mapped_column(String(80))
+    filters: Mapped[dict] = mapped_column(JSON)
+    position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+    is_default: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
+
+
+class ComfortPin(Base, TimestampMixin):
+    """One entry on a user's comfort shelf — the anti-paralysis fallback: "just put on X".
+
+    Deliberately NOT a boolean on `VideoState`/`PlexState`, and NOT a polymorphic `(kind, ref)` string
+    pair. A flag cannot ORDER a shelf and cannot hold a channel or a show — and comfort media is
+    usually "put on X", a series or a channel, not one particular video. A string ref would order and
+    hold anything, but it would also lose the foreign key: a Plex item dropped from the library, or an
+    unsubscribed channel, would leave a dead tile on the one surface whose whole job is to be
+    reliable. So instead: one nullable CASCADE-ing FK per pinnable kind, and a check constraint that
+    exactly one of them is set.
+
+    That constraint is the house rule applied to a five-way union — the invariant lives in the schema,
+    not in a comment every insert site has to remember. `kind`/`ref` below derive FROM the columns, so
+    there is no second vocabulary to keep in sync.
+
+    NULLs are DISTINCT in a Postgres unique index, so the five per-kind unique constraints each stop a
+    double-pin of the same thing without colliding across kinds."""
+
+    __tablename__ = "comfort_pins"
+    __table_args__ = (
+        CheckConstraint(
+            "num_nonnulls(video_id, channel_id, plex_item_id, plex_show_id, playlist_id) = 1",
+            name="ck_comfort_pin_one_ref",
+        ),
+        UniqueConstraint("user_id", "video_id", name="uq_comfort_pin_video"),
+        UniqueConstraint("user_id", "channel_id", name="uq_comfort_pin_channel"),
+        UniqueConstraint("user_id", "plex_item_id", name="uq_comfort_pin_plex_item"),
+        UniqueConstraint("user_id", "plex_show_id", name="uq_comfort_pin_plex_show"),
+        UniqueConstraint("user_id", "playlist_id", name="uq_comfort_pin_playlist"),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
+
+    video_id: Mapped[str | None] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
+    channel_id: Mapped[str | None] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
+    plex_item_id: Mapped[int | None] = mapped_column(
+        ForeignKey("plex_items.id", ondelete="CASCADE")
+    )
+    plex_show_id: Mapped[int | None] = mapped_column(
+        ForeignKey("plex_shows.id", ondelete="CASCADE")
+    )
+    playlist_id: Mapped[int | None] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
+
+    # 0-based shelf order. The shelf is short and hand-curated, so a dense integer the reorder
+    # endpoint rewrites wholesale beats a fractional index nobody would ever exercise here.
+    position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
+
+
+class TakeoutKind(StrEnum):
+    """What a Google Takeout row says the user did with a video."""
+
+    WATCHED = "watched"  # history/watch-history.json
+    WATCH_LATER = "watch_later"  # playlists/Watch later-videos.csv
+
+
+class TakeoutOutcome(StrEnum):
+    """What the import did with a row; NULL = still waiting for its video to be known (and, for a
+    watched one, probed for Shorts)."""
+
+    WATCHED = "watched"  # the video is marked watched at the Takeout time
+    KEPT = "kept"  # the state the user set here (watched, hidden, or changed later) stays
+    SHORT = "short"  # a Short: swiping one past is not watching it
+    ADDED = "added"  # added to Watch later
+    PRESENT = "present"  # already in Watch later
+    MISSING = "missing"  # YouTube no longer has the video
+
+
+class TakeoutItem(Base, TimestampMixin):
+    """One video from a user's Google Takeout export: watched (with when) or in their YouTube Watch
+    later list (with when it was added). The rows are both the import job's input and its memory: a
+    video the catalog does not know yet waits here, with no outcome, until it enters the catalog.
+    The id is not a foreign key for that reason. A watched row's channel and length feed the
+    "channels you watch but do not follow" list: the length is asked of YouTube for that list alone
+    and kept here, so its videos need not enter the shared catalog to be counted."""
+
+    __tablename__ = "takeout_items"
+    __table_args__ = (
+        UniqueConstraint("user_id", "kind", "video_id", name="uq_takeout_item"),
+        CheckConstraint("kind IN ('watched', 'watch_later')", name="ck_takeout_item_kind"),
+        # The waiting rows, by video: what a newly known video looks up.
+        Index(
+            "ix_takeout_items_waiting",
+            "video_id",
+            postgresql_where=text("outcome IS NULL"),
+        ),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    kind: Mapped[str] = mapped_column(String(12))
+    video_id: Mapped[str] = mapped_column(String(16))
+    channel_id: Mapped[str | None] = mapped_column(String(32))
+    # Watched: the last time in the export; Watch later: when it was added.
+    at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
+    outcome: Mapped[str | None] = mapped_column(String(12))
+    # When YouTube was asked for the video's length (NULL = not yet), and the length it gave; a
+    # video it no longer has keeps a NULL length.
+    looked_up_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    duration_seconds: Mapped[int | None] = mapped_column(Integer)
+
+
+class TakeoutChannelDismissal(Base, TimestampMixin):
+    """A channel the user put out of their "watched but not followed" list. The id is not a foreign
+    key: the channel may never have entered the catalog, or may leave it, and the choice stays."""
+
+    __tablename__ = "takeout_channel_dismissals"
+    __table_args__ = (
+        UniqueConstraint("user_id", "channel_id", name="uq_takeout_channel_dismissal"),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    channel_id: Mapped[str] = mapped_column(String(32))

backend/app/reader/queries.py

@@ -0,0 +1,793 @@
+"""Reader's reads: a user's labels, saved views, watch state, playlists and Takeout import, for the HTTP
+layer and for the other units that ask about them.
+
+Every read of a user's row goes through `userscope`, so "someone else's row" is the same answer as
+a missing one. Reader reads Catalog's videos along its declared edge (a playlist's cover and
+length, whether a video exists); another unit asks the questions at the end of this module rather
+than joining Reader's tables itself.
+"""
+
+from collections.abc import Iterable
+from datetime import datetime
+
+from sqlalchemy import ColumnElement, and_, func, or_, select
+from sqlalchemy.orm import InstrumentedAttribute, Session, aliased
+
+from app.collation import collated
+from app.feedquery import feed_columns, serialize
+from app.models import User
+from app.catalog.models import BlockedChannel, Channel, Subscription, Video
+from app.reader.models import (
+    ChannelTag,
+    Playlist,
+    PlaylistItem,
+    SavedView,
+    Tag,
+    TakeoutChannelDismissal,
+    TakeoutItem,
+    TakeoutKind,
+    VideoState,
+    VideoUnmark,
+)
+from app.userscope import owned_by, owned_row
+
+# --- catalog, read along the edge ------------------------------------------------------------
+
+
+def video(db: Session, video_id: str) -> Video | None:
+    return db.get(Video, video_id)
+
+
+# --- tags ------------------------------------------------------------------------------------
+
+
+def tags_with_counts(db: Session, user: User) -> list[tuple[Tag, int, int]]:
+    """The tags this user sees (the shared system ones and their own), by category then name, each
+    with how many channel links it has and how many of the user's own links sit on a channel they
+    subscribe to and have not hidden — the rows the Channels page's tag filter shows (a hidden
+    channel has its own tab). A link outlives an unsubscribe and a hide, so the two differ."""
+    subscribed = select(Subscription.channel_id).where(
+        owned_by(Subscription.user_id, user), Subscription.hidden.is_(False)
+    )
+    on_subscribed = dict(
+        db.execute(
+            select(ChannelTag.tag_id, func.count(ChannelTag.id))
+            .where(owned_by(ChannelTag.user_id, user), ChannelTag.channel_id.in_(subscribed))
+            .group_by(ChannelTag.tag_id)
+        )
+        .tuples()
+        .all()
+    )
+    return [
+        (tag, count, on_subscribed.get(tag.id, 0))
+        for tag, count in db.execute(
+            select(Tag, func.count(ChannelTag.id))
+            .outerjoin(ChannelTag, ChannelTag.tag_id == Tag.id)
+            .where(or_(Tag.user_id.is_(None), owned_by(Tag.user_id, user)))
+            .group_by(Tag.id)
+            .order_by(Tag.category, Tag.name)
+        ).tuples()
+    ]
+
+
+def own_tag(db: Session, user: User, tag_id: int) -> Tag | None:
+    """One of the user's own tags. A system tag (no owner) is nobody's to edit, so it reads as
+    missing too."""
+    return owned_row(db, Tag, tag_id, user)
+
+
+# --- saved views -----------------------------------------------------------------------------
+
+
+def views(db: Session, user: User, source: str) -> list[SavedView]:
+    return list(
+        db.execute(
+            select(SavedView)
+            .where(owned_by(SavedView.user_id, user), SavedView.source == source)
+            .order_by(SavedView.position, SavedView.id)
+        ).scalars()
+    )
+
+
+def all_views(db: Session, user: User) -> list[SavedView]:
+    return list(db.execute(select(SavedView).where(owned_by(SavedView.user_id, user))).scalars())
+
+
+def own_view(db: Session, user: User, view_id: int) -> SavedView | None:
+    return owned_row(db, SavedView, view_id, user)
+
+
+# --- watch state -----------------------------------------------------------------------------
+
+
+def video_state(db: Session, user: User, video_id: str) -> VideoState | None:
+    return db.execute(
+        select(VideoState).where(
+            owned_by(VideoState.user_id, user), VideoState.video_id == video_id
+        )
+    ).scalar_one_or_none()
+
+
+# --- playlists -------------------------------------------------------------------------------
+
+
+def own_playlist(
+    db: Session, user: User, playlist_id: int, *, lock: bool = False
+) -> Playlist | None:
+    return owned_row(db, Playlist, playlist_id, user, lock=lock)
+
+
+def playlists_for(db: Session, user: User) -> list[Playlist]:
+    """The user's playlists in their rail order."""
+    return list(
+        db.execute(
+            select(Playlist)
+            .where(owned_by(Playlist.user_id, user))
+            .order_by(Playlist.position, Playlist.created_at)
+        ).scalars()
+    )
+
+
+def watch_later(db: Session, user: User) -> Playlist | None:
+    return db.scalar(
+        select(Playlist).where(owned_by(Playlist.user_id, user), Playlist.kind == "watch_later")
+    )
+
+
+def item_counts(db: Session, ids: list[int]) -> dict[int, int]:
+    return dict(
+        db.execute(
+            select(PlaylistItem.playlist_id, func.count())
+            .where(PlaylistItem.playlist_id.in_(ids))
+            .group_by(PlaylistItem.playlist_id)
+        )
+        .tuples()
+        .all()
+    )
+
+
+def durations(db: Session, ids: list[int]) -> dict[int, tuple[int, int]]:
+    """Each playlist's running time — the sum of its videos' known lengths — and how many videos
+    it left out for having none (`Video.without_length`)."""
+    return {
+        pid: (int(total or 0), int(left_out or 0))
+        for pid, total, left_out in db.execute(
+            select(
+                PlaylistItem.playlist_id,
+                func.coalesce(func.sum(Video.length_seconds), 0),
+                func.count().filter(Video.without_length),
+            )
+            .join(Video, Video.id == PlaylistItem.video_id)
+            .where(PlaylistItem.playlist_id.in_(ids))
+            .group_by(PlaylistItem.playlist_id)
+        ).tuples()
+    }
+
+
+def containing(db: Session, ids: list[int], video_id: str) -> set[int]:
+    """Which of these playlists already hold the video (the add-to-playlist popover's ticks)."""
+    return set(
+        db.execute(
+            select(PlaylistItem.playlist_id).where(
+                PlaylistItem.playlist_id.in_(ids), PlaylistItem.video_id == video_id
+            )
+        ).scalars()
+    )
+
+
+def covers(db: Session, ids: list[int]) -> dict[int, str | None]:
+    """Each playlist's cover: its first item's thumbnail, for every list in ONE window query rather
+    than a lookup per playlist."""
+    rn = (
+        func.row_number()
+        .over(partition_by=PlaylistItem.playlist_id, order_by=PlaylistItem.position)
+        .label("rn")
+    )
+    first_items = (
+        select(PlaylistItem.playlist_id.label("pid"), Video.thumbnail_url.label("thumb"), rn)
+        .join(Video, Video.id == PlaylistItem.video_id)
+        .where(PlaylistItem.playlist_id.in_(ids))
+        .subquery()
+    )
+    return dict(
+        db.execute(select(first_items.c.pid, first_items.c.thumb).where(first_items.c.rn == 1))
+        .tuples()
+        .all()
+    )
+
+
+def playlist_totals(db: Session, playlist_id: int) -> tuple[int, int, int, str | None]:
+    """One playlist's item count, running time, the videos the running time left out for having no
+    length (`Video.without_length`) and cover. An outer join, so an empty list still answers
+    (0, 0, 0, None)."""
+    count, total, left_out = db.execute(
+        select(
+            func.count(PlaylistItem.id),
+            func.coalesce(func.sum(Video.length_seconds), 0),
+            func.count().filter(Video.without_length),
+        )
+        .select_from(PlaylistItem)
+        .outerjoin(Video, Video.id == PlaylistItem.video_id)
+        .where(PlaylistItem.playlist_id == playlist_id)
+    ).one()
+    cover = db.scalar(
+        select(Video.thumbnail_url)
+        .join(PlaylistItem, PlaylistItem.video_id == Video.id)
+        .where(PlaylistItem.playlist_id == playlist_id)
+        .order_by(PlaylistItem.position)
+        .limit(1)
+    )
+    return count or 0, total or 0, left_out or 0, cover
+
+
+def playlist_rows(db: Session, user: User, playlist_id: int) -> list:
+    """A playlist's videos as feed rows (the same columns the feed serializes), in list order."""
+    state = aliased(VideoState)
+    return list(
+        db.execute(
+            select(*feed_columns(user, state))
+            .join(PlaylistItem, PlaylistItem.video_id == Video.id)
+            .join(Channel, Channel.id == Video.channel_id)
+            .outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
+            .where(PlaylistItem.playlist_id == playlist_id)
+            # `id` breaks a position tie the same way on every read (the renumbering add does too).
+            .order_by(PlaylistItem.position, PlaylistItem.id)
+        ).all()
+    )
+
+
+def items(db: Session, playlist_id: int) -> list[PlaylistItem]:
+    """A playlist's item rows, in list order."""
+    return list(
+        db.execute(
+            select(PlaylistItem)
+            .where(PlaylistItem.playlist_id == playlist_id)
+            .order_by(PlaylistItem.position, PlaylistItem.id)
+        ).scalars()
+    )
+
+
+def item(db: Session, playlist_id: int, video_id: str) -> PlaylistItem | None:
+    return db.scalar(
+        select(PlaylistItem).where(
+            PlaylistItem.playlist_id == playlist_id, PlaylistItem.video_id == video_id
+        )
+    )
+
+
+# --- for the other units ---------------------------------------------------------------------
+
+
+def kept_by_a_reader(video_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
+    """A video someone holds: watched, in progress or hidden (a `VideoState`), or in one of their
+    playlists. The catalog's cleanup jobs keep such a video; as a clause, so their DELETE stays
+    one statement."""
+    return or_(
+        select(VideoState.id).where(VideoState.video_id == video_id).exists(),
+        select(PlaylistItem.id).where(PlaylistItem.video_id == video_id).exists(),
+    )
+
+
+def in_a_watch_history(channel_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
+    """A channel someone's uploaded Takeout history shows watched. The "watched but not followed"
+    list names a channel from its catalog row, so the explore cleanup keeps the row of one (its
+    videos it may still reclaim: opening the list's videos brings them back). As a clause, so the
+    cleanup's DELETE stays one statement."""
+    return (
+        select(TakeoutItem.id)
+        .where(TakeoutItem.channel_id == channel_id, TakeoutItem.kind == TakeoutKind.WATCHED)
+        .exists()
+    )
+
+
+def playlisted_by(db: Session, video_ids: list[str]) -> list[tuple[int, str]]:
+    """(user id, video id) for every user who has one of these videos in a playlist."""
+    return list(
+        db.execute(
+            select(Playlist.user_id, PlaylistItem.video_id)
+            .join(PlaylistItem, PlaylistItem.playlist_id == Playlist.id)
+            .where(PlaylistItem.video_id.in_(video_ids))
+        ).tuples()
+    )
+
+
+def has_auto_tags(channel_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
+    """A channel the auto-tagger has already labelled, as a clause for its "only the missing" scan."""
+    return (
+        select(ChannelTag.id)
+        .where(
+            ChannelTag.channel_id == channel_id,
+            ChannelTag.source == "auto",
+            ChannelTag.user_id.is_(None),
+        )
+        .exists()
+    )
+
+
+def view_tag_ids(filters: object) -> list[int]:
+    """The tag ids a stored view's filters name; anything else in that slot names none."""
+    tags = filters.get("tags") if isinstance(filters, dict) else None
+    if not isinstance(tags, list):
+        return []
+    return [t for t in tags if type(t) is int]  # `type`, not isinstance: True is not a tag id
+
+
+def tags_in_feed_views(db: Session) -> set[int]:
+    """Every tag id any user's saved feed view filters on. Read defensively (`view_tag_ids`): a view
+    stored before its route checked the shape must not raise for everyone's job."""
+    return {
+        t
+        for (filters,) in db.execute(select(SavedView.filters).where(SavedView.source == "feed"))
+        for t in view_tag_ids(filters)
+    }
+
+
+def held_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
+    """Which of these videos the user already holds: watched, in progress or hidden, or in one of
+    their playlists (a live search sinks them below the new finds)."""
+    if not video_ids:
+        return set()
+    has_state = (
+        select(VideoState.id)
+        .where(VideoState.video_id == Video.id, owned_by(VideoState.user_id, user))
+        .exists()
+    )
+    in_playlist = (
+        select(PlaylistItem.id)
+        .join(Playlist, Playlist.id == PlaylistItem.playlist_id)
+        .where(PlaylistItem.video_id == Video.id, owned_by(Playlist.user_id, user))
+        .exists()
+    )
+    return set(
+        db.scalars(select(Video.id).where(Video.id.in_(video_ids), or_(has_state, in_playlist)))
+    )
+
+
+def seen_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
+    """Which of these videos the user has already seen or set aside: watched or hidden here, or in
+    their uploaded Takeout history unless they took it back to "new" here since. The ids need not
+    be in the catalog — a Takeout row names a video the catalog may never have had — so a list
+    of what YouTube suggests can drop them before anything is ingested."""
+    if not video_ids:
+        return set()
+    marked = select(VideoState.video_id).where(
+        owned_by(VideoState.user_id, user),
+        VideoState.video_id.in_(video_ids),
+        VideoState.status.in_(("watched", "hidden")),
+    )
+    unmarked_since = (
+        select(VideoUnmark.id)
+        .where(
+            owned_by(VideoUnmark.user_id, user),
+            VideoUnmark.video_id == TakeoutItem.video_id,
+            VideoUnmark.at > TakeoutItem.at,
+        )
+        .exists()
+    )
+    in_history = select(TakeoutItem.video_id).where(
+        owned_by(TakeoutItem.user_id, user),
+        TakeoutItem.kind == TakeoutKind.WATCHED,
+        TakeoutItem.video_id.in_(video_ids),
+        ~unmarked_since,
+    )
+    return set(db.scalars(marked)) | set(db.scalars(in_history))
+
+
+def feed_rows_by_ids(db: Session, user: User, video_ids: list[str]) -> list[dict]:
+    """These videos serialized with the feed's projection (their channel, this user's watch state,
+    saved), in the order given."""
+    if not video_ids:
+        return []
+    state = aliased(VideoState)
+    rows = db.execute(
+        select(*feed_columns(user, state))
+        .join(Channel, Channel.id == Video.channel_id)
+        .outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
+        .where(Video.id.in_(video_ids))
+    ).all()
+    by_id = {r.id: serialize(r) for r in rows}
+    return [by_id[i] for i in video_ids if i in by_id]
+
+
+def playlist_channels(db: Session, user: User, exclude: str | None, lang: str) -> list:
+    """The channels of the videos in the user's playlists that they don't subscribe to (`exclude`:
+    their own channel), each with how many of those videos it has and across how many playlists;
+    most present first, then by title in the reader's alphabet."""
+    subscribed = select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
+    q = (
+        select(
+            Channel,
+            func.count(func.distinct(PlaylistItem.video_id)),
+            func.count(func.distinct(Playlist.id)),
+        )
+        .select_from(PlaylistItem)
+        .join(Playlist, Playlist.id == PlaylistItem.playlist_id)
+        .join(Video, Video.id == PlaylistItem.video_id)
+        .join(Channel, Channel.id == Video.channel_id)
+        .where(owned_by(Playlist.user_id, user), Channel.id.not_in(subscribed))
+    )
+    if exclude:
+        q = q.where(Channel.id != exclude)
+    return list(
+        db.execute(
+            q.group_by(Channel.id).order_by(
+                func.count(func.distinct(PlaylistItem.video_id)).desc(),
+                collated(Channel.title, lang),
+            )
+        ).all()
+    )
+
+
+def user_tag_links(db: Session, user: User, channel_id: str | None = None) -> dict[str, list[int]]:
+    """The user's own tag links by channel (one channel's, when given)."""
+    stmt = select(ChannelTag.channel_id, ChannelTag.tag_id).where(
+        owned_by(ChannelTag.user_id, user)
+    )
+    if channel_id is not None:
+        stmt = stmt.where(ChannelTag.channel_id == channel_id)
+    out: dict[str, list[int]] = {}
+    for cid, tag_id in db.execute(stmt).tuples():
+        out.setdefault(cid, []).append(tag_id)
+    return out
+
+
+def user_tag_link(db: Session, user: User, channel_id: str, tag_id: int) -> ChannelTag | None:
+    return db.execute(
+        select(ChannelTag).where(
+            ChannelTag.channel_id == channel_id,
+            ChannelTag.tag_id == tag_id,
+            owned_by(ChannelTag.user_id, user),
+        )
+    ).scalar_one_or_none()
+
+
+# --- Google Takeout --------------------------------------------------------------------------
+
+
+def _settled_here() -> ColumnElement[bool]:
+    """A video a waiting "watched" row may be marked on: one the catalog keeps of its own, not a
+    discovery video only a search or an explore brought in. A mark is a `VideoState`, which the
+    cleanup jobs read as a reader keeping it (`kept_by_a_reader`): on one of those it would keep the
+    video, and its explored channel, for good. Such a row waits until the video is the catalog's own
+    (its channel subscribed to) — or is gone."""
+    subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
+    return and_(Video.via_explore.is_(False), or_(Video.via_search.is_(False), subscribed))
+
+
+def takeout_watched_ready(db: Session, user: User) -> list[tuple[int, str, datetime, bool]]:
+    """The user's waiting "watched" rows whose video the catalog knows and has probed for Shorts:
+    (row id, video id, Takeout time, is Short)."""
+    return list(
+        db.execute(
+            select(TakeoutItem.id, TakeoutItem.video_id, TakeoutItem.at, Video.is_short)
+            .join(Video, Video.id == TakeoutItem.video_id)
+            .where(
+                owned_by(TakeoutItem.user_id, user),
+                TakeoutItem.kind == TakeoutKind.WATCHED,
+                TakeoutItem.outcome.is_(None),
+                Video.shorts_probed,
+                _settled_here(),
+            )
+        ).tuples()
+    )
+
+
+def takeout_waiting(db: Session, user: User, kind: TakeoutKind) -> list[tuple[int, str]]:
+    """The user's rows of `kind` the import has not settled, oldest first: (row id, video id)."""
+    return list(
+        db.execute(
+            select(TakeoutItem.id, TakeoutItem.video_id)
+            .where(
+                owned_by(TakeoutItem.user_id, user),
+                TakeoutItem.kind == kind,
+                TakeoutItem.outcome.is_(None),
+            )
+            .order_by(TakeoutItem.at, TakeoutItem.id)
+        ).tuples()
+    )
+
+
+# Ids per IN list: a list is bound one parameter an id, and a statement takes at most 65,535.
+_IN_CHUNK = 5000
+
+
+def known_video_ids(db: Session, ids: list[str]) -> set[str]:
+    """Which of `ids` the catalog holds."""
+    known: set[str] = set()
+    for start in range(0, len(ids), _IN_CHUNK):
+        chunk = ids[start : start + _IN_CHUNK]
+        known.update(db.execute(select(Video.id).where(Video.id.in_(chunk))).scalars())
+    return known
+
+
+def unmarked_at(db: Session, user: User, ids: list[str]) -> dict[str, datetime]:
+    """When the user last took each of `ids` back to "new" here, for those they ever did."""
+    out: dict[str, datetime] = {}
+    for start in range(0, len(ids), _IN_CHUNK):
+        rows = db.execute(
+            select(VideoUnmark.video_id, VideoUnmark.at).where(
+                owned_by(VideoUnmark.user_id, user),
+                VideoUnmark.video_id.in_(ids[start : start + _IN_CHUNK]),
+            )
+        ).tuples()
+        out.update((vid, at) for vid, at in rows)
+    return out
+
+
+def in_playlist(db: Session, playlist_id: int, ids: list[str]) -> set[str]:
+    """Which of `ids` the playlist already holds."""
+    held: set[str] = set()
+    for start in range(0, len(ids), _IN_CHUNK):
+        chunk = ids[start : start + _IN_CHUNK]
+        held.update(
+            db.execute(
+                select(PlaylistItem.video_id).where(
+                    PlaylistItem.playlist_id == playlist_id, PlaylistItem.video_id.in_(chunk)
+                )
+            ).scalars()
+        )
+    return held
+
+
+def takeout_rows_besides(
+    db: Session, user: User, kind: TakeoutKind, video_ids: Iterable[str]
+) -> int:
+    """How many Takeout rows the user keeps other than `video_ids` of `kind`: what an upload of
+    those videos would be added to."""
+    total = (
+        db.scalar(select(func.count(TakeoutItem.id)).where(owned_by(TakeoutItem.user_id, user)))
+        or 0
+    )
+    ids = list(video_ids)
+    for start in range(0, len(ids), _IN_CHUNK):
+        total -= (
+            db.scalar(
+                select(func.count(TakeoutItem.id)).where(
+                    owned_by(TakeoutItem.user_id, user),
+                    TakeoutItem.kind == kind,
+                    TakeoutItem.video_id.in_(ids[start : start + _IN_CHUNK]),
+                )
+            )
+            or 0
+        )
+    return total
+
+
+def takeout_outcomes(db: Session, user: User) -> dict[tuple[str, str | None], int]:
+    """How many of the user's rows each kind and outcome holds (None = waiting)."""
+    rows = db.execute(
+        select(TakeoutItem.kind, TakeoutItem.outcome, func.count())
+        .where(owned_by(TakeoutItem.user_id, user))
+        .group_by(TakeoutItem.kind, TakeoutItem.outcome)
+    ).all()
+    return {(kind, outcome): n for kind, outcome, n in rows}
+
+
+def takeout_summary(db: Session, user: User) -> dict[str, dict[str, int]]:
+    """The user's kept Takeout videos by kind and outcome, for their settings page: `{"watched":
+    {"watched": n, "short": n, "waiting": n, …}, "watch_later": {…}}` — the import's outcome words,
+    "waiting" for a video not known yet."""
+    out: dict[str, dict[str, int]] = {kind.value: {} for kind in TakeoutKind}
+    for (kind, outcome), n in takeout_outcomes(db, user).items():
+        out[kind][outcome or "waiting"] = n
+    return out
+
+
+def takeout_users_ready(db: Session) -> list[int]:
+    """Every user with a waiting "watched" row whose video is now known and probed for Shorts: the
+    rows a later run of the Shorts probe made ready."""
+    return list(
+        db.execute(
+            select(TakeoutItem.user_id)
+            .join(Video, Video.id == TakeoutItem.video_id)
+            .where(
+                TakeoutItem.kind == TakeoutKind.WATCHED,
+                TakeoutItem.outcome.is_(None),
+                Video.shorts_probed,
+                _settled_here(),
+            )
+            .distinct()
+        ).scalars()
+    )
+
+
+def takeout_users_watch_later_waiting(db: Session) -> list[int]:
+    """Every user with a Watch later row a run could not put in yet."""
+    return list(
+        db.execute(
+            select(TakeoutItem.user_id)
+            .where(TakeoutItem.kind == TakeoutKind.WATCH_LATER, TakeoutItem.outcome.is_(None))
+            .distinct()
+        ).scalars()
+    )
+
+
+# --- channels watched but not followed ---------------------------------------------------------
+
+# A channel joins the list with this many watched videos of at least this length. A Short is three
+# minutes at most, so the length alone keeps them out, and a video under five minutes says too
+# little of what a channel is about to count towards it.
+WATCHED_CHANNEL_MIN_VIDEOS = 5
+WATCHED_CHANNEL_MIN_SECONDS = 300
+
+
+def _unfollowed(user: User) -> ColumnElement[bool]:
+    """A watched row whose channel the user neither follows, blocked, put out of the list, nor is."""
+    subscribed = select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
+    blocked = select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user))
+    dismissed = select(TakeoutChannelDismissal.channel_id).where(
+        owned_by(TakeoutChannelDismissal.user_id, user)
+    )
+    clauses = [
+        owned_by(TakeoutItem.user_id, user),
+        TakeoutItem.kind == TakeoutKind.WATCHED,
+        TakeoutItem.channel_id.is_not(None),
+        TakeoutItem.channel_id.not_in(subscribed),
+        TakeoutItem.channel_id.not_in(blocked),
+        TakeoutItem.channel_id.not_in(dismissed),
+    ]
+    if user.yt_channel_id:
+        clauses.append(TakeoutItem.channel_id != user.yt_channel_id)
+    return and_(*clauses)
+
+
+def _candidate_channels(user: User):
+    """The unfollowed channels with enough watched videos to make the list once their lengths are
+    known: (channel id, how many watched)."""
+    return (
+        select(TakeoutItem.channel_id, func.count().label("watched"))
+        .where(_unfollowed(user))
+        .group_by(TakeoutItem.channel_id)
+        .having(func.count() >= WATCHED_CHANNEL_MIN_VIDEOS)
+        .subquery()
+    )
+
+
+def takeout_lengths_to_ask(db: Session, user: User) -> list[str]:
+    """The watched videos whose length the list still needs: those of the candidate channels not
+    asked about yet, the most watched channels first, so a lookup cut short fills the top."""
+    cand = _candidate_channels(user)
+    return list(
+        db.execute(
+            select(TakeoutItem.video_id)
+            .join(cand, cand.c.channel_id == TakeoutItem.channel_id)
+            .where(_unfollowed(user), TakeoutItem.looked_up_at.is_(None))
+            .order_by(cand.c.watched.desc(), TakeoutItem.channel_id, TakeoutItem.at.desc())
+        ).scalars()
+    )
+
+
+def watched_channels_pending(db: Session, user: User) -> int:
+    """How many channels a later run may still add to the list: the candidates with a watched video
+    whose length is not known, and the ones that made it but have no catalog row to be listed from
+    yet. One the catalog holds without details is listed already (by the name it came in with), so
+    it is not one to come."""
+    cand = _candidate_channels(user)
+    unasked = db.execute(
+        select(TakeoutItem.channel_id)
+        .join(cand, cand.c.channel_id == TakeoutItem.channel_id)
+        .where(_unfollowed(user), TakeoutItem.looked_up_at.is_(None))
+        .distinct()
+    ).scalars()
+    counted = _counted_channels(user)
+    unlisted = db.execute(
+        select(counted.c.channel_id)
+        .outerjoin(Channel, Channel.id == counted.c.channel_id)
+        .where(Channel.id.is_(None))
+    ).scalars()
+    return len(set(unasked) | set(unlisted))
+
+
+def _counted_channels(user: User):
+    """The channels that make the list by their counted videos, whatever the catalog holds of them."""
+    return (
+        select(TakeoutItem.channel_id)
+        .where(_unfollowed(user), TakeoutItem.duration_seconds >= WATCHED_CHANNEL_MIN_SECONDS)
+        .group_by(TakeoutItem.channel_id)
+        .having(func.count() >= WATCHED_CHANNEL_MIN_VIDEOS)
+        .subquery()
+    )
+
+
+def watched_channels_to_describe(db: Session, user: User) -> list[str]:
+    """The channels on the list (by their counted videos) that the catalog lacks or has no details
+    of yet: what the list cannot name, or names with no more than a video brought."""
+    counted = _counted_channels(user)
+    return list(
+        db.execute(
+            select(counted.c.channel_id)
+            .outerjoin(Channel, Channel.id == counted.c.channel_id)
+            .where(Channel.details_synced_at.is_(None))
+        ).scalars()
+    )
+
+
+def watched_channels(db: Session, user: User) -> list[tuple[Channel, int, datetime]]:
+    """The channels the user watched often but does not follow: (channel, how many of its videos
+    over five minutes they watched, when last). Most watched first; on a tie the one watched more
+    lately, which is the likelier interest."""
+    n = func.count(TakeoutItem.id)
+    last = func.max(TakeoutItem.at)
+    return list(
+        db.execute(
+            select(Channel, n, last)
+            .join(Channel, Channel.id == TakeoutItem.channel_id)
+            .where(_unfollowed(user), TakeoutItem.duration_seconds >= WATCHED_CHANNEL_MIN_SECONDS)
+            .group_by(Channel.id)
+            .having(n >= WATCHED_CHANNEL_MIN_VIDEOS)
+            .order_by(n.desc(), last.desc(), Channel.id)
+        ).tuples()
+    )
+
+
+def has_watch_history(db: Session, user: User) -> bool:
+    """Whether the user has uploaded a Takeout watch history at all."""
+    return bool(
+        db.scalar(
+            select(
+                select(TakeoutItem.id)
+                .where(owned_by(TakeoutItem.user_id, user), TakeoutItem.kind == TakeoutKind.WATCHED)
+                .exists()
+            )
+        )
+    )
+
+
+def watched_in_history(db: Session, user: User, channel_id: str) -> bool:
+    """Whether the user's Takeout history shows them watching this channel at all."""
+    return bool(
+        db.scalar(
+            select(
+                select(TakeoutItem.id)
+                .where(
+                    owned_by(TakeoutItem.user_id, user),
+                    TakeoutItem.kind == TakeoutKind.WATCHED,
+                    TakeoutItem.channel_id == channel_id,
+                )
+                .exists()
+            )
+        )
+    )
+
+
+def watched_channel_videos(
+    db: Session, user: User, channel_id: str
+) -> tuple[list[tuple[str, datetime]], int]:
+    """The user's watched videos of one channel that the list counts, last watched first: (video
+    id, when). And how many YouTube no longer has."""
+    rows = db.execute(
+        select(TakeoutItem.video_id, TakeoutItem.at, TakeoutItem.duration_seconds)
+        .where(
+            owned_by(TakeoutItem.user_id, user),
+            TakeoutItem.kind == TakeoutKind.WATCHED,
+            TakeoutItem.channel_id == channel_id,
+            TakeoutItem.looked_up_at.is_not(None),
+        )
+        .order_by(TakeoutItem.at.desc(), TakeoutItem.video_id)
+    ).tuples()
+    counted: list[tuple[str, datetime]] = []
+    gone = 0
+    for vid, at, seconds in rows:
+        if seconds is None:
+            gone += 1
+        elif seconds >= WATCHED_CHANNEL_MIN_SECONDS:
+            counted.append((vid, at))
+    return counted, gone
+
+
+def takeout_users_with_list_work(db: Session) -> list[int]:
+    """Every user whose "watched but not followed" list a run can still add to: lengths to ask, or
+    channels to name. Asked of each user whose history holds a channel of enough watched videos,
+    since what is off a list (followed, blocked, put out, their own) is that user's own — a row of
+    one is never asked about, and would otherwise bring the user back on every run."""
+    per_channel = (
+        select(TakeoutItem.user_id)
+        .where(TakeoutItem.kind == TakeoutKind.WATCHED, TakeoutItem.channel_id.is_not(None))
+        .group_by(TakeoutItem.user_id, TakeoutItem.channel_id)
+        .having(func.count() >= WATCHED_CHANNEL_MIN_VIDEOS)
+        .subquery()
+    )
+    users = db.execute(
+        select(User).where(User.id.in_(select(per_channel.c.user_id))).order_by(User.id)
+    ).scalars()
+    return [user.id for user in users if watched_channels_pending(db, user)]

backend/app/reader/service.py

@@ -0,0 +1,809 @@
+"""Reader's writes: a user's labels and saved views, where they are in a video, their playlists'
+items and order — and the system tags the catalog's auto-tagger hands over.
+
+Every function here flushes and leaves the commit to its caller (the handler, or the job that owns
+its session). A write that must not double up keeps its guard where it is written: an upsert on the
+unique key, a row lock taken before a presence check. The comfort shelf's writes are Home's
+(`discover.pins`); a playlist's YouTube sync is `sync.playlists`.
+"""
+
+from datetime import datetime, timezone
+
+from typing import Literal, cast
+
+from sqlalchemy import Table, bindparam, delete, exists, func, select, text, update
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+from sqlalchemy.orm import Session
+
+from app.models import User
+from app.reader import queries
+from app.reader.models import (
+    ChannelTag,
+    Playlist,
+    PlaylistItem,
+    SavedView,
+    Tag,
+    TakeoutChannelDismissal,
+    TakeoutItem,
+    TakeoutKind,
+    TakeoutOutcome,
+    VideoState,
+    VideoUnmark,
+)
+from app.sync.playlists import lock_playlists, recompute_dirty
+from app.userscope import owned_by
+
+TAG_CATEGORIES = ("language", "topic", "other")
+VIEW_NAME_MAX = 80
+
+# --- tags ------------------------------------------------------------------------------------
+
+
+def create_tag(db: Session, user: User, name: str, category: str, color: str | None) -> Tag:
+    """A tag of the user's own. An unknown category falls back to "other" rather than refusing. A
+    name the user already has raises `IntegrityError` at the flush."""
+    tag = Tag(
+        user_id=user.id,
+        name=name,
+        color=color,
+        category=category if category in TAG_CATEGORIES else "other",
+    )
+    db.add(tag)
+    db.flush()
+    return tag
+
+
+def update_tag(tag: Tag, *, sent: set[str], name: str | None, color: str | None) -> None:
+    """Change the fields the caller sent: `color` sent as None clears it."""
+    if "name" in sent and name is not None:
+        tag.name = name
+    if "color" in sent:
+        tag.color = color
+
+
+def delete_tag(db: Session, tag: Tag) -> None:
+    db.delete(tag)  # its channel links cascade
+    db.flush()
+
+
+def ensure_system_tag(db: Session, name: str, category: str, color: str | None) -> Tag:
+    """The shared system tag of this name, made the first time a channel earns it."""
+    tag = db.execute(
+        select(Tag).where(Tag.user_id.is_(None), Tag.name == name)
+    ).scalar_one_or_none()
+    if tag is None:
+        tag = Tag(user_id=None, name=name, category=category, color=color)
+        db.add(tag)
+        db.flush()
+    return tag
+
+
+def set_auto_tags(db: Session, channel_id: str, desired: dict[int, float | None]) -> None:
+    """Make a channel's automatic tag links exactly `desired` (tag id → confidence). Only the
+    auto-tagger's own links move; a user's links on the channel are never touched."""
+    existing = {
+        ct.tag_id: ct
+        for ct in db.execute(
+            select(ChannelTag).where(
+                ChannelTag.channel_id == channel_id,
+                ChannelTag.user_id.is_(None),
+                ChannelTag.source == "auto",
+            )
+        ).scalars()
+    }
+    for tag_id, ct in existing.items():
+        if tag_id not in desired:
+            db.delete(ct)
+    for tag_id, conf in desired.items():
+        ct = existing.get(tag_id)
+        if ct is None:
+            db.add(
+                ChannelTag(
+                    channel_id=channel_id,
+                    tag_id=tag_id,
+                    user_id=None,
+                    source="auto",
+                    confidence=conf,
+                )
+            )
+        else:
+            ct.confidence = conf
+    db.flush()
+
+
+def prune_orphan_system_tags(db: Session) -> int:
+    """Delete the system tags no channel links to any more (after a mapping change) — but not one a
+    saved feed view filters on. Views hold tag IDS, and a deleted tag comes back under a new id when
+    a channel earns it again, so the view's filter would point at nothing; the feed drops an unknown
+    id, and the view showed the whole feed under its old name. The kept ids are filtered here rather
+    than in SQL: a `NOT IN` over every view's ids could pass the driver's bind-parameter limit."""
+    in_views = queries.tags_in_feed_views(db)
+    orphans = [
+        tag
+        for tag in db.execute(
+            select(Tag).where(Tag.user_id.is_(None), ~exists().where(ChannelTag.tag_id == Tag.id))
+        ).scalars()
+        if tag.id not in in_views
+    ]
+    for tag in orphans:
+        db.delete(tag)
+    db.flush()
+    return len(orphans)
+
+
+# --- saved views -----------------------------------------------------------------------------
+
+
+def create_view(db: Session, user: User, source: str, name: str, filters: dict) -> SavedView:
+    """A view appended after the user's others of the same module (each module numbers its own
+    0..N). A name the user already has there raises `IntegrityError` at the flush."""
+    maxpos = db.execute(
+        select(func.coalesce(func.max(SavedView.position), -1)).where(
+            owned_by(SavedView.user_id, user), SavedView.source == source
+        )
+    ).scalar_one()
+    view = SavedView(
+        user_id=user.id,
+        source=source,
+        name=name[:VIEW_NAME_MAX],
+        filters=filters,
+        position=maxpos + 1,
+    )
+    db.add(view)
+    db.flush()
+    return view
+
+
+def update_view(
+    db: Session,
+    view: SavedView,
+    *,
+    sent: set[str],
+    name: str | None = None,
+    filters: dict | None = None,
+    is_default: bool | None = None,
+) -> None:
+    """Change the fields the caller sent. Making a view the default clears the flag on the user's
+    other views of the same module first: at most one default per (user, source)."""
+    if "name" in sent and name is not None:
+        view.name = name[:VIEW_NAME_MAX]
+    if "filters" in sent and filters is not None:
+        view.filters = filters
+    if "is_default" in sent:
+        make_default = bool(is_default)
+        if make_default:
+            db.execute(
+                update(SavedView)
+                .where(
+                    owned_by(SavedView.user_id, view.user_id),
+                    SavedView.source == view.source,
+                    SavedView.id != view.id,
+                )
+                .values(is_default=False)
+            )
+        view.is_default = make_default
+    db.flush()
+
+
+def delete_view(db: Session, view: SavedView) -> None:
+    db.delete(view)
+    db.flush()
+
+
+def reorder_views(db: Session, user: User, ids: list[int]) -> None:
+    """Number the listed views 0..n in the listed order; ids that are not the user's are skipped."""
+    by_id = {v.id: v for v in queries.all_views(db, user)}
+    pos = 0
+    for vid in ids:
+        v = by_id.get(vid)
+        if v is not None:
+            v.position = pos
+            pos += 1
+    db.flush()
+
+
+# --- watch state -----------------------------------------------------------------------------
+
+
+def unmark(db: Session, user: User, video_id: str) -> None:
+    """Back to "new". A row that still holds a resume point keeps it (un-marking "watched" restores
+    the in-progress state, not where the user left off); otherwise the row goes. The player fires
+    this beside a checkpoint that may delete the same row: the caller's commit then raises
+    `StaleDataError`, which is that lost race and nothing worse."""
+    lock_takeout(db, user)  # an import settling meanwhile reads the un-mark below, or comes first
+    row = queries.video_state(db, user, video_id)
+    if row is None:
+        return
+    if row.position_seconds:
+        row.status = "new"
+        row.watched_at = None
+    else:
+        db.delete(row)
+    _remember_unmark(db, user, video_id)
+    db.flush()
+
+
+def _remember_unmark(db: Session, user: User, video_id: str) -> None:
+    """Note that the user took the video back to "new" now: the state they set leaves no row, and
+    an import of their YouTube history must not mark it watched again (`apply_takeout_watched`)."""
+    now = datetime.now(timezone.utc)
+    db.execute(
+        pg_insert(VideoUnmark)
+        .values(user_id=user.id, video_id=video_id, at=now)
+        .on_conflict_do_update(constraint="uq_video_unmark", set_={"at": now})
+    )
+
+
+def mark(db: Session, user: User, video_id: str, status: str) -> None:
+    """ "watched" or "hidden", as one atomic upsert on (user, video). The player marks "watched" at
+    the end of playback at the instant a checkpoint may delete the in-progress row; a SELECT then
+    UPDATE raised `StaleDataError` on that race, and an INSERT … ON CONFLICT is immune to it.
+    "hidden" keeps any earlier `watched_at`."""
+    now = datetime.now(timezone.utc)
+    set_: dict = {"status": status, "updated_at": now}
+    if status == "watched":
+        set_["watched_at"] = now
+    db.execute(
+        pg_insert(VideoState)
+        .values(
+            user_id=user.id,
+            video_id=video_id,
+            status=status,
+            watched_at=now if status == "watched" else None,
+        )
+        .on_conflict_do_update(constraint="uq_user_video", set_=set_)
+    )
+
+
+def set_progress(db: Session, user: User, video_id: str, position: int) -> None:
+    """The player's resume point. 0 clears it: a row holding nothing else goes, one with a status
+    (watched, hidden) keeps it with the position zeroed. The status itself is never changed here.
+    A position is one upsert on (user, video): the first checkpoint and the end-of-playback mark
+    beside it both read "no row", and a plain add let the second one fail on `uq_user_video`."""
+    now = datetime.now(timezone.utc)
+    if position:
+        db.execute(
+            pg_insert(VideoState)
+            .values(
+                user_id=user.id,
+                video_id=video_id,
+                position_seconds=position,
+                progress_updated_at=now,
+            )
+            .on_conflict_do_update(
+                constraint="uq_user_video",
+                set_={"position_seconds": position, "progress_updated_at": now, "updated_at": now},
+            )
+        )
+        return
+    row = queries.video_state(db, user, video_id)
+    if row is None:
+        return
+    if row.status == "new":
+        db.delete(row)
+    else:
+        row.position_seconds = 0
+        row.progress_updated_at = now
+    db.flush()
+
+
+# --- playlists -------------------------------------------------------------------------------
+
+
+# Every reason a move is refused; the UI names each one (`playlists.moveErrors.<reason>`, held by the
+# frontend's backendIds contract, which reads this Literal).
+MoveReason = Literal["same_playlist"]
+
+
+class MoveRefused(Exception):
+    """A move between playlists that cannot happen: `reason` is the code the UI translates."""
+
+    def __init__(self, reason: MoveReason):
+        super().__init__(reason)
+        self.reason: MoveReason = reason
+
+
+def touch(pl: Playlist) -> None:
+    """Bump the playlist's `updated_at` for an edit that only writes `playlist_items`.
+
+    `UpdatedAtMixin`'s `onupdate` fires on an UPDATE of the PLAYLIST row, and add / remove /
+    reorder / move write the item rows instead — so without this the rail's "recently changed"
+    sort would only ever see renames. The assignment is what makes the column mean "last time
+    this list changed", which is what the rail promises the reader.
+
+    The DATABASE's clock, like the mixin's own `onupdate`: one sort key written from two clocks
+    (this process's and Postgres's) could order a rename and an item edit the wrong way round."""
+    pl.updated_at = func.now()
+
+
+def _next_playlist_position(db: Session, user_id: int) -> int:
+    """Append position for a new playlist in this user's rail."""
+    return (
+        db.execute(
+            select(func.coalesce(func.max(Playlist.position), -1)).where(
+                owned_by(Playlist.user_id, user_id)
+            )
+        ).scalar_one()
+        + 1
+    )
+
+
+def next_item_position(db: Session, playlist_id: int) -> int:
+    """Append position for a new item at the end of a playlist. Locks the playlist row first: two
+    adds (two tabs, the rail and the page) otherwise both read the same max and land on one
+    position, and the list's order between them is then arbitrary."""
+    lock_playlists(db, playlist_id)
+    return (
+        db.execute(
+            select(func.coalesce(func.max(PlaylistItem.position), -1)).where(
+                PlaylistItem.playlist_id == playlist_id
+            )
+        ).scalar_one()
+        + 1
+    )
+
+
+def create_playlist(db: Session, user: User, name: str) -> Playlist:
+    pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id))
+    db.add(pl)
+    db.flush()
+    return pl
+
+
+def watch_later(db: Session, user: User) -> Playlist:
+    """The user's built-in Watch later playlist, created on first use. Created by the insert itself
+    (ON CONFLICT DO NOTHING on the one-per-user index) and then read: the first two saves at once
+    both read "none", and each used to make a list of its own."""
+    pl = queries.watch_later(db, user)
+    if pl is None:
+        db.execute(
+            pg_insert(Playlist)
+            .values(
+                user_id=user.id,
+                name="Watch later",
+                kind="watch_later",
+                position=_next_playlist_position(db, user.id),
+            )
+            .on_conflict_do_nothing(
+                index_elements=["user_id"], index_where=text("kind = 'watch_later'")
+            )
+        )
+        pl = queries.watch_later(db, user)
+        assert pl is not None  # inserted now, or committed by the request that won
+    return pl
+
+
+def add_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
+    """Append a video to a playlist if not already present (idempotent). Returns whether it
+    was added. `mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later).
+    Locked BEFORE the presence check: two adds of one video both saw "absent" and the second one's
+    insert hit uq_playlist_video as a 500."""
+    lock_playlists(db, pl.id)
+    if queries.item(db, pl.id, video_id) is not None:
+        return False
+    db.add(
+        PlaylistItem(playlist_id=pl.id, video_id=video_id, position=next_item_position(db, pl.id))
+    )
+    if mark_dirty:
+        recompute_dirty(db, pl)
+    touch(pl)
+    db.flush()
+    return True
+
+
+def remove_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
+    """Remove a video from a playlist if present (idempotent). Returns whether it was removed.
+    `mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later)."""
+    lock_playlists(db, pl.id)
+    found = queries.item(db, pl.id, video_id)
+    if found is None:
+        return False
+    db.delete(found)
+    if mark_dirty:
+        recompute_dirty(db, pl)
+    touch(pl)
+    db.flush()
+    return True
+
+
+def rename_playlist(db: Session, pl: Playlist, name: str) -> None:
+    """A new name moves `dirty`; the caller holds the playlist's lock (the scheduled mirror refresh
+    reads `dirty` under it)."""
+    if name != pl.name:
+        pl.name = name
+        recompute_dirty(db, pl)
+    db.flush()
+
+
+def delete_playlist(db: Session, pl: Playlist) -> None:
+    """The playlist first, as every item write: the cascade deletes the items after the list row
+    is locked."""
+    lock_playlists(db, pl.id)
+    db.delete(pl)  # items cascade
+    db.flush()
+
+
+def remove_items(db: Session, pl: Playlist, video_ids: list[str]) -> list[str]:
+    """Remove these videos from a playlist, in one transaction; a video not in it any more is simply
+    not among them. Says which went: an Undo must not bring back one this removal never took."""
+    lock_playlists(db, pl.id)
+    wanted = set(video_ids)
+    rows = [it for it in queries.items(db, pl.id) if it.video_id in wanted]
+    for row in rows:
+        db.delete(row)
+    if rows:
+        recompute_dirty(db, pl)
+        touch(pl)
+    db.flush()
+    return [row.video_id for row in rows]
+
+
+def _insert_at(
+    db: Session,
+    pl: Playlist,
+    placed: list[tuple[int, str]],
+    leaving: dict[str, PlaylistItem] | None = None,
+    staying_out: frozenset[str] = frozenset(),
+) -> int:
+    """Add videos at their indices (ascending, so each index is where the row stood when the rows
+    before it were back) and renumber the list 0..n-1, as positions can have gaps. A video already in
+    the list stays where it is, and one the catalog has lost or in `staying_out` is left out, the
+    ones after it closing up. A video with a row in `leaving` brings that row along, for the reason
+    `move_items` re-parents. Says how many were added; the caller holds the playlist's lock."""
+    leaving = leaving or {}
+    rows = queries.items(db, pl.id)
+    present = {r.video_id for r in rows}
+    known = queries.known_video_ids(db, [vid for _, vid in placed])
+    added = 0
+    left_out = 0
+    for at, vid in sorted(placed, key=lambda p: p[0]):
+        if vid in present or vid not in known or vid in staying_out:
+            left_out += 1
+            continue
+        row = leaving.pop(vid, None)
+        if row is None:
+            row = PlaylistItem(playlist_id=pl.id, video_id=vid, position=0)
+            db.add(row)
+        else:
+            row.playlist_id = pl.id
+        rows.insert(min(at - left_out, len(rows)), row)
+        present.add(vid)
+        added += 1
+    for pos, row in enumerate(rows):
+        row.position = pos
+    return added
+
+
+def restore_items(db: Session, pl: Playlist, video_ids: list[str], positions: list[int]) -> int:
+    """Put removed videos back at their places, in ONE request: an append plus a whole-list reorder
+    leaves them at the end whenever the second call fails. Says how many came back."""
+    lock_playlists(db, pl.id)
+    added = _insert_at(db, pl, list(zip(positions, video_ids, strict=True)))
+    if added:
+        recompute_dirty(db, pl)
+        touch(pl)
+    db.flush()
+    return added
+
+
+class ListGone(Exception):
+    """A list of a two-list write was deleted between the route's read and the lock."""
+
+    def __init__(self, *, source: bool):
+        super().__init__("source" if source else "target")
+        self.source = source
+
+
+def _lock_both(db: Session, src: Playlist, dst: Playlist) -> None:
+    """Both lists, in id order: a move A→B beside a move B→A would otherwise lock crosswise. A list
+    already gone is said here: a write into it would answer with its foreign key's 500."""
+    found = lock_playlists(db, src.id, dst.id)
+    if src.id not in found:
+        raise ListGone(source=True)
+    if dst.id not in found:
+        raise ListGone(source=False)
+
+
+def move_items(
+    db: Session, src: Playlist, dst: Playlist, video_ids: list[str]
+) -> tuple[list[str], list[str]]:
+    """Move these videos from `src` into `dst`, in the caller's one transaction, in the source's
+    order whatever order they were picked in. Returns the ones that landed in the target and the ones
+    it already held, which only left the source: the Undo needs to tell them apart. A video a
+    concurrent edit already took out of the source is in neither. Raises `MoveRefused` for the same
+    list twice."""
+    if src.id == dst.id:
+        raise MoveRefused("same_playlist")
+    _lock_both(db, src, dst)
+    wanted = set(video_ids)
+    rows = [it for it in queries.items(db, src.id) if it.video_id in wanted]
+    held = queries.in_playlist(db, dst.id, [r.video_id for r in rows])
+    position = next_item_position(db, dst.id)
+    moved: list[str] = []
+    already_there: list[str] = []
+    for row in rows:
+        if row.video_id in held:
+            already_there.append(row.video_id)
+            db.delete(row)
+        else:
+            moved.append(row.video_id)
+            # Re-parented rather than deleted and inserted: `added_at` ("when I saved this") survives
+            # the move, and there is no window where the video belongs to neither list. `held` stays
+            # true: both lists are locked, and an insert into the target (another tab's add, a
+            # YouTube merge) waits for the lock, as its foreign key needs the target row.
+            row.playlist_id = dst.id
+            row.position = position
+            position += 1
+    if rows:
+        recompute_dirty(db, src)
+        touch(src)
+    # Only when a row landed in it: touching a target that already had the video promoted an untouched
+    # list in the rail's "recently changed" sort.
+    if moved:
+        recompute_dirty(db, dst)
+        touch(dst)
+    db.flush()
+    return moved, already_there
+
+
+def undo_move(
+    db: Session,
+    src: Playlist,
+    dst: Playlist,
+    moved: list[str],
+    already_there: list[str],
+    positions: list[int],
+) -> int:
+    """Take a move back in one transaction: the `moved` videos leave `dst` and return to their places
+    in `src`, keeping their rows, as the move did. The `already_there` ones return too and stay in
+    `dst`, which held them before. A moved video `dst` no longer holds has gone on from there, and
+    bringing it back would leave it in lists the reader never put it in, so it stays out, as a
+    move skips a video a concurrent edit took out. `positions` pairs with `moved` + `already_there`.
+    Says how many came back. Raises `MoveRefused` for the same list twice, as a move does."""
+    if src.id == dst.id:
+        raise MoveRefused("same_playlist")
+    _lock_both(db, src, dst)
+    wanted = set(moved)
+    rows = {it.video_id: it for it in queries.items(db, dst.id) if it.video_id in wanted}
+    changed = bool(rows)
+    placed = list(zip(positions, [*moved, *already_there], strict=True))
+    back = _insert_at(db, src, placed, leaving=rows, staying_out=frozenset(wanted - rows.keys()))
+    # What `src` holds again by now only leaves `dst`.
+    for row in rows.values():
+        db.delete(row)
+    db.flush()
+    if changed:
+        recompute_dirty(db, dst)
+        touch(dst)
+    if back:
+        recompute_dirty(db, src)
+        touch(src)
+    db.flush()
+    return back
+
+
+def reorder_items(db: Session, pl: Playlist, order: list[str]) -> int:
+    """Number the listed videos 0..n in the listed order. Items the list left out (added in another
+    tab meanwhile) keep their relative order after them, so no two items share a position. The
+    caller holds the playlist's lock. Returns how many items the list has."""
+    by_video = {it.video_id: it for it in queries.items(db, pl.id)}
+    pos = 0
+    seen: set[str] = set()
+    for vid in order:
+        it = by_video.get(vid)
+        if it is not None:
+            it.position = pos
+            pos += 1
+            seen.add(vid)
+    for it in sorted(
+        (it for vid, it in by_video.items() if vid not in seen), key=lambda i: i.position
+    ):
+        it.position = pos
+        pos += 1
+    recompute_dirty(db, pl)
+    touch(pl)
+    db.flush()
+    return pos
+
+
+def attach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
+    """The user's own label on a channel (idempotent). The caller has checked the tag is theirs."""
+    if queries.user_tag_link(db, user, channel_id, tag_id) is None:
+        db.add(ChannelTag(channel_id=channel_id, tag_id=tag_id, user_id=user.id, source="user"))
+        db.flush()
+
+
+def detach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
+    row = queries.user_tag_link(db, user, channel_id, tag_id)
+    if row is not None:
+        db.delete(row)
+        db.flush()
+
+
+# --- Google Takeout --------------------------------------------------------------------------
+
+_TAKEOUT_BATCH = 1000
+# The advisory-lock namespace of one user's Takeout rows ("TO"), so a key cannot meet another lock's.
+_TAKEOUT_LOCK_NS = 0x544F
+
+
+def lock_takeout(db: Session, user: User) -> None:
+    """Hold the user's Takeout rows for the rest of the transaction. An upload's staging and an
+    import's settling each lock many of the same rows, one statement after another and in no common
+    order, so without one lock over both the two can deadlock. An un-mark takes it too: the import
+    reads the un-marks before it marks, and one landing between the two would be overwritten."""
+    db.execute(select(func.pg_advisory_xact_lock(_TAKEOUT_LOCK_NS, user.id)))
+
+
+def stage_takeout(
+    db: Session, user: User, kind: TakeoutKind, rows: dict[str, tuple[datetime, str | None]]
+) -> None:
+    """Keep the videos of one Takeout file, `{video id: (time, channel id)}`, for the import job.
+    A second upload of the same export, or a newer one, adds what is new: a video already kept keeps
+    the later time, and a row the import has settled keeps its outcome."""
+    lock_takeout(db, user)
+    values = [
+        {"user_id": user.id, "kind": kind, "video_id": vid, "at": at, "channel_id": channel}
+        for vid, (at, channel) in rows.items()
+    ]
+    for start in range(0, len(values), _TAKEOUT_BATCH):
+        stmt = pg_insert(TakeoutItem).values(values[start : start + _TAKEOUT_BATCH])
+        db.execute(
+            stmt.on_conflict_do_update(
+                constraint="uq_takeout_item",
+                set_={
+                    "at": func.greatest(TakeoutItem.at, stmt.excluded.at),
+                    "channel_id": func.coalesce(stmt.excluded.channel_id, TakeoutItem.channel_id),
+                },
+            )
+        )
+    db.flush()
+
+
+def _settle(db: Session, ids: list[int], outcome: TakeoutOutcome) -> None:
+    for start in range(0, len(ids), _TAKEOUT_BATCH):
+        db.execute(
+            update(TakeoutItem)
+            .where(TakeoutItem.id.in_(ids[start : start + _TAKEOUT_BATCH]))
+            .values(outcome=outcome)
+        )
+
+
+def apply_takeout_watched(db: Session, user: User) -> dict[str, int]:
+    """Mark watched what the user watched on YouTube, for the videos the catalog knows and has probed
+    for Shorts; the rest wait. `watched_at` is the Takeout time. A Short is not marked (swiping one
+    past is not watching it), and a state the user set here wins: watched or hidden, anything
+    changed after the Takeout time, or an un-mark since then (`VideoUnmark`). The upsert's own
+    condition is that rule, so a state set while the import runs wins too."""
+    lock_takeout(db, user)
+    short: list[int] = []
+    todo: dict[str, tuple[int, datetime]] = {}
+    ready = queries.takeout_watched_ready(db, user)
+    unmarked = queries.unmarked_at(db, user, [video_id for _, video_id, _, _ in ready])
+    kept: list[int] = []
+    for row_id, video_id, at, is_short in ready:
+        if is_short:
+            short.append(row_id)
+        elif video_id in unmarked and unmarked[video_id] >= at:
+            # Un-marked here since they watched it there: theirs, as a state they set is.
+            kept.append(row_id)
+        else:
+            todo[video_id] = (row_id, at)
+    marked: set[str] = set()
+    items = list(todo.items())
+    now = datetime.now(timezone.utc)
+    for start in range(0, len(items), _TAKEOUT_BATCH):
+        stmt = pg_insert(VideoState).values(
+            [
+                {"user_id": user.id, "video_id": vid, "status": "watched", "watched_at": at}
+                for vid, (_, at) in items[start : start + _TAKEOUT_BATCH]
+            ]
+        )
+        marked.update(
+            db.execute(
+                stmt.on_conflict_do_update(
+                    constraint="uq_user_video",
+                    set_={
+                        "status": "watched",
+                        "watched_at": stmt.excluded.watched_at,
+                        "updated_at": now,
+                    },
+                    where=(VideoState.status == "new")
+                    & (VideoState.updated_at < stmt.excluded.watched_at),
+                ).returning(VideoState.video_id)
+            ).scalars()
+        )
+    kept += [row_id for vid, (row_id, _) in items if vid not in marked]
+    _settle(db, [todo[vid][0] for vid in marked], TakeoutOutcome.WATCHED)
+    _settle(db, kept, TakeoutOutcome.KEPT)
+    _settle(db, short, TakeoutOutcome.SHORT)
+    db.flush()
+    return {"watched": len(marked), "kept": len(kept), "shorts": len(short)}
+
+
+def apply_takeout_watch_later(
+    db: Session, user: User, rows: list[tuple[int, str]], asked: set[str]
+) -> dict[str, int]:
+    """Put the videos of the user's waiting Watch later `rows`, (row id, video id) oldest first, into
+    their list here, after what it already holds. `asked` is the videos YouTube was just asked for: one
+    of those the catalog still lacks is gone from YouTube; any other unknown one keeps waiting."""
+    if not rows:
+        return {}
+    lock_takeout(db, user)
+    ids = [vid for _, vid in rows]
+    known = queries.known_video_ids(db, ids)
+    pl = watch_later(db, user)
+    # What add_item does a video at a time, for the whole list at once: one presence read and one
+    # insert, not four statements a video (a full list is thousands). Locked before the read, as there.
+    next_position = next_item_position(db, pl.id)
+    held = queries.in_playlist(db, pl.id, [vid for vid in ids if vid in known])
+    done: dict[TakeoutOutcome, list[int]] = {o: [] for o in TakeoutOutcome}
+    for row_id, vid in rows:
+        if vid not in known:
+            if vid in asked:
+                done[TakeoutOutcome.MISSING].append(row_id)
+        elif vid in held:
+            done[TakeoutOutcome.PRESENT].append(row_id)
+        else:
+            db.add(PlaylistItem(playlist_id=pl.id, video_id=vid, position=next_position))
+            next_position += 1
+            held.add(vid)
+            done[TakeoutOutcome.ADDED].append(row_id)
+    if done[TakeoutOutcome.ADDED]:
+        touch(pl)
+    for outcome, ids in done.items():
+        _settle(db, ids, outcome)
+    db.flush()
+    return {
+        "watch_later_added": len(done[TakeoutOutcome.ADDED]),
+        "watch_later_present": len(done[TakeoutOutcome.PRESENT]),
+        "watch_later_missing": len(done[TakeoutOutcome.MISSING]),
+    }
+
+
+# --- channels watched but not followed ---------------------------------------------------------
+
+
+def record_takeout_lengths(
+    db: Session, user: User, asked: list[str], lengths: dict[str, int | None]
+) -> None:
+    """Keep what YouTube said of the user's watched videos `asked` about: each one's length, or none
+    for a video it no longer has (`lengths` leaves it out). Every asked row is marked asked, so a
+    video YouTube lost is not asked about again."""
+    if not asked:
+        return
+    lock_takeout(db, user)  # these rows are an upload's to update too
+    now = datetime.now(timezone.utc)
+    table = cast(Table, TakeoutItem.__table__)
+    db.execute(
+        table.update()
+        .where(
+            owned_by(TakeoutItem.user_id, user),
+            table.c.kind == TakeoutKind.WATCHED,
+            table.c.video_id == bindparam("vid"),
+        )
+        .values(looked_up_at=now, duration_seconds=bindparam("seconds")),
+        [{"vid": vid, "seconds": lengths.get(vid)} for vid in asked],
+    )
+    db.flush()
+
+
+def dismiss_watched_channel(db: Session, user: User, channel_id: str) -> None:
+    """Put a channel out of the user's "watched but not followed" list (again is a no-op)."""
+    db.execute(
+        pg_insert(TakeoutChannelDismissal)
+        .values(user_id=user.id, channel_id=channel_id)
+        .on_conflict_do_nothing(constraint="uq_takeout_channel_dismissal")
+    )
+    db.flush()
+
+
+def restore_watched_channel(db: Session, user: User, channel_id: str) -> None:
+    """Let a channel the user put out of the list back on it."""
+    db.execute(
+        delete(TakeoutChannelDismissal).where(
+            owned_by(TakeoutChannelDismissal.user_id, user),
+            TakeoutChannelDismissal.channel_id == channel_id,
+        )
+    )
+    db.flush()

backend/app/reader/takeout.py

@@ -0,0 +1,497 @@
+"""A Google Takeout export, imported: what the user watched on YouTube becomes watched here, and their
+YouTube Watch later list joins theirs. The YouTube API reads neither of them, so the export is the
+only way in.
+
+Two files of the export are read, each uploaded on its own: `history/watch-history.json` and
+`playlists/Watch later-videos.csv`. Their videos are kept (`TakeoutItem`) and the import job applies
+what it can; a video the catalog does not know yet waits for it. The watched videos' channels also
+make the list of channels the user watches but does not follow (`queries.watched_channels`): the
+job asks YouTube how long those videos are, and what the channels are, for it.
+
+The format, measured on a real export (2026-10): the history is a JSON list, one entry per view,
+newest first. A view of a video has a `titleUrl` with `watch?v=<id>` (on www.youtube.com or
+music.youtube.com) and a `time`, with or without milliseconds. The channel is the first `subtitles`
+entry's url, absent for a video since removed. An ad is an entry whose `details` say "From Google
+Ads". The title is in the account's language, so nothing here reads it. A Short is a plain
+`watch?v=` view, told apart only once the catalog has probed the video. The Watch later csv is a
+header row (in the account's language) and `<id>,<ISO time added>` rows.
+"""
+
+import codecs
+import csv
+import io
+import json
+import re
+import threading
+from collections.abc import Iterator
+from contextlib import contextmanager
+from dataclasses import dataclass, field
+from datetime import datetime
+from itertools import chain
+
+from sqlalchemy.orm import Session
+
+from app import logs, quota
+from app.bodylimit import TAKEOUT_MAX_BYTES
+from app.db import SessionLocal
+from app.models import User
+from app.ratelimit import RateLimiter
+from app.reader import queries, service
+from app.reader.models import TakeoutKind
+from app.sync.runner import get_service_user
+from app.sync.subscriptions import describe_channels
+from app.sync.videos import ensure_videos, parse_iso8601_duration
+from app.youtube.client import YouTubeClient, YouTubeError
+
+log = logs.get_logger(__name__)
+
+# The largest file the upload takes (its edge cap and the route's own are one number).
+MAX_BYTES = TAKEOUT_MAX_BYTES
+# The most videos a Watch later file may hold: YouTube's own cap on one playlist. Each unknown one is
+# looked up on the shared API budget, so a longer list is not one YouTube made.
+WATCH_LATER_MAX = 5000
+# The most Takeout videos one user keeps: twenty times the real export measured here (~9,000 videos
+# across both files), so it only stops uploads of made-up ones piling up for good.
+ROWS_MAX = 200_000
+
+_VIDEO_ID = re.compile(r"^[A-Za-z0-9_-]{11}$")
+_WATCH_URL = re.compile(
+    r"^https?://(?:www\.|music\.|m\.)?youtube\.com/watch\?v=([A-Za-z0-9_-]{11})"
+)
+_CHANNEL_URL = re.compile(r"/channel/(UC[A-Za-z0-9_-]{22})")
+
+
+class TakeoutError(ValueError):
+    """The file is not one the import reads; `reason` is the client's word for why."""
+
+    def __init__(self, reason: str):
+        super().__init__(reason)
+        self.reason = reason
+
+
+@dataclass
+class Parsed:
+    """One file's videos, `{video id: (time, channel id)}`, and what it left out, counted."""
+
+    kind: TakeoutKind
+    videos: dict[str, tuple[datetime, str | None]] = field(default_factory=dict)
+    entries: int = 0
+    ads: int = 0
+    other: int = 0  # not a view of a video (a post, a story) or a video with no link left
+    malformed: int = 0
+
+
+def _time(raw: object) -> datetime | None:
+    if not isinstance(raw, str):
+        return None
+    try:
+        at = datetime.fromisoformat(raw.replace("Z", "+00:00"))
+    # silent-ok: the row is counted as malformed, and the count is in the upload's answer.
+    except ValueError:
+        return None
+    return at if at.tzinfo is not None else None
+
+
+def _keep(parsed: Parsed, vid: str, at: datetime, channel: str | None) -> None:
+    old = parsed.videos.get(vid)
+    if old is None or at > old[0]:
+        parsed.videos[vid] = (at, channel or (old[1] if old else None))
+
+
+def _history(data: Iterator[object]) -> Parsed:
+    parsed = Parsed(TakeoutKind.WATCHED)
+    for entry in data:
+        parsed.entries += 1
+        if not isinstance(entry, dict):
+            parsed.malformed += 1
+            continue
+        details = entry.get("details")
+        if isinstance(details, list) and any(
+            isinstance(d, dict) and d.get("name") == "From Google Ads" for d in details
+        ):
+            parsed.ads += 1
+            continue
+        url = entry.get("titleUrl")
+        match = _WATCH_URL.match(url) if isinstance(url, str) else None
+        if match is None:
+            parsed.other += 1
+            continue
+        at = _time(entry.get("time"))
+        if at is None:
+            parsed.malformed += 1
+            continue
+        channel = None
+        subtitles = entry.get("subtitles")
+        if isinstance(subtitles, list) and subtitles and isinstance(subtitles[0], dict):
+            found = _CHANNEL_URL.search(str(subtitles[0].get("url", "")))
+            channel = found.group(1) if found else None
+        _keep(parsed, match.group(1), at, channel)
+    return parsed
+
+
+def _watch_later(text: str) -> Parsed:
+    parsed = Parsed(TakeoutKind.WATCH_LATER)
+    rows = (row for row in csv.reader(io.StringIO(text)) if any(cell.strip() for cell in row))
+    first = next(rows, None)
+    if first is None or len(first) < 2:
+        raise TakeoutError("not_takeout")
+    # The first row is the header, in the account's language, unless it is already a video.
+    for row in chain([first], rows) if _VIDEO_ID.match(first[0].strip()) else rows:
+        parsed.entries += 1
+        vid = row[0].strip()
+        at = _time(row[1].strip()) if len(row) > 1 else None
+        if not _VIDEO_ID.match(vid) or at is None:
+            parsed.malformed += 1
+            continue
+        _keep(parsed, vid, at, None)
+        if len(parsed.videos) > WATCH_LATER_MAX:
+            raise TakeoutError("too_many")
+    return parsed
+
+
+def _skip(text: str, at: int) -> int:
+    """The first index at or after `at` that is not JSON whitespace."""
+    while at < len(text) and text[at] in " \t\n\r":
+        at += 1
+    return at
+
+
+# How much of the file is decoded at a time. Decoded whole, the text costs up to four bytes a
+# character (one emoji in a title is enough), several times the file on top of its bytes.
+_WINDOW = 1024 * 1024
+
+
+class _Window:
+    """The file's text, decoded a window at a time: `text` holds what is not read yet from `at` on.
+    The bytes are the caller's, so only the window is this reader's memory."""
+
+    def __init__(self, body: bytes, start: int):
+        self._body, self._next = body, start
+        self._decoder = codecs.getincrementaldecoder("utf-8")()  # strict: bad UTF-8 raises
+        self.text, self.at = "", 0
+
+    @property
+    def done(self) -> bool:
+        return self._next >= len(self._body)
+
+    def more(self, at_least: int = 0) -> None:
+        """Decode the next window (at least `at_least` bytes), dropping what is read already."""
+        self.text, self.at = self.text[self.at :], 0
+        size = max(_WINDOW, at_least)
+        chunk = self._body[self._next : self._next + size]
+        self._next += len(chunk)
+        self.text += self._decoder.decode(chunk, final=self.done)
+
+    def skip(self) -> None:
+        """Past the whitespace, reading on as far as it goes."""
+        while True:
+            self.at = _skip(self.text, self.at)
+            if self.at < len(self.text) or self.done:
+                return
+            self.more()
+
+    def startswith(self, char: str) -> bool:
+        self.skip()
+        return self.text.startswith(char, self.at)
+
+
+def _decode(decoder: json.JSONDecoder, text: str, at: int) -> tuple[object, int] | None:
+    """The JSON value at `at` and where it ends, or None where none reads (yet)."""
+    try:
+        return decoder.raw_decode(text, at)
+    # silent-ok: not a failure here — the caller reads on, or refuses the file once it has all.
+    except json.JSONDecodeError:
+        return None
+
+
+def _json_list(body: bytes, start: int) -> Iterator[object]:
+    """The entries of the JSON list at byte `start` (its "["), one at a time: the whole tree is never
+    built, nor the whole text, so a file of a million tiny entries costs a window and one entry, not
+    a million dicts at once."""
+    decoder = json.JSONDecoder()
+    window = _Window(body, start + 1)  # past the "[" the caller saw
+    if window.startswith("]"):
+        window.at += 1
+    else:
+        while True:
+            window.skip()
+            read = _decode(decoder, window.text, window.at)
+            # An entry cut by the window's edge fails, or (a number) reads short: read on and try
+            # again, with a window twice what is left, so one entry larger than any window is read
+            # in a few tries, not one a window.
+            if read is None or (read[1] == len(window.text) and not window.done):
+                if window.done:
+                    raise ValueError("not a JSON list")
+                window.more(2 * (len(window.text) - window.at))
+                continue
+            entry, window.at = read
+            yield entry
+            if window.startswith(","):
+                window.at += 1
+            elif window.startswith("]"):
+                window.at += 1
+                break
+            else:
+                raise ValueError("not a JSON list")
+    window.skip()
+    if window.at != len(window.text):
+        raise ValueError("text after the JSON list")
+
+
+# The largest Watch later file read: YouTube's longest list is some 200 KB of rows, so a file ten
+# times that is not one — and, unlike the history, it is decoded whole.
+CSV_MAX_BYTES = 2 * 1024 * 1024
+_BOM = codecs.BOM_UTF8
+
+
+def parse(body: bytes) -> Parsed:
+    """Read an uploaded Takeout file: the watch history (JSON) or the Watch later list (csv), told
+    apart by their content, not their name. Raises `TakeoutError` for anything else, and for a file
+    with nothing to import in it."""
+    start = len(_BOM) if body.startswith(_BOM) else 0
+    while start < len(body) and body[start] in b" \t\n\r\f\v":
+        start += 1
+    first = body[start : start + 1]
+    if first in (b"{", b"<"):
+        # Another JSON file of the export, or the history's HTML variant.
+        raise TakeoutError("not_takeout")
+    if first != b"[" and len(body) - start > CSV_MAX_BYTES:
+        raise TakeoutError("too_many")
+    try:
+        # A bad UTF-8 sequence is a ValueError too (UnicodeDecodeError), read below as not a file.
+        parsed = (
+            _history(_json_list(body, start))
+            if first == b"["
+            else _watch_later(body[start:].decode("utf-8"))
+        )
+    # A file the import cannot read, however it fails: bad JSON (a ValueError, as is a number too
+    # long to convert), nesting deeper than the decoder recurses, a csv field past csv's limit.
+    except (ValueError, RecursionError, csv.Error) as exc:
+        if isinstance(exc, TakeoutError):
+            raise
+        raise TakeoutError("not_takeout") from None
+    if not parsed.videos:
+        raise TakeoutError("no_videos")
+    return parsed
+
+
+# One import per user at a time: a second file uploaded while the first one's run is under way runs
+# after it (the scheduler's ad-hoc queue), but the Shorts job settles the same users' rows on a
+# thread of its own, and the two would settle the same rows. One API process runs the scheduler, so a
+# lock in this module holds it (as the channel backfill's does).
+_user_locks: dict[int, threading.Lock] = {}
+_user_locks_guard = threading.Lock()
+
+
+@contextmanager
+def _one_import(user_id: int) -> Iterator[None]:
+    with _user_locks_guard:
+        lock = _user_locks.setdefault(user_id, threading.Lock())
+    with lock:
+        yield
+
+
+def run_target(user_id: int) -> str:
+    """The import run's target, `job_runs.target`: one user's import."""
+    return f"user:{user_id}"
+
+
+def target_title(db: Session, target: str) -> str | None:
+    """Who an import run's target is, for the admin's run lists: the user's name, else their email."""
+    _, _, raw = target.partition("user:")
+    user = db.get(User, int(raw)) if raw.isdigit() else None
+    return (user.display_name or user.email) if user is not None else None
+
+
+class TooMany(Exception):
+    """The upload would take the user past `ROWS_MAX` kept videos."""
+
+
+def stage(db: Session, user: User, parsed: Parsed) -> None:
+    """Keep an uploaded file's videos for the import, unless that would take the user past
+    `ROWS_MAX` (`TooMany`). The caller commits."""
+    service.lock_takeout(db, user)  # the count and the insert are one user's, not two uploads' each
+    if (
+        queries.takeout_rows_besides(db, user, parsed.kind, parsed.videos) + len(parsed.videos)
+        > ROWS_MAX
+    ):
+        raise TooMany
+    service.stage_takeout(db, user, parsed.kind, parsed.videos)
+
+
+# How many unknown Watch later videos one user may have looked up on the shared API budget in a day:
+# two full Watch later lists (two units a hundred ids). The rest wait for the next day's runs. Both
+# shares pace the work, they do not bound its cost: they are held in memory, so a restart starts
+# them afresh, which only brings the rest forward — every video is asked about once (a looked-up
+# row is marked), and `quota.backfill_reserve_reached` still guards the shared budget.
+_lookups = RateLimiter(max_events=2 * WATCH_LATER_MAX, window_seconds=24 * 3600)
+# How many watched videos' lengths one user may ask about in a day for the "watched but not followed"
+# list (a unit a fifty). A share of its own: a large history asks for thousands, and on the Watch
+# later share it would leave the export's Watch later file waiting a day behind it.
+_lengths = RateLimiter(max_events=2 * WATCH_LATER_MAX, window_seconds=24 * 3600)
+
+
+def _put_in_watch_later(db: Session, user: User) -> dict:
+    """Put the user's waiting Watch later videos into theirs here: the ones the catalog knows at once,
+    the others once YouTube was asked for them — when a YouTube-connected account can ask, today's
+    budget is above the backfill reserve, and the user's own daily share of lookups allows it. A video
+    not asked about yet waits for a later run. The lookup commits what it fetched; the caller commits
+    the rest."""
+    waiting = queries.takeout_waiting(db, user, TakeoutKind.WATCH_LATER)
+    if not waiting:
+        return {}
+    ids = [vid for _, vid in waiting]
+    known = queries.known_video_ids(db, ids)
+    ask = [vid for vid in ids if vid not in known][:WATCH_LATER_MAX]
+    asked: set[str] = set()
+    fetcher = get_service_user(db) if ask else None
+    if (
+        fetcher is not None
+        and not quota.backfill_reserve_reached()
+        and _lookups.room(str(user.id)) >= len(ask)
+    ):
+        with quota.attribute(user.id, quota.QuotaAction.VIDEOS_LOOKUP):
+            with YouTubeClient(db, fetcher) as yt:
+                ensure_videos(db, yt, ask)
+        # Counted once done: a lookup that failed spent none of the share (one run per user at a
+        # time, so the room asked above is still there).
+        _lookups.allow(str(user.id), cost=len(ask))
+        asked = set(ask)
+    # Only the rows read above: one a second upload staged meanwhile was never asked about.
+    return service.apply_takeout_watch_later(db, user, waiting, asked)
+
+
+def _ask_lengths(db: Session, user: User) -> dict:
+    """Ask YouTube how long the user's watched videos are that the "watched but not followed" list
+    still needs, and keep the answer — on the same terms as the Watch later lookups, on a daily
+    share of its own. What is not asked yet (no YouTube-connected account, the reserve reached, the
+    share spent) waits for a later run, and the list says it is not complete. A YouTube failure is
+    logged and left for a later run too: it is no reason to fail the import it follows. The caller
+    commits."""
+    ask = queries.takeout_lengths_to_ask(db, user)
+    fetcher = get_service_user(db) if ask else None
+    if fetcher is None or quota.backfill_reserve_reached():
+        return {}
+    ask = ask[: int(_lengths.room(str(user.id)))]
+    if not ask:
+        return {}
+    try:
+        with quota.attribute(user.id, quota.QuotaAction.VIDEOS_LOOKUP):
+            with YouTubeClient(db, fetcher) as yt:
+                items = yt.get_videos(ask)
+    except YouTubeError as exc:
+        log.warning(
+            "Takeout: asking the watched videos' lengths failed: %s",
+            exc,
+            extra=logs.event("takeout.lengths.failed", user_id=user.id, reason=exc.reason),
+        )
+        return {}
+    _lengths.allow(str(user.id), cost=len(ask))
+    lengths = {
+        it["id"]: parse_iso8601_duration(it.get("contentDetails", {}).get("duration"))
+        for it in items
+        if it.get("id")
+    }
+    service.record_takeout_lengths(db, user, ask, lengths)
+    return {"lengths_asked": len(ask)}
+
+
+def _describe_channels(db: Session, user: User) -> dict:
+    """Bring the channels the user's list holds into the catalog with their details (name, picture,
+    description), the ones it lacks: without them the list cannot show a channel. A YouTube failure
+    is logged and left for a later run. The caller commits."""
+    need = queries.watched_channels_to_describe(db, user)
+    fetcher = get_service_user(db) if need else None
+    if fetcher is None or quota.backfill_reserve_reached():
+        return {}
+    try:
+        with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
+            with YouTubeClient(db, fetcher) as yt:
+                describe_channels(db, yt, need)
+    except YouTubeError as exc:
+        db.rollback()
+        log.warning(
+            "Takeout: describing the watched channels failed: %s",
+            exc,
+            extra=logs.event("takeout.channels.failed", user_id=user.id, reason=exc.reason),
+        )
+        return {}
+    return {"channels_described": len(need)}
+
+
+def _channel_list(db: Session, user: User) -> dict:
+    """The "watched but not followed" list's part of a run: the lengths, landed, then the channels.
+    Commits each."""
+    result = _ask_lengths(db, user)
+    db.commit()
+    result |= _describe_channels(db, user)
+    db.commit()
+    return result
+
+
+def run_import(user_id: int) -> dict:
+    """The import job: apply what the user's kept Takeout rows can say now, as one recorded run.
+    The watched part commits before the Watch later part asks YouTube for the videos the catalog
+    lacks, so a YouTube failure there loses nothing already marked; the "watched but not followed"
+    list's lookups come last. Returns the counts."""
+    with _one_import(user_id), SessionLocal() as db:
+        user = db.get(User, user_id)
+        if user is None:
+            return {"skipped": "no user"}
+        result = service.apply_takeout_watched(db, user)
+        db.commit()
+        result |= _put_in_watch_later(db, user)
+        db.commit()
+        result |= _channel_list(db, user)
+        outcomes = queries.takeout_outcomes(db, user)
+        result["waiting"] = outcomes.get((TakeoutKind.WATCHED, None), 0)
+        if outcomes.get((TakeoutKind.WATCH_LATER, None)):
+            result["watch_later_waiting"] = outcomes[(TakeoutKind.WATCH_LATER, None)]
+        return result
+
+
+def apply_waiting(db: Session) -> dict:
+    """Settle what every user's import left waiting and can be settled now, one commit each: a
+    watched video the catalog has come to know and the Shorts probe has cleared (by its job or a
+    search's), the Watch later videos a run could not put in yet, and what the "watched but not
+    followed" list still needs asked of YouTube. Run after each run of the
+    `shorts` job, which is what clears the most of them. Returns its counts, or an empty dict when
+    nothing was waiting that could move."""
+    marked = added = asked = failed = 0
+    ready = set(queries.takeout_users_ready(db))
+    later = set(queries.takeout_users_watch_later_waiting(db))
+    lengths = set(queries.takeout_users_with_list_work(db))
+    for user_id in sorted(ready | later | lengths):
+        user = db.get(User, user_id)
+        if user is None:
+            continue
+        # One user's failure (a lookup YouTube refused) is theirs: it neither fails the job that
+        # runs this nor keeps the users after them waiting. What it did not settle waits.
+        try:
+            with _one_import(user_id):
+                if user_id in ready:
+                    marked += service.apply_takeout_watched(db, user)["watched"]
+                    db.commit()
+                if user_id in later:
+                    added += _put_in_watch_later(db, user).get("watch_later_added", 0)
+                    db.commit()
+                if user_id in lengths:
+                    asked += _channel_list(db, user).get("lengths_asked", 0)
+        except Exception:
+            db.rollback()
+            failed += 1
+            log.exception(
+                "Takeout settling failed for user %s",
+                user_id,
+                extra=logs.event("takeout.settle.failed", user_id=user_id),
+            )
+    out = {}
+    if ready:
+        out["takeout_watched"] = marked
+    if added:
+        out["takeout_watch_later_added"] = added
+    if asked:
+        out["takeout_lengths_asked"] = asked
+    if failed:
+        out["takeout_failed"] = failed
+    return out

backend/app/refusal.py

@@ -1,29 +1,39 @@
 """A request the app refuses, said in a way the UI can put in the reader's language.
 
 Every refusal the API answers carries a structured `detail` — `{code, reason, …params}` — and the client
-translates it (`lib/api/core.ts` `localizeDetail`). A bare English string reached the Hungarian UI
+translates it (`lib/api/refusalText.ts` `localizeDetail`). A bare English string reached the Hungarian UI
 verbatim: the blocking error dialog, and every inline surface that printed `HttpError.detail`, showed
 whatever sentence the route happened to hand-write. `refused` is the generic family
 (`errors.refused.<reason>`, the params interpolated into the sentence); the older specific codes
 (`not_found`, `youtube`, `playlist`, the admin refusals, …) stay, since each is already translated.
 
-A test (`test_http_details.py`) reads every `HTTPException` in the app and fails on a string detail,
-and the frontend's contract test requires every `refused(…, "<reason>")` to be named in both
-languages — so the rule is enforced, not described.
+A test (`test_http_details.py`) reads every place an `HTTPException` or a `JSONResponse` sends a detail
+and fails on one the client cannot say: a sentence, a dict without a `code`, a variable, none at all.
+`tests/refusal_sites.py` reads every `{"code": …}` detail from the syntax tree, written where it is sent
+(or returned by a factory that only ever is), with any `**` spread before its `code` and `reason`, and
+follows each reason along a closed list of shapes: a literal, a `Literal` type on the parameter or
+attribute it travels in, every call of a helper like `refused` below, a map's values; anything else is
+reported, never guessed. The frontend's contract test
+(`i18n/backendIds.contract.test.ts`) runs it and requires each code in one record of
+`lib/api/refusalCodes.ts`, and each reason's sentence in both languages; a code or reason it cannot
+follow fails it, unless no screen shows that code. So the rule is enforced, not described.
 """
 
 from fastapi import HTTPException
 
+from app.sysconfig import InvalidSetting, SettingReason
+
 
 def refused(status: int, reason: str, **params: object) -> HTTPException:
     """The refusal to raise: `raise refused(409, "view_name_taken", name=name)`. `params` travel in
-    the detail for the sentence to interpolate (`{{name}}`); keep them to values the reader may see."""
-    return HTTPException(status_code=status, detail={"code": "refused", "reason": reason, **params})
+    the detail for the sentence to interpolate (`{{name}}`); keep them to values the reader may see.
+    They are spread first, so a `code` or `reason` among them could never replace these two."""
+    return HTTPException(status_code=status, detail={**params, "code": "refused", "reason": reason})
 
 
-# `sysconfig.InvalidSetting.reason` → the refusal it answers as. A dict of literals, so the frontend's
-# contract test can read every reason this can produce.
-_SETTING_REASONS = {
+# `sysconfig.InvalidSetting.reason` → the refusal it answers as. A dict of literals, so the refusal
+# scan reads every reason this can produce, and holds its keys to `SettingReason`.
+_SETTING_REASONS: dict[SettingReason, str] = {
     "not_a_number": "setting_not_a_number",
     "too_low": "setting_too_low",
     "too_high": "setting_too_high",
@@ -32,6 +42,6 @@ _SETTING_REASONS = {
 }
 
 
-def setting_refused(exc) -> HTTPException:
+def setting_refused(exc: InvalidSetting) -> HTTPException:
     """A value the admin entered that `sysconfig.set_value` refused (`exc`: its `InvalidSetting`)."""
     return refused(400, _SETTING_REASONS[exc.reason], key=exc.key, **exc.params)

backend/app/routes/_account.py

@@ -0,0 +1,17 @@
+"""The account filter the activity page's lists share (runs, admin actions, server log, quota): one
+filter bar drives them all, so the value it sends means the same on every tab.
+
+The value is an account id, or `none` — the rows no account made (the background) — a value to filter
+on, not the absence of a filter.
+"""
+
+from sqlalchemy import ColumnElement
+
+NONE = "none"
+# The `user` query parameter's shape on every route that takes it.
+PATTERN = r"^(none|[0-9]{1,10})$"
+
+
+def condition(column, value: str) -> ColumnElement[bool]:
+    """`column` (an account id column) narrowed to `value`."""
+    return column.is_(None) if value == NONE else column == int(value)

backend/app/routes/_common.py

@@ -6,9 +6,9 @@ and "fetch a row this user owns or 404" idioms have one definition, one status c
 one existence-leak policy.
 """
 
-from typing import Any, Protocol, TypeVar
+from typing import Annotated, Any, Protocol, TypeVar
 
-from fastapi import HTTPException, Request
+from fastapi import HTTPException, Query, Request
 from pydantic import BaseModel, ConfigDict, Field
 from sqlalchemy.orm import Session
 
@@ -16,10 +16,15 @@ from app.db import Base
 from app.models import POSITION_MAX_S, User
 from app.issues import gitea
 from app.refusal import refused
+from app.userscope import owned_row
 from app.youtube.client import YouTubeError, refusal_reason
 
 M = TypeVar("M", bound=Base)
 
+# A shuffle / surprise seed off the query string. The client draws one under a billion; bounded so a
+# hand-typed 400-digit one is a 422, not an int no float or bigint can hold further down.
+Seed = Annotated[int, Query(ge=-(2**53), le=2**53)]
+
 # Resume-position thresholds of a player that only keeps a resume point (the YouTube player, the
 # library's): positions below this are "didn't really start" and within this of the end are
 # "basically finished" — neither is worth storing, so the position is cleared instead (no progress
@@ -88,10 +93,11 @@ def owned_or_404(
 ) -> M:
     """Fetch a row by primary key that `user` owns, or raise 404 (`lock`: see `get_or_404`). A row owned by someone
     else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger.
-    It replaced the near-identical per-route ownership helpers, and is called directly — a thin
-    `_own_x` wrapper around it is the ad-hoc shape CLAUDE.md's patterns table rules out."""
-    obj = get_or_404(db, model, id_, what=what, lock=lock)
-    if getattr(obj, owner_attr) != user.id:
+    A core route calls it directly. A unit's route cannot (it may not name a unit's model as a value):
+    it asks its unit's `queries.own_x`, which goes through the same `owned_row`, and answers
+    `not_found` itself."""
+    obj = owned_row(db, model, id_, user, owner_attr=owner_attr, lock=lock)
+    if obj is None:
         raise not_found(what)
     return obj
 
@@ -122,6 +128,14 @@ async def capped_body(request: Request, limit: int) -> bytes:
     return bytes(body)
 
 
+async def drain_body(request: Request) -> None:
+    """Read the rest of the body to nowhere, before a refusal is raised: answered mid-upload, the
+    client reports a network failure, not the refusal (`capped_stream_to` says why). `bodylimit`
+    bounds how much there can be."""
+    async for _ in request.stream():
+        pass
+
+
 class ByteSink(Protocol):
     """Where `capped_stream_to` writes: a file, or anything that takes the bytes on their way to one."""
 

backend/app/routes/_keyset.py

@@ -0,0 +1,66 @@
+"""Keyset paging over `(timestamp desc, id desc)`, shared by the admin history lists (server log,
+audit log, background runs).
+
+A timestamp alone is not unique (a browser log row's `ts` is clamped into its consent window; a run
+and its siblings can start in the same microsecond), and an offset would skip or repeat rows while
+writers add new ones at the top — so a page continues from the last row's `(timestamp, id)`.
+"""
+
+import base64
+import binascii
+import json
+from datetime import datetime, timezone
+
+from fastapi import HTTPException
+from sqlalchemy import and_, or_, select
+from sqlalchemy.orm import Session
+
+from app.models import User
+
+PAGE_MAX = 200
+
+
+def utc(value: datetime | None) -> datetime | None:
+    """A bound without a zone is UTC — the columns are `timestamptz`, and a naive value would be read
+    in the database session's zone instead."""
+    if value is None or value.tzinfo is not None:
+        return value
+    return value.replace(tzinfo=timezone.utc)
+
+
+def encode_cursor(ts: datetime, row_id: int) -> str:
+    return base64.urlsafe_b64encode(json.dumps([ts.isoformat(), row_id]).encode()).decode()
+
+
+def decode_cursor(cursor: str) -> tuple[datetime, int]:
+    try:
+        ts, row_id = json.loads(base64.urlsafe_b64decode(cursor.encode()))
+        at = datetime.fromisoformat(ts)
+        return (at if at.tzinfo is not None else at.replace(tzinfo=timezone.utc)), int(row_id)
+    except (binascii.Error, json.JSONDecodeError, ValueError, TypeError) as exc:
+        raise HTTPException(status_code=400, detail={"code": "bad_cursor"}) from exc
+
+
+def older_than(ts_col, id_col, cursor: str):
+    """The rows after `cursor` in `(ts desc, id desc)` order. The `ts <=` conjunct is implied by the
+    OR; it is there so the planner can bound the timestamp index scan instead of filtering every
+    newer row first."""
+    ts, row_id = decode_cursor(cursor)
+    return and_(ts_col <= ts, or_(ts_col < ts, and_(ts_col == ts, id_col < row_id)))
+
+
+def page(rows: list, limit: int, ts_of) -> tuple[list, str | None]:
+    """`rows` was read with `limit + 1` (to know whether there is a next page without a count query):
+    the page itself, and the cursor that continues after it (None on the last page)."""
+    more = len(rows) > limit
+    rows = rows[:limit]
+    return rows, encode_cursor(ts_of(rows[-1]), rows[-1].id) if more else None
+
+
+def accounts(db: Session, user_ids: set[int]) -> dict[str, str]:
+    """The accounts a page names, by id → address. An id with no account left (erased after the row
+    was written) is simply absent, and the client shows the bare id."""
+    if not user_ids:
+        return {}
+    rows = db.execute(select(User.id, User.email).where(User.id.in_(user_ids))).all()
+    return {str(uid): email for uid, email in rows}

backend/app/routes/admin_logs.py

@@ -8,33 +8,29 @@ What a row holds was redacted when it was written (`logs.record_fields`, and the
 browser's rows), so this serves the columns as they are; a value the redactor took out reads as
 `[redacted]` / `[email]`, which is the point — the console shows that something was there.
 
-Paging is a keyset over `(ts desc, id desc)`: a browser row's `ts` is clamped into its consent
-window, so `ts` alone is not unique, and an offset would skip or repeat rows while the writer adds
-new ones at the top. The live tail re-reads the first page rather than asking for "ids after N":
+Paging is a keyset over `(ts desc, id desc)` (`_keyset`). The live tail re-reads the first page rather than asking for "ids after N":
 several processes write this table, and a batch's ids are taken when it inserts, not when it
 commits, so a slower batch can land below an id the tail already passed — and would never show.
 """
 
-import base64
-import binascii
-import json
 from dataclasses import dataclass
-from datetime import datetime, timezone
+from datetime import datetime
 from typing import Literal
 
-from fastapi import APIRouter, Depends, HTTPException, Query
-from sqlalchemy import Text, and_, cast, func, or_, select
+from fastapi import APIRouter, Depends, Query
+from sqlalchemy import Text, cast, func, or_, select
 from sqlalchemy.orm import Session
 
 from app import logsink
 from app.auth import admin_user
 from app.db import get_db
 from app.models import LogEvent, User
+from app.routes import _account as account
+from app.routes._keyset import PAGE_MAX, accounts, older_than, page, utc
 from app.utils import like_escape
 
 router = APIRouter(prefix="/api/admin/logs", tags=["admin"])
 
-PAGE_MAX = 200
 # A facet list is a chip row, not a report: past this many distinct events the rest are reachable
 # through the free-text search, and the response stays small whatever a self-host logs.
 FACET_EVENTS_MAX = 100
@@ -42,38 +38,9 @@ FACET_EVENTS_MAX = 100
 Level = Literal["TRACE", "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
 # The filters a facet can leave out of its own count (`LogFilters.conditions`).
 Facet = Literal["levels", "sources", "user", "event"]
-# `none` selects the rows that have no account / no event code (a scheduler line, an anonymous
-# request) — a real value to filter on, not the absence of a filter.
-NONE = "none"
-
-
-def _utc(value: datetime | None) -> datetime | None:
-    """A bound without a zone is UTC — the column is `timestamptz`, and a naive value would be read
-    in the database session's zone instead."""
-    if value is None or value.tzinfo is not None:
-        return value
-    return value.replace(tzinfo=timezone.utc)
-
-
-def _encode_cursor(row: LogEvent) -> str:
-    return base64.urlsafe_b64encode(json.dumps([row.ts.isoformat(), row.id]).encode()).decode()
-
-
-def _decode_cursor(cursor: str) -> tuple[datetime, int]:
-    try:
-        ts, row_id = json.loads(base64.urlsafe_b64decode(cursor.encode()))
-        return _utc(datetime.fromisoformat(ts)), int(row_id)
-    except (binascii.Error, json.JSONDecodeError, ValueError, TypeError) as exc:
-        raise HTTPException(status_code=400, detail={"code": "bad_cursor"}) from exc
-
-
-def _accounts(db: Session, user_ids: set[int]) -> dict[str, str]:
-    """The accounts a page names, by id → address. An id with no account left (erased after the
-    line was written) is simply absent, and the client shows the bare id."""
-    if not user_ids:
-        return {}
-    rows = db.execute(select(User.id, User.email).where(User.id.in_(user_ids))).all()
-    return {str(uid): email for uid, email in rows}
+# `none` selects the rows that have no event code, as the account filter's `none` (app/routes/_account)
+# selects the rows of no account — a real value to filter on, not the absence of a filter.
+NONE = account.NONE
 
 
 @dataclass(frozen=True)
@@ -86,6 +53,7 @@ class LogFilters:
     event: str | None
     request_id: str | None
     job_id: int | None
+    run_id: int | None
     q: str | None
     since: datetime | None
     until: datetime | None
@@ -100,11 +68,7 @@ class LogFilters:
         if self.sources and skip != "sources":
             out.append(LogEvent.source.in_(self.sources))
         if self.user is not None and skip != "user":
-            out.append(
-                LogEvent.user_id.is_(None)
-                if self.user == NONE
-                else LogEvent.user_id == int(self.user)
-            )
+            out.append(account.condition(LogEvent.user_id, self.user))
         if self.event is not None and skip != "event":
             out.append(
                 LogEvent.event.is_(None) if self.event == NONE else LogEvent.event == self.event
@@ -113,10 +77,12 @@ class LogFilters:
             out.append(LogEvent.request_id == self.request_id)
         if self.job_id is not None:
             out.append(LogEvent.job_id == self.job_id)
+        if self.run_id is not None:
+            out.append(LogEvent.run_id == self.run_id)
         if self.since is not None:
-            out.append(LogEvent.ts >= _utc(self.since))
+            out.append(LogEvent.ts >= utc(self.since))
         if self.until is not None:
-            out.append(LogEvent.ts < _utc(self.until))
+            out.append(LogEvent.ts < utc(self.until))
         if self.q and self.q.strip():
             like = f"%{like_escape(self.q.strip())}%"
             out.append(
@@ -134,17 +100,20 @@ class LogFilters:
 def log_filters(
     levels: list[Level] = Query(default=[]),
     sources: list[str] = Query(default=[], max_length=8),
-    user: str | None = Query(default=None, pattern=r"^(none|[0-9]{1,10})$"),
+    user: str | None = Query(default=None, pattern=account.PATTERN),
     event: str | None = Query(
         default=None, max_length=64, pattern=r"^(none|[a-z][a-z0-9_]*(\.[a-z0-9_]+)*)$"
     ),
     request_id: str | None = Query(default=None, pattern=r"^[A-Za-z0-9._-]{8,64}$"),
     job_id: int | None = Query(default=None, ge=1),
+    run_id: int | None = Query(default=None, ge=1),
     q: str | None = Query(default=None, max_length=200),
     since: datetime | None = None,
     until: datetime | None = None,
 ) -> LogFilters:
-    return LogFilters(list(levels), list(sources), user, event, request_id, job_id, q, since, until)
+    return LogFilters(
+        list(levels), list(sources), user, event, request_id, job_id, run_id, q, since, until
+    )
 
 
 @router.get("")
@@ -159,28 +128,20 @@ def list_logs(
     continues from a previous page's `next_cursor`."""
     query = select(LogEvent).where(*filters.conditions())
     if before is not None:
-        ts, row_id = _decode_cursor(before)
-        # The `ts <=` conjunct is implied by the OR; it is there so the planner can bound the
-        # `ts` index scan instead of filtering every newer row first.
-        query = query.where(
-            and_(
-                LogEvent.ts <= ts,
-                or_(LogEvent.ts < ts, and_(LogEvent.ts == ts, LogEvent.id < row_id)),
-            )
-        )
-
-    # One more than asked, to know whether there is a next page without a second count query.
-    rows = (
-        db.execute(query.order_by(LogEvent.ts.desc(), LogEvent.id.desc()).limit(limit + 1))
-        .scalars()
-        .all()
+        query = query.where(older_than(LogEvent.ts, LogEvent.id, before))
+    rows, cursor = page(
+        list(
+            db.execute(query.order_by(LogEvent.ts.desc(), LogEvent.id.desc()).limit(limit + 1))
+            .scalars()
+            .all()
+        ),
+        limit,
+        lambda r: r.ts,
     )
-    more = len(rows) > limit
-    rows = rows[:limit]
     return {
         "items": [logsink.serialize(r) for r in rows],
-        "accounts": _accounts(db, {r.user_id for r in rows if r.user_id is not None}),
-        "next_cursor": _encode_cursor(rows[-1]) if more else None,
+        "accounts": accounts(db, {r.user_id for r in rows if r.user_id is not None}),
+        "next_cursor": cursor,
     }
 
 
@@ -211,6 +172,6 @@ def log_facets(
         "sources": counts(LogEvent.source, "sources"),
         "events": counts(LogEvent.event, "event", FACET_EVENTS_MAX),
         "users": [{**u, "value": str(u["value"])} for u in users],
-        "accounts": _accounts(db, {u["value"] for u in users if u["value"] != NONE}),
+        "accounts": accounts(db, {u["value"] for u in users if u["value"] != NONE}),
         "total": db.scalar(total) or 0,
     }

backend/app/routes/audit.py

@@ -1,67 +1,144 @@
 """Admin audit log API: read the append-only who/what/when trail, and clear it.
 
-Admin-only. The client renders these rows in a DataTable (client-side sort/filter/paginate), so
-this returns the most recent window of rows (capped) plus the true total, rather than paging
-server-side — the log stays small in practice (state-changing runs + admin actions only)."""
+Admin-only. The activity page reads it a page at a time, newest first, filtered on the server like
+the server log (`_keyset` paging), so the whole trail is reachable however long it grows."""
 
-from fastapi import APIRouter, Depends
-from sqlalchemy import delete, func, select
+from dataclasses import dataclass
+from datetime import datetime
+from typing import Literal
+
+from fastapi import APIRouter, Depends, Query
+from sqlalchemy import Text, cast, delete, func, or_, select
 from sqlalchemy.orm import Session
 
 from app import audit
 from app.audit import AuditAction
 from app.auth import admin_user
 from app.db import get_db
-from app.models import AuditLog, User
-from app.utils import iso
+from app.models import AuditLog, LogEvent, User
+from app.routes import _account as account
+from app.routes._keyset import PAGE_MAX, accounts, older_than, page, utc
+from app.utils import iso, like_escape
 
 router = APIRouter(prefix="/api/admin/audit", tags=["admin"])
 
-# Cap the client-side window. The audit log is small by design; if it ever outgrows this we'd
-# switch the DataTable to server-side paging.
-MAX_ROWS = 2000
 
-
-def _serialize(row: AuditLog, actors: dict[int, str]) -> dict:
+def _serialize(row: AuditLog, lines: dict[str, int]) -> dict:
     return {
         "id": row.id,
         "created_at": iso(row.created_at),
-        # actor_id NULL = the scheduler/background did it (client shows "System"); a set id that
-        # can't be resolved would mean a deleted user, but SET NULL makes that NULL — so present.
-        "actor": actors.get(row.actor_id) if row.actor_id is not None else None,
+        # NULL = the background did it (the client shows "System"); the address is in `accounts`.
+        "actor_id": row.actor_id,
         "action": row.action,
         "target_type": row.target_type,
         "target_id": row.target_id,
         "summary": row.summary,
         "before": row.before,
         "after": row.after,
+        "request_id": row.request_id,
+        # How many of that request's log lines are still kept: they go after days, the row after
+        # months, and a link to none would open an empty list.
+        "log_lines": lines.get(row.request_id, 0) if row.request_id else 0,
     }
 
 
+@dataclass(frozen=True)
+class AuditFilters:
+    action: str | None
+    actor: str | None
+    q: str | None
+    since: datetime | None
+    until: datetime | None
+
+    def conditions(self, skip: Literal["action"] | None = None) -> list:
+        """The WHERE conditions, ANDed. `skip="action"` leaves the action filter out, so the action
+        menu counts every action under the other filters (the way back from a pick)."""
+        out = []
+        if self.action is not None and skip != "action":
+            out.append(AuditLog.action == self.action)
+        if self.actor is not None:
+            out.append(account.condition(AuditLog.actor_id, self.actor))
+        if self.since is not None:
+            out.append(AuditLog.created_at >= utc(self.since))
+        if self.until is not None:
+            out.append(AuditLog.created_at < utc(self.until))
+        if self.q and self.q.strip():
+            like = f"%{like_escape(self.q.strip())}%"
+            out.append(
+                or_(
+                    AuditLog.summary.ilike(like),
+                    AuditLog.target_id.ilike(like),
+                    AuditLog.action.ilike(like),
+                    AuditLog.request_id.ilike(like),
+                    cast(AuditLog.before, Text).ilike(like),
+                    cast(AuditLog.after, Text).ilike(like),
+                )
+            )
+        return out
+
+
+def audit_filters(
+    action: str | None = Query(default=None, max_length=48, pattern=r"^[a-z][a-z0-9_]*$"),
+    user: str | None = Query(default=None, pattern=account.PATTERN),
+    q: str | None = Query(default=None, max_length=200),
+    since: datetime | None = None,
+    until: datetime | None = None,
+) -> AuditFilters:
+    return AuditFilters(action, user, q, since, until)
+
+
 @router.get("")
 def list_audit(
-    limit: int = 500,
+    filters: AuditFilters = Depends(audit_filters),
+    before: str | None = Query(default=None, max_length=200),
+    limit: int = Query(default=100, ge=1, le=PAGE_MAX),
     _: User = Depends(admin_user),
     db: Session = Depends(get_db),
 ) -> dict:
-    limit = max(1, min(limit, MAX_ROWS))
-    total = db.scalar(select(func.count()).select_from(AuditLog)) or 0
-    rows = (
-        db.execute(select(AuditLog).order_by(AuditLog.created_at.desc()).limit(limit))
-        .scalars()
-        .all()
+    """One page of rows, newest first, matching every filter given; `before` continues from a
+    previous page's `next_cursor`."""
+    query = select(AuditLog).where(*filters.conditions())
+    if before is not None:
+        query = query.where(older_than(AuditLog.created_at, AuditLog.id, before))
+    order = (AuditLog.created_at.desc(), AuditLog.id.desc())
+    rows, cursor = page(
+        list(db.execute(query.order_by(*order).limit(limit + 1)).scalars().all()),
+        limit,
+        lambda r: r.created_at,
     )
-    actor_ids = {r.actor_id for r in rows if r.actor_id is not None}
-    actors: dict[int, str] = {}
-    if actor_ids:
-        actors = dict(db.execute(select(User.id, User.email).where(User.id.in_(actor_ids))).all())
+    requests = {r.request_id for r in rows if r.request_id}
+    lines: dict[str, int] = {}
+    if requests:
+        counted = db.execute(
+            select(LogEvent.request_id, func.count())
+            .where(LogEvent.request_id.in_(requests))
+            .group_by(LogEvent.request_id)
+        )
+        lines = {rid: n for rid, n in counted.tuples() if rid is not None}
     return {
-        "items": [_serialize(r, actors) for r in rows],
-        "total": total,
-        "returned": len(rows),
+        "items": [_serialize(r, lines) for r in rows],
+        "accounts": accounts(db, {r.actor_id for r in rows if r.actor_id is not None}),
+        "next_cursor": cursor,
     }
 
 
+@router.get("/facets")
+def audit_facets(
+    filters: AuditFilters = Depends(audit_filters),
+    _: User = Depends(admin_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """The actions present under the other filters, with counts — what the action menu offers."""
+    n = func.count().label("n")
+    stmt = (
+        select(AuditLog.action, n)
+        .where(*filters.conditions("action"))
+        .group_by(AuditLog.action)
+        .order_by(n.desc(), AuditLog.action)
+    )
+    return {"actions": [{"value": a, "count": c} for a, c in db.execute(stmt).all()]}
+
+
 @router.delete("")
 def clear_audit(admin: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
     """Clear the audit log. Leaves ONE row recording who cleared it + how many (tamper-evidence)."""

backend/app/routes/auth.py

@@ -5,9 +5,10 @@ helpers) stays in app/auth.py, imported below."""
 
 import secrets
 from datetime import datetime, timedelta, timezone
-from typing import NamedTuple
+from typing import Literal, NamedTuple
 from urllib.parse import urlencode
 
+from authlib.common.security import generate_token
 from authlib.integrations.starlette_client import OAuth, OAuthError
 from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
 from fastapi.responses import JSONResponse, RedirectResponse
@@ -16,6 +17,8 @@ from sqlalchemy import select, update
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import Session
 
+from app.userscope import owned_by
+from app import apitokens
 from app import email as email_mod
 from app import logs
 from app import mailtext
@@ -23,10 +26,11 @@ from app import sysconfig
 from app.auth import (
     _YOUTUBE_SCOPES,
     BASE_SCOPES,
-    MAX_SESSION_ACCOUNTS,
+    EXPECTED_ACCOUNT_HEADER,
     PASSWORD_MIN_LEN,
     READ_SCOPES,
     WRITE_SCOPES,
+    built_for_another_account,
     current_user,
     env_admin_canonicals,
     env_allowed_canonicals,
@@ -35,7 +39,9 @@ from app.auth import (
     is_allowed,
     log,
     operator_contact,
-    resolved_user_id,
+    optional_current_user,
+    session_user_id,
+    stamp_session,
 )
 from app.config import settings
 from app.db import SessionLocal, get_db
@@ -318,6 +324,25 @@ class SetPasswordIn(BaseModel):
     current_password: str | None = None
 
 
+# A connect in flight, in the session: `{state, uid, explicit}`. It names the OAuth state of the flow
+# that set it, because the session is the browser's, not the flow's: a callback coming back with
+# another state (an older flow, which authlib then refuses) must not spend the marker of the connect
+# still open, or that connect would return as a plain sign-in — signing someone else in, or storing
+# a YouTube grant on whichever account the Google identity already belongs to.
+_LINK_MARKER = "oauth_link"
+
+# Where a navigation from a tab showing another account than the one signed in lands: the app, which
+# boots as the account signed in now and says why (the frontend's `ACCOUNT_CHANGED_FLAG`).
+ACCOUNT_CHANGED_LANDING = "/?account=changed"
+
+
+def _take_link_marker(request: Request) -> dict | None:
+    marker = request.session.get(_LINK_MARKER)
+    if not isinstance(marker, dict) or marker.get("state") != request.query_params.get("state"):
+        return None
+    return request.session.pop(_LINK_MARKER)
+
+
 @router.get("/login")
 async def login(request: Request, db: Session = Depends(get_db)):
     # access_type=offline ensures a refresh_token on first authorization. We avoid
@@ -328,6 +353,10 @@ async def login(request: Request, db: Session = Depends(get_db)):
         # Google sign-in isn't configured on this instance — land back on the login page (the UI
         # hides the button, so this is just a safety net for a stale/direct hit).
         return RedirectResponse(url="/")
+    # A sign-in, never a link. Starting it retires every earlier flow's state (authlib keeps only the
+    # newest), so a connect's marker left from before — abandoned at Google, or still open in
+    # another tab — names a flow that can no longer finish.
+    request.session.pop(_LINK_MARKER, None)
     return await client.authorize_redirect(
         request,
         _oauth_redirect_uri(request),
@@ -348,24 +377,23 @@ async def callback(
     if client is None:
         return RedirectResponse(url="/")
     # Linking mode: an already-authenticated account is attaching this Google identity (a
-    # password account enabling YouTube or connecting SSO). The markers are set by /auth/link and
-    # /auth/upgrade; when present we attach to the current user instead of identifying by sub.
-    # Pop them BEFORE the token exchange: if the user aborts Google's consent screen the exchange
-    # throws below, and a marker left behind would silently turn the NEXT clean sign-in into a link
-    # attempt. Popping up front means an aborted link flow leaves no residue in the session.
-    link_uid = request.session.pop("oauth_link_uid", None)
-    link_explicit = request.session.pop("oauth_link_explicit", False)
+    # password account enabling YouTube or connecting SSO). The marker is set by
+    # /auth/google/connect; when it is this flow's we attach to the current user instead of
+    # identifying by sub. Taken BEFORE the token exchange: if the user aborts Google's consent screen
+    # the exchange throws below, and a marker left behind would silently turn the NEXT clean sign-in
+    # into a link attempt. A connect that never comes back here at all is cleared by /auth/login.
+    link = _take_link_marker(request)
     try:
         token = await client.authorize_access_token(request)
     except OAuthError as exc:
         # The user cancelled or denied consent at Google — land back with a friendly notice instead
-        # of a raw 400 JSON error page. An already-signed-in user (adding another account via
-        # /auth/login, or linking/upgrading via /auth/link|upgrade) never sees the pre-auth login
-        # banner, so surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on
-        # the login page with the oauth_cancelled banner instead. Keyed on the live session, not on
-        # link_uid, so the add-account path (no marker) is covered too.
+        # of a raw 400 JSON error page. An already-signed-in user (signing in again via /auth/login,
+        # or connecting via /auth/google/connect) never sees the pre-auth login banner, so
+        # surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on the login page
+        # with the oauth_cancelled banner instead. Keyed on the live session, not on link_uid, so a
+        # sign-in started while signed in (no marker) is covered too.
         log.info("Google OAuth cancelled/failed: %s", exc.error)
-        if request.session.get("user_id") is not None:
+        if session_user_id(request) is not None:
             return RedirectResponse(url="/?oauth=cancelled")
         return RedirectResponse(url="/?login=oauth_cancelled")
 
@@ -376,8 +404,8 @@ async def callback(
     sub = userinfo["sub"]
     email = (userinfo.get("email") or "").lower()
 
-    if link_uid is not None:
-        return _complete_link(request, db, link_uid, link_explicit, sub, userinfo, token)
+    if link is not None:
+        return _complete_link(request, db, link["uid"], link["explicit"], sub, userinfo, token)
 
     if not is_allowed(db, email):
         # No uid to name — having no account here is the branch. And the address is left out on
@@ -446,7 +474,7 @@ async def callback(
             # SEC-10: drop any password nobody proved the mailbox for, now that Google attests
             # ownership (bumps the epoch to revoke any session it minted). Runs before the
             # activation below, which is what it reads.
-            revoke_untrusted_preset_password(user)
+            revoke_untrusted_preset_password(db, user)
             # Google verified the email and is_allowed granted access, so finish activating a
             # previously pending password registration (otherwise current_user would bounce it).
             user.email_verified = True
@@ -486,7 +514,7 @@ async def callback(
     # real owner, signed in through Google right now, sets a new password in Settings.
     if canonical_email(email) in env_admin_canonicals():
         if user.role != "admin" and user.password_hash is not None:
-            _revoke_password(user)
+            _revoke_password(db, user)
         user.role = "admin"
     elif not user.role:
         user.role = "user"
@@ -578,19 +606,16 @@ def _complete_link(
     token: dict,
 ):
     """Attach the just-authorized Google identity to the already-signed-in account that started
-    the link/upgrade. No is_allowed gate — they're already an active user. Refuses to hijack a
+    the connect. No is_allowed gate — they're already an active user. Refuses to hijack a
     Google identity owned by another account, or to silently swap a different linked identity.
-    `explicit` (the popped oauth_link_explicit marker) distinguishes a "Connect Google" click from
-    a YouTube upgrade — the caller pops it up front so an aborted flow leaves nothing behind."""
-    # SEC-11: the marker must name an account signed into THIS browser — the session default OR any
-    # account in the wallet — not only the default. A tab acting as a secondary account links Google to
-    # ITSELF (the marker was set from current_user, which resolves the ?account= it navigated with), so
-    # accepting wallet membership stops the identity/token landing on the default account A instead of B.
-    accounts = request.session.get("account_ids") or []
-    if link_uid != request.session.get("user_id") and link_uid not in accounts:
-        return RedirectResponse(url="/?link=error")
-    user = db.get(User, link_uid)
-    if user is None:
+    `explicit` (from the link marker) distinguishes a "Connect Google" click from a YouTube grant —
+    the caller takes the marker up front so an aborted flow leaves nothing behind."""
+    # The marker must name the account still signed into THIS browser, and that session must still
+    # hold: a sign-in as someone else between the start and the callback replaced it (the identity
+    # must not land on the newcomer), and a session revoked meanwhile ("log out everywhere", a
+    # password change, a suspension) must not finish what it started.
+    user = optional_current_user(request, db)
+    if user is None or user.id != link_uid:
         return RedirectResponse(url="/?link=error")
     other = db.execute(
         select(User).where(User.google_sub == sub, User.id != user.id)
@@ -610,45 +635,31 @@ def _complete_link(
     _store_token(db, user, token)
     db.commit()
     log.info("Google linked: uid=%s scopes=%s", user.id, token.get("scope"))
-    # An explicit "Connect Google" lands on Settings with a confirmation; a YouTube upgrade
+    # An explicit "Connect Google" lands on Settings with a confirmation; a YouTube grant
     # (wizard / Settings access rows) returns to the app so the onboarding flow resumes as before.
     return RedirectResponse(url="/?link=ok" if explicit else "/")
 
 
 @router.post("/logout")
 async def logout(request: Request):
-    """Sign the requesting tab's active account out of this browser (removing it from the wallet).
-    Per-tab aware: the account is taken from the X-Siftlode-Account header when present, so logging
-    out in one tab doesn't disturb another tab running a different account. If the account being
-    removed was the session default, promote the most recent remaining account (or clear the
-    session when none remain)."""
-    # A tab still pinned to an account already signed out in another tab resolves to nobody, so it
-    # signs nobody out — it used to fall back to the default and sign THAT account out in its place.
-    active, is_default = resolved_user_id(request)
-    remaining = [a for a in (request.session.get("account_ids") or []) if a != active]
-    request.session["account_ids"] = remaining
-    if is_default:
-        if remaining:
-            request.session["user_id"] = remaining[-1]
-            return JSONResponse({"ok": True, "switched": True})
-        request.session.clear()
-        return JSONResponse({"ok": True, "switched": False})
-    # A non-default (per-tab) account signed out: it's gone from the wallet and the session default
-    # is untouched. The tab drops its own override and falls back to the default on reload.
-    return JSONResponse({"ok": True, "switched": False})
+    """Sign this browser out: the session holds one account, and it ends — unless the tab asking was
+    built for another account than the one signed in now (a sign-in as someone else in another tab):
+    its Sign out would end THAT person's session (see `EXPECTED_ACCOUNT_HEADER`)."""
+    signed_in = session_user_id(request)
+    if signed_in is not None and built_for_another_account(
+        signed_in, request.headers.get(EXPECTED_ACCOUNT_HEADER)
+    ):
+        raise refused(401, "account_changed")
+    request.session.clear()
+    return JSONResponse({"ok": True})
 
 
 def _establish_session(request: Request, user: User) -> None:
     """Sign `user` in for this browser session — the ONE sign-in path, shared by the Google callback
-    and password login, so both join the same account switcher and both stamp `last_login_at` (the
-    caller commits). Multi-session: every account that authenticated in this browser is remembered so
-    the user can switch between them; only accounts that actually proved themselves HERE land in that
-    wallet (it is the proof they may be switched to)."""
-    accounts = [a for a in (request.session.get("account_ids") or []) if a != user.id]
-    accounts.append(user.id)
-    request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
+    and password login, so both stamp `last_login_at` (the caller commits). One account per session:
+    the sign-in replaces whoever was signed in here before."""
     request.session["user_id"] = user.id
-    _remember_epoch(request, user)
+    stamp_session(request, user)
     user.last_login_at = datetime.now(timezone.utc)
 
 
@@ -965,7 +976,16 @@ def _resend_verify_if_eligible(email: str) -> None:
             send_verification_link(db, user)
 
 
-def _login_refused(status: int, reason: str, user: User | None = None, **extra) -> HTTPException:
+# Every refusal a password sign-in answers with; the UI names each one (`welcome.auth.blocked.<reason>`,
+# held by the frontend's backendIds contract, which reads this Literal).
+LoginRefusal = Literal[
+    "rate_limited", "invalid", "suspended", "email_unverified", "pending_approval"
+]
+
+
+def _login_refused(
+    status: int, reason: LoginRefusal, user: User | None = None, **extra
+) -> HTTPException:
     """Every refusal password_login can answer with, in the codebase's structured `{code, reason}`
     shape so the SPA can say it in the user's own language (the old bare English strings were
     unreadable under a bilingual UI, and unusable as anything but a one-line error).
@@ -981,15 +1001,15 @@ def _login_refused(status: int, reason: str, user: User | None = None, **extra)
     `email_verified` is false, and vice versa — the SPA renders whichever is actually outstanding
     rather than assuming a sequence.
     """
-    detail: dict = {"code": "login", "reason": reason}
-    if user is not None:
-        detail |= {
-            "email": user.email,
-            "email_verified": user.email_verified,
-            "approved": user.is_active,
-        }
-    detail |= {k: v for k, v in extra.items() if v is not None}
-    return HTTPException(status_code=status, detail=detail)
+    account = (
+        {"email": user.email, "email_verified": user.email_verified, "approved": user.is_active}
+        if user is not None
+        else {}
+    )
+    given = {k: v for k, v in extra.items() if v is not None}
+    return HTTPException(
+        status_code=status, detail={**account, **given, "code": "login", "reason": reason}
+    )
 
 
 @router.post("/password-login")
@@ -1118,13 +1138,12 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
     # about. The person still confirms their address the normal way.
     if kind == "reset":
         user.email_verified = True
-    bump_session_epoch(
-        user
-    )  # SA4: a reset (often a compromise response) kills all existing sessions.
+    # SA4: a reset (often a compromise response) kills all existing sessions — and API tokens.
+    bump_session_epoch(db, user)
     # Burn any other outstanding reset tokens for this user.
     for row in db.execute(
         select(AuthToken).where(
-            AuthToken.user_id == user.id,
+            owned_by(AuthToken.user_id, user),
             AuthToken.kind.in_(_PASSWORD_TOKEN_KINDS),
             AuthToken.used_at.is_(None),
         )
@@ -1134,22 +1153,27 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
     return {"ok": True}
 
 
-def _remember_epoch(request: Request, user: User) -> None:
-    """Record this account's current `session_epoch` in the signed cookie (SA4), so `current_user`
-    can later reject the cookie if the account's epoch was bumped (password reset/change, or
-    "log out everywhere"). Per-account so one account's revocation doesn't evict the others."""
-    epochs = dict(request.session.get("epochs") or {})
-    epochs[str(user.id)] = user.session_epoch
-    request.session["epochs"] = epochs
-
-
-def bump_session_epoch(user: User) -> None:
+def bump_session_epoch(db: Session, user: User) -> None:
     """Invalidate every EXISTING signed cookie for this account by advancing its epoch (SA4). Cookies
-    minted before the bump carry a now-stale epoch and are rejected by `current_user`."""
-    user.session_epoch = (user.session_epoch or 0) + 1
+    minted before the bump carry a now-stale epoch and are rejected by `current_user`. The account's
+    API tokens go with them (`apitokens.revoke_all`): a token minted from a stolen session must not
+    outlive the answer to that theft. Not committed — the caller commits it with its change.
+
+    The user's row is locked before the tokens go, the lock `create_token` takes too: a token made at
+    the same moment either commits first and is deleted here, or waits and sees the new epoch. FOR NO
+    KEY UPDATE (`key_share`): the two still exclude each other, but a row that only refers to the user
+    (a notice, an audit line) is not held up — a token's expiry notice would deadlock with it."""
+    if user.id is None:
+        user.session_epoch = (user.session_epoch or 0) + 1
+        return
+    epoch = db.execute(
+        select(User.session_epoch).where(User.id == user.id).with_for_update(key_share=True)
+    ).scalar_one()
+    user.session_epoch = (epoch or 0) + 1
+    apitokens.revoke_all(db, user.id)
 
 
-def revoke_untrusted_preset_password(user: User) -> None:
+def revoke_untrusted_preset_password(db: Session, user: User) -> None:
     """SEC-10 (account pre-emption): a password set on an account whose email was NEVER verified is
     untrusted — whoever registered the address (to pre-empt the real owner) could have chosen it. When
     the real owner attests ownership through Google, drop that password and bump the epoch to revoke any
@@ -1183,15 +1207,15 @@ def revoke_untrusted_preset_password(user: User) -> None:
     _send_reset_if_eligible keyed off exactly that, so a revoked account silently got no reset link and
     was locked out for good (it could not sign in either). The stamp separates the two cases."""
     if not user.email_verified or (not user.is_active and user.password_hash is not None):
-        _revoke_password(user)
+        _revoke_password(db, user)
 
 
-def _revoke_password(user: User) -> None:
+def _revoke_password(db: Session, user: User) -> None:
     """Drop the account's password, stamp why (so password-reset still accepts it) and end every
     session it signed in."""
     user.password_hash = None
     user.password_revoked_at = datetime.now(timezone.utc)
-    bump_session_epoch(user)
+    bump_session_epoch(db, user)
 
 
 @router.post("/logout-others")
@@ -1203,37 +1227,13 @@ def logout_others(
     in somewhere" / a suspected session compromise."""
     # Commit the bump BEFORE re-stamping this cookie, so a failed commit can't strand the current
     # session at a newer epoch than the DB (see set_password).
-    bump_session_epoch(user)
+    bump_session_epoch(db, user)
     db.commit()
-    _remember_epoch(request, user)
+    stamp_session(request, user)
     log.info("Logout-others: uid=%s (epoch=%s)", user.id, user.session_epoch)
     return {"ok": True}
 
 
-@router.get("/link")
-async def link_google(
-    request: Request,
-    user: User = Depends(current_user),
-    db: Session = Depends(get_db),
-):
-    """Start linking a Google account to the signed-in (e.g. password) account. Requests only
-    the identity scopes — YouTube access is granted separately via /auth/upgrade. The callback
-    sees `oauth_link_uid` and attaches the identity instead of creating a new account."""
-    client = google_oauth(db)
-    if client is None:
-        return RedirectResponse(url="/?link=error")
-    request.session["oauth_link_uid"] = user.id
-    request.session["oauth_link_explicit"] = True
-    return await client.authorize_redirect(
-        request,
-        _oauth_redirect_uri(request),
-        access_type="offline",
-        prompt="select_account",
-        include_granted_scopes="true",
-        scope=BASE_SCOPES,
-    )
-
-
 @router.post("/set-password")
 def set_password(
     body: SetPasswordIn,
@@ -1255,37 +1255,59 @@ def set_password(
     # SA4: a password change logs out every OTHER session; re-stamp this cookie at the new epoch so
     # the person making the change stays signed in here. Commit BEFORE re-stamping so a failed commit
     # can't leave the cookie at a newer epoch than the DB (which would wrongly 401 THIS session).
-    bump_session_epoch(user)
+    bump_session_epoch(db, user)
     db.commit()
-    _remember_epoch(request, user)
+    stamp_session(request, user)
     log.info("Password set/changed: uid=%s", user.id)
     return {"ok": True}
 
 
-@router.get("/upgrade")
-async def upgrade(
+# What each Google connect asks Google for. `identity` attaches a Google sign-in to the account
+# (Settings → Connect Google) and lets the person pick which Google account; `read` and `write` grant
+# YouTube access (the onboarding wizard, the Settings access rows) and always show the consent screen,
+# so a scope granted earlier and since revoked at Google is asked for again.
+_CONNECT: dict[str, tuple[str, str]] = {
+    "identity": (BASE_SCOPES, "select_account"),
+    "read": (READ_SCOPES, "consent"),
+    "write": (WRITE_SCOPES, "consent"),
+}
+
+
+@router.get("/google/connect")
+async def google_connect(
     request: Request,
-    access: str = "read",
+    access: Literal["identity", "read", "write"],
+    expect: str | None = None,
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
 ):
-    """Incremental consent for the onboarding wizard. `access=read` grants YouTube
-    read-only (enough to build the feed); `access=write` additionally grants management
-    (unsubscribe). The shared callback stores whatever Google returns, so `can_read` /
-    `can_write` flip on once granted. Anything other than an explicit `write` defaults to
-    the least-privileged read scope."""
+    """Attach Google to the signed-in account: its identity, or YouTube read/write access on top.
+    The shared callback sees the link marker and attaches what Google returns to THIS account —
+    without it a password account (no google_sub) would get a brand-new, separate Google user
+    created instead. It stores whatever scopes Google grants, so `can_read` / `can_write` flip on
+    once granted.
+
+    `expect` is the account the tab was built for. A navigation carries no headers, so the check
+    every request gets (`EXPECTED_ACCOUNT_HEADER`) rides the query here: a tab still showing the
+    account signed in before would otherwise attach a Google identity to the one signed in now."""
+    if built_for_another_account(user.id, expect):
+        return RedirectResponse(url=ACCOUNT_CHANGED_LANDING)
     client = google_oauth(db)
     if client is None:
-        return RedirectResponse(url="/")
-    # Attach the grant to THIS account in the callback. Without it a password account (no
-    # google_sub) would get a brand-new, separate Google user created instead of being linked.
-    request.session["oauth_link_uid"] = user.id
-    scope = WRITE_SCOPES if access == "write" else READ_SCOPES
+        return RedirectResponse(url="/?link=error")
+    scope, prompt = _CONNECT[access]
+    state = generate_token(48)
+    request.session[_LINK_MARKER] = {
+        "state": state,
+        "uid": user.id,
+        "explicit": access == "identity",
+    }
     return await client.authorize_redirect(
         request,
         _oauth_redirect_uri(request),
+        state=state,
         access_type="offline",
-        prompt="consent",
+        prompt=prompt,
         include_granted_scopes="true",
         scope=scope,
     )

backend/app/routes/channels.py

@@ -3,31 +3,23 @@ hidden, tags) and per-channel sync state. Local reads/writes plus YouTube-side w
 (subscribe / unsubscribe) that require the optional write scope."""
 
 from datetime import datetime, timezone
+from functools import partial
+from typing import TYPE_CHECKING
 
-from fastapi import APIRouter, BackgroundTasks, Depends
+from fastapi import APIRouter, Depends
 from pydantic import BaseModel
-from sqlalchemy import Select, func, select, update
 from sqlalchemy.orm import Session
 
-from app.collation import collated, sorted_words
+from app.collation import sorted_words
 from app.mailtext import lang_for_user
-from app import logs, quota
+from app import logs, quota, scheduler
 from app.auth import admin_user, current_user, require_write_scope
 from app.db import get_db
-from app.models import (
-    LIVE_OR_UPCOMING,
-    BlockedChannel,
-    Channel,
-    ChannelTag,
-    ExploredChannel,
-    Playlist,
-    PlaylistItem,
-    Subscription,
-    Tag,
-    User,
-    Video,
-)
-from app.routes._common import get_or_404, not_found, owned_or_404, youtube_refusal
+from app.models import User
+from app.catalog import queries, service
+from app.reader import queries as reader_queries
+from app.reader import service as reader_service
+from app.routes._common import not_found, youtube_refusal
 from app.sync.explore import explore_ingest_page
 from app.sync.runner import backfill_channel_now, run_recent_backfill
 from app.sync.subscriptions import apply_channel_details, details_stale
@@ -35,35 +27,17 @@ from app.utils import iso
 from app.youtube.client import YouTubeClient, YouTubeError
 from app.refusal import refused
 
+if TYPE_CHECKING:
+    from app.catalog.models import Channel, Subscription
+
 router = APIRouter(prefix="/api/channels", tags=["channels"])
 
 log = logs.get_logger(__name__)
 
 
-def channel_aggregates(channel_ids: list[str]) -> Select:
-    """Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
-    reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
-    alone; a test holds that (test_channel_list_index_only)."""
-    return (
-        select(
-            Video.channel_id,
-            func.count(Video.id),
-            func.max(Video.published_at),
-            func.coalesce(func.sum(Video.duration_seconds), 0),
-        )
-        .where(Video.channel_id.in_(channel_ids))
-        .group_by(Video.channel_id)
-    )
-
-
 @router.get("")
 def list_channels(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
-    rows = db.execute(
-        select(Subscription, Channel)
-        .join(Channel, Channel.id == Subscription.channel_id)
-        .where(Subscription.user_id == user.id)
-        .order_by(Subscription.priority.desc(), collated(Channel.title, lang_for_user(user)))
-    ).all()
+    rows = queries.subscriptions(db, user, lang_for_user(user))
 
     # Per-channel aggregates over the shared catalog (channel_aggregates), computed on read rather than
     # denormalised: an Index Only Scan over a ~20 MB index, where it used to read every video's heap page
@@ -72,36 +46,21 @@ def list_channels(user: User = Depends(current_user), db: Session = Depends(get_
     # computed here — the manager table dropped its N/S/L column.)
     channel_ids = [c.id for _, c in rows]
     agg: dict[str, dict] = {}
-    if channel_ids:
-        for cid, n, last_up, total_dur in db.execute(channel_aggregates(channel_ids)).all():
-            agg[cid] = {
-                "stored": n,
-                "last_video_at": iso(last_up),
-                "total_duration_seconds": int(total_dur or 0),
-            }
+    for cid, n, last_up, total_dur in queries.aggregates(db, channel_ids):
+        agg[cid] = {
+            "stored": n,
+            "last_video_at": iso(last_up),
+            "total_duration_seconds": int(total_dur or 0),
+        }
 
     # Channels already in the *global* deep-backfill queue — i.e. at least one user (any
     # user) has requested full history and it isn't done yet. Their whole back-catalogue is
     # coming for everyone (video data is shared), so the UI shouldn't offer "get full
     # history" to a user who simply hasn't personally opted in.
-    deep_queued: set[str] = set()
-    if channel_ids:
-        for (cid,) in db.execute(
-            select(Subscription.channel_id)
-            .where(
-                Subscription.channel_id.in_(channel_ids),
-                Subscription.deep_requested.is_(True),
-            )
-            .distinct()
-        ).all():
-            deep_queued.add(cid)
+    deep_queued = queries.deep_queued(db, channel_ids)
 
     # The user's personal tag links, grouped by channel.
-    tags_by_channel: dict[str, list[int]] = {}
-    for cid, tag_id in db.execute(
-        select(ChannelTag.channel_id, ChannelTag.tag_id).where(ChannelTag.user_id == user.id)
-    ).all():
-        tags_by_channel.setdefault(cid, []).append(tag_id)
+    tags_by_channel = reader_queries.user_tag_links(db, user)
 
     return [
         {
@@ -117,6 +76,10 @@ def list_channels(user: User = Depends(current_user), db: Session = Depends(get_
             "details_synced": ch.details_synced_at is not None,
             "recent_synced": ch.recent_synced_at is not None,
             "backfill_done": ch.backfill_done,
+            # When the follow reached this list: written by the subscription import the run that
+            # first saw it, or by a subscribe made here — not the channel row's own date, which an
+            # explored or discovered channel had before it was followed.
+            "added_at": iso(sub.created_at),
         }
         for sub, ch in rows
     ]
@@ -130,27 +93,7 @@ DISCOVERY_ENRICH_CAP = 200
 def _discovery_rows(db: Session, user: User) -> list:
     """The user's playlist channels they don't subscribe to (and that aren't their own),
     each with how many playlist videos come from it and across how many playlists."""
-    subscribed = select(Subscription.channel_id).where(Subscription.user_id == user.id)
-    q = (
-        select(
-            Channel,
-            func.count(func.distinct(PlaylistItem.video_id)),
-            func.count(func.distinct(Playlist.id)),
-        )
-        .select_from(PlaylistItem)
-        .join(Playlist, Playlist.id == PlaylistItem.playlist_id)
-        .join(Video, Video.id == PlaylistItem.video_id)
-        .join(Channel, Channel.id == Video.channel_id)
-        .where(Playlist.user_id == user.id, Channel.id.not_in(subscribed))
-    )
-    if user.yt_channel_id:
-        q = q.where(Channel.id != user.yt_channel_id)
-    return db.execute(
-        q.group_by(Channel.id).order_by(
-            func.count(func.distinct(PlaylistItem.video_id)).desc(),
-            collated(Channel.title, lang_for_user(user)),
-        )
-    ).all()
+    return reader_queries.playlist_channels(db, user, user.yt_channel_id, lang_for_user(user))
 
 
 @router.get("/discovery")
@@ -172,7 +115,7 @@ def discover_channels(
             ):
                 own = yt.get_my_channel_id()
             if own:
-                user.yt_channel_id = own
+                service.remember_own_channel(user, own)
                 db.commit()
         except YouTubeError as exc:
             # No traceback anywhere in this file's YouTube handlers: for an upstream answer
@@ -228,7 +171,7 @@ def discover_channels(
     ]
 
 
-def _channel_summary(ch: Channel) -> dict:
+def _channel_summary(ch: "Channel") -> dict:
     """The channel fields common to the list and discovery projections."""
     return {
         "id": ch.id,
@@ -242,7 +185,7 @@ def _channel_summary(ch: Channel) -> dict:
 
 
 def _channel_detail_dict(
-    channel: Channel,
+    channel: "Channel",
     *,
     subscribed: bool,
     explored: bool,
@@ -311,7 +254,7 @@ def channel_detail(
     needed (a stub) and refreshes it once it is stale (`DETAILS_MAX_AGE`), which is how a channel nobody subscribes to (an explored one, outside the
     subscription import) gets a banner that still resolves. The enrich uses the API key (1 unit,
     cached after)."""
-    channel = get_or_404(db, Channel, channel_id, what="channel")
+    channel = _channel_or_404(db, channel_id)
     # No `published_at is None` sentinel: a terminated channel never gets one, so it re-fetched on
     # every view; rows enriched before that field existed are long past DETAILS_MAX_AGE anyway.
     stale = details_stale(channel, datetime.now(timezone.utc))
@@ -323,7 +266,7 @@ def channel_detail(
             ):
                 apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
             db.commit()
-            channel = db.get(Channel, channel_id)
+            channel = _channel_or_404(db, channel_id)
         except YouTubeError as exc:
             log.warning(
                 "channel detail enrich failed: %s",
@@ -333,41 +276,15 @@ def channel_detail(
                 ),
             )
             db.rollback()
-    sub = db.execute(
-        select(Subscription).where(
-            Subscription.user_id == user.id, Subscription.channel_id == channel_id
-        )
-    ).scalar_one_or_none()
+    sub = queries.subscription(db, user, channel_id)
     subscribed = sub is not None
-    explored = (
-        db.execute(
-            select(ExploredChannel.id).where(
-                ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
-            )
-        ).first()
-        is not None
-    )
-    blocked = _is_blocked(db, user, channel_id)
+    explored = queries.exploration(db, user, channel_id) is not None
+    blocked = queries.is_blocked(db, user, channel_id)
     # Stored total + the Normal/Shorts/Live split in one grouped pass (same filters as the manager
     # list), so the channel page can show the per-type breakdown that moved off the table.
-    n_all, n_short, n_live = db.execute(
-        select(
-            func.count(Video.id),
-            func.count(Video.id).filter(Video.is_short.is_(True)),
-            func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
-        ).where(Video.channel_id == channel_id)
-    ).one()
-    stored = int(n_all or 0)
-    count_short = int(n_short or 0)
-    count_live = int(n_live or 0)
+    stored, count_short, count_live = queries.video_counts(db, channel_id)
     count_normal = max(0, stored - count_short - count_live)
-    tag_ids = list(
-        db.scalars(
-            select(ChannelTag.tag_id).where(
-                ChannelTag.user_id == user.id, ChannelTag.channel_id == channel_id
-            )
-        ).all()
-    )
+    tag_ids = reader_queries.user_tag_links(db, user, channel_id).get(channel_id, [])
     return _channel_detail_dict(
         channel,
         subscribed=subscribed,
@@ -394,31 +311,18 @@ def explore_channel(
     """Open an un-subscribed channel for browsing. Records the exploration (a per-user,
     grace-clocked row that gates this user's access to the channel's ephemeral videos and
     drives the later cleanup), flags the channel ephemeral unless someone organically follows
-    it, then ingests one page of its uploads (newest-first) marked via_explore + enriched.
-    Returns the next page token so the page can "load more". Guarded against the shared
-    backfill quota reserve."""
-    channel = get_or_404(db, Channel, channel_id, what="channel")
-    if _is_blocked(db, user, channel_id):
+    it, then ingests one page of its uploads (newest-first), enriched — marked via_explore while
+    nobody follows it. Returns the next page token so the page can "load more". Guarded against
+    the shared backfill quota reserve."""
+    channel = _channel_or_404(db, channel_id)
+    if queries.is_blocked(db, user, channel_id):
         raise refused(403, "channel_blocked")
     if quota.backfill_reserve_reached():
         raise refused(429, "quota_reserve")
 
     # Record/refresh the exploration (DB only) before ingesting, so the explorer can see the
     # videos as soon as they land. Flag the channel ephemeral unless it's organically followed.
-    existing = db.execute(
-        select(ExploredChannel).where(
-            ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
-        )
-    ).scalar_one_or_none()
-    if existing is None:
-        db.add(ExploredChannel(user_id=user.id, channel_id=channel_id))
-    else:
-        existing.created_at = datetime.now(timezone.utc)  # refresh the grace clock
-    followed = db.execute(
-        select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
-    ).first()
-    if followed is None and not channel.from_explore:
-        channel.from_explore = True
+    service.explore(db, user, channel)
     db.commit()
 
     try:
@@ -432,7 +336,7 @@ def explore_channel(
             if details_stale(channel, datetime.now(timezone.utc)):
                 apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
                 db.commit()
-                channel = db.get(Channel, channel_id)
+                channel = _channel_or_404(db, channel_id)
             result = explore_ingest_page(db, yt, channel, page_token)
     except YouTubeError as exc:
         raise youtube_refusal(exc) from exc
@@ -444,30 +348,20 @@ def explore_channel(
     }
 
 
-def _user_subscription(db: Session, user: User, channel_id: str) -> Subscription:
-    sub = db.execute(
-        select(Subscription).where(
-            Subscription.user_id == user.id, Subscription.channel_id == channel_id
-        )
-    ).scalar_one_or_none()
+def _channel_or_404(db: Session, channel_id: str) -> "Channel":
+    channel = queries.channel(db, channel_id)
+    if channel is None:
+        raise not_found("channel")
+    return channel
+
+
+def _user_subscription(db: Session, user: User, channel_id: str) -> "Subscription":
+    sub = queries.subscription(db, user, channel_id)
     if sub is None:
         raise not_found("subscription")
     return sub
 
 
-def _is_blocked(db: Session, user: User, channel_id: str) -> bool:
-    """Whether this user has blocked this channel."""
-    return (
-        db.execute(
-            select(BlockedChannel.id).where(
-                BlockedChannel.user_id == user.id,
-                BlockedChannel.channel_id == channel_id,
-            )
-        ).first()
-        is not None
-    )
-
-
 class ChannelUpdateIn(BaseModel):
     # PATCH semantics: a field left unset (None) is untouched. Every field is optional so the
     # UI can send just the one control it changed. int/bool coercion + rejection of garbage
@@ -481,24 +375,18 @@ class ChannelUpdateIn(BaseModel):
 def update_channel(
     channel_id: str,
     body: ChannelUpdateIn,
-    background: BackgroundTasks,
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
 ) -> dict:
     sub = _user_subscription(db, user, channel_id)
-    channel = db.get(Channel, channel_id)
-    deep_turned_on = False
-    if body.priority is not None:
-        sub.priority = body.priority
-    if body.hidden is not None:
-        sub.hidden = body.hidden
-    if body.deep_requested is not None:
-        # Opt this channel into (or out of) full-history backfill. The deep scheduler
-        # picks the flag up on its next run; turning it off won't undo already-fetched
-        # videos, it just stops further deep paging if nobody else still wants it.
-        want = body.deep_requested
-        deep_turned_on = want and not sub.deep_requested
-        sub.deep_requested = want
+    channel = queries.channel(db, channel_id)
+    deep_turned_on = service.update_subscription(
+        db,
+        sub,
+        priority=body.priority,
+        hidden=body.hidden,
+        deep_requested=body.deep_requested,
+    )
     db.commit()
 
     # Opting a channel into full history should give an immediate feed: if we haven't even
@@ -508,12 +396,19 @@ def update_channel(
         with quota.attribute(user.id, quota.QuotaAction.VIDEOS_BACKFILL_RECENT):
             run_recent_backfill(db, [channel], max_channels=1)
 
-    # …and kick the FULL deep backfill for this one channel immediately in the background, so the
-    # whole catalogue lands in seconds instead of over the scheduler's paced 10-minute ticks. The
-    # request returns right away; the channel page's live poll shows the count climb. The scheduler
-    # still finishes a very large channel's tail on its normal runs (deep_requested stays set).
-    if deep_turned_on:
-        background.add_task(backfill_channel_now, channel_id, user.id)
+    # …and kick the FULL deep backfill for this one channel immediately, as a recorded run of its own,
+    # so the whole catalogue lands in seconds instead of over the scheduler's paced 10-minute ticks.
+    # The request returns right away; the channel page's live poll shows the count climb. The
+    # scheduler still finishes a very large channel's tail on its normal runs (deep_requested stays set).
+    # A channel whose history is all in gets no run: it would only say "nothing to do", one kept row
+    # per flip of the flag.
+    if deep_turned_on and not (channel is not None and channel.backfill_done):
+        scheduler.run_ad_hoc(
+            scheduler.CHANNEL_BACKFILL,
+            partial(backfill_channel_now, channel_id, user.id),
+            actor_id=user.id,
+            target=channel_id,
+        )
 
     return {
         "ok": True,
@@ -535,19 +430,8 @@ def reset_backfill(
     state (a "reset" trigger). Clears the channel's backfill markers, opts it back into deep
     backfill, and re-runs its recent pull immediately; the deep scheduler re-pages the full
     back-catalog on its next run. Idempotent — videos upsert by id, so nothing duplicates."""
-    channel = get_or_404(db, Channel, channel_id, what="channel")
-    channel.backfill_done = False
-    channel.backfill_cursor = None
-    channel.recent_synced_at = None
-    # Re-arm the demand-driven deep scheduler for this channel. run_deep_backfill gates on an
-    # EXISTS over subscriptions with deep_requested=True, so flip it on EVERY subscriber rather
-    # than requiring the admin to be personally subscribed (they reset from the manager, not
-    # their own feed). No-op row count if the channel has no subscribers.
-    db.execute(
-        update(Subscription)
-        .where(Subscription.channel_id == channel_id)
-        .values(deep_requested=True)
-    )
+    channel = _channel_or_404(db, channel_id)
+    service.reset_backfill(db, channel)
     db.commit()
     with quota.attribute(user.id, quota.QuotaAction.VIDEOS_BACKFILL_RECENT):
         run_recent_backfill(db, [channel], max_channels=1)
@@ -565,13 +449,8 @@ def subscribe(
     local subscription with YouTube's returned resource id and enriches the channel's metadata
     if it's still a stub, so the new row renders properly in the manager. It does NOT pull the
     channel's videos — the scheduler picks those up on its next run, like any subscribed channel."""
-    channel = get_or_404(db, Channel, channel_id, what="channel")
-    existing = db.execute(
-        select(Subscription).where(
-            Subscription.user_id == user.id, Subscription.channel_id == channel_id
-        )
-    ).scalar_one_or_none()
-    if existing is not None:
+    channel = _channel_or_404(db, channel_id)
+    if queries.subscription(db, user, channel_id) is not None:
         raise refused(409, "already_subscribed")
     with (
         quota.attribute(user.id, quota.QuotaAction.CHANNELS_SUBSCRIBE),
@@ -611,24 +490,7 @@ def subscribe(
                         reason=exc.reason,
                     ),
                 )
-    db.add(
-        Subscription(
-            user_id=user.id,
-            channel_id=channel_id,
-            yt_subscription_id=yt_sub_id or None,
-            subscribed_at=datetime.now(timezone.utc),
-        )
-    )
-    # Subscribing is the "keep" signal: a previously-explored channel becomes a normal organic
-    # one. Drop its ephemeral flags so the cleanup job leaves it alone and its videos stop being
-    # per-user private (they join the shared Library like any subscribed channel's uploads).
-    if channel.from_explore:
-        channel.from_explore = False
-        db.execute(
-            update(Video)
-            .where(Video.channel_id == channel_id, Video.via_explore.is_(True))
-            .values(via_explore=False)
-        )
+    service.record_subscription(db, user, channel, yt_sub_id)
     db.commit()
     return {"ok": True, "subscribed": channel_id}
 
@@ -655,7 +517,7 @@ def unsubscribe(
         # 422 (a real, explainable error → speaking modal), not 502 which the client treats as
         # a transient "connection lost" gateway blip.
         raise youtube_refusal(exc) from exc
-    db.delete(sub)
+    service.drop_subscription(db, sub)
     db.commit()
     return {"ok": True, "unsubscribed": channel_id}
 
@@ -663,16 +525,7 @@ def unsubscribe(
 @router.get("/blocked/list")
 def list_blocked(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
     """The user's blocked channels (videos from these are excluded from their search/feed/explore)."""
-    rows = (
-        db.execute(
-            select(Channel)
-            .join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
-            .where(BlockedChannel.user_id == user.id)
-            .order_by(collated(Channel.title, lang_for_user(user)))
-        )
-        .scalars()
-        .all()
-    )
+    rows = queries.blocked_channels(db, user, lang_for_user(user))
     return [
         {"id": c.id, "title": c.title, "handle": c.handle, "thumbnail_url": c.thumbnail_url}
         for c in rows
@@ -688,16 +541,9 @@ def block_channel(
     """Block a channel for this user: its videos won't appear in their live search (and won't be
     ingested for them) and are hidden from their feed / explore. Idempotent. The now-hidden
     un-kept search/explore videos are reclaimed by the discovery-cleanup job in due course."""
-    get_or_404(db, Channel, channel_id, what="channel")
-    if not _is_blocked(db, user, channel_id):
-        db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
-        # Stop any active exploration of it so it can be cleaned up.
-        db.execute(
-            ExploredChannel.__table__.delete().where(
-                (ExploredChannel.user_id == user.id) & (ExploredChannel.channel_id == channel_id)
-            )
-        )
-        db.commit()
+    _channel_or_404(db, channel_id)
+    service.block(db, user, channel_id)
+    db.commit()
     return {"ok": True, "blocked": channel_id}
 
 
@@ -707,11 +553,7 @@ def unblock_channel(
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
 ) -> dict:
-    db.execute(
-        BlockedChannel.__table__.delete().where(
-            (BlockedChannel.user_id == user.id) & (BlockedChannel.channel_id == channel_id)
-        )
-    )
+    service.unblock(db, user, channel_id)
     db.commit()
     return {"ok": True, "unblocked": channel_id}
 
@@ -728,26 +570,12 @@ def attach_tag(
     db: Session = Depends(get_db),
 ) -> dict:
     _user_subscription(db, user, channel_id)
-    # Only the user's own tags may be attached as personal links (owned_or_404 → 404 if the tag
-    # is missing OR belongs to someone else, without leaking which).
-    owned_or_404(db, Tag, body.tag_id, user, what="tag")
-    exists = db.execute(
-        select(ChannelTag).where(
-            ChannelTag.channel_id == channel_id,
-            ChannelTag.tag_id == body.tag_id,
-            ChannelTag.user_id == user.id,
-        )
-    ).scalar_one_or_none()
-    if exists is None:
-        db.add(
-            ChannelTag(
-                channel_id=channel_id,
-                tag_id=body.tag_id,
-                user_id=user.id,
-                source="user",
-            )
-        )
-        db.commit()
+    # Only the user's own tags may be attached as personal links (a 404 if the tag is missing OR
+    # belongs to someone else, without leaking which).
+    if reader_queries.own_tag(db, user, body.tag_id) is None:
+        raise not_found("tag")
+    reader_service.attach_user_tag(db, user, channel_id, body.tag_id)
+    db.commit()
     return {"ok": True, "channel_id": channel_id, "tag_id": body.tag_id}
 
 
@@ -758,14 +586,6 @@ def detach_tag(
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
 ) -> dict:
-    row = db.execute(
-        select(ChannelTag).where(
-            ChannelTag.channel_id == channel_id,
-            ChannelTag.tag_id == tag_id,
-            ChannelTag.user_id == user.id,
-        )
-    ).scalar_one_or_none()
-    if row is not None:
-        db.delete(row)
-        db.commit()
+    reader_service.detach_user_tag(db, user, channel_id, tag_id)
+    db.commit()
     return {"ok": True, "channel_id": channel_id, "tag_id": tag_id}