Merge branch 'feature/e45-related-videos'

Search YouTube again with the same term read the failure from the cache and sent nothing, so a busy or unreachable YouTube left the search on its error until the term changed. The first page's failure is now the error state with Retry, offered only where a retry can answer (not the day's cap, the feature turned off or the shared budget).

.gitleaksignore

@@ -0,0 +1,8 @@
+# gitleaks false positives, one fingerprint per finding (commit:file:rule:line). Each says why it is no secret.
+#
+# YouTube's public WEB InnerTube key: embedded in every youtube.com results page, read live and cached;
+# this constant is only the fallback (backend/app/youtube/innertube.py).
+8b37c470ba5d11c45757f0b2e62d660b782e816d:backend/app/youtube/innertube.py:gcp-api-key:23
+# A recorded watch-next page's continuation cursors (continuationCommand.token): page pointers, not credentials.
+8b37c470ba5d11c45757f0b2e62d660b782e816d:backend/tests/fixtures/innertube_next_first.json:generic-api-key:768
+8b37c470ba5d11c45757f0b2e62d660b782e816d:backend/tests/fixtures/innertube_next_first.json:generic-api-key:792

CHANGELOG.md

@@ -7,6 +7,84 @@ edit the release notes and re-run the generator.
 
 Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
 
+## 0.112.0 — 2026-10-25
+
+### New
+
+- Player: a Related tab lists videos related to the one playing, leaving out what you have already watched (your uploaded YouTube history included). Channels you follow are marked, and you can switch between On topic and Trending.
+- Play a related video in place; Back returns to the one you opened. Admins set a daily per-user limit in Config › Quota (default 200, 0 turns it off).
+- Side panel: the list controls and the watched toggle are glyphs on one line, and Auto and Loop show when they are on, also under the pointer.
+
+### Fixed
+
+- Live search: a search that never reached YouTube no longer uses up a daily slot, a busy moment says so after a few seconds instead of waiting on, and a first page that failed offers Retry.
+- Menus: Play from the beginning wears the same ↺ glyph everywhere, and Reset its own eraser glyph.
+- On the day of this update, API searches made earlier that day stop counting toward the daily search limit, until Pacific midnight.
+
+## 0.111.0 — 2026-10-24
+
+### New
+
+- Channels › Discover from your history: the channels your uploaded YouTube history shows you watching often (five videos of five minutes or more) but not following.
+- Follow one, put it out of the list (with Undo), or open the videos you watched there and play them one by one or all in a row.
+
+### Fixed
+
+- A channel someone explored and then followed on YouTube is now part of the shared Library like any followed channel, and so are its videos; before, they could stay hidden and their watched marks never applied.
+
+## 0.110.0 — 2026-10-23
+
+### New
+
+- YouTube history: upload your Google Takeout export in Settings › Account. What you watched on YouTube is marked watched here (never a Short, never over a state you set), and your YouTube Watch later list joins yours.
+- A video the catalog does not know yet is remembered and marked once it arrives. The import's outcome reaches your inbox, and admins see each run on the Scheduler page.
+
+### Fixed
+
+- Scheduler: a channel's full-history backfill asked for again while one is running now runs after it, instead of being silently dropped.
+
+## 0.109.2 — 2026-10-23
+
+### Fixed
+
+- Plex player: when the server restarts mid-episode (an update), playback carries on from the buffer and picks up where you are once it is back, instead of showing "Playback couldn't start" and stopping where the buffer ended.
+
+## 0.109.1 — 2026-10-23
+
+### New
+
+- Development: the frontend is one folder per page (its page, rail, store and definition), with the layers held by lint and the shared /watch page held apart by a contract test; the header knows no page, and the feed and Plex each keep their own saved views.
+
+### Fixed
+
+- Saved views: a view you applied and then changed is still active after a visit to another page, in the feed and in Plex.
+
+## 0.109.0 — 2026-10-21
+
+### New
+
+- Accounts: a browser is signed in to one account at a time. The account menu's "Sign in as someone else" replaces it; the account list and switcher are gone. After this update everyone signs in once more.
+- Settings follow you: the Home start page, the channel manager and Plex playlist layouts, the discovery sorts and a dismissed onboarding or release banner are saved to your account, on every device. Performance mode stays per device.
+- Development: every key the browser stores declares its kind, held by a contract test; one Google connect route serves the sign-in and the YouTube access grants.
+
+### Fixed
+
+- Accounts: a tab still showing another account than the one signed in now reloads as that one and says why; a failed "Sign in as someone else" says what happened.
+- A link whose path starts with // no longer opens the error screen.
+- A member following a link to an admin page lands on the feed, with the rail and the address bar in step.
+- Google: an unknown access request is refused instead of quietly asking for read access, and a YouTube grant on an instance without Google says so.
+
+## 0.108.1 — 2026-10-20
+
+### New
+
+- Development: the backend is split into four units (downloads, reader, catalog, Plex), each owning its tables. A contract test holds what each one reads and writes, and that every query names whose rows it asks for.
+
+### Fixed
+
+- Watch later: saving the first two videos at once no longer makes two lists. An existing second list is merged into the first, its videos and its comfort-shelf pin with it.
+- Playback: the first resume point of a video or a Plex item no longer fails when it is saved together with the watched mark, a whole season marked at once, or a Plex sync.
+
 ## 0.108.0 — 2026-10-18
 
 ### New

CLAUDE.md

@@ -70,54 +70,66 @@ The domain is four **units**, each owning its tables: Capture (`downloads/`), Re
 sync), Catalog (`catalog/`, the `sync/` ingests) and Plex (`plex/`). A unit's route does no database work: it reads through
 the unit's `queries`, writes through its `service` (which flushes) and commits. A unit's code reads another unit's tables
 only along a declared edge and writes them only through that unit's service; core reaches a unit's tables through its
-`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`); conventions in
-[`backend/app/README.md`](backend/app/README.md).
-Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
-entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
-components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
-its menus, `Modal`/`Overlay`, `Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`, …). That
-boundary is one-way and **lint-enforced**, not just described: a platform component may not import a
-feature component or a domain API slice, and one layer down `lib/` may not import from `components/` at
-all, not even a type (`no-restricted-imports` in `eslint.config.js`, with its own test). A new generic
-component goes in `ui/`; see [`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
-Data via react-query (`qk` factory), per-account `localStorage`, and server prefs; API through
-`lib/api/*` slices over `core.req<T>`.
+`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`) — as far as an AST sees: what it
+cannot (an ORM write on another unit's loaded row, SQL in `text()`, a commit by `begin()`) is review's to catch, listed in
+[`backend/app/README.md`](backend/app/README.md) with the conventions.
+Frontend, by folder under `src/`: `main.tsx` provider tree → `shell/` (`App`, header, nav rail, palette,
+boot; a shared `/watch` link boots from its own entry, `watch.html` → `watchMain.tsx` → `watch/`, held
+app-free by a contract test) → **`modules/<page>/`** (one folder per page: the page, its rail, its store, its tests,
+and its `module.ts` definition — label, icon, gate, the lazy page and rail, its store — collected in nav order by
+`features/navigation/moduleRegistry`, which also mounts every store at the root; a new page is its folder, its id in
+`lib/urlState` and its entry there, plus its panel in `lib/panelLayout` if it has a rail) →
+**`features/<name>/`** (what several pages share: video cards, filters, downloads, the player, navigation) → the
+**platform layer `components/ui/`** (domain-free primitives: the popover engine and its menus, `Modal`/`Overlay`,
+`Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`,
+…) → `lib/`. The direction is one-way and **lint-enforced**, not just described (`local/layers`, with
+its own test): an import only goes down; another module is reached only through its **`index.ts`** (its
+public face — the stores, mounted at the root and read app-wide, which `features/` may read too) or
+lazily by `import()`; a platform component may not import a domain API slice, and `lib/` nothing above
+it, not even a type. A lazily imported page, rail or dialog that a static import drags onto the boot
+path fails the build (`chunkGuard.ts`). A new generic component goes in `ui/`; see
+[`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
+Data via react-query (`qk` factory), the signed-in account's `localStorage` (`storage.ts`), and server
+prefs; API through `lib/api/*` slices over `core.req<T>`.
 
 **Not wanted** (reviewed and rejected as over-engineering for this size): microservices, event bus,
 CQRS, a repository/DI framework, a router library, a generic CRUD-router factory or base serializer.
 
 ## Canonical patterns — when you have a choice, pick THIS
 
-| Area                                 | Use                                                                                                                                                                                               | Not                                                                                                                                                     |
-| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| ORM                                  | SQLAlchemy 2.0 `select()`                                                                                                                                                                         | `db.query(...)`                                                                                                                                         |
-| Request body                         | a Pydantic model                                                                                                                                                                                  | `payload: dict`                                                                                                                                         |
-| Request SIZE                         | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES`                                                                                               | `capped_body` on a route that does not read raw bytes itself                                                                                            |
-| Ownership check                      | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404`                                 | an inline `Model.user_id == …` (fence-held, in every module) or a `db.get` and a compare                                                                |
-| Admin gate import                    | `app.auth`                                                                                                                                                                                        | the `app.routes.admin` re-export                                                                                                                        |
-| Upstream failure status              | `422` with JSON detail                                                                                                                                                                            | `502` (reserve 502 for raw byte-proxies)                                                                                                                |
-| User-facing error                    | structured `detail` (`{code, reason}`)                                                                                                                                                            | a bare English string under a bilingual UI                                                                                                              |
-| Mutation response                    | `{"ok": true, …}` (`{"deleted": id}`), test-held                                                                                                                                                  | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok`                                                                                                     |
-| Transaction commit                   | the handler commits; a unit's `service` and every helper `flush` (fence-held)                                                                                                                     | a helper that commits (only `quota`, `jobruns` and the jobs the fence's `UNIT_COMMITS` names own their session)                                         |
-| Admin knob storage                   | `sysconfig` (values)                                                                                                                                                                              | env / `app_state` / a table of its own (the job intervals had one)                                                                                      |
-| Job status word                      | the `downloads/status.py` StrEnum                                                                                                                                                                 | loose `"canceled"`/`"cancelled"` literals                                                                                                               |
-| Current time                         | inline `datetime.now(timezone.utc)`                                                                                                                                                               | new `now_utc`/`_now` helpers                                                                                                                            |
-| FE per-account state                 | `usePref`                                                                                                                                                                                         | raw `readAccount`/`writeAccount` or bespoke hooks                                                                                                       |
-| FE keyboard shortcut                 | a row in `lib/keymap.ts` + `dispatch()`                                                                                                                                                           | a private `keydown` listener with its own key literals                                                                                                  |
-| FE list/grid keys                    | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open`                                                                                                                                        | a private arrow-key / roving-tabindex handler                                                                                                           |
-| FE view push (history)               | `rememberNavPosition()` right before `history.pushState`                                                                                                                                          | a push that forgets: Back strands focus on `<body>`                                                                                                     |
-| FE static label / button-like link   | `Badge` (flat, no edge) · `LinkButton` / `IconLink`                                                                                                                                               | a glass pill that reads as a chip; an `<a>` hand-painted like a button                                                                                  |
-| FE loading/empty/error               | `QueryState` / `StateMessage`                                                                                                                                                                     | ad-hoc conditionals                                                                                                                                     |
-| FE failed write                      | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect                                            | a bare `catch {}` or `notify()` with no error attached                                                                                                  |
-| FE floating or full-screen layer     | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers")                      | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]`                              |
-| FE player on a native media element  | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE                                                                                  | a private `navigator.mediaSession` / `keydown` / position-saving effect per player                                                                      |
-| FE diagnostics                       | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts`                                                                           | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded                                                                    |
-| FE state that MIRRORS a prop/query   | derive it, or hold a draft (`draft ?? server`)                                                                                                                                                    | a copy an effect re-syncs — that paints the stale value first                                                                                           |
-| FE state reset on a key change       | adjust it DURING render against a remembered key                                                                                                                                                  | `useEffect(() => setX(init), [key])`                                                                                                                    |
-| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback`                                                                                                             | a render-phase `ref.current = x` — the React Compiler skips the component                                                                               |
-| FE hook suppression                  | an inline `eslint-disable-next-line` NAMING the invariant                                                                                                                                         | `eslint-suppressions.json` (deliberately absent — E27 S2)                                                                                               |
-| FE popover / scroll-fade ref         | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}`                                                                                                                    | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips                                         |
-| FE async handler with cleanup        | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
+| Area                                 | Use                                                                                                                                                                                                                                           | Not                                                                                                                                                                                         |
+| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ORM                                  | SQLAlchemy 2.0 `select()`                                                                                                                                                                                                                     | `db.query(...)`                                                                                                                                                                             |
+| Request body                         | a Pydantic model                                                                                                                                                                                                                              | `payload: dict`                                                                                                                                                                             |
+| Request SIZE                         | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES`                                                                                                                                           | `capped_body` on a route that does not read raw bytes itself                                                                                                                                |
+| Ownership check                      | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404`. A composite rule (own or shared, built-in or own) says so in its `queries` | an inline owner filter — `Model.user_id == …`, an alias's, `.in_()`, `filter_by(user_id=…)` (fence-held, every module) — or a `db.get` and a compare where `owned_row` fits                 |
+| Admin gate import                    | `app.auth`                                                                                                                                                                                                                                    | the `app.routes.admin` re-export                                                                                                                                                            |
+| Upstream failure status              | `422` with JSON detail                                                                                                                                                                                                                        | `502` (reserve 502 for raw byte-proxies)                                                                                                                                                    |
+| User-facing error                    | structured `detail` (`{code, reason}`)                                                                                                                                                                                                        | a bare English string under a bilingual UI                                                                                                                                                  |
+| Mutation response                    | `{"ok": true, …}` (`{"deleted": id}`), test-held                                                                                                                                                                                              | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok`                                                                                                                                         |
+| Transaction commit                   | the handler commits; a unit's `service` and every helper `flush` (fence-held, every module)                                                                                                                                                   | a helper that commits — only a listed one (what owns its session, or a core step that must land whatever its caller does next), each with its reason: the fence's `UNIT_COMMITS` / `CORE_COMMITS`, a route's `COMMITTING_HELPERS` (`test_route_conventions`) |
+| Admin knob storage                   | `sysconfig` (values)                                                                                                                                                                                                                          | env / `app_state` / a table of its own (the job intervals had one)                                                                                                                          |
+| Job status word                      | the `downloads/status.py` StrEnum                                                                                                                                                                                                             | loose `"canceled"`/`"cancelled"` literals                                                                                                                                                   |
+| Current time                         | inline `datetime.now(timezone.utc)`                                                                                                                                                                                                           | new `now_utc`/`_now` helpers                                                                                                                                                                |
+| FE browser storage                   | a key in `storage.ts` with its kind (contract-held): a `pref` through `usePref` (the server's, follows the account); `device` / `view` / `cache` this browser's; `public` kept for the /watch page                                            | a `"siftlode.x"` literal; a preference kept only in `localStorage` (it stays on one device)                                                                                                 |
+| FE keyboard shortcut                 | a row in `lib/keymap.ts` + `dispatch()`                                                                                                                                                                                                       | a private `keydown` listener with its own key literals                                                                                                                                      |
+| FE list/grid keys                    | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open`                                                                                                                                                                                    | a private arrow-key / roving-tabindex handler                                                                                                                                               |
+| FE view push (history)               | `rememberNavPosition()` right before `history.pushState`                                                                                                                                                                                      | a push that forgets: Back strands focus on `<body>`                                                                                                                                         |
+| FE a page's header content           | the page renders it into the floating header itself (`PageHeader`, `ui/PageShell`)                                                                                                                                                            | a `page === …` branch in the shell's header, or a bridge carrying a page's state up to it                                                                                                   |
+| FE module store                      | its definition's `Store` (`module.ts`), mounted at the root by `ModuleStores`; it reads no other module's store (contract-held)                                                                                                               | a provider hand-nested in `main.tsx`                                                                                                                                                        |
+| FE saved views                       | the module's own store from `features/filters` (`createSavedViewsStore` + `useSavedViewsStore`)                                                                                                                                               | one store switched by the active page — every visit elsewhere reset it                                                                                                                      |
+| FE static label / button-like link   | `Badge` (flat, no edge) · `LinkButton` / `IconLink`                                                                                                                                                                                           | a glass pill that reads as a chip; an `<a>` hand-painted like a button                                                                                                                      |
+| FE loading/empty/error               | `QueryState` / `StateMessage`                                                                                                                                                                                                                 | ad-hoc conditionals                                                                                                                                                                         |
+| FE failed write                      | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect                                                                                        | a bare `catch {}` or `notify()` with no error attached                                                                                                                                      |
+| FE floating or full-screen layer     | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers")                                                                  | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]`                                                                  |
+| FE player on a native media element  | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE                                                                                                                              | a private `navigator.mediaSession` / `keydown` / position-saving effect per player                                                                                                          |
+| FE diagnostics                       | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts`                                                                                                                       | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded                                                                                                        |
+| FE state that MIRRORS a prop/query   | derive it, or hold a draft (`draft ?? server`)                                                                                                                                                                                                | a copy an effect re-syncs — that paints the stale value first                                                                                                                               |
+| FE state reset on a key change       | adjust it DURING render against a remembered key                                                                                                                                                                                              | `useEffect(() => setX(init), [key])`                                                                                                                                                        |
+| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback`                                                                                                                                                         | a render-phase `ref.current = x` — the React Compiler skips the component                                                                                                                   |
+| FE hook suppression                  | an inline `eslint-disable-next-line` NAMING the invariant                                                                                                                                                                                     | `eslint-suppressions.json` (deliberately absent — E27 S2)                                                                                                                                   |
+| FE popover / scroll-fade ref         | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}`                                                                                                                                                                | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips                                                                             |
+| FE async handler with cleanup        | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper                                             | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these)                                     |
 
 Not repeated here, because a lint rule refuses the wrong one and its message names the right one:
 buttons (`ui/Button`, `IconButton`, `Pressable`, `Chip`, `DragHandle`), text fields and native

VERSION

@@ -1 +1 @@
-0.108.0
+0.112.0

backend/alembic/versions/0112_one_watch_later.py

@@ -0,0 +1,70 @@
+"""One Watch later list per user, held by the database
+
+Revision ID: 0112_one_watch_later
+Revises: 0111_audit_request_id
+Create Date: 2026-10-20
+
+Watch later was made on a user's first save by a read then an insert, so two first saves at once
+each made a list, and the reader then picked between them at random. A partial unique index holds
+"one per user" now. An instance that already has a second list keeps the oldest: the others' videos
+move into it (after its own, in their order; one it already holds is not added twice), a Home pin
+of one of them moves to it (unless it is pinned itself; of several, the first on the shelf), then
+they go — a pin left on them goes with them, as the kept list already holds one.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0112_one_watch_later"
+down_revision: str | None = "0111_audit_request_id"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+_EXTRA = """
+    SELECT p.id, k.keep_id
+    FROM playlists p
+    JOIN (
+        SELECT user_id, min(id) AS keep_id FROM playlists WHERE kind = 'watch_later' GROUP BY user_id
+    ) k ON k.user_id = p.user_id
+    WHERE p.kind = 'watch_later' AND p.id <> k.keep_id
+"""
+
+
+def upgrade() -> None:
+    op.execute(
+        f"""
+        INSERT INTO playlist_items (playlist_id, video_id, position, added_at)
+        SELECT e.keep_id, i.video_id,
+               (SELECT coalesce(max(position), -1) FROM playlist_items WHERE playlist_id = e.keep_id)
+               + row_number() OVER (PARTITION BY e.keep_id ORDER BY i.playlist_id, i.position, i.id),
+               i.added_at
+        FROM playlist_items i
+        JOIN ({_EXTRA}) e ON e.id = i.playlist_id
+        ON CONFLICT ON CONSTRAINT uq_playlist_video DO NOTHING
+        """
+    )
+    op.execute(
+        f"""
+        UPDATE comfort_pins c SET playlist_id = m.keep_id
+        FROM (
+            SELECT DISTINCT ON (e.keep_id) p.id AS pin_id, e.keep_id
+            FROM comfort_pins p
+            JOIN ({_EXTRA}) e ON e.id = p.playlist_id
+            WHERE NOT EXISTS (SELECT 1 FROM comfort_pins k WHERE k.playlist_id = e.keep_id)
+            ORDER BY e.keep_id, p.position, p.id
+        ) m
+        WHERE c.id = m.pin_id
+        """
+    )
+    op.execute(f"DELETE FROM playlists WHERE id IN (SELECT id FROM ({_EXTRA}) e)")
+    op.create_index(
+        "uq_playlist_watch_later",
+        "playlists",
+        ["user_id"],
+        unique=True,
+        postgresql_where=sa.text("kind = 'watch_later'"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("uq_playlist_watch_later", table_name="playlists")

backend/alembic/versions/0113_takeout_items.py

@@ -0,0 +1,51 @@
+"""A user's Google Takeout export: what they watched on YouTube, and their Watch later list
+
+Revision ID: 0113_takeout_items
+Revises: 0112_one_watch_later
+Create Date: 2026-10-23
+
+One row per video and kind. A video the catalog does not know yet waits with no outcome, so the id
+is not a foreign key; the partial index is what a newly known video looks its waiting rows up by.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0113_takeout_items"
+down_revision: str | None = "0112_one_watch_later"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "takeout_items",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column("kind", sa.String(12), nullable=False),
+        sa.Column("video_id", sa.String(16), nullable=False),
+        sa.Column("channel_id", sa.String(32), nullable=True),
+        sa.Column("at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("outcome", sa.String(12), nullable=True),
+        sa.Column(
+            "created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
+        ),
+        sa.UniqueConstraint("user_id", "kind", "video_id", name="uq_takeout_item"),
+        sa.CheckConstraint("kind IN ('watched', 'watch_later')", name="ck_takeout_item_kind"),
+    )
+    op.create_index(
+        "ix_takeout_items_waiting",
+        "takeout_items",
+        ["video_id"],
+        postgresql_where=sa.text("outcome IS NULL"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_takeout_items_waiting", table_name="takeout_items")
+    op.drop_table("takeout_items")

backend/alembic/versions/0114_video_unmarks.py

@@ -0,0 +1,44 @@
+"""When a user last took a video back to "new"
+
+Revision ID: 0114_video_unmarks
+Revises: 0113_takeout_items
+Create Date: 2026-10-23
+
+Un-marking or resetting a video leaves no `video_states` row, so this is what tells an import of the
+user's YouTube history a video they un-marked here from one never touched.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0114_video_unmarks"
+down_revision: str | None = "0113_takeout_items"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "video_unmarks",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column(
+            "video_id",
+            sa.String(16),
+            sa.ForeignKey("videos.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column("at", sa.DateTime(timezone=True), nullable=False),
+        sa.UniqueConstraint("user_id", "video_id", name="uq_video_unmark"),
+    )
+    op.create_index("ix_video_unmarks_video_id", "video_unmarks", ["video_id"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_video_unmarks_video_id", table_name="video_unmarks")
+    op.drop_table("video_unmarks")

backend/alembic/versions/0115_takeout_channels.py

@@ -0,0 +1,49 @@
+"""A watched Takeout video's length, and the channels a user put out of their list
+
+Revision ID: 0115_takeout_channels
+Revises: 0114_video_unmarks
+Create Date: 2026-10-24
+
+The "channels you watch but do not follow" list counts a user's watched videos over five minutes
+by channel. The length is asked of YouTube for that list and kept on the Takeout row; a channel the
+user does not want on the list is kept apart.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0115_takeout_channels"
+down_revision: str | None = "0114_video_unmarks"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "takeout_items", sa.Column("looked_up_at", sa.DateTime(timezone=True), nullable=True)
+    )
+    op.add_column("takeout_items", sa.Column("duration_seconds", sa.Integer(), nullable=True))
+    op.create_table(
+        "takeout_channel_dismissals",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column("channel_id", sa.String(32), nullable=False),
+        sa.Column(
+            "created_at",
+            sa.DateTime(timezone=True),
+            server_default=sa.func.now(),
+            nullable=False,
+        ),
+        sa.UniqueConstraint("user_id", "channel_id", name="uq_takeout_channel_dismissal"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("takeout_channel_dismissals")
+    op.drop_column("takeout_items", "duration_seconds")
+    op.drop_column("takeout_items", "looked_up_at")

backend/alembic/versions/0116_followed_explored_channels.py

@@ -0,0 +1,34 @@
+"""An explored channel someone follows is the catalog's own, and so are its videos
+
+Revision ID: 0116_followed_explored_channels
+Revises: 0115_takeout_channels
+Create Date: 2026-10-24
+
+A follow pulled in from YouTube (the subscription import) left an explored channel flagged as an
+exploration's, and a subscribe landing while an exploration's videos came in left those videos
+flagged on a channel that no longer was one. Both are held in the code now; this clears what they
+left: the channel of a subscription is not an exploration's, and no video is an exploration's on a
+channel that is not one. Data only, so the downgrade has nothing to undo.
+"""
+
+from alembic import op
+
+revision: str = "0116_followed_explored_channels"
+down_revision: str | None = "0115_takeout_channels"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    op.execute(
+        "UPDATE channels SET from_explore = false WHERE from_explore"
+        " AND EXISTS (SELECT 1 FROM subscriptions s WHERE s.channel_id = channels.id)"
+    )
+    op.execute(
+        "UPDATE videos SET via_explore = false WHERE via_explore"
+        " AND channel_id IN (SELECT id FROM channels WHERE NOT from_explore)"
+    )
+
+
+def downgrade() -> None:
+    pass

backend/app/README.md

@@ -56,11 +56,19 @@ The domain is four units, each owning its tables (`models.py` in its package):
 - **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id,
   user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the
   same answer as a missing one). A unit's `queries.own_x` returns None and its route answers
-  `not_found`; a core route calls `owned_or_404`.
+  `not_found`; a core route calls `owned_or_404`. A rule `owned_row` cannot say — a job its owner
+  shared, a built-in profile nobody owns, a folder read under `FOR SHARE` — is written out in the
+  unit's own code, beside the reason.
 
 All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's
-packages and edges (`UNIT_PACKAGES`, `READS`) and the jobs that commit (`UNIT_COMMITS`) are listed
-there.
+packages and edges (`UNIT_PACKAGES`, `READS`) and what may commit (`UNIT_COMMITS`, `CORE_COMMITS`)
+are listed there. It reads the code's AST, so it holds the forms written here: an owner column
+compared inline (a model's, an import alias's or an `aliased()` one's, by `==`, `!=`, `.in_()`, or a
+`filter_by(user_id=…)`), a model built or an `insert`/`update`/`delete` (under any import name, of a
+model or its `__table__`) across units, and a `.commit()` call off the lists all fail it. What an AST
+cannot tell is review's to catch: an ORM write on another unit's loaded row (an attribute, a
+`db.delete`), SQL in `text()`, a column or model reached another way (bound to a variable, through
+`__table__.c` or a module path), and a commit made by a `begin()` block or a `commit` handed in.
 
 ## Transaction ownership
 
@@ -73,10 +81,12 @@ mutation it records, so an audit entry can never exist without its change (or vi
 
 - The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it
   makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the
-  request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, and a unit's
-  jobs (the worker's `jobdb` writes, the syncs and ingests that commit per page or batch so a later
-  failure loses nothing already stored) — the fence's `UNIT_COMMITS` names each. Everything else
-  flushes.
+  request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, the
+  scheduler's jobs, a unit's jobs (the worker's `jobdb` writes, the syncs and ingests that commit per
+  page or batch so a later failure loses nothing already stored), and a few core steps that must land
+  whatever their caller does next (a token's last-used stamp, a refreshed YouTube grant) — the fence's
+  `UNIT_COMMITS` and `CORE_COMMITS` name each with its reason, `test_route_conventions` a route's own
+  committing helpers. Everything else flushes.
 - `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
   loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
   attribute — see `service.decr_ref_count`.

backend/app/audit.py

@@ -110,10 +110,9 @@ def record(
 
 def gc(db: Session, retention_days: int) -> int:
     """Delete audit rows older than `retention_days`. Returns the number removed. A
-    retention_days <= 0 disables pruning (keep forever)."""
+    retention_days <= 0 disables pruning (keep forever). Flushes; the job commits."""
     if retention_days <= 0:
         return 0
     cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
     result = db.execute(delete(AuditLog).where(AuditLog.created_at < cutoff))
-    db.commit()
     return result.rowcount or 0

backend/app/auth.py

@@ -25,8 +25,6 @@ def operator_contact() -> str | None:
     return next(iter(sorted(settings.admin_email_set)), None)
 
 
-MAX_SESSION_ACCOUNTS = 6
-
 # OAuth scope vocabulary — the atoms + the request bundles, kept together as core. has_*_scope check
 # the atoms against a user's stored grant; the *_SCOPES bundles are what the login/callback endpoints
 # request (imported by app/routes/auth.py).
@@ -253,94 +251,63 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
     background.add_task(email_mod.send_account_deleted, email, lang, operator_contact())
 
 
-ACTIVE_ACCOUNT_HEADER = "x-siftlode-account"
+def session_user_id(conn: HTTPConnection) -> int | None:
+    """The account this browser session signed in as, read from the signed cookie alone (no DB).
+    One account per session: signing in as another replaces it (`routes.auth._establish_session`)."""
+    return conn.session.get("user_id")
 
 
-def resolved_user_id(conn: HTTPConnection, *, strict: bool = True) -> tuple[int | None, bool]:
-    """Which account this connection acts as, and whether that's the session's default account.
-    Takes an HTTPConnection rather than a Request: it reads only the session and the headers.
-
-    The signed cookie holds the *wallet* (`account_ids`, every account signed into this browser)
-    plus a default `user_id`. A caller may name its account with the X-Siftlode-Account header (or
-    `?account=`), but ONLY an account in the wallet — so a tab can't impersonate an account that
-    never authenticated here. A connection that names no account acts as the cookie default.
-
-    A connection naming an account NOT in the wallet acts as nobody (`None`) — typically a tab still
-    pinned to an account that was signed out in another tab. It must not fall back to the default:
-    that tab would go on acting as an account it never showed, its player checkpoints and pref saves
-    landing there and its Sign out signing THAT account out. Only the bootstrap /api/me probe passes
-    `strict=False` and gets the default, which is how such a tab learns who it now is (the frontend
-    re-pins it and reloads). An account whose session ended while it stayed in the wallet (deleted,
-    suspended, revoked) leaves the wallet when it is refused (`_session_user`), so a tab pinned to it
-    takes the same way back.
-    """
-    default_id = conn.session.get("user_id")
-    wallet = conn.session.get("account_ids") or []
-    # Header for normal XHR; ?account= for a context that can't set headers (a plain <a> file
-    # download). Both are wallet-gated below.
-    hdr = conn.headers.get(ACTIVE_ACCOUNT_HEADER) or conn.query_params.get("account")
-    if hdr:
-        try:
-            hid = int(hdr)
-        except ValueError:
-            # Under `strict` this becomes a 401 and the tab authenticates as nobody, which reads
-            # in the browser exactly like a session that expired. The value itself stays out —
-            # it came from a request, and `wallet` says which accounts were actually on offer.
-            log.debug(
-                "the active-account header could not be read as an id",
-                extra=logs.event("auth.account_header.invalid", accounts=len(wallet)),
-            )
-            hid = None
-        if hid is not None and hid in wallet:
-            return hid, hid == default_id
-        if strict:
-            return None, False
-    return default_id, True
+def stamp_session(request: Request, user: User) -> None:
+    """Record the account's current `session_epoch` in the signed cookie, so `current_user` can later
+    reject the cookie once the epoch moves (password reset/change, or "log out everywhere")."""
+    request.session["epoch"] = user.session_epoch or 0
 
 
-def drop_from_wallet(session: dict, user_id: int) -> None:
-    """Take one account out of this browser's wallet (`account_ids`); the session default is untouched."""
-    session["account_ids"] = [a for a in (session.get("account_ids") or []) if a != user_id]
+# The account a tab was built for, sent with every request it makes (`identityHeaders`, the
+# frontend). The cookie alone says who is asking, and one browser holds one account: a sign-in as
+# someone else in another tab replaces it under every open tab. A tab still showing the previous
+# account would otherwise act as the new one — its player checkpoints, pref saves, Sign out and
+# Delete account landing there — until it noticed. It never selects an account; a mismatch is
+# refused, and the tab reloads as whoever is signed in now.
+EXPECTED_ACCOUNT_HEADER = "x-siftlode-expect-account"
 
 
-def _end_account_session(request: Request, user_id: int, is_default: bool) -> None:
-    """An account whose session here is no longer valid. Only nuke the whole session when it is the
-    SESSION DEFAULT; a per-tab account just leaves the wallet (other tabs' default may still be valid).
-    Leaving is what lets a tab pinned to it recover: its next /api/me names an account no longer signed
-    in here and gets the default. Left in, it answered "signed out" — the landing page, with the default
-    still signed in, and no way off it but closing the tab."""
-    if is_default:
-        request.session.clear()
-    else:
-        drop_from_wallet(request.session, user_id)
+def built_for_another_account(user_id: int, expected: str | None) -> bool:
+    """Whether a tab naming `expected` is showing another account than the one signed in. No name is
+    no claim (a caller from before this check, a script): only a mismatch is refused, and the
+    session stays — it is the other tab's sign-in, and it holds. The one comparison for every way a
+    tab names its account: this header on a request, `expect` on a navigation."""
+    if expected is None or expected == str(user_id):
+        return False
+    log.info(
+        "a request from a tab built for another account was refused",
+        extra=logs.event("auth.session.account_changed", user_id=user_id),
+    )
+    return True
 
 
 def current_user(request: Request, db: Session = Depends(get_db)) -> User:
-    return _session_user(request, db, *resolved_user_id(request))
+    user = _session_user(request, db, session_user_id(request))
+    if built_for_another_account(user.id, request.headers.get(EXPECTED_ACCOUNT_HEADER)):
+        raise refused(401, "account_changed")
+    return user
 
 
-def _session_user(request: Request, db: Session, user_id: int | None, is_default: bool) -> User:
+def _session_user(request: Request, db: Session, user_id: int | None) -> User:
     if not user_id:
         raise refused(401, "not_authenticated")
     user = db.get(User, user_id)
     if user is None or not user.is_active or user.is_suspended:
         # Suspended/deactivated mid-session → end it so the block takes effect at once.
-        _end_account_session(request, user_id, is_default)
+        request.session.clear()
         raise refused(401, "not_authenticated")
-    # SA4: reject a cookie whose recorded epoch is behind the account's current one (revoked by a
-    # password reset/change or "log out everywhere"). A cookie with no recorded epoch (pre-SA4, or an
-    # account signed in before this shipped) is treated as epoch 0 — so the FIRST bump revokes it too.
-    epochs = request.session.get("epochs") or {}
-    if int(epochs.get(str(user_id), 0)) != (user.session_epoch or 0):
-        _end_account_session(request, user_id, is_default)
+    # A cookie whose epoch is behind the account's (revoked by a password reset/change or "log out
+    # everywhere") is refused — and so is one with no epoch at all, which only a build before the
+    # one-account session wrote: its holder signs in once more.
+    epoch = request.session.get("epoch")
+    if epoch is None or int(epoch) != (user.session_epoch or 0):
+        request.session.clear()
         raise refused(401, "not_authenticated")
-    # Always keep the session default in the switchable list — covers sessions created before
-    # multi-session existed (their account_ids was never seeded), so the switcher isn't empty.
-    default_id = request.session.get("user_id")
-    accounts = request.session.get("account_ids") or []
-    if default_id and default_id not in accounts:
-        accounts.append(default_id)
-        request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
     # Attribute this request's log lines to the account. Here, not in the request middleware:
     # this is the one place a session becomes an account, and the middleware runs outside both the
     # session middleware and this task's context. `request.state` is shared across middleware layers
@@ -358,25 +325,17 @@ def _session_user(request: Request, db: Session, user_id: int | None, is_default
 def optional_current_user(request: Request, db: Session = Depends(get_db)) -> User | None:
     """Like `current_user` but returns None instead of raising 401 when there's no valid session.
     Used by the bootstrap /api/me probe so the logged-out landing doesn't log a failed request to
-    the browser console (a non-2xx fetch is a console error no matter how the app handles it).
-    The one lenient resolution (`strict=False`): a tab naming an account no longer signed in here gets
-    the default account, which is how it learns who it now is — see `resolved_user_id`. That includes an
-    account refused right here, which `_session_user` has just taken out of the wallet."""
-    user_id, is_default = resolved_user_id(request, strict=False)
+    the browser console (a non-2xx fetch is a console error no matter how the app handles it)."""
+    user_id = session_user_id(request)
+    if user_id is None:
+        return None
     try:
-        return _session_user(request, db, user_id, is_default)
-    # silent-ok: this is the anonymous case as much as the refused one — every landing page hit
-    # comes through here. The second attempt below is the handling, and it reports.
+        return _session_user(request, db, user_id)
     except HTTPException:
-        if is_default:
-            return None
-    try:
-        return _session_user(request, db, *resolved_user_id(request, strict=False))
-    except HTTPException:
-        # Both the pinned account AND the default one were refused, so the tab is signed out
-        # without being told why — and it got here from a session that named a real account.
+        # A session that named an account, refused: the browser is signed out without being told
+        # why, so say it here.
         log.info(
-            "a session was resolved to nobody: neither the named nor the default account held",
+            "a session was refused: its account no longer holds",
             extra=logs.event("auth.session.probe_rejected", user_id=user_id),
         )
         return None

backend/app/bodylimit.py

@@ -72,6 +72,13 @@ DEFAULT_MAX_BODY_BYTES = 1024 * 1024
 # A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
 # `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
 # default, so listing it could only widen what it accepts.
+# A Google Takeout file (`reader.takeout`). A real watch history measured 3.7 MB for 9500 views, ~390
+# bytes a view, so 32 MiB holds ~86k views. YouTube keeps the history 36 months by default on an
+# account made since mid-2020 and for good on an older one, so a heavy viewer's can pass 16 MiB. The
+# file's text is decoded a window at a time and read an entry at a time (`takeout._json_list`), so
+# it costs its bytes and a window, not a tree of it; the route takes two at once across every user.
+TAKEOUT_MAX_BYTES = 32 * 1024 * 1024
+
 ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
     # Ingesting a local file. The body IS the file now — no multipart envelope to allow for —
     # so this ceiling and the route's own cap count exactly the same bytes. `sysconfig`'s
@@ -95,6 +102,7 @@ ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
     # sending it at all, off the `upload_max_bytes` that `GET /api/files/usage` reports.
     ("POST", "/api/files/upload"): settings.download_upload_max_bytes,
     ("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES,
+    ("POST", "/api/takeout"): TAKEOUT_MAX_BYTES,
 }
 
 # The refusal, byte-for-byte what `capped_body`'s `HTTPException(413, {"code": "too_large"})` renders

backend/app/catalog/models.py

@@ -71,10 +71,12 @@ class Channel(Base, TimestampMixin):
         Boolean, default=False, server_default="false", index=True
     )
     # True when this channel exists only because a user EXPLORED it (opened its page without
-    # subscribing) — an ephemeral, per-explorer addition. The cleanup job hard-deletes such a
-    # channel (and its via_explore videos) once nobody keeps it (no subscription, no live
-    # ExploredChannel row, no per-user state) after a grace period. Cleared when someone
-    # subscribes (the "keep" signal promotes it to organic).
+    # subscribing) — an ephemeral, per-explorer addition. After a grace period the cleanup job
+    # hard-deletes its via_explore videos nobody keeps (no subscription, no live ExploredChannel
+    # row, no per-user state), and then the channel row itself once it holds no videos — unless a
+    # Takeout watch history shows it watched: the "watched but not followed" list names it from
+    # that row. Cleared when someone subscribes, here or on YouTube (the "keep" signal promotes it
+    # to organic).
     from_explore: Mapped[bool] = mapped_column(
         Boolean, default=False, server_default="false", index=True
     )
@@ -235,8 +237,9 @@ class ExploredChannel(Base, TimestampMixin):
     Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
     user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
     explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
-    purge: a from_explore channel with no live ExploredChannel row (and no subscription / no
-    per-user state on its videos) is hard-deleted by the cleanup job. Subscribing is the "keep"
+    purge: of a from_explore channel with no live ExploredChannel row (and no subscription) the
+    cleanup job hard-deletes the videos without per-user state, then the channel once it holds none
+    and no Takeout watch history shows it watched. Subscribing is the "keep"
     signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
     (from TimestampMixin) is the grace clock; a re-visit refreshes it."""
 

backend/app/catalog/queries.py

@@ -158,6 +158,19 @@ def blocked_ids(db: Session, user: User) -> set[str]:
     )
 
 
+def followed_among(db: Session, user: User, channel_ids: set[str]) -> set[str]:
+    """Which of these channels the user subscribes to."""
+    if not channel_ids:
+        return set()
+    return set(
+        db.scalars(
+            select(Subscription.channel_id).where(
+                owned_by(Subscription.user_id, user), Subscription.channel_id.in_(channel_ids)
+            )
+        )
+    )
+
+
 def blocked_channels(db: Session, user: User, lang: str) -> list[Channel]:
     return list(
         db.execute(

backend/app/catalog/search.py

@@ -75,7 +75,8 @@ def ingest_candidates(
 ) -> list[str]:
     """Materialise one page of search candidates into the catalog (channel stubs + enrich,
     video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
-    video ids — non-Short, non-live — in the input relevance order."""
+    video ids — non-Short, non-live — in the input relevance order. A videos.list that fails
+    raises its YouTubeError: with nothing enriched, nothing could be kept."""
     ids = [it["id"] for it in candidates]
     if not ids:
         return []
@@ -132,13 +133,16 @@ def ingest_candidates(
         with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
             detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
     except YouTubeError:
-        # Every candidate then stays unenriched, and an unenriched video cannot be judged live or
-        # Short — so it is dropped. A working search returns nothing and reads as "no results".
+        # An unenriched video cannot be judged live or Short, so none of this page could be kept:
+        # swallowed, the failure answered as a page with nothing on it — "no results" for a search
+        # that never ran (the shared budget spent, no API key). The stubs stay (`insert_stubs`
+        # committed them), as the channels' do above; the caller answers the failure (its refusal
+        # says which).
         log.warning(
             "the videos found by a search could not be enriched",
             extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
         )
-        detail_by_id = {}
+        raise
     enriched_at = datetime.now(
         timezone.utc
     )  # when THIS enrich ran (distinct from the stub placeholder)
@@ -212,17 +216,25 @@ def clear(db: Session, user: User, video_ids: list[str]) -> int:
     """Discard these results "as if never added": drop this user's finds of them, then delete the
     ones nobody kept — a `via_search` video no reader holds (`reader.queries.kept_by_a_reader`),
     whose channel nobody subscribes to, and that no other user still has as a find. Returns how
-    many videos went."""
-    db.execute(
-        delete(SearchFind).where(
-            owned_by(SearchFind.user_id, user), SearchFind.video_id.in_(video_ids)
+    many videos went.
+
+    Only the ones this user found: a video in the catalog with no find at all (a related list's,
+    which records none) is in someone's list right now, and an id any user could name would let
+    them delete it from under that list."""
+    mine = list(
+        db.scalars(
+            delete(SearchFind)
+            .where(owned_by(SearchFind.user_id, user), SearchFind.video_id.in_(video_ids))
+            .returning(SearchFind.video_id)
         )
     )
+    if not mine:
+        return 0
     subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
     other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
     result = db.execute(
         delete(Video).where(
-            Video.id.in_(video_ids),
+            Video.id.in_(mine),
             Video.via_search.is_(True),
             ~reader_queries.kept_by_a_reader(Video.id),
             ~subscribed,

backend/app/catalog/service.py

@@ -96,13 +96,24 @@ def record_subscription(db: Session, user: User, channel: Channel, yt_subscripti
             subscribed_at=datetime.now(timezone.utc),
         )
     )
-    if channel.from_explore:
-        channel.from_explore = False
-        db.execute(
-            update(Video)
-            .where(Video.channel_id == channel.id, Video.via_explore.is_(True))
-            .values(via_explore=False)
-        )
+    db.flush()
+    settle_explored(db, channel)
+
+
+def settle_explored(db: Session, channel: Channel) -> None:
+    """Hold the rule that a channel's videos are an exploration's only while nobody follows it,
+    decided by who follows it now, not by the flag held in memory: a subscribe in another request
+    may have cleared that while YouTube answered this one, and the videos that came in meanwhile
+    would stay flagged. A channel someone follows, however the follow came (here or pulled in from
+    YouTube), is the catalog's own, and so are its videos."""
+    if channel.from_explore and not queries.followed(db, channel.id):
+        return
+    channel.from_explore = False
+    db.execute(
+        update(Video)
+        .where(Video.channel_id == channel.id, Video.via_explore.is_(True))
+        .values(via_explore=False)
+    )
     db.flush()
 
 

backend/app/config.py

@@ -113,6 +113,11 @@ class Settings(BaseSettings):
     # each search.list costs 100 units, so the shared budget only affords ~80-90/day total; with
     # the zero-quota scrape source it's a pure anti-abuse rate limit. Admin-tunable.
     search_daily_limit_per_user: int = 70
+    # Per-user cap on related-video requests per day — a list opened and each "Show more" (the
+    # InnerTube watch page, no quota; its enrich is up to ~8 units: up to 4 pages, each a videos.list
+    # and a channels.list). A failed one gives its slot back. 0 turns related videos off.
+    # Admin-tunable.
+    related_daily_limit_per_user: int = 200
     # Source for live YouTube search: "scrape" (InnerTube, no API quota) or "api" (search.list,
     # 100 units/page). Scrape is the default; flip to "api" if scraping is blocked/breaks.
     search_source: SearchSource = "scrape"

backend/app/debugsession.py

@@ -26,7 +26,7 @@ from uuid import uuid4
 from sqlalchemy import and_, delete, func, not_, select, update
 from sqlalchemy.orm import Session, defer
 
-from app.userscope import owned_by
+from app.userscope import owned_by, owned_row
 from app import diagnostics
 from app.models import DebugScreenshot, DebugSession, LogEvent, User
 
@@ -129,12 +129,8 @@ def from_header(db: Session, user: User, value: str | None) -> DebugSession | No
     """The session a request names, if it is the caller's own and still live; else None."""
     if not value or len(value) != _ID_LEN or not all(c in "0123456789abcdef" for c in value):
         return None
-    session = db.get(DebugSession, value)
-    if (
-        session is None
-        or session.user_id != user.id
-        or not is_live(session, datetime.now(timezone.utc))
-    ):
+    session = owned_row(db, DebugSession, value, user)
+    if session is None or not is_live(session, datetime.now(timezone.utc)):
         return None
     return session
 

backend/app/discover/pins.py

@@ -72,11 +72,11 @@ def resolve_ref(db: Session, user: User, kind: str, ref: str) -> tuple[str, obje
     # such pin target", which is the same answer this function gives for one that does not exist.
     except ValueError:
         return None
-    pl = db.get(Playlist, pid)
     # A playlist is per-user, unlike the other four (which are shared catalog rows whose PRIVACY
     # lives in the state tables). Someone else's list must read as "no such thing", not as a 403 —
     # the same reason `owned_or_404` 404s rather than 403s.
-    return (col, pl.id) if pl is not None and pl.user_id == user.id else None
+    pl = owned_row(db, Playlist, pid, user)
+    return (col, pl.id) if pl is not None else None
 
 
 def reorder(db: Session, user: User, ids: list[int]) -> None:

backend/app/discover/queries.py

@@ -93,7 +93,7 @@ def yt_in_progress(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[tup
     q = (
         select(*feed_columns(user, st), st.progress_updated_at.label("progressed_at"))
         .join(Channel, Channel.id == Video.channel_id)
-        .join(st, and_(st.video_id == Video.id, st.user_id == user.id))
+        .join(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
         .where(
             st.position_seconds > 0,
             st.status.notin_(("watched", "hidden")),
@@ -136,7 +136,7 @@ def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
     row = db.execute(
         select(*feed_columns(user, st))
         .join(Channel, Channel.id == Video.channel_id)
-        .outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
+        .outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
         .where(Video.id == video_id)
     ).first()
     return serialize(row) if row is not None else None
@@ -179,7 +179,7 @@ def playlists_summary(
         select(*feed_columns(user, st), PlaylistItem.playlist_id.label("_playlist"))
         .join(PlaylistItem, PlaylistItem.video_id == Video.id)
         .join(Channel, Channel.id == Video.channel_id)
-        .outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
+        .outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
         .where(PlaylistItem.playlist_id.in_(playlist_ids))
         .order_by(PlaylistItem.playlist_id, PlaylistItem.position.asc())
     ).all()

backend/app/downloads/folders.py

@@ -328,9 +328,14 @@ def create_own(db: Session, user_id: int, name: str) -> FileFolder:
     return folder
 
 
+def renamable(folder: FileFolder) -> bool:
+    """Only a folder of the user's own making takes a name: a source folder's is its origin's."""
+    return folder.kind == FolderKind.OWN
+
+
 def rename(db: Session, folder: FileFolder, name: str) -> None:
     """Rename one of the user's own folders (a source folder's name is its origin's)."""
-    if folder.kind != FolderKind.OWN:
+    if not renamable(folder):
         raise FolderError("file_folder_not_own")
     folder.name = name
     try:

backend/app/downloads/queries.py

@@ -296,13 +296,11 @@ def own_link(db: Session, user: User, link_id: int) -> DownloadLink | None:
     if link is None:
         return None
     target = (
-        db.get(DownloadJob, link.job_id)
+        owned_row(db, DownloadJob, link.job_id, user)
         if link.job_id is not None
-        else db.get(FileFolder, link.folder_id)
+        else owned_row(db, FileFolder, link.folder_id, user)
     )
-    if target is None or target.user_id != user.id:
-        return None
-    return link
+    return link if target is not None else None
 
 
 def links_to(

backend/app/downloads/runner.py

@@ -16,6 +16,7 @@ import shutil
 import subprocess
 import sys
 
+import app.tables  # noqa: F401 — a fresh interpreter: every table, before the first query
 from app import logs
 from app import logsink
 from app.config import settings

backend/app/feedquery.py

@@ -212,7 +212,7 @@ def filtered_query(
     # exactly. 'unwatched'/'all' include row-less 'new' videos, so they keep the outer join.
     state_required = show in ("watched", "hidden", "in_progress")
     join_state = query.join if state_required else query.outerjoin
-    query = join_state(state, and_(state.video_id == Video.id, state.user_id == user.id))
+    query = join_state(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
 
     # Hide videos the maintenance job has flagged as currently unplayable (deleted/private/
     # abandoned). They're either deleted after a grace period or unhidden if they recover;

backend/app/jobruns.py

@@ -119,6 +119,16 @@ def recent_runs(db: Session, jobs: list[str], per_job: int) -> dict[str, list[Jo
     return out
 
 
+def last_run(db: Session, job: str, target: str) -> JobRun | None:
+    """The newest run of an ad-hoc `job` on `target` (a user's own import), or None."""
+    return db.execute(
+        select(JobRun)
+        .where(JobRun.job == job, JobRun.target == target)
+        .order_by(JobRun.started_at.desc(), JobRun.id.desc())
+        .limit(1)
+    ).scalar_one_or_none()
+
+
 def gc(db: Session, retention_days: int) -> int:
     """Delete finished runs older than `retention_days` (0 = keep forever). A run still going is
     never reaped. The caller commits."""

backend/app/main.py

@@ -28,6 +28,7 @@ log = logs.get_logger(__name__)
 from app import bodylimit, collation, logsink, state, sysconfig  # noqa: E402 — logging is configured first
 from app.config import settings  # noqa: E402 — logging is configured first
 from app.db import SessionLocal, engine  # noqa: E402 — logging is configured first
+import app.tables  # noqa: E402 — every table, before the first query
 from app.downloads import service as download_service  # noqa: E402 — logging is configured first
 from app.downloads import storage  # noqa: E402 — logging is configured first
 from app.plex import stream as plex_stream  # noqa: E402 — logging is configured first

backend/app/notifications.py

@@ -49,7 +49,9 @@ def create_notification(
 
 def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
     """Delete a user's oldest READ notifications beyond `cap`. Returns the number removed.
-    Idempotent and cheap; safe to call after marking things read."""
+    Idempotent and cheap; safe to call after marking things read. Flushes first, so a notice the
+    caller has just marked read counts; the caller commits."""
+    db.flush()
     ids = (
         db.execute(
             select(Notification.id)
@@ -64,5 +66,4 @@ def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
         return 0
     # One bulk DELETE rather than a get+delete per row (the ids are already in hand).
     db.execute(delete(Notification).where(Notification.id.in_(ids)))
-    db.commit()
     return len(ids)

backend/app/plex/queries.py

@@ -108,20 +108,6 @@ def state(db: Session, user: User | int, item_id: int) -> PlexState | None:
     )
 
 
-def states(db: Session, user: User | int, item_ids: list[int]) -> dict[int, PlexState]:
-    """The user's state on each of these items that has one, by item id."""
-    if not item_ids:
-        return {}
-    return {
-        s.item_id: s
-        for s in db.execute(
-            select(PlexState).where(
-                owned_by(PlexState.user_id, user), PlexState.item_id.in_(item_ids)
-            )
-        ).scalars()
-    }
-
-
 # --- a show ----------------------------------------------------------------------------------
 
 

backend/app/plex/rollup.py

@@ -8,6 +8,7 @@ from datetime import datetime, timezone
 from sqlalchemy import and_, case, func, or_, select
 from sqlalchemy.orm import Session, aliased
 
+from app.models import User
 from app.userscope import owned_by
 from app.plex.models import PlexItem, PlexState
 from app.plex.cards import ShowState
@@ -106,7 +107,7 @@ def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, ShowS
     return {r.show_id: show_state(r.total, r.watched, r.inprog) for r in rows}
 
 
-def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int, PlexState]:
+def item_state_map(db: Session, user_id: User | int, item_ids: list[int]) -> dict[int, PlexState]:
     """A user's PlexState rows for a set of leaf items (movies or episodes), keyed by item_id — the
     leaf analogue of `state_map` (which rolls a show's episodes up to a `ShowState`). One batched
     fetch, so the caller serializes cards without a per-card query. Empty in → empty dict (no `IN ()`)."""

backend/app/plex/service.py

@@ -19,7 +19,7 @@ from app.plex import people as plex_people
 from app.plex import tmdb as plex_tmdb
 from app.plex.lookups import by_rating_key
 from app.models import User
-from app.plex import queries
+from app.plex import queries, rollup
 from app.plex.models import (
     PlexCollection,
     PlexItem,
@@ -302,6 +302,36 @@ def mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> boo
     return False
 
 
+def state_row(
+    db: Session,
+    user: User | int,
+    item_id: int,
+    st: PlexState | None = None,
+    *,
+    synced: bool = False,
+) -> PlexState:
+    """The user's state row on this item, made if the read before found none. Made by the insert
+    itself (ON CONFLICT DO NOTHING) and then read: the first checkpoint of an item and the `final`
+    flush beside it routinely both read "none", and a plain add let the second one fail on
+    `uq_plex_user_item` — a 500, and the push it was carrying lost. Every first row goes through
+    here: a checkpoint, a status, a season marked at once, a pull from Plex. `synced` is what a row
+    made now says about Plex; a row another writer made first keeps its own."""
+    if st is not None:
+        return st
+    db.execute(
+        pg_insert(PlexState)
+        .values(
+            user_id=user if isinstance(user, int) else user.id,
+            item_id=item_id,
+            synced_to_plex=synced,
+        )
+        .on_conflict_do_nothing(constraint="uq_plex_user_item")
+    )
+    made = queries.state(db, user, item_id)
+    assert made is not None  # inserted now, or committed by the request that won
+    return made
+
+
 def clear_progress(db: Session, st: PlexState | None, now: datetime) -> None:
     """A checkpoint too early to keep: a state that held nothing else goes, any other forgets its
     position."""
@@ -325,9 +355,7 @@ def checkpoint(
 ) -> PlexState:
     """Record a resume position. `dirty` marks it not yet mirrored to Plex — set before a push, so
     the reconcile re-pushes it if the push fails and the push can settle it once Plex confirms."""
-    if st is None:
-        st = PlexState(user_id=user.id, item_id=item_id)
-        db.add(st)
+    st = state_row(db, user, item_id, st)
     st.position_seconds = position
     st.progress_updated_at = now
     if dirty:
@@ -345,9 +373,7 @@ def set_status(
     db: Session, user: User, item_id: int, st: PlexState | None, status: str
 ) -> PlexState:
     """Set a watched or hidden status; watched also clears the resume point."""
-    if st is None:
-        st = PlexState(user_id=user.id, item_id=item_id)
-        db.add(st)
+    st = state_row(db, user, item_id, st)
     st.status = status
     if status == "watched":
         st.watched_at = datetime.now(timezone.utc)
@@ -364,15 +390,14 @@ def bulk_state(
     (item id, rating key, action) pushes a linked Plex account needs. Unwatching keeps the user's
     private "hidden" marks."""
     now = datetime.now(timezone.utc)
-    existing = queries.states(db, user_id, [e.id for e in eps])
+    existing = rollup.item_state_map(db, user_id, [e.id for e in eps])
     to_push: list[tuple[int, str, str]] = []
     for e in eps:
         st = existing.get(e.id)
         prev = st.status if st is not None else "new"
         if watched:
-            if st is None:
-                st = PlexState(user_id=user_id, item_id=e.id)
-                db.add(st)
+            st = state_row(db, user_id, e.id, st)
+            prev = st.status  # a first checkpoint beside this may have made it meanwhile
             st.status = "watched"
             st.watched_at = now
             st.position_seconds = 0

backend/app/plex/watch_sync.py

@@ -32,6 +32,8 @@ from app.db import SessionLocal
 from app.userscope import owned_by
 from app.plex.models import PlexItem, PlexLibrary, PlexLink, PlexState
 from app.plex.client import PlexClient, PlexError, PlexNotConfigured
+from app.plex import queries
+from app.plex.service import state_row
 from app.plex.sync import (
     _TYPE_EPISODE,
     _TYPE_MOVIE,
@@ -252,15 +254,7 @@ def push_state_to_plex(
         # Flag the row synced — but an "unwatched" via status=new deletes the row, so there may be
         # nothing to flag; that's fine (unscrobble still happened). Skip the flag if the row was
         # re-edited since we captured `expect_ts` (leave it dirty for its own push / the reconcile).
-        st = (
-            db.execute(
-                select(PlexState).where(
-                    owned_by(PlexState.user_id, user_id), PlexState.item_id == item_id
-                )
-            )
-            .scalars()
-            .first()
-        )
+        st = queries.state(db, user_id, item_id)
         if st is not None and _row_matches(st, expect_ts):
             st.synced_to_plex = True
             db.commit()
@@ -394,15 +388,7 @@ def _pull_apply(
     """Bring one Plex-side positive signal into Siftlode under last-write-wins. Returns
     "watched"/"in_progress" if it changed the row, else None. Never clears a row (un-watch is the
     full reconcile's job) — this only pulls in watched/resume."""
-    st = (
-        db.execute(
-            select(PlexState).where(
-                owned_by(PlexState.user_id, user_id), PlexState.item_id == item_id
-            )
-        )
-        .scalars()
-        .first()
-    )
+    st = queries.state(db, user_id, item_id)
     if _same_state(st, status, pos):
         if st is not None and not st.synced_to_plex:
             st.synced_to_plex = True  # already matches Plex → just settle the flag, no churn
@@ -423,8 +409,9 @@ def _pull_apply(
         # "almost always already pushed" assumption above excludes.
         return None
     if st is None:
-        st = PlexState(user_id=user_id, item_id=item_id)
-        db.add(st)
+        st = state_row(db, user_id, item_id, synced=True)
+        if not st.synced_to_plex:  # a local change made it first, beside this pull: the guard above
+            return None
     if status == "watched":
         st.status, st.position_seconds, st.watched_at, st.progress_updated_at = (
             "watched",

backend/app/quota.py

@@ -44,6 +44,7 @@ class QuotaAction:
     VIDEOS_ENRICH = "videos_enrich"
     VIDEOS_LOOKUP = "videos_lookup"
     VIDEOS_SEARCH = "videos_search"
+    VIDEOS_RELATED = "videos_related"
     CHANNELS_DISCOVER = "channels_discover"
     CHANNELS_EXPLORE = "channels_explore"
     CHANNELS_SUBSCRIBE = "channels_subscribe"
@@ -142,7 +143,7 @@ def backfill_reserve_reached() -> bool:
     condition for every backfill/explore path (recent, deep, immediate full-history, explore, and
     rolling revalidation).
 
-    Reads in its OWN short-lived session (like record_usage / log_action), for two reasons: it
+    Reads in its OWN short-lived session (like record_usage / claim_action), for two reasons: it
     replaces the remaining/reserve comparison that was open-coded at every call site, and — critically
     for the per-page guards in paging loops — a guard must not hold the caller's transaction open
     across the network fetch that follows it. Quota is committed independently of any caller, so a
@@ -151,24 +152,40 @@ def backfill_reserve_reached() -> bool:
         return remaining_today(qs) <= sysconfig.get_int(qs, "backfill_quota_reserve")
 
 
-def log_action(user_id: int, action: str) -> None:
-    """Record a zero-cost action event so per-user daily caps still count it.
+def claim_action(user_id: int, action: str, limit: int) -> int | None:
+    """Take one of this user's `limit` daily `action` slots, or None when they are all taken — the
+    check and the take in one transaction, under a lock per user and action. Checked first and
+    logged after the work, N requests sent together at the last free slot all passed the check
+    (the work in between is seconds of scraping). Returns the event's id, for `refund_action` when
+    the work then produced nothing.
 
-    `record_usage` only logs when units are actually charged; a scrape-based search spends no
-    quota yet must still be rate-limited per user, so it logs its event here directly.
-
-    Writes in its OWN session (like record_usage) — quota accounting is durable independent of the
-    caller's transaction and must never commit the caller's session (that would flush a job's
-    partial state mid-run, defeating its own rollback)."""
+    The ONE way to count a capped action: a claim is a zero-unit event, and `actions_today` counts
+    those alone, so a call the action then pays for (search.list's own `record_usage` event, under
+    the same action) is accounting, never a second slot. Own session, committed at once — quota
+    accounting is durable whatever the caller's transaction does next, and must never commit it."""
     with SessionLocal() as qs:
-        qs.add(QuotaEvent(user_id=user_id, action=action, units=0))
+        qs.execute(select(func.pg_advisory_xact_lock(func.hashtext(action), user_id)))
+        if actions_today(qs, user_id, action) >= limit:
+            return None
+        event = QuotaEvent(user_id=user_id, action=action, units=0)
+        qs.add(event)
+        qs.commit()
+        return event.id
+
+
+def refund_action(event_id: int) -> None:
+    """Give a slot `claim_action` took back: the work it was taken for failed outright, and a
+    request that showed the user nothing must not count against them."""
+    with SessionLocal() as qs:
+        qs.execute(delete(QuotaEvent).where(QuotaEvent.id == event_id))
         qs.commit()
 
 
 def actions_today(db: Session, user_id: int, action: str) -> int:
-    """How many quota events of `action` this user has logged so far in the current Pacific
-    day — for per-user, per-action daily caps (e.g. the live-search limit). Counts events,
-    not units, so it only works for actions charged exactly once per user action."""
+    """How many slots of `action` this user has claimed so far in the current Pacific day — for
+    per-user, per-action daily caps (the live search, the related lists). Counts the zero-unit
+    claim events (`claim_action`) only: `record_usage`'s events for units the action spent are not
+    slots (it logs only when units were charged)."""
     return (
         db.scalar(
             select(func.count())
@@ -176,6 +193,7 @@ def actions_today(db: Session, user_id: int, action: str) -> int:
             .where(
                 owned_by(QuotaEvent.user_id, user_id),
                 QuotaEvent.action == action,
+                QuotaEvent.units == 0,
                 QuotaEvent.created_at >= pacific_day_start_utc(),
             )
         )

backend/app/ratelimit.py

@@ -42,6 +42,14 @@ class RateLimiter:
         with self._lock:
             return sum(c for t, c in self._hits.get(key, []) if t > cutoff) >= self.max_events
 
+    def room(self, key: str) -> float:
+        """How much cost ``key`` may still spend in the current window, WITHOUT counting anything —
+        for work counted only once it has succeeded: ask it first, ``allow`` what was done."""
+        cutoff = time.monotonic() - self.window
+        with self._lock:
+            used = sum(c for t, c in self._hits.get(key, []) if t > cutoff)
+        return max(0.0, self.max_events - used)
+
     def allow(self, key: str, cost: float = 1) -> bool:
         """Record an attempt costing ``cost`` for ``key`` and return whether it is within the limit.
 

backend/app/reader/models.py

@@ -4,6 +4,7 @@ and the comfort shelf on Home. Only this unit writes them; another unit reads th
 edge, or asks this unit's service."""
 
 from datetime import datetime
+from enum import StrEnum
 
 from sqlalchemy import (
     Boolean,
@@ -11,12 +12,14 @@ from sqlalchemy import (
     DateTime,
     Float,
     ForeignKey,
+    Index,
     Integer,
     JSON,
     String,
     Text,
     UniqueConstraint,
     func,
+    text,
 )
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
@@ -82,6 +85,21 @@ class VideoState(Base, UpdatedAtMixin):
     progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
 
 
+class VideoUnmark(Base):
+    """When the user last took a video back to "new" here — un-marked it, or reset it. That leaves
+    no `VideoState` (its absence is "new"), so without this row a later import of what they watched
+    on YouTube could not tell a video they un-marked from one never touched, and marked it watched
+    again. Kept by user and video; the video's own deletion takes it."""
+
+    __tablename__ = "video_unmarks"
+    __table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_video_unmark"),)
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
+    at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
+
+
 class Playlist(Base, TimestampMixin, UpdatedAtMixin):
     """A per-user named, ordered collection of videos from the shared catalog.
 
@@ -93,6 +111,15 @@ class Playlist(Base, TimestampMixin, UpdatedAtMixin):
     sync phases; the foundation phase only uses local playlists.)"""
 
     __tablename__ = "playlists"
+    __table_args__ = (
+        # One Watch later per user, held by the database: two first saves at once each made one.
+        Index(
+            "uq_playlist_watch_later",
+            "user_id",
+            unique=True,
+            postgresql_where=text("kind = 'watch_later'"),
+        ),
+    )
 
     id: Mapped[int] = mapped_column(primary_key=True)
     user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
@@ -202,3 +229,70 @@ class ComfortPin(Base, TimestampMixin):
     # 0-based shelf order. The shelf is short and hand-curated, so a dense integer the reorder
     # endpoint rewrites wholesale beats a fractional index nobody would ever exercise here.
     position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
+
+
+class TakeoutKind(StrEnum):
+    """What a Google Takeout row says the user did with a video."""
+
+    WATCHED = "watched"  # history/watch-history.json
+    WATCH_LATER = "watch_later"  # playlists/Watch later-videos.csv
+
+
+class TakeoutOutcome(StrEnum):
+    """What the import did with a row; NULL = still waiting for its video to be known (and, for a
+    watched one, probed for Shorts)."""
+
+    WATCHED = "watched"  # the video is marked watched at the Takeout time
+    KEPT = "kept"  # the state the user set here (watched, hidden, or changed later) stays
+    SHORT = "short"  # a Short: swiping one past is not watching it
+    ADDED = "added"  # added to Watch later
+    PRESENT = "present"  # already in Watch later
+    MISSING = "missing"  # YouTube no longer has the video
+
+
+class TakeoutItem(Base, TimestampMixin):
+    """One video from a user's Google Takeout export: watched (with when) or in their YouTube Watch
+    later list (with when it was added). The rows are both the import job's input and its memory: a
+    video the catalog does not know yet waits here, with no outcome, until it enters the catalog.
+    The id is not a foreign key for that reason. A watched row's channel and length feed the
+    "channels you watch but do not follow" list: the length is asked of YouTube for that list alone
+    and kept here, so its videos need not enter the shared catalog to be counted."""
+
+    __tablename__ = "takeout_items"
+    __table_args__ = (
+        UniqueConstraint("user_id", "kind", "video_id", name="uq_takeout_item"),
+        CheckConstraint("kind IN ('watched', 'watch_later')", name="ck_takeout_item_kind"),
+        # The waiting rows, by video: what a newly known video looks up.
+        Index(
+            "ix_takeout_items_waiting",
+            "video_id",
+            postgresql_where=text("outcome IS NULL"),
+        ),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    kind: Mapped[str] = mapped_column(String(12))
+    video_id: Mapped[str] = mapped_column(String(16))
+    channel_id: Mapped[str | None] = mapped_column(String(32))
+    # Watched: the last time in the export; Watch later: when it was added.
+    at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
+    outcome: Mapped[str | None] = mapped_column(String(12))
+    # When YouTube was asked for the video's length (NULL = not yet), and the length it gave; a
+    # video it no longer has keeps a NULL length.
+    looked_up_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    duration_seconds: Mapped[int | None] = mapped_column(Integer)
+
+
+class TakeoutChannelDismissal(Base, TimestampMixin):
+    """A channel the user put out of their "watched but not followed" list. The id is not a foreign
+    key: the channel may never have entered the catalog, or may leave it, and the choice stays."""
+
+    __tablename__ = "takeout_channel_dismissals"
+    __table_args__ = (
+        UniqueConstraint("user_id", "channel_id", name="uq_takeout_channel_dismissal"),
+    )
+
+    id: Mapped[int] = mapped_column(primary_key=True)
+    user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
+    channel_id: Mapped[str] = mapped_column(String(32))

backend/app/reader/queries.py

@@ -1,5 +1,5 @@
-"""Reader's reads: a user's labels, saved views, watch state and playlists, for the HTTP layer and
-for the other units that ask about them.
+"""Reader's reads: a user's labels, saved views, watch state, playlists and Takeout import, for the HTTP
+layer and for the other units that ask about them.
 
 Every read of a user's row goes through `userscope`, so "someone else's row" is the same answer as
 a missing one. Reader reads Catalog's videos along its declared edge (a playlist's cover and
@@ -7,14 +7,28 @@ length, whether a video exists); another unit asks the questions at the end of t
 than joining Reader's tables itself.
 """
 
+from collections.abc import Iterable
+from datetime import datetime
+
 from sqlalchemy import ColumnElement, and_, func, or_, select
 from sqlalchemy.orm import InstrumentedAttribute, Session, aliased
 
 from app.collation import collated
 from app.feedquery import feed_columns, serialize
 from app.models import User
-from app.catalog.models import Channel, Subscription, Video
-from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
+from app.catalog.models import BlockedChannel, Channel, Subscription, Video
+from app.reader.models import (
+    ChannelTag,
+    Playlist,
+    PlaylistItem,
+    SavedView,
+    Tag,
+    TakeoutChannelDismissal,
+    TakeoutItem,
+    TakeoutKind,
+    VideoState,
+    VideoUnmark,
+)
 from app.userscope import owned_by, owned_row
 
 # --- catalog, read along the edge ------------------------------------------------------------
@@ -230,6 +244,18 @@ def kept_by_a_reader(video_id: InstrumentedAttribute[str]) -> ColumnElement[bool
     )
 
 
+def in_a_watch_history(channel_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
+    """A channel someone's uploaded Takeout history shows watched. The "watched but not followed"
+    list names a channel from its catalog row, so the explore cleanup keeps the row of one (its
+    videos it may still reclaim: opening the list's videos brings them back). As a clause, so the
+    cleanup's DELETE stays one statement."""
+    return (
+        select(TakeoutItem.id)
+        .where(TakeoutItem.channel_id == channel_id, TakeoutItem.kind == TakeoutKind.WATCHED)
+        .exists()
+    )
+
+
 def playlisted_by(db: Session, video_ids: list[str]) -> list[tuple[int, str]]:
     """(user id, video id) for every user who has one of these videos in a playlist."""
     return list(
@@ -293,6 +319,36 @@ def held_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
     )
 
 
+def seen_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
+    """Which of these videos the user has already seen or set aside: watched or hidden here, or in
+    their uploaded Takeout history unless they took it back to "new" here since. The ids need not
+    be in the catalog — a Takeout row names a video the catalog may never have had — so a list
+    of what YouTube suggests can drop them before anything is ingested."""
+    if not video_ids:
+        return set()
+    marked = select(VideoState.video_id).where(
+        owned_by(VideoState.user_id, user),
+        VideoState.video_id.in_(video_ids),
+        VideoState.status.in_(("watched", "hidden")),
+    )
+    unmarked_since = (
+        select(VideoUnmark.id)
+        .where(
+            owned_by(VideoUnmark.user_id, user),
+            VideoUnmark.video_id == TakeoutItem.video_id,
+            VideoUnmark.at > TakeoutItem.at,
+        )
+        .exists()
+    )
+    in_history = select(TakeoutItem.video_id).where(
+        owned_by(TakeoutItem.user_id, user),
+        TakeoutItem.kind == TakeoutKind.WATCHED,
+        TakeoutItem.video_id.in_(video_ids),
+        ~unmarked_since,
+    )
+    return set(db.scalars(marked)) | set(db.scalars(in_history))
+
+
 def feed_rows_by_ids(db: Session, user: User, video_ids: list[str]) -> list[dict]:
     """These videos serialized with the feed's projection (their channel, this user's watch state,
     saved), in the order given."""
@@ -359,3 +415,355 @@ def user_tag_link(db: Session, user: User, channel_id: str, tag_id: int) -> Chan
             owned_by(ChannelTag.user_id, user),
         )
     ).scalar_one_or_none()
+
+
+# --- Google Takeout --------------------------------------------------------------------------
+
+
+def _settled_here() -> ColumnElement[bool]:
+    """A video a waiting "watched" row may be marked on: one the catalog keeps of its own, not a
+    discovery video only a search or an explore brought in. A mark is a `VideoState`, which the
+    cleanup jobs read as a reader keeping it (`kept_by_a_reader`): on one of those it would keep the
+    video, and its explored channel, for good. Such a row waits until the video is the catalog's own
+    (its channel subscribed to) — or is gone."""
+    subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
+    return and_(Video.via_explore.is_(False), or_(Video.via_search.is_(False), subscribed))
+
+
+def takeout_watched_ready(db: Session, user: User) -> list[tuple[int, str, datetime, bool]]:
+    """The user's waiting "watched" rows whose video the catalog knows and has probed for Shorts:
+    (row id, video id, Takeout time, is Short)."""
+    return list(
+        db.execute(
+            select(TakeoutItem.id, TakeoutItem.video_id, TakeoutItem.at, Video.is_short)
+            .join(Video, Video.id == TakeoutItem.video_id)
+            .where(
+                owned_by(TakeoutItem.user_id, user),
+                TakeoutItem.kind == TakeoutKind.WATCHED,
+                TakeoutItem.outcome.is_(None),
+                Video.shorts_probed,
+                _settled_here(),
+            )
+        ).tuples()
+    )
+
+
+def takeout_waiting(db: Session, user: User, kind: TakeoutKind) -> list[tuple[int, str]]:
+    """The user's rows of `kind` the import has not settled, oldest first: (row id, video id)."""
+    return list(
+        db.execute(
+            select(TakeoutItem.id, TakeoutItem.video_id)
+            .where(
+                owned_by(TakeoutItem.user_id, user),
+                TakeoutItem.kind == kind,
+                TakeoutItem.outcome.is_(None),
+            )
+            .order_by(TakeoutItem.at, TakeoutItem.id)
+        ).tuples()
+    )
+
+
+# Ids per IN list: a list is bound one parameter an id, and a statement takes at most 65,535.
+_IN_CHUNK = 5000
+
+
+def known_video_ids(db: Session, ids: list[str]) -> set[str]:
+    """Which of `ids` the catalog holds."""
+    known: set[str] = set()
+    for start in range(0, len(ids), _IN_CHUNK):
+        chunk = ids[start : start + _IN_CHUNK]
+        known.update(db.execute(select(Video.id).where(Video.id.in_(chunk))).scalars())
+    return known
+
+
+def unmarked_at(db: Session, user: User, ids: list[str]) -> dict[str, datetime]:
+    """When the user last took each of `ids` back to "new" here, for those they ever did."""
+    out: dict[str, datetime] = {}
+    for start in range(0, len(ids), _IN_CHUNK):
+        rows = db.execute(
+            select(VideoUnmark.video_id, VideoUnmark.at).where(
+                owned_by(VideoUnmark.user_id, user),
+                VideoUnmark.video_id.in_(ids[start : start + _IN_CHUNK]),
+            )
+        ).tuples()
+        out.update((vid, at) for vid, at in rows)
+    return out
+
+
+def in_playlist(db: Session, playlist_id: int, ids: list[str]) -> set[str]:
+    """Which of `ids` the playlist already holds."""
+    held: set[str] = set()
+    for start in range(0, len(ids), _IN_CHUNK):
+        chunk = ids[start : start + _IN_CHUNK]
+        held.update(
+            db.execute(
+                select(PlaylistItem.video_id).where(
+                    PlaylistItem.playlist_id == playlist_id, PlaylistItem.video_id.in_(chunk)
+                )
+            ).scalars()
+        )
+    return held
+
+
+def takeout_rows_besides(
+    db: Session, user: User, kind: TakeoutKind, video_ids: Iterable[str]
+) -> int:
+    """How many Takeout rows the user keeps other than `video_ids` of `kind`: what an upload of
+    those videos would be added to."""
+    total = (
+        db.scalar(select(func.count(TakeoutItem.id)).where(owned_by(TakeoutItem.user_id, user)))
+        or 0
+    )
+    ids = list(video_ids)
+    for start in range(0, len(ids), _IN_CHUNK):
+        total -= (
+            db.scalar(
+                select(func.count(TakeoutItem.id)).where(
+                    owned_by(TakeoutItem.user_id, user),
+                    TakeoutItem.kind == kind,
+                    TakeoutItem.video_id.in_(ids[start : start + _IN_CHUNK]),
+                )
+            )
+            or 0
+        )
+    return total
+
+
+def takeout_outcomes(db: Session, user: User) -> dict[tuple[str, str | None], int]:
+    """How many of the user's rows each kind and outcome holds (None = waiting)."""
+    rows = db.execute(
+        select(TakeoutItem.kind, TakeoutItem.outcome, func.count())
+        .where(owned_by(TakeoutItem.user_id, user))
+        .group_by(TakeoutItem.kind, TakeoutItem.outcome)
+    ).all()
+    return {(kind, outcome): n for kind, outcome, n in rows}
+
+
+def takeout_summary(db: Session, user: User) -> dict[str, dict[str, int]]:
+    """The user's kept Takeout videos by kind and outcome, for their settings page: `{"watched":
+    {"watched": n, "short": n, "waiting": n, …}, "watch_later": {…}}` — the import's outcome words,
+    "waiting" for a video not known yet."""
+    out: dict[str, dict[str, int]] = {kind.value: {} for kind in TakeoutKind}
+    for (kind, outcome), n in takeout_outcomes(db, user).items():
+        out[kind][outcome or "waiting"] = n
+    return out
+
+
+def takeout_users_ready(db: Session) -> list[int]:
+    """Every user with a waiting "watched" row whose video is now known and probed for Shorts: the
+    rows a later run of the Shorts probe made ready."""
+    return list(
+        db.execute(
+            select(TakeoutItem.user_id)
+            .join(Video, Video.id == TakeoutItem.video_id)
+            .where(
+                TakeoutItem.kind == TakeoutKind.WATCHED,
+                TakeoutItem.outcome.is_(None),
+                Video.shorts_probed,
+                _settled_here(),
+            )
+            .distinct()
+        ).scalars()
+    )
+
+
+def takeout_users_watch_later_waiting(db: Session) -> list[int]:
+    """Every user with a Watch later row a run could not put in yet."""
+    return list(
+        db.execute(
+            select(TakeoutItem.user_id)
+            .where(TakeoutItem.kind == TakeoutKind.WATCH_LATER, TakeoutItem.outcome.is_(None))
+            .distinct()
+        ).scalars()
+    )
+
+
+# --- channels watched but not followed ---------------------------------------------------------
+
+# A channel joins the list with this many watched videos of at least this length. A Short is three
+# minutes at most, so the length alone keeps them out, and a video under five minutes says too
+# little of what a channel is about to count towards it.
+WATCHED_CHANNEL_MIN_VIDEOS = 5
+WATCHED_CHANNEL_MIN_SECONDS = 300
+
+
+def _unfollowed(user: User) -> ColumnElement[bool]:
+    """A watched row whose channel the user neither follows, blocked, put out of the list, nor is."""
+    subscribed = select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
+    blocked = select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user))
+    dismissed = select(TakeoutChannelDismissal.channel_id).where(
+        owned_by(TakeoutChannelDismissal.user_id, user)
+    )
+    clauses = [
+        owned_by(TakeoutItem.user_id, user),
+        TakeoutItem.kind == TakeoutKind.WATCHED,
+        TakeoutItem.channel_id.is_not(None),
+        TakeoutItem.channel_id.not_in(subscribed),
+        TakeoutItem.channel_id.not_in(blocked),
+        TakeoutItem.channel_id.not_in(dismissed),
+    ]
+    if user.yt_channel_id:
+        clauses.append(TakeoutItem.channel_id != user.yt_channel_id)
+    return and_(*clauses)
+
+
+def _candidate_channels(user: User):
+    """The unfollowed channels with enough watched videos to make the list once their lengths are
+    known: (channel id, how many watched)."""
+    return (
+        select(TakeoutItem.channel_id, func.count().label("watched"))
+        .where(_unfollowed(user))
+        .group_by(TakeoutItem.channel_id)
+        .having(func.count() >= WATCHED_CHANNEL_MIN_VIDEOS)
+        .subquery()
+    )
+
+
+def takeout_lengths_to_ask(db: Session, user: User) -> list[str]:
+    """The watched videos whose length the list still needs: those of the candidate channels not
+    asked about yet, the most watched channels first, so a lookup cut short fills the top."""
+    cand = _candidate_channels(user)
+    return list(
+        db.execute(
+            select(TakeoutItem.video_id)
+            .join(cand, cand.c.channel_id == TakeoutItem.channel_id)
+            .where(_unfollowed(user), TakeoutItem.looked_up_at.is_(None))
+            .order_by(cand.c.watched.desc(), TakeoutItem.channel_id, TakeoutItem.at.desc())
+        ).scalars()
+    )
+
+
+def watched_channels_pending(db: Session, user: User) -> int:
+    """How many channels a later run may still add to the list: the candidates with a watched video
+    whose length is not known, and the ones that made it but have no catalog row to be listed from
+    yet. One the catalog holds without details is listed already (by the name it came in with), so
+    it is not one to come."""
+    cand = _candidate_channels(user)
+    unasked = db.execute(
+        select(TakeoutItem.channel_id)
+        .join(cand, cand.c.channel_id == TakeoutItem.channel_id)
+        .where(_unfollowed(user), TakeoutItem.looked_up_at.is_(None))
+        .distinct()
+    ).scalars()
+    counted = _counted_channels(user)
+    unlisted = db.execute(
+        select(counted.c.channel_id)
+        .outerjoin(Channel, Channel.id == counted.c.channel_id)
+        .where(Channel.id.is_(None))
+    ).scalars()
+    return len(set(unasked) | set(unlisted))
+
+
+def _counted_channels(user: User):
+    """The channels that make the list by their counted videos, whatever the catalog holds of them."""
+    return (
+        select(TakeoutItem.channel_id)
+        .where(_unfollowed(user), TakeoutItem.duration_seconds >= WATCHED_CHANNEL_MIN_SECONDS)
+        .group_by(TakeoutItem.channel_id)
+        .having(func.count() >= WATCHED_CHANNEL_MIN_VIDEOS)
+        .subquery()
+    )
+
+
+def watched_channels_to_describe(db: Session, user: User) -> list[str]:
+    """The channels on the list (by their counted videos) that the catalog lacks or has no details
+    of yet: what the list cannot name, or names with no more than a video brought."""
+    counted = _counted_channels(user)
+    return list(
+        db.execute(
+            select(counted.c.channel_id)
+            .outerjoin(Channel, Channel.id == counted.c.channel_id)
+            .where(Channel.details_synced_at.is_(None))
+        ).scalars()
+    )
+
+
+def watched_channels(db: Session, user: User) -> list[tuple[Channel, int, datetime]]:
+    """The channels the user watched often but does not follow: (channel, how many of its videos
+    over five minutes they watched, when last). Most watched first; on a tie the one watched more
+    lately, which is the likelier interest."""
+    n = func.count(TakeoutItem.id)
+    last = func.max(TakeoutItem.at)
+    return list(
+        db.execute(
+            select(Channel, n, last)
+            .join(Channel, Channel.id == TakeoutItem.channel_id)
+            .where(_unfollowed(user), TakeoutItem.duration_seconds >= WATCHED_CHANNEL_MIN_SECONDS)
+            .group_by(Channel.id)
+            .having(n >= WATCHED_CHANNEL_MIN_VIDEOS)
+            .order_by(n.desc(), last.desc(), Channel.id)
+        ).tuples()
+    )
+
+
+def has_watch_history(db: Session, user: User) -> bool:
+    """Whether the user has uploaded a Takeout watch history at all."""
+    return bool(
+        db.scalar(
+            select(
+                select(TakeoutItem.id)
+                .where(owned_by(TakeoutItem.user_id, user), TakeoutItem.kind == TakeoutKind.WATCHED)
+                .exists()
+            )
+        )
+    )
+
+
+def watched_in_history(db: Session, user: User, channel_id: str) -> bool:
+    """Whether the user's Takeout history shows them watching this channel at all."""
+    return bool(
+        db.scalar(
+            select(
+                select(TakeoutItem.id)
+                .where(
+                    owned_by(TakeoutItem.user_id, user),
+                    TakeoutItem.kind == TakeoutKind.WATCHED,
+                    TakeoutItem.channel_id == channel_id,
+                )
+                .exists()
+            )
+        )
+    )
+
+
+def watched_channel_videos(
+    db: Session, user: User, channel_id: str
+) -> tuple[list[tuple[str, datetime]], int]:
+    """The user's watched videos of one channel that the list counts, last watched first: (video
+    id, when). And how many YouTube no longer has."""
+    rows = db.execute(
+        select(TakeoutItem.video_id, TakeoutItem.at, TakeoutItem.duration_seconds)
+        .where(
+            owned_by(TakeoutItem.user_id, user),
+            TakeoutItem.kind == TakeoutKind.WATCHED,
+            TakeoutItem.channel_id == channel_id,
+            TakeoutItem.looked_up_at.is_not(None),
+        )
+        .order_by(TakeoutItem.at.desc(), TakeoutItem.video_id)
+    ).tuples()
+    counted: list[tuple[str, datetime]] = []
+    gone = 0
+    for vid, at, seconds in rows:
+        if seconds is None:
+            gone += 1
+        elif seconds >= WATCHED_CHANNEL_MIN_SECONDS:
+            counted.append((vid, at))
+    return counted, gone
+
+
+def takeout_users_with_list_work(db: Session) -> list[int]:
+    """Every user whose "watched but not followed" list a run can still add to: lengths to ask, or
+    channels to name. Asked of each user whose history holds a channel of enough watched videos,
+    since what is off a list (followed, blocked, put out, their own) is that user's own — a row of
+    one is never asked about, and would otherwise bring the user back on every run."""
+    per_channel = (
+        select(TakeoutItem.user_id)
+        .where(TakeoutItem.kind == TakeoutKind.WATCHED, TakeoutItem.channel_id.is_not(None))
+        .group_by(TakeoutItem.user_id, TakeoutItem.channel_id)
+        .having(func.count() >= WATCHED_CHANNEL_MIN_VIDEOS)
+        .subquery()
+    )
+    users = db.execute(
+        select(User).where(User.id.in_(select(per_channel.c.user_id))).order_by(User.id)
+    ).scalars()
+    return [user.id for user in users if watched_channels_pending(db, user)]

backend/app/reader/service.py

@@ -9,13 +9,27 @@ unique key, a row lock taken before a presence check. The comfort shelf's writes
 
 from datetime import datetime, timezone
 
-from sqlalchemy import exists, func, select, update
+from typing import cast
+
+from sqlalchemy import Table, bindparam, delete, exists, func, select, text, update
 from sqlalchemy.dialects.postgresql import insert as pg_insert
 from sqlalchemy.orm import Session
 
 from app.models import User
 from app.reader import queries
-from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
+from app.reader.models import (
+    ChannelTag,
+    Playlist,
+    PlaylistItem,
+    SavedView,
+    Tag,
+    TakeoutChannelDismissal,
+    TakeoutItem,
+    TakeoutKind,
+    TakeoutOutcome,
+    VideoState,
+    VideoUnmark,
+)
 from app.sync.playlists import lock_playlists, recompute_dirty
 from app.userscope import owned_by
 
@@ -196,6 +210,7 @@ def unmark(db: Session, user: User, video_id: str) -> None:
     the in-progress state, not where the user left off); otherwise the row goes. The player fires
     this beside a checkpoint that may delete the same row: the caller's commit then raises
     `StaleDataError`, which is that lost race and nothing worse."""
+    lock_takeout(db, user)  # an import settling meanwhile reads the un-mark below, or comes first
     row = queries.video_state(db, user, video_id)
     if row is None:
         return
@@ -204,9 +219,21 @@ def unmark(db: Session, user: User, video_id: str) -> None:
         row.watched_at = None
     else:
         db.delete(row)
+    _remember_unmark(db, user, video_id)
     db.flush()
 
 
+def _remember_unmark(db: Session, user: User, video_id: str) -> None:
+    """Note that the user took the video back to "new" now: the state they set leaves no row, and
+    an import of their YouTube history must not mark it watched again (`apply_takeout_watched`)."""
+    now = datetime.now(timezone.utc)
+    db.execute(
+        pg_insert(VideoUnmark)
+        .values(user_id=user.id, video_id=video_id, at=now)
+        .on_conflict_do_update(constraint="uq_video_unmark", set_={"at": now})
+    )
+
+
 def mark(db: Session, user: User, video_id: str, status: str) -> None:
     """ "watched" or "hidden", as one atomic upsert on (user, video). The player marks "watched" at
     the end of playback at the instant a checkpoint may delete the in-progress row; a SELECT then
@@ -231,32 +258,43 @@ def mark(db: Session, user: User, video_id: str, status: str) -> None:
 def clear_state(db: Session, user: User, video_id: str) -> None:
     """The video back to pristine for this user: status, resume point and `watched_at` all go.
     Playlist membership lives elsewhere and stays."""
+    lock_takeout(db, user)  # as `unmark`
     row = queries.video_state(db, user, video_id)
     if row is not None:
         db.delete(row)
+        _remember_unmark(db, user, video_id)
         db.flush()
 
 
 def set_progress(db: Session, user: User, video_id: str, position: int) -> None:
     """The player's resume point. 0 clears it: a row holding nothing else goes, one with a status
-    (watched, hidden) keeps it with the position zeroed. The status itself is never changed here."""
-    row = queries.video_state(db, user, video_id)
+    (watched, hidden) keeps it with the position zeroed. The status itself is never changed here.
+    A position is one upsert on (user, video): the first checkpoint and the end-of-playback mark
+    beside it both read "no row", and a plain add let the second one fail on `uq_user_video`."""
     now = datetime.now(timezone.utc)
-    if not position:
-        if row is None:
-            return
-        if row.status == "new":
-            db.delete(row)
-        else:
-            row.position_seconds = 0
-            row.progress_updated_at = now
-        db.flush()
+    if position:
+        db.execute(
+            pg_insert(VideoState)
+            .values(
+                user_id=user.id,
+                video_id=video_id,
+                position_seconds=position,
+                progress_updated_at=now,
+            )
+            .on_conflict_do_update(
+                constraint="uq_user_video",
+                set_={"position_seconds": position, "progress_updated_at": now, "updated_at": now},
+            )
+        )
         return
+    row = queries.video_state(db, user, video_id)
     if row is None:
-        row = VideoState(user_id=user.id, video_id=video_id)
-        db.add(row)
-    row.position_seconds = position
-    row.progress_updated_at = now
+        return
+    if row.status == "new":
+        db.delete(row)
+    else:
+        row.position_seconds = 0
+        row.progress_updated_at = now
     db.flush()
 
 
@@ -319,17 +357,25 @@ def create_playlist(db: Session, user: User, name: str) -> Playlist:
 
 
 def watch_later(db: Session, user: User) -> Playlist:
-    """The user's built-in Watch later playlist, created on first use."""
+    """The user's built-in Watch later playlist, created on first use. Created by the insert itself
+    (ON CONFLICT DO NOTHING on the one-per-user index) and then read: the first two saves at once
+    both read "none", and each used to make a list of its own."""
     pl = queries.watch_later(db, user)
     if pl is None:
-        pl = Playlist(
-            user_id=user.id,
-            name="Watch later",
-            kind="watch_later",
-            position=_next_playlist_position(db, user.id),
+        db.execute(
+            pg_insert(Playlist)
+            .values(
+                user_id=user.id,
+                name="Watch later",
+                kind="watch_later",
+                position=_next_playlist_position(db, user.id),
+            )
+            .on_conflict_do_nothing(
+                index_elements=["user_id"], index_where=text("kind = 'watch_later'")
+            )
         )
-        db.add(pl)
-        db.flush()  # the id, for the item the caller adds next
+        pl = queries.watch_later(db, user)
+        assert pl is not None  # inserted now, or committed by the request that won
     return pl
 
 
@@ -478,3 +524,192 @@ def detach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> No
     if row is not None:
         db.delete(row)
         db.flush()
+
+
+# --- Google Takeout --------------------------------------------------------------------------
+
+_TAKEOUT_BATCH = 1000
+# The advisory-lock namespace of one user's Takeout rows ("TO"), so a key cannot meet another lock's.
+_TAKEOUT_LOCK_NS = 0x544F
+
+
+def lock_takeout(db: Session, user: User) -> None:
+    """Hold the user's Takeout rows for the rest of the transaction. An upload's staging and an
+    import's settling each lock many of the same rows, one statement after another and in no common
+    order, so without one lock over both the two can deadlock. An un-mark takes it too: the import
+    reads the un-marks before it marks, and one landing between the two would be overwritten."""
+    db.execute(select(func.pg_advisory_xact_lock(_TAKEOUT_LOCK_NS, user.id)))
+
+
+def stage_takeout(
+    db: Session, user: User, kind: TakeoutKind, rows: dict[str, tuple[datetime, str | None]]
+) -> None:
+    """Keep the videos of one Takeout file, `{video id: (time, channel id)}`, for the import job.
+    A second upload of the same export, or a newer one, adds what is new: a video already kept keeps
+    the later time, and a row the import has settled keeps its outcome."""
+    lock_takeout(db, user)
+    values = [
+        {"user_id": user.id, "kind": kind, "video_id": vid, "at": at, "channel_id": channel}
+        for vid, (at, channel) in rows.items()
+    ]
+    for start in range(0, len(values), _TAKEOUT_BATCH):
+        stmt = pg_insert(TakeoutItem).values(values[start : start + _TAKEOUT_BATCH])
+        db.execute(
+            stmt.on_conflict_do_update(
+                constraint="uq_takeout_item",
+                set_={
+                    "at": func.greatest(TakeoutItem.at, stmt.excluded.at),
+                    "channel_id": func.coalesce(stmt.excluded.channel_id, TakeoutItem.channel_id),
+                },
+            )
+        )
+    db.flush()
+
+
+def _settle(db: Session, ids: list[int], outcome: TakeoutOutcome) -> None:
+    for start in range(0, len(ids), _TAKEOUT_BATCH):
+        db.execute(
+            update(TakeoutItem)
+            .where(TakeoutItem.id.in_(ids[start : start + _TAKEOUT_BATCH]))
+            .values(outcome=outcome)
+        )
+
+
+def apply_takeout_watched(db: Session, user: User) -> dict[str, int]:
+    """Mark watched what the user watched on YouTube, for the videos the catalog knows and has probed
+    for Shorts; the rest wait. `watched_at` is the Takeout time. A Short is not marked (swiping one
+    past is not watching it), and a state the user set here wins: watched or hidden, anything
+    changed after the Takeout time, or an un-mark since then (`VideoUnmark`). The upsert's own
+    condition is that rule, so a state set while the import runs wins too."""
+    lock_takeout(db, user)
+    short: list[int] = []
+    todo: dict[str, tuple[int, datetime]] = {}
+    ready = queries.takeout_watched_ready(db, user)
+    unmarked = queries.unmarked_at(db, user, [video_id for _, video_id, _, _ in ready])
+    kept: list[int] = []
+    for row_id, video_id, at, is_short in ready:
+        if is_short:
+            short.append(row_id)
+        elif video_id in unmarked and unmarked[video_id] >= at:
+            # Un-marked here since they watched it there: theirs, as a state they set is.
+            kept.append(row_id)
+        else:
+            todo[video_id] = (row_id, at)
+    marked: set[str] = set()
+    items = list(todo.items())
+    now = datetime.now(timezone.utc)
+    for start in range(0, len(items), _TAKEOUT_BATCH):
+        stmt = pg_insert(VideoState).values(
+            [
+                {"user_id": user.id, "video_id": vid, "status": "watched", "watched_at": at}
+                for vid, (_, at) in items[start : start + _TAKEOUT_BATCH]
+            ]
+        )
+        marked.update(
+            db.execute(
+                stmt.on_conflict_do_update(
+                    constraint="uq_user_video",
+                    set_={
+                        "status": "watched",
+                        "watched_at": stmt.excluded.watched_at,
+                        "updated_at": now,
+                    },
+                    where=(VideoState.status == "new")
+                    & (VideoState.updated_at < stmt.excluded.watched_at),
+                ).returning(VideoState.video_id)
+            ).scalars()
+        )
+    kept += [row_id for vid, (row_id, _) in items if vid not in marked]
+    _settle(db, [todo[vid][0] for vid in marked], TakeoutOutcome.WATCHED)
+    _settle(db, kept, TakeoutOutcome.KEPT)
+    _settle(db, short, TakeoutOutcome.SHORT)
+    db.flush()
+    return {"watched": len(marked), "kept": len(kept), "shorts": len(short)}
+
+
+def apply_takeout_watch_later(
+    db: Session, user: User, rows: list[tuple[int, str]], asked: set[str]
+) -> dict[str, int]:
+    """Put the videos of the user's waiting Watch later `rows`, (row id, video id) oldest first, into
+    their list here, after what it already holds. `asked` is the videos YouTube was just asked for: one
+    of those the catalog still lacks is gone from YouTube; any other unknown one keeps waiting."""
+    if not rows:
+        return {}
+    lock_takeout(db, user)
+    ids = [vid for _, vid in rows]
+    known = queries.known_video_ids(db, ids)
+    pl = watch_later(db, user)
+    # What add_item does a video at a time, for the whole list at once: one presence read and one
+    # insert, not four statements a video (a full list is thousands). Locked before the read, as there.
+    next_position = next_item_position(db, pl.id)
+    held = queries.in_playlist(db, pl.id, [vid for vid in ids if vid in known])
+    done: dict[TakeoutOutcome, list[int]] = {o: [] for o in TakeoutOutcome}
+    for row_id, vid in rows:
+        if vid not in known:
+            if vid in asked:
+                done[TakeoutOutcome.MISSING].append(row_id)
+        elif vid in held:
+            done[TakeoutOutcome.PRESENT].append(row_id)
+        else:
+            db.add(PlaylistItem(playlist_id=pl.id, video_id=vid, position=next_position))
+            next_position += 1
+            held.add(vid)
+            done[TakeoutOutcome.ADDED].append(row_id)
+    if done[TakeoutOutcome.ADDED]:
+        touch(pl)
+    for outcome, ids in done.items():
+        _settle(db, ids, outcome)
+    db.flush()
+    return {
+        "watch_later_added": len(done[TakeoutOutcome.ADDED]),
+        "watch_later_present": len(done[TakeoutOutcome.PRESENT]),
+        "watch_later_missing": len(done[TakeoutOutcome.MISSING]),
+    }
+
+
+# --- channels watched but not followed ---------------------------------------------------------
+
+
+def record_takeout_lengths(
+    db: Session, user: User, asked: list[str], lengths: dict[str, int | None]
+) -> None:
+    """Keep what YouTube said of the user's watched videos `asked` about: each one's length, or none
+    for a video it no longer has (`lengths` leaves it out). Every asked row is marked asked, so a
+    video YouTube lost is not asked about again."""
+    if not asked:
+        return
+    lock_takeout(db, user)  # these rows are an upload's to update too
+    now = datetime.now(timezone.utc)
+    table = cast(Table, TakeoutItem.__table__)
+    db.execute(
+        table.update()
+        .where(
+            owned_by(TakeoutItem.user_id, user),
+            table.c.kind == TakeoutKind.WATCHED,
+            table.c.video_id == bindparam("vid"),
+        )
+        .values(looked_up_at=now, duration_seconds=bindparam("seconds")),
+        [{"vid": vid, "seconds": lengths.get(vid)} for vid in asked],
+    )
+    db.flush()
+
+
+def dismiss_watched_channel(db: Session, user: User, channel_id: str) -> None:
+    """Put a channel out of the user's "watched but not followed" list (again is a no-op)."""
+    db.execute(
+        pg_insert(TakeoutChannelDismissal)
+        .values(user_id=user.id, channel_id=channel_id)
+        .on_conflict_do_nothing(constraint="uq_takeout_channel_dismissal")
+    )
+    db.flush()
+
+
+def restore_watched_channel(db: Session, user: User, channel_id: str) -> None:
+    """Let a channel the user put out of the list back on it."""
+    db.execute(
+        delete(TakeoutChannelDismissal).where(
+            owned_by(TakeoutChannelDismissal.user_id, user),
+            TakeoutChannelDismissal.channel_id == channel_id,
+        )
+    )
+    db.flush()

backend/app/reader/takeout.py

@@ -0,0 +1,497 @@
+"""A Google Takeout export, imported: what the user watched on YouTube becomes watched here, and their
+YouTube Watch later list joins theirs. The YouTube API reads neither of them, so the export is the
+only way in.
+
+Two files of the export are read, each uploaded on its own: `history/watch-history.json` and
+`playlists/Watch later-videos.csv`. Their videos are kept (`TakeoutItem`) and the import job applies
+what it can; a video the catalog does not know yet waits for it. The watched videos' channels also
+make the list of channels the user watches but does not follow (`queries.watched_channels`): the
+job asks YouTube how long those videos are, and what the channels are, for it.
+
+The format, measured on a real export (2026-10): the history is a JSON list, one entry per view,
+newest first. A view of a video has a `titleUrl` with `watch?v=<id>` (on www.youtube.com or
+music.youtube.com) and a `time`, with or without milliseconds. The channel is the first `subtitles`
+entry's url, absent for a video since removed. An ad is an entry whose `details` say "From Google
+Ads". The title is in the account's language, so nothing here reads it. A Short is a plain
+`watch?v=` view, told apart only once the catalog has probed the video. The Watch later csv is a
+header row (in the account's language) and `<id>,<ISO time added>` rows.
+"""
+
+import codecs
+import csv
+import io
+import json
+import re
+import threading
+from collections.abc import Iterator
+from contextlib import contextmanager
+from dataclasses import dataclass, field
+from datetime import datetime
+from itertools import chain
+
+from sqlalchemy.orm import Session
+
+from app import logs, quota
+from app.bodylimit import TAKEOUT_MAX_BYTES
+from app.db import SessionLocal
+from app.models import User
+from app.ratelimit import RateLimiter
+from app.reader import queries, service
+from app.reader.models import TakeoutKind
+from app.sync.runner import get_service_user
+from app.sync.subscriptions import describe_channels
+from app.sync.videos import ensure_videos, parse_iso8601_duration
+from app.youtube.client import YouTubeClient, YouTubeError
+
+log = logs.get_logger(__name__)
+
+# The largest file the upload takes (its edge cap and the route's own are one number).
+MAX_BYTES = TAKEOUT_MAX_BYTES
+# The most videos a Watch later file may hold: YouTube's own cap on one playlist. Each unknown one is
+# looked up on the shared API budget, so a longer list is not one YouTube made.
+WATCH_LATER_MAX = 5000
+# The most Takeout videos one user keeps: twenty times the real export measured here (~9,000 videos
+# across both files), so it only stops uploads of made-up ones piling up for good.
+ROWS_MAX = 200_000
+
+_VIDEO_ID = re.compile(r"^[A-Za-z0-9_-]{11}$")
+_WATCH_URL = re.compile(
+    r"^https?://(?:www\.|music\.|m\.)?youtube\.com/watch\?v=([A-Za-z0-9_-]{11})"
+)
+_CHANNEL_URL = re.compile(r"/channel/(UC[A-Za-z0-9_-]{22})")
+
+
+class TakeoutError(ValueError):
+    """The file is not one the import reads; `reason` is the client's word for why."""
+
+    def __init__(self, reason: str):
+        super().__init__(reason)
+        self.reason = reason
+
+
+@dataclass
+class Parsed:
+    """One file's videos, `{video id: (time, channel id)}`, and what it left out, counted."""
+
+    kind: TakeoutKind
+    videos: dict[str, tuple[datetime, str | None]] = field(default_factory=dict)
+    entries: int = 0
+    ads: int = 0
+    other: int = 0  # not a view of a video (a post, a story) or a video with no link left
+    malformed: int = 0
+
+
+def _time(raw: object) -> datetime | None:
+    if not isinstance(raw, str):
+        return None
+    try:
+        at = datetime.fromisoformat(raw.replace("Z", "+00:00"))
+    # silent-ok: the row is counted as malformed, and the count is in the upload's answer.
+    except ValueError:
+        return None
+    return at if at.tzinfo is not None else None
+
+
+def _keep(parsed: Parsed, vid: str, at: datetime, channel: str | None) -> None:
+    old = parsed.videos.get(vid)
+    if old is None or at > old[0]:
+        parsed.videos[vid] = (at, channel or (old[1] if old else None))
+
+
+def _history(data: Iterator[object]) -> Parsed:
+    parsed = Parsed(TakeoutKind.WATCHED)
+    for entry in data:
+        parsed.entries += 1
+        if not isinstance(entry, dict):
+            parsed.malformed += 1
+            continue
+        details = entry.get("details")
+        if isinstance(details, list) and any(
+            isinstance(d, dict) and d.get("name") == "From Google Ads" for d in details
+        ):
+            parsed.ads += 1
+            continue
+        url = entry.get("titleUrl")
+        match = _WATCH_URL.match(url) if isinstance(url, str) else None
+        if match is None:
+            parsed.other += 1
+            continue
+        at = _time(entry.get("time"))
+        if at is None:
+            parsed.malformed += 1
+            continue
+        channel = None
+        subtitles = entry.get("subtitles")
+        if isinstance(subtitles, list) and subtitles and isinstance(subtitles[0], dict):
+            found = _CHANNEL_URL.search(str(subtitles[0].get("url", "")))
+            channel = found.group(1) if found else None
+        _keep(parsed, match.group(1), at, channel)
+    return parsed
+
+
+def _watch_later(text: str) -> Parsed:
+    parsed = Parsed(TakeoutKind.WATCH_LATER)
+    rows = (row for row in csv.reader(io.StringIO(text)) if any(cell.strip() for cell in row))
+    first = next(rows, None)
+    if first is None or len(first) < 2:
+        raise TakeoutError("not_takeout")
+    # The first row is the header, in the account's language, unless it is already a video.
+    for row in chain([first], rows) if _VIDEO_ID.match(first[0].strip()) else rows:
+        parsed.entries += 1
+        vid = row[0].strip()
+        at = _time(row[1].strip()) if len(row) > 1 else None
+        if not _VIDEO_ID.match(vid) or at is None:
+            parsed.malformed += 1
+            continue
+        _keep(parsed, vid, at, None)
+        if len(parsed.videos) > WATCH_LATER_MAX:
+            raise TakeoutError("too_many")
+    return parsed
+
+
+def _skip(text: str, at: int) -> int:
+    """The first index at or after `at` that is not JSON whitespace."""
+    while at < len(text) and text[at] in " \t\n\r":
+        at += 1
+    return at
+
+
+# How much of the file is decoded at a time. Decoded whole, the text costs up to four bytes a
+# character (one emoji in a title is enough), several times the file on top of its bytes.
+_WINDOW = 1024 * 1024
+
+
+class _Window:
+    """The file's text, decoded a window at a time: `text` holds what is not read yet from `at` on.
+    The bytes are the caller's, so only the window is this reader's memory."""
+
+    def __init__(self, body: bytes, start: int):
+        self._body, self._next = body, start
+        self._decoder = codecs.getincrementaldecoder("utf-8")()  # strict: bad UTF-8 raises
+        self.text, self.at = "", 0
+
+    @property
+    def done(self) -> bool:
+        return self._next >= len(self._body)
+
+    def more(self, at_least: int = 0) -> None:
+        """Decode the next window (at least `at_least` bytes), dropping what is read already."""
+        self.text, self.at = self.text[self.at :], 0
+        size = max(_WINDOW, at_least)
+        chunk = self._body[self._next : self._next + size]
+        self._next += len(chunk)
+        self.text += self._decoder.decode(chunk, final=self.done)
+
+    def skip(self) -> None:
+        """Past the whitespace, reading on as far as it goes."""
+        while True:
+            self.at = _skip(self.text, self.at)
+            if self.at < len(self.text) or self.done:
+                return
+            self.more()
+
+    def startswith(self, char: str) -> bool:
+        self.skip()
+        return self.text.startswith(char, self.at)
+
+
+def _decode(decoder: json.JSONDecoder, text: str, at: int) -> tuple[object, int] | None:
+    """The JSON value at `at` and where it ends, or None where none reads (yet)."""
+    try:
+        return decoder.raw_decode(text, at)
+    # silent-ok: not a failure here — the caller reads on, or refuses the file once it has all.
+    except json.JSONDecodeError:
+        return None
+
+
+def _json_list(body: bytes, start: int) -> Iterator[object]:
+    """The entries of the JSON list at byte `start` (its "["), one at a time: the whole tree is never
+    built, nor the whole text, so a file of a million tiny entries costs a window and one entry, not
+    a million dicts at once."""
+    decoder = json.JSONDecoder()
+    window = _Window(body, start + 1)  # past the "[" the caller saw
+    if window.startswith("]"):
+        window.at += 1
+    else:
+        while True:
+            window.skip()
+            read = _decode(decoder, window.text, window.at)
+            # An entry cut by the window's edge fails, or (a number) reads short: read on and try
+            # again, with a window twice what is left, so one entry larger than any window is read
+            # in a few tries, not one a window.
+            if read is None or (read[1] == len(window.text) and not window.done):
+                if window.done:
+                    raise ValueError("not a JSON list")
+                window.more(2 * (len(window.text) - window.at))
+                continue
+            entry, window.at = read
+            yield entry
+            if window.startswith(","):
+                window.at += 1
+            elif window.startswith("]"):
+                window.at += 1
+                break
+            else:
+                raise ValueError("not a JSON list")
+    window.skip()
+    if window.at != len(window.text):
+        raise ValueError("text after the JSON list")
+
+
+# The largest Watch later file read: YouTube's longest list is some 200 KB of rows, so a file ten
+# times that is not one — and, unlike the history, it is decoded whole.
+CSV_MAX_BYTES = 2 * 1024 * 1024
+_BOM = codecs.BOM_UTF8
+
+
+def parse(body: bytes) -> Parsed:
+    """Read an uploaded Takeout file: the watch history (JSON) or the Watch later list (csv), told
+    apart by their content, not their name. Raises `TakeoutError` for anything else, and for a file
+    with nothing to import in it."""
+    start = len(_BOM) if body.startswith(_BOM) else 0
+    while start < len(body) and body[start] in b" \t\n\r\f\v":
+        start += 1
+    first = body[start : start + 1]
+    if first in (b"{", b"<"):
+        # Another JSON file of the export, or the history's HTML variant.
+        raise TakeoutError("not_takeout")
+    if first != b"[" and len(body) - start > CSV_MAX_BYTES:
+        raise TakeoutError("too_many")
+    try:
+        # A bad UTF-8 sequence is a ValueError too (UnicodeDecodeError), read below as not a file.
+        parsed = (
+            _history(_json_list(body, start))
+            if first == b"["
+            else _watch_later(body[start:].decode("utf-8"))
+        )
+    # A file the import cannot read, however it fails: bad JSON (a ValueError, as is a number too
+    # long to convert), nesting deeper than the decoder recurses, a csv field past csv's limit.
+    except (ValueError, RecursionError, csv.Error) as exc:
+        if isinstance(exc, TakeoutError):
+            raise
+        raise TakeoutError("not_takeout") from None
+    if not parsed.videos:
+        raise TakeoutError("no_videos")
+    return parsed
+
+
+# One import per user at a time: a second file uploaded while the first one's run is under way runs
+# after it (the scheduler's ad-hoc queue), but the Shorts job settles the same users' rows on a
+# thread of its own, and the two would settle the same rows. One API process runs the scheduler, so a
+# lock in this module holds it (as the channel backfill's does).
+_user_locks: dict[int, threading.Lock] = {}
+_user_locks_guard = threading.Lock()
+
+
+@contextmanager
+def _one_import(user_id: int) -> Iterator[None]:
+    with _user_locks_guard:
+        lock = _user_locks.setdefault(user_id, threading.Lock())
+    with lock:
+        yield
+
+
+def run_target(user_id: int) -> str:
+    """The import run's target, `job_runs.target`: one user's import."""
+    return f"user:{user_id}"
+
+
+def target_title(db: Session, target: str) -> str | None:
+    """Who an import run's target is, for the admin's run lists: the user's name, else their email."""
+    _, _, raw = target.partition("user:")
+    user = db.get(User, int(raw)) if raw.isdigit() else None
+    return (user.display_name or user.email) if user is not None else None
+
+
+class TooMany(Exception):
+    """The upload would take the user past `ROWS_MAX` kept videos."""
+
+
+def stage(db: Session, user: User, parsed: Parsed) -> None:
+    """Keep an uploaded file's videos for the import, unless that would take the user past
+    `ROWS_MAX` (`TooMany`). The caller commits."""
+    service.lock_takeout(db, user)  # the count and the insert are one user's, not two uploads' each
+    if (
+        queries.takeout_rows_besides(db, user, parsed.kind, parsed.videos) + len(parsed.videos)
+        > ROWS_MAX
+    ):
+        raise TooMany
+    service.stage_takeout(db, user, parsed.kind, parsed.videos)
+
+
+# How many unknown Watch later videos one user may have looked up on the shared API budget in a day:
+# two full Watch later lists (two units a hundred ids). The rest wait for the next day's runs. Both
+# shares pace the work, they do not bound its cost: they are held in memory, so a restart starts
+# them afresh, which only brings the rest forward — every video is asked about once (a looked-up
+# row is marked), and `quota.backfill_reserve_reached` still guards the shared budget.
+_lookups = RateLimiter(max_events=2 * WATCH_LATER_MAX, window_seconds=24 * 3600)
+# How many watched videos' lengths one user may ask about in a day for the "watched but not followed"
+# list (a unit a fifty). A share of its own: a large history asks for thousands, and on the Watch
+# later share it would leave the export's Watch later file waiting a day behind it.
+_lengths = RateLimiter(max_events=2 * WATCH_LATER_MAX, window_seconds=24 * 3600)
+
+
+def _put_in_watch_later(db: Session, user: User) -> dict:
+    """Put the user's waiting Watch later videos into theirs here: the ones the catalog knows at once,
+    the others once YouTube was asked for them — when a YouTube-connected account can ask, today's
+    budget is above the backfill reserve, and the user's own daily share of lookups allows it. A video
+    not asked about yet waits for a later run. The lookup commits what it fetched; the caller commits
+    the rest."""
+    waiting = queries.takeout_waiting(db, user, TakeoutKind.WATCH_LATER)
+    if not waiting:
+        return {}
+    ids = [vid for _, vid in waiting]
+    known = queries.known_video_ids(db, ids)
+    ask = [vid for vid in ids if vid not in known][:WATCH_LATER_MAX]
+    asked: set[str] = set()
+    fetcher = get_service_user(db) if ask else None
+    if (
+        fetcher is not None
+        and not quota.backfill_reserve_reached()
+        and _lookups.room(str(user.id)) >= len(ask)
+    ):
+        with quota.attribute(user.id, quota.QuotaAction.VIDEOS_LOOKUP):
+            with YouTubeClient(db, fetcher) as yt:
+                ensure_videos(db, yt, ask)
+        # Counted once done: a lookup that failed spent none of the share (one run per user at a
+        # time, so the room asked above is still there).
+        _lookups.allow(str(user.id), cost=len(ask))
+        asked = set(ask)
+    # Only the rows read above: one a second upload staged meanwhile was never asked about.
+    return service.apply_takeout_watch_later(db, user, waiting, asked)
+
+
+def _ask_lengths(db: Session, user: User) -> dict:
+    """Ask YouTube how long the user's watched videos are that the "watched but not followed" list
+    still needs, and keep the answer — on the same terms as the Watch later lookups, on a daily
+    share of its own. What is not asked yet (no YouTube-connected account, the reserve reached, the
+    share spent) waits for a later run, and the list says it is not complete. A YouTube failure is
+    logged and left for a later run too: it is no reason to fail the import it follows. The caller
+    commits."""
+    ask = queries.takeout_lengths_to_ask(db, user)
+    fetcher = get_service_user(db) if ask else None
+    if fetcher is None or quota.backfill_reserve_reached():
+        return {}
+    ask = ask[: int(_lengths.room(str(user.id)))]
+    if not ask:
+        return {}
+    try:
+        with quota.attribute(user.id, quota.QuotaAction.VIDEOS_LOOKUP):
+            with YouTubeClient(db, fetcher) as yt:
+                items = yt.get_videos(ask)
+    except YouTubeError as exc:
+        log.warning(
+            "Takeout: asking the watched videos' lengths failed: %s",
+            exc,
+            extra=logs.event("takeout.lengths.failed", user_id=user.id, reason=exc.reason),
+        )
+        return {}
+    _lengths.allow(str(user.id), cost=len(ask))
+    lengths = {
+        it["id"]: parse_iso8601_duration(it.get("contentDetails", {}).get("duration"))
+        for it in items
+        if it.get("id")
+    }
+    service.record_takeout_lengths(db, user, ask, lengths)
+    return {"lengths_asked": len(ask)}
+
+
+def _describe_channels(db: Session, user: User) -> dict:
+    """Bring the channels the user's list holds into the catalog with their details (name, picture,
+    description), the ones it lacks: without them the list cannot show a channel. A YouTube failure
+    is logged and left for a later run. The caller commits."""
+    need = queries.watched_channels_to_describe(db, user)
+    fetcher = get_service_user(db) if need else None
+    if fetcher is None or quota.backfill_reserve_reached():
+        return {}
+    try:
+        with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
+            with YouTubeClient(db, fetcher) as yt:
+                describe_channels(db, yt, need)
+    except YouTubeError as exc:
+        db.rollback()
+        log.warning(
+            "Takeout: describing the watched channels failed: %s",
+            exc,
+            extra=logs.event("takeout.channels.failed", user_id=user.id, reason=exc.reason),
+        )
+        return {}
+    return {"channels_described": len(need)}
+
+
+def _channel_list(db: Session, user: User) -> dict:
+    """The "watched but not followed" list's part of a run: the lengths, landed, then the channels.
+    Commits each."""
+    result = _ask_lengths(db, user)
+    db.commit()
+    result |= _describe_channels(db, user)
+    db.commit()
+    return result
+
+
+def run_import(user_id: int) -> dict:
+    """The import job: apply what the user's kept Takeout rows can say now, as one recorded run.
+    The watched part commits before the Watch later part asks YouTube for the videos the catalog
+    lacks, so a YouTube failure there loses nothing already marked; the "watched but not followed"
+    list's lookups come last. Returns the counts."""
+    with _one_import(user_id), SessionLocal() as db:
+        user = db.get(User, user_id)
+        if user is None:
+            return {"skipped": "no user"}
+        result = service.apply_takeout_watched(db, user)
+        db.commit()
+        result |= _put_in_watch_later(db, user)
+        db.commit()
+        result |= _channel_list(db, user)
+        outcomes = queries.takeout_outcomes(db, user)
+        result["waiting"] = outcomes.get((TakeoutKind.WATCHED, None), 0)
+        if outcomes.get((TakeoutKind.WATCH_LATER, None)):
+            result["watch_later_waiting"] = outcomes[(TakeoutKind.WATCH_LATER, None)]
+        return result
+
+
+def apply_waiting(db: Session) -> dict:
+    """Settle what every user's import left waiting and can be settled now, one commit each: a
+    watched video the catalog has come to know and the Shorts probe has cleared (by its job or a
+    search's), the Watch later videos a run could not put in yet, and what the "watched but not
+    followed" list still needs asked of YouTube. Run after each run of the
+    `shorts` job, which is what clears the most of them. Returns its counts, or an empty dict when
+    nothing was waiting that could move."""
+    marked = added = asked = failed = 0
+    ready = set(queries.takeout_users_ready(db))
+    later = set(queries.takeout_users_watch_later_waiting(db))
+    lengths = set(queries.takeout_users_with_list_work(db))
+    for user_id in sorted(ready | later | lengths):
+        user = db.get(User, user_id)
+        if user is None:
+            continue
+        # One user's failure (a lookup YouTube refused) is theirs: it neither fails the job that
+        # runs this nor keeps the users after them waiting. What it did not settle waits.
+        try:
+            with _one_import(user_id):
+                if user_id in ready:
+                    marked += service.apply_takeout_watched(db, user)["watched"]
+                    db.commit()
+                if user_id in later:
+                    added += _put_in_watch_later(db, user).get("watch_later_added", 0)
+                    db.commit()
+                if user_id in lengths:
+                    asked += _channel_list(db, user).get("lengths_asked", 0)
+        except Exception:
+            db.rollback()
+            failed += 1
+            log.exception(
+                "Takeout settling failed for user %s",
+                user_id,
+                extra=logs.event("takeout.settle.failed", user_id=user_id),
+            )
+    out = {}
+    if ready:
+        out["takeout_watched"] = marked
+    if added:
+        out["takeout_watch_later_added"] = added
+    if asked:
+        out["takeout_lengths_asked"] = asked
+    if failed:
+        out["takeout_failed"] = failed
+    return out

backend/app/routes/_common.py

@@ -89,8 +89,9 @@ def owned_or_404(
 ) -> M:
     """Fetch a row by primary key that `user` owns, or raise 404 (`lock`: see `get_or_404`). A row owned by someone
     else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger.
-    It replaced the near-identical per-route ownership helpers, and is called directly — a thin
-    `_own_x` wrapper around it is the ad-hoc shape CLAUDE.md's patterns table rules out."""
+    A core route calls it directly. A unit's route cannot (it may not name a unit's model as a value):
+    it asks its unit's `queries.own_x`, which goes through the same `owned_row`, and answers
+    `not_found` itself."""
     obj = owned_row(db, model, id_, user, owner_attr=owner_attr, lock=lock)
     if obj is None:
         raise not_found(what)
@@ -123,6 +124,14 @@ async def capped_body(request: Request, limit: int) -> bytes:
     return bytes(body)
 
 
+async def drain_body(request: Request) -> None:
+    """Read the rest of the body to nowhere, before a refusal is raised: answered mid-upload, the
+    client reports a network failure, not the refusal (`capped_stream_to` says why). `bodylimit`
+    bounds how much there can be."""
+    async for _ in request.stream():
+        pass
+
+
 class ByteSink(Protocol):
     """Where `capped_stream_to` writes: a file, or anything that takes the bytes on their way to one."""
 

backend/app/routes/auth.py

@@ -5,9 +5,10 @@ helpers) stays in app/auth.py, imported below."""
 
 import secrets
 from datetime import datetime, timedelta, timezone
-from typing import NamedTuple
+from typing import Literal, NamedTuple
 from urllib.parse import urlencode
 
+from authlib.common.security import generate_token
 from authlib.integrations.starlette_client import OAuth, OAuthError
 from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
 from fastapi.responses import JSONResponse, RedirectResponse
@@ -25,10 +26,11 @@ from app import sysconfig
 from app.auth import (
     _YOUTUBE_SCOPES,
     BASE_SCOPES,
-    MAX_SESSION_ACCOUNTS,
+    EXPECTED_ACCOUNT_HEADER,
     PASSWORD_MIN_LEN,
     READ_SCOPES,
     WRITE_SCOPES,
+    built_for_another_account,
     current_user,
     env_admin_canonicals,
     env_allowed_canonicals,
@@ -37,7 +39,9 @@ from app.auth import (
     is_allowed,
     log,
     operator_contact,
-    resolved_user_id,
+    optional_current_user,
+    session_user_id,
+    stamp_session,
 )
 from app.config import settings
 from app.db import SessionLocal, get_db
@@ -320,6 +324,25 @@ class SetPasswordIn(BaseModel):
     current_password: str | None = None
 
 
+# A connect in flight, in the session: `{state, uid, explicit}`. It names the OAuth state of the flow
+# that set it, because the session is the browser's, not the flow's: a callback coming back with
+# another state (an older flow, which authlib then refuses) must not spend the marker of the connect
+# still open, or that connect would return as a plain sign-in — signing someone else in, or storing
+# a YouTube grant on whichever account the Google identity already belongs to.
+_LINK_MARKER = "oauth_link"
+
+# Where a navigation from a tab showing another account than the one signed in lands: the app, which
+# boots as the account signed in now and says why (the frontend's `ACCOUNT_CHANGED_FLAG`).
+ACCOUNT_CHANGED_LANDING = "/?account=changed"
+
+
+def _take_link_marker(request: Request) -> dict | None:
+    marker = request.session.get(_LINK_MARKER)
+    if not isinstance(marker, dict) or marker.get("state") != request.query_params.get("state"):
+        return None
+    return request.session.pop(_LINK_MARKER)
+
+
 @router.get("/login")
 async def login(request: Request, db: Session = Depends(get_db)):
     # access_type=offline ensures a refresh_token on first authorization. We avoid
@@ -330,6 +353,10 @@ async def login(request: Request, db: Session = Depends(get_db)):
         # Google sign-in isn't configured on this instance — land back on the login page (the UI
         # hides the button, so this is just a safety net for a stale/direct hit).
         return RedirectResponse(url="/")
+    # A sign-in, never a link. Starting it retires every earlier flow's state (authlib keeps only the
+    # newest), so a connect's marker left from before — abandoned at Google, or still open in
+    # another tab — names a flow that can no longer finish.
+    request.session.pop(_LINK_MARKER, None)
     return await client.authorize_redirect(
         request,
         _oauth_redirect_uri(request),
@@ -350,24 +377,23 @@ async def callback(
     if client is None:
         return RedirectResponse(url="/")
     # Linking mode: an already-authenticated account is attaching this Google identity (a
-    # password account enabling YouTube or connecting SSO). The markers are set by /auth/link and
-    # /auth/upgrade; when present we attach to the current user instead of identifying by sub.
-    # Pop them BEFORE the token exchange: if the user aborts Google's consent screen the exchange
-    # throws below, and a marker left behind would silently turn the NEXT clean sign-in into a link
-    # attempt. Popping up front means an aborted link flow leaves no residue in the session.
-    link_uid = request.session.pop("oauth_link_uid", None)
-    link_explicit = request.session.pop("oauth_link_explicit", False)
+    # password account enabling YouTube or connecting SSO). The marker is set by
+    # /auth/google/connect; when it is this flow's we attach to the current user instead of
+    # identifying by sub. Taken BEFORE the token exchange: if the user aborts Google's consent screen
+    # the exchange throws below, and a marker left behind would silently turn the NEXT clean sign-in
+    # into a link attempt. A connect that never comes back here at all is cleared by /auth/login.
+    link = _take_link_marker(request)
     try:
         token = await client.authorize_access_token(request)
     except OAuthError as exc:
         # The user cancelled or denied consent at Google — land back with a friendly notice instead
-        # of a raw 400 JSON error page. An already-signed-in user (adding another account via
-        # /auth/login, or linking/upgrading via /auth/link|upgrade) never sees the pre-auth login
-        # banner, so surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on
-        # the login page with the oauth_cancelled banner instead. Keyed on the live session, not on
-        # link_uid, so the add-account path (no marker) is covered too.
+        # of a raw 400 JSON error page. An already-signed-in user (signing in again via /auth/login,
+        # or connecting via /auth/google/connect) never sees the pre-auth login banner, so
+        # surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on the login page
+        # with the oauth_cancelled banner instead. Keyed on the live session, not on link_uid, so a
+        # sign-in started while signed in (no marker) is covered too.
         log.info("Google OAuth cancelled/failed: %s", exc.error)
-        if request.session.get("user_id") is not None:
+        if session_user_id(request) is not None:
             return RedirectResponse(url="/?oauth=cancelled")
         return RedirectResponse(url="/?login=oauth_cancelled")
 
@@ -378,8 +404,8 @@ async def callback(
     sub = userinfo["sub"]
     email = (userinfo.get("email") or "").lower()
 
-    if link_uid is not None:
-        return _complete_link(request, db, link_uid, link_explicit, sub, userinfo, token)
+    if link is not None:
+        return _complete_link(request, db, link["uid"], link["explicit"], sub, userinfo, token)
 
     if not is_allowed(db, email):
         # No uid to name — having no account here is the branch. And the address is left out on
@@ -580,19 +606,16 @@ def _complete_link(
     token: dict,
 ):
     """Attach the just-authorized Google identity to the already-signed-in account that started
-    the link/upgrade. No is_allowed gate — they're already an active user. Refuses to hijack a
+    the connect. No is_allowed gate — they're already an active user. Refuses to hijack a
     Google identity owned by another account, or to silently swap a different linked identity.
-    `explicit` (the popped oauth_link_explicit marker) distinguishes a "Connect Google" click from
-    a YouTube upgrade — the caller pops it up front so an aborted flow leaves nothing behind."""
-    # SEC-11: the marker must name an account signed into THIS browser — the session default OR any
-    # account in the wallet — not only the default. A tab acting as a secondary account links Google to
-    # ITSELF (the marker was set from current_user, which resolves the ?account= it navigated with), so
-    # accepting wallet membership stops the identity/token landing on the default account A instead of B.
-    accounts = request.session.get("account_ids") or []
-    if link_uid != request.session.get("user_id") and link_uid not in accounts:
-        return RedirectResponse(url="/?link=error")
-    user = db.get(User, link_uid)
-    if user is None:
+    `explicit` (from the link marker) distinguishes a "Connect Google" click from a YouTube grant —
+    the caller takes the marker up front so an aborted flow leaves nothing behind."""
+    # The marker must name the account still signed into THIS browser, and that session must still
+    # hold: a sign-in as someone else between the start and the callback replaced it (the identity
+    # must not land on the newcomer), and a session revoked meanwhile ("log out everywhere", a
+    # password change, a suspension) must not finish what it started.
+    user = optional_current_user(request, db)
+    if user is None or user.id != link_uid:
         return RedirectResponse(url="/?link=error")
     other = db.execute(
         select(User).where(User.google_sub == sub, User.id != user.id)
@@ -612,45 +635,31 @@ def _complete_link(
     _store_token(db, user, token)
     db.commit()
     log.info("Google linked: uid=%s scopes=%s", user.id, token.get("scope"))
-    # An explicit "Connect Google" lands on Settings with a confirmation; a YouTube upgrade
+    # An explicit "Connect Google" lands on Settings with a confirmation; a YouTube grant
     # (wizard / Settings access rows) returns to the app so the onboarding flow resumes as before.
     return RedirectResponse(url="/?link=ok" if explicit else "/")
 
 
 @router.post("/logout")
 async def logout(request: Request):
-    """Sign the requesting tab's active account out of this browser (removing it from the wallet).
-    Per-tab aware: the account is taken from the X-Siftlode-Account header when present, so logging
-    out in one tab doesn't disturb another tab running a different account. If the account being
-    removed was the session default, promote the most recent remaining account (or clear the
-    session when none remain)."""
-    # A tab still pinned to an account already signed out in another tab resolves to nobody, so it
-    # signs nobody out — it used to fall back to the default and sign THAT account out in its place.
-    active, is_default = resolved_user_id(request)
-    remaining = [a for a in (request.session.get("account_ids") or []) if a != active]
-    request.session["account_ids"] = remaining
-    if is_default:
-        if remaining:
-            request.session["user_id"] = remaining[-1]
-            return JSONResponse({"ok": True, "switched": True})
-        request.session.clear()
-        return JSONResponse({"ok": True, "switched": False})
-    # A non-default (per-tab) account signed out: it's gone from the wallet and the session default
-    # is untouched. The tab drops its own override and falls back to the default on reload.
-    return JSONResponse({"ok": True, "switched": False})
+    """Sign this browser out: the session holds one account, and it ends — unless the tab asking was
+    built for another account than the one signed in now (a sign-in as someone else in another tab):
+    its Sign out would end THAT person's session (see `EXPECTED_ACCOUNT_HEADER`)."""
+    signed_in = session_user_id(request)
+    if signed_in is not None and built_for_another_account(
+        signed_in, request.headers.get(EXPECTED_ACCOUNT_HEADER)
+    ):
+        raise refused(401, "account_changed")
+    request.session.clear()
+    return JSONResponse({"ok": True})
 
 
 def _establish_session(request: Request, user: User) -> None:
     """Sign `user` in for this browser session — the ONE sign-in path, shared by the Google callback
-    and password login, so both join the same account switcher and both stamp `last_login_at` (the
-    caller commits). Multi-session: every account that authenticated in this browser is remembered so
-    the user can switch between them; only accounts that actually proved themselves HERE land in that
-    wallet (it is the proof they may be switched to)."""
-    accounts = [a for a in (request.session.get("account_ids") or []) if a != user.id]
-    accounts.append(user.id)
-    request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
+    and password login, so both stamp `last_login_at` (the caller commits). One account per session:
+    the sign-in replaces whoever was signed in here before."""
     request.session["user_id"] = user.id
-    _remember_epoch(request, user)
+    stamp_session(request, user)
     user.last_login_at = datetime.now(timezone.utc)
 
 
@@ -1135,15 +1144,6 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
     return {"ok": True}
 
 
-def _remember_epoch(request: Request, user: User) -> None:
-    """Record this account's current `session_epoch` in the signed cookie (SA4), so `current_user`
-    can later reject the cookie if the account's epoch was bumped (password reset/change, or
-    "log out everywhere"). Per-account so one account's revocation doesn't evict the others."""
-    epochs = dict(request.session.get("epochs") or {})
-    epochs[str(user.id)] = user.session_epoch
-    request.session["epochs"] = epochs
-
-
 def bump_session_epoch(db: Session, user: User) -> None:
     """Invalidate every EXISTING signed cookie for this account by advancing its epoch (SA4). Cookies
     minted before the bump carry a now-stale epoch and are rejected by `current_user`. The account's
@@ -1220,35 +1220,11 @@ def logout_others(
     # session at a newer epoch than the DB (see set_password).
     bump_session_epoch(db, user)
     db.commit()
-    _remember_epoch(request, user)
+    stamp_session(request, user)
     log.info("Logout-others: uid=%s (epoch=%s)", user.id, user.session_epoch)
     return {"ok": True}
 
 
-@router.get("/link")
-async def link_google(
-    request: Request,
-    user: User = Depends(current_user),
-    db: Session = Depends(get_db),
-):
-    """Start linking a Google account to the signed-in (e.g. password) account. Requests only
-    the identity scopes — YouTube access is granted separately via /auth/upgrade. The callback
-    sees `oauth_link_uid` and attaches the identity instead of creating a new account."""
-    client = google_oauth(db)
-    if client is None:
-        return RedirectResponse(url="/?link=error")
-    request.session["oauth_link_uid"] = user.id
-    request.session["oauth_link_explicit"] = True
-    return await client.authorize_redirect(
-        request,
-        _oauth_redirect_uri(request),
-        access_type="offline",
-        prompt="select_account",
-        include_granted_scopes="true",
-        scope=BASE_SCOPES,
-    )
-
-
 @router.post("/set-password")
 def set_password(
     body: SetPasswordIn,
@@ -1272,35 +1248,57 @@ def set_password(
     # can't leave the cookie at a newer epoch than the DB (which would wrongly 401 THIS session).
     bump_session_epoch(db, user)
     db.commit()
-    _remember_epoch(request, user)
+    stamp_session(request, user)
     log.info("Password set/changed: uid=%s", user.id)
     return {"ok": True}
 
 
-@router.get("/upgrade")
-async def upgrade(
+# What each Google connect asks Google for. `identity` attaches a Google sign-in to the account
+# (Settings → Connect Google) and lets the person pick which Google account; `read` and `write` grant
+# YouTube access (the onboarding wizard, the Settings access rows) and always show the consent screen,
+# so a scope granted earlier and since revoked at Google is asked for again.
+_CONNECT: dict[str, tuple[str, str]] = {
+    "identity": (BASE_SCOPES, "select_account"),
+    "read": (READ_SCOPES, "consent"),
+    "write": (WRITE_SCOPES, "consent"),
+}
+
+
+@router.get("/google/connect")
+async def google_connect(
     request: Request,
-    access: str = "read",
+    access: Literal["identity", "read", "write"],
+    expect: str | None = None,
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
 ):
-    """Incremental consent for the onboarding wizard. `access=read` grants YouTube
-    read-only (enough to build the feed); `access=write` additionally grants management
-    (unsubscribe). The shared callback stores whatever Google returns, so `can_read` /
-    `can_write` flip on once granted. Anything other than an explicit `write` defaults to
-    the least-privileged read scope."""
+    """Attach Google to the signed-in account: its identity, or YouTube read/write access on top.
+    The shared callback sees the link marker and attaches what Google returns to THIS account —
+    without it a password account (no google_sub) would get a brand-new, separate Google user
+    created instead. It stores whatever scopes Google grants, so `can_read` / `can_write` flip on
+    once granted.
+
+    `expect` is the account the tab was built for. A navigation carries no headers, so the check
+    every request gets (`EXPECTED_ACCOUNT_HEADER`) rides the query here: a tab still showing the
+    account signed in before would otherwise attach a Google identity to the one signed in now."""
+    if built_for_another_account(user.id, expect):
+        return RedirectResponse(url=ACCOUNT_CHANGED_LANDING)
     client = google_oauth(db)
     if client is None:
-        return RedirectResponse(url="/")
-    # Attach the grant to THIS account in the callback. Without it a password account (no
-    # google_sub) would get a brand-new, separate Google user created instead of being linked.
-    request.session["oauth_link_uid"] = user.id
-    scope = WRITE_SCOPES if access == "write" else READ_SCOPES
+        return RedirectResponse(url="/?link=error")
+    scope, prompt = _CONNECT[access]
+    state = generate_token(48)
+    request.session[_LINK_MARKER] = {
+        "state": state,
+        "uid": user.id,
+        "explicit": access == "identity",
+    }
     return await client.authorize_redirect(
         request,
         _oauth_redirect_uri(request),
+        state=state,
         access_type="offline",
-        prompt="consent",
+        prompt=prompt,
         include_granted_scopes="true",
         scope=scope,
     )

backend/app/routes/channels.py

@@ -307,9 +307,9 @@ def explore_channel(
     """Open an un-subscribed channel for browsing. Records the exploration (a per-user,
     grace-clocked row that gates this user's access to the channel's ephemeral videos and
     drives the later cleanup), flags the channel ephemeral unless someone organically follows
-    it, then ingests one page of its uploads (newest-first) marked via_explore + enriched.
-    Returns the next page token so the page can "load more". Guarded against the shared
-    backfill quota reserve."""
+    it, then ingests one page of its uploads (newest-first), enriched — marked via_explore while
+    nobody follows it. Returns the next page token so the page can "load more". Guarded against
+    the shared backfill quota reserve."""
     channel = _channel_or_404(db, channel_id)
     if queries.is_blocked(db, user, channel_id):
         raise refused(403, "channel_blocked")

backend/app/routes/debug_sessions.py

@@ -220,6 +220,7 @@ def send_issue(sid: str, user: User = Depends(current_user), db: Session = Depen
         raise refused(409, "issue_sending")
     new = session.issue_url is None
     if not new and not debugsession.unfinished(session):
+        db.commit()  # nothing to send: the lock ends here, not when the request's session closes
         return {"ok": True, "session": _view(db, session)}
     key = str(user.id)
     if new:

backend/app/routes/feed.py

@@ -1,19 +1,31 @@
+import threading
+from collections.abc import AsyncIterator
 from datetime import date
+from functools import partial
 
-from fastapi import APIRouter, Depends, Query
+from fastapi import APIRouter, Depends, Path, Query, Request
 from pydantic import BaseModel
 from sqlalchemy.orm import Session
 from sqlalchemy.orm.exc import StaleDataError
 
-from app import quota
+from app import jobruns, quota, scheduler
+from app.catalog import queries as catalog_queries, service as catalog_service
 from app.auth import current_user
-from app.db import get_db
+from app.db import get_db, release_read_connection
 from app.mailtext import lang_for_user
+from app.ratelimit import RateLimiter
 from app.feedquery import BadCursor, facet_counts, feed_count, feed_page, feed_summary, feed_years
 from app.models import User
-from app.reader import queries, service
-from app.routes._common import ProgressIn, kept_position, not_found, youtube_refusal
-from app.sync.videos import parse_iso8601_duration
+from app.reader import queries, service, takeout
+from app.routes._common import (
+    ProgressIn,
+    capped_body,
+    drain_body,
+    kept_position,
+    not_found,
+    youtube_refusal,
+)
+from app.sync.videos import ensure_videos, parse_iso8601_duration
 from app.utils import iso
 from app.youtube.client import YouTubeClient, YouTubeError
 from app.refusal import refused
@@ -293,3 +305,214 @@ def get_video_detail(
             items[0].get("contentDetails", {}).get("duration")
         ),
     }
+
+
+# --- Google Takeout ------------------------------------------------------------------------------
+
+
+# One upload at a time per user, ten an hour, and two at once across every user: a file's bytes are
+# held whole while it is read (its text a window at a time), so these bound what the accounts can
+# make the process hold. The section uploads its files one after another.
+_takeout_limiter = RateLimiter(max_events=10, window_seconds=3600)
+_takeout_uploading: set[int] = set()
+_takeout_uploading_guard = threading.Lock()
+_TAKEOUT_AT_ONCE = 2
+
+
+async def _takeout_body(
+    request: Request, user: User = Depends(current_user), db: Session = Depends(get_db)
+) -> AsyncIterator[bytes]:
+    # The sign-in check has run, and it was a SELECT: hand the connection back before a file of
+    # megabytes arrives, so a slow upload does not hold one `idle in transaction`. The user's id is
+    # taken first: the release expires the user, and reading it after would reload it, taking a
+    # connection straight back out.
+    uid = user.id
+    release_read_connection(db)
+    with _takeout_uploading_guard:
+        busy = uid in _takeout_uploading
+        full = not busy and len(_takeout_uploading) >= _TAKEOUT_AT_ONCE
+        taken = not busy and not full
+        if taken:
+            _takeout_uploading.add(uid)
+    if not taken or not _takeout_limiter.allow(str(uid)):
+        if taken:
+            with _takeout_uploading_guard:
+                _takeout_uploading.discard(uid)
+        # Read to the end before answering: refused mid-upload, the browser reports a network
+        # failure instead of the refusal (capped_stream_to says why).
+        await drain_body(request)
+        if busy:
+            raise refused(429, "takeout_busy")
+        if full:
+            raise refused(429, "takeout_server_busy")
+        raise refused(429, "takeout_cooldown")
+    try:
+        yield await capped_body(request, takeout.MAX_BYTES)
+    finally:
+        with _takeout_uploading_guard:
+            _takeout_uploading.discard(uid)
+
+
+@router.post("/takeout")
+def upload_takeout(
+    # The sign-in check FIRST: the body's own dependency needs the signed-in user, so a stranger's
+    # file is refused before a byte of it is read.
+    user: User = Depends(current_user),
+    body: bytes = Depends(_takeout_body),
+    db: Session = Depends(get_db),
+):
+    """One file of the user's Google Takeout export — the watch history or the Watch later list.
+    Its videos are kept and the import runs in the background; its outcome reaches the user's inbox."""
+    try:
+        parsed = takeout.parse(body)
+    except takeout.TakeoutError as exc:
+        if exc.reason == "no_videos":
+            raise refused(422, "takeout_no_videos") from None
+        if exc.reason == "too_many":
+            raise refused(422, "takeout_list_too_long", limit=takeout.WATCH_LATER_MAX) from None
+        raise refused(422, "takeout_unreadable") from None
+    try:
+        takeout.stage(db, user, parsed)
+    except takeout.TooMany:
+        db.rollback()
+        raise refused(422, "takeout_too_many", limit=takeout.ROWS_MAX) from None
+    db.commit()
+    scheduler.run_ad_hoc(
+        scheduler.TAKEOUT_IMPORT,
+        partial(takeout.run_import, user.id),
+        actor_id=user.id,
+        target=takeout.run_target(user.id),
+        notify=True,
+    )
+    return {
+        "ok": True,
+        "kind": parsed.kind,
+        "videos": len(parsed.videos),
+        "entries": parsed.entries,
+        "ads": parsed.ads,
+        "other": parsed.other,
+        "malformed": parsed.malformed,
+    }
+
+
+@router.get("/takeout")
+def takeout_status(user: User = Depends(current_user), db: Session = Depends(get_db)):
+    """What the user's Takeout import has done: its last run (none before the first upload), and
+    every video kept from the export, by kind and outcome — a video the catalog did not know yet
+    counts as "waiting" until it does, and is marked then."""
+    target = takeout.run_target(user.id)
+    run = jobruns.last_run(db, scheduler.TAKEOUT_IMPORT, target)
+    return {
+        # Queued and not started: its run row is not written yet, and the last run is still the
+        # previous one, so the page waits for it rather than taking that one for this upload's.
+        "queued": scheduler.ad_hoc_queued(scheduler.TAKEOUT_IMPORT, target),
+        "max_bytes": takeout.MAX_BYTES,
+        "run": None
+        if run is None
+        else {
+            "status": run.status,
+            "started_at": iso(run.started_at),
+            "finished_at": iso(run.finished_at),
+            "result": run.result,
+            "failure": run.failure,
+        },
+        **queries.takeout_summary(db, user),
+    }
+
+
+# --- channels watched but not followed -----------------------------------------------------------
+
+
+@router.get("/takeout/channels")
+def watched_channels(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
+    """The channels the user's Takeout history shows them watching often but not following: each
+    with how many of its videos over five minutes they watched, and when last. `pending` is how many
+    more channels may join once the import has asked YouTube for their videos' lengths or their
+    names, so a list still being made does not read as the whole of it; `imported` tells "no history
+    uploaded" from "nothing to suggest"."""
+    return {
+        "imported": queries.has_watch_history(db, user),
+        "pending": queries.watched_channels_pending(db, user),
+        "channels": [
+            {
+                "id": ch.id,
+                "title": ch.title,
+                "handle": ch.handle,
+                "thumbnail_url": ch.thumbnail_url,
+                "subscriber_count": ch.subscriber_count,
+                "video_count": ch.video_count,
+                "description": ch.description,
+                "watched": int(n),
+                "last_watched_at": iso(last),
+            }
+            for ch, n, last in queries.watched_channels(db, user)
+        ],
+    }
+
+
+@router.post("/takeout/channels/{channel_id}/videos")
+def watched_channel_videos(
+    channel_id: str, user: User = Depends(current_user), db: Session = Depends(get_db)
+) -> dict:
+    """The videos of a channel that the user's list counts, ready to play, last watched first. The
+    channel is explored for the user (as its page does), and the videos the catalog lacks come in as
+    the exploration's: the player needs them in the catalog, the shared Library does not get them,
+    and the cleanup reclaims them once nobody keeps them. Of a channel someone follows they come in
+    as the catalog's own, as its uploads do. `gone`: how many YouTube no longer has."""
+    channel = catalog_queries.channel(db, channel_id)
+    if channel is None:
+        raise not_found("channel")
+    if catalog_queries.is_blocked(db, user, channel_id):
+        raise refused(403, "channel_blocked")
+    counted, gone = queries.watched_channel_videos(db, user, channel_id)
+    ids = [vid for vid, _ in counted]
+    known = queries.known_video_ids(db, ids)
+    missing = [vid for vid in ids if vid not in known]
+    if missing and quota.backfill_reserve_reached():
+        raise refused(429, "quota_reserve")
+    catalog_service.explore(db, user, channel)
+    db.commit()
+    if missing:
+        try:
+            with (
+                quota.attribute(user.id, quota.QuotaAction.CHANNELS_EXPLORE),
+                YouTubeClient(db, user) as yt,
+            ):
+                ensure_videos(db, yt, missing, explore=channel.from_explore)
+        except YouTubeError as exc:
+            raise youtube_refusal(exc) from exc
+        catalog_service.settle_explored(db, channel)
+        db.commit()
+    items = queries.feed_rows_by_ids(db, user, ids)
+    watched_at = dict(counted)
+    for item in items:
+        item["takeout_watched_at"] = iso(watched_at[item["id"]])
+    # A video YouTube still had when its length was asked may have gone since.
+    return {"ok": True, "items": items, "gone": gone + len(ids) - len(items)}
+
+
+@router.put("/takeout/channels/{channel_id}/dismissal")
+def dismiss_watched_channel(
+    channel_id: str = Path(max_length=32),
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Put a channel out of the user's list (it stays out until restored). Only one their own history
+    shows them watching: a made-up id would only pile up rows every list read then has to skip."""
+    if not queries.watched_in_history(db, user, channel_id):
+        raise not_found("channel")
+    service.dismiss_watched_channel(db, user, channel_id)
+    db.commit()
+    return {"ok": True, "channel_id": channel_id}
+
+
+@router.delete("/takeout/channels/{channel_id}/dismissal")
+def restore_watched_channel(
+    channel_id: str = Path(max_length=32),
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """Let a channel the user put out of the list back on it."""
+    service.restore_watched_channel(db, user, channel_id)
+    db.commit()
+    return {"ok": True, "channel_id": channel_id}

backend/app/routes/files.py

@@ -375,6 +375,9 @@ def update_folder(
     folder = own_folder(db, user, folder_id)
     sent = body.model_fields_set
     if "name" in sent:
+        # Which folder may be renamed is answered before what the name is, as it always was.
+        if not folders.renamable(folder):
+            raise refused(409, "file_folder_not_own")
         try:
             folders.rename(db, folder, _folder_name(body.name or ""))
         except folders.FolderError as e:

backend/app/routes/me.py

@@ -9,11 +9,9 @@ from app.auth import (
     count_admins,
     count_pending_activations,
     current_user,
-    drop_from_wallet,
     google_enabled,
     has_read_scope,
     has_write_scope,
-    is_allowed,
     optional_current_user,
     purge_user,
 )
@@ -26,79 +24,10 @@ from app.db import get_db
 from app.models import Invite, User
 from app.utils import iso
 from app.refusal import refused
-from app.routes._common import not_found
 
 router = APIRouter(prefix="/api/me", tags=["me"])
 
 
-@router.get("/accounts")
-def my_accounts(
-    request: Request,
-    user: User = Depends(current_user),
-    db: Session = Depends(get_db),
-) -> list[dict]:
-    """The accounts that have authenticated in this browser session (switchable without a
-    new Google sign-in). The active one is flagged."""
-    ids = request.session.get("account_ids") or [user.id]
-    rows = {u.id: u for u in db.execute(select(User).where(User.id.in_(ids))).scalars()}
-    out = []
-    for uid in ids:  # preserve recency order from the session
-        u = rows.get(uid)
-        if u is not None:
-            out.append(
-                {
-                    "id": u.id,
-                    "email": u.email,
-                    "display_name": u.display_name,
-                    "avatar_url": u.avatar_url,
-                    "active": u.id == user.id,
-                }
-            )
-    return out
-
-
-class SwitchAccountIn(BaseModel):
-    user_id: int
-
-
-@router.post("/switch")
-def switch_account(
-    body: SwitchAccountIn,
-    request: Request,
-    user: User = Depends(current_user),
-    db: Session = Depends(get_db),
-) -> dict:
-    """Switch the active account to another one already authenticated in this browser. No
-    Google round-trip — but only accounts present in the session list (proof they signed in
-    here) are allowed, and access is re-checked in case the invite was revoked since."""
-    target = body.user_id
-    if target not in (request.session.get("account_ids") or []):
-        raise refused(403, "not_your_account")
-    # An account `_session_user` would refuse — gone, deactivated, suspended, or its session revoked — is
-    # refused here too, and dropped from the switch list the way `_session_user` drops it. Made the session
-    # default instead, the next request's current_user would refuse it as the DEFAULT and clear the whole
-    # wallet session, logging out every account here. (B-core-9)
-    u = db.get(User, target)
-    if u is None:
-        drop_from_wallet(request.session, target)
-        raise not_found("account")
-    if not is_allowed(db, u.email):
-        raise refused(403, "account_no_access")
-    if not u.is_active:
-        drop_from_wallet(request.session, target)
-        raise refused(403, "account_no_access")
-    if u.is_suspended:
-        drop_from_wallet(request.session, target)
-        raise refused(403, "account_suspended")
-    # Revoked: a password reset/change or "log out everywhere" bumped the account's epoch.
-    epochs = request.session.get("epochs") or {}
-    if int(epochs.get(str(target), 0)) != (u.session_epoch or 0):
-        drop_from_wallet(request.session, target)
-        raise refused(403, "account_signed_out")
-    request.session["user_id"] = target
-    return {"ok": True, "user_id": target}
-
-
 @router.get("")
 def get_me(
     user: User | None = Depends(optional_current_user), db: Session = Depends(get_db)
@@ -150,6 +79,10 @@ def get_me(
         # Which live-search source runs, so the search button can say what a search costs: "scrape"
         # spends no API quota, "api" spends 100 units a page.
         "search_source": sysconfig.get_str(db, "search_source"),
+        # Whether live search and related videos are on: an admin turns either off with a daily
+        # limit of 0, and the UI then leaves out what would only be refused.
+        "search_enabled": sysconfig.get_int(db, "search_daily_limit_per_user") > 0,
+        "related_enabled": sysconfig.get_int(db, "related_daily_limit_per_user") > 0,
         "can_read": has_read_scope(user),
         "can_write": has_write_scope(user),
         "pending_invites": pending_invites,
@@ -187,17 +120,11 @@ def delete_my_account(
     surfaces)."""
     if user.role == "admin" and count_admins(db) <= 1:
         raise refused(400, "only_admin_delete")
-    user_id = user.id
     # Full erasure (cascades) + access-request cleanup + Google-grant revocation; shared with the
-    # admin delete path. Capture the id first — `user` is detached once purge_user deletes the row.
+    # admin delete path.
     purge_user(db, user, background)
-    # Drop this account from the browser session; switch to another signed-in account if any.
-    remaining = [a for a in (request.session.get("account_ids") or []) if a != user_id]
-    if remaining:
-        request.session["account_ids"] = remaining
-        request.session["user_id"] = remaining[-1]
-    else:
-        request.session.clear()
+    # The session's one account is gone: so is the session.
+    request.session.clear()
     return {"deleted": True}
 
 
@@ -213,7 +140,13 @@ def update_preferences(
     user: User = Depends(current_user),
     db: Session = Depends(get_db),
 ) -> dict:
-    # Merge so partial updates don't wipe other keys.
+    # Merge so partial updates don't wipe other keys — over the stored row read again under a row
+    # lock: a client saves keys one at a time (several at once on a boot that moves this browser's
+    # copies to the account), and two merges over the same earlier read would keep only the last.
+    # FOR NO KEY UPDATE, as bump_session_epoch: it still orders two merges, and a transaction that
+    # only added a row referring to the account (a playlist pull, then minutes of YouTube) does not
+    # hold the save up.
+    db.refresh(user, with_for_update={"key_share": True})
     merged = dict(user.preferences or {})
     merged.update(preferences.root)
     user.preferences = merged

backend/app/routes/notifications.py

@@ -78,8 +78,8 @@ def mark_read(
 ) -> dict:
     n = owned_or_404(db, Notification, notification_id, user, what="notification")
     n.read = True
-    db.commit()
     trim_read(db, user.id)
+    db.commit()
     return {"ok": True}
 
 
@@ -90,8 +90,8 @@ def mark_all_read(user: User = Depends(current_user), db: Session = Depends(get_
         .where(owned_by(Notification.user_id, user), Notification.read.is_(False))
         .values(read=True)
     )
-    db.commit()
     trim_read(db, user.id)
+    db.commit()
     return {"ok": True}
 
 

backend/app/routes/plex/detail.py

@@ -100,7 +100,7 @@ def show_detail(
         raise not_found("show")
     seasons = queries.seasons(db, sh)
     eps = queries.episodes(db, sh)
-    states = queries.states(db, user, [e.id for e in eps])
+    states = rollup.item_state_map(db, user, [e.id for e in eps])
     by_season: dict[int | None, list] = {}
     all_cards: list[dict] = []  # every episode, in season/episode order → show-level rollup
     for e in eps:
@@ -247,7 +247,7 @@ def _collection_strips(db: Session, it: "PlexItem", user_id: int, lang: str) ->
     for col in ordered:
         rk = col.rating_key
         members = queries.library_movies_in(db, it.library_id, rk, lang, limit=60)
-        states = queries.states(db, user_id, [m.id for m in members])
+        states = rollup.item_state_map(db, user_id, [m.id for m in members])
         out.append(
             {
                 "id": rk,

backend/app/routes/plex/playback.py

@@ -19,7 +19,7 @@ from fastapi.responses import FileResponse
 from pydantic import BaseModel
 from sqlalchemy.orm import Session
 
-from app.auth import current_user, resolved_user_id
+from app.auth import current_user, session_user_id
 from app.db import SessionLocal, get_db
 from app.models import User
 from app.plex import imgcache, queries, service, subtitles
@@ -42,7 +42,7 @@ def _require_session(request: Request) -> int:
     """Lightweight auth for the high-fan-out image proxies: validate the signed session cookie WITHOUT
     a DB user-load, so these endpoints never hold a pool connection for the auth. Poster images are
     low-sensitivity; a valid session is enough."""
-    uid, _ = resolved_user_id(request)
+    uid = session_user_id(request)
     if not uid:
         raise refused(401, "not_authenticated")
     return uid

backend/app/routes/plex/playlists.py

@@ -9,7 +9,7 @@ from sqlalchemy.orm import Session
 from app.auth import current_user
 from app.db import get_db
 from app.models import User
-from app.plex import queries, service
+from app.plex import queries, rollup, service
 from app.plex.cards import display_title, leaf_card, norm_lang
 from app.refusal import refused
 from app.routes._common import not_found
@@ -99,7 +99,7 @@ def playlist_detail(
     lang = norm_lang(lang)
     pl = _own_playlist(db, user, pid)
     items = queries.playlist_items(db, pl.id)
-    states = queries.states(db, user, [it.id for it in items])
+    states = rollup.item_state_map(db, user, [it.id for it in items])
     shows = queries.shows_by_ids(db, {it.show_id for it in items if it.show_id})
     cards = [
         leaf_card(

backend/app/routes/runs.py

@@ -3,7 +3,7 @@
 Admin-only and read-only — rows leave the table through the retention job
 (`job_run_retention_days`). Filtered on the server and paged by `_keyset`, like the server log and
 the audit log beside it on the activity page. A row names what a reader needs without a second
-request: the channel a user's ad-hoc run worked on, and how many log lines the run left (only
+request: what a user's ad-hoc run worked on (the channel, the importing user), and how many log lines the run left (only
 WARNING and up are stored, so most runs have none — the count says so before the click).
 """
 
@@ -116,8 +116,12 @@ def list_runs(
         limit,
         lambda r: r.started_at,
     )
-    channels = {r.target for r in runs if r.job in scheduler.AD_HOC_JOBS and r.target}
+    channels = {r.target for r in runs if r.job == scheduler.CHANNEL_BACKFILL and r.target}
     titles = catalog_queries.channel_titles(db, channels)
+    for target in {r.target for r in runs if r.job == scheduler.TAKEOUT_IMPORT and r.target}:
+        title = scheduler.ad_hoc_target_title(db, scheduler.TAKEOUT_IMPORT, target)
+        if title is not None:
+            titles[target] = title
     lines: dict[int, int] = {}
     if runs:
         counted = db.execute(

backend/app/routes/search.py

@@ -13,10 +13,18 @@ this search-discovered "noise" by default. Shorts and currently-live/upcoming vi
 ingested this way (Shorts are confirmed by the youtube.com/shorts probe; live/upcoming are
 dropped from the search response up front).
 
+A video's related videos (`/related/{video_id}`, the list YouTube shows beside it) come in the same
+way: read from the website (`related_scrape`), ingested as search finds are, under their own daily
+cap — with what this user has already seen left out.
+
 This is the HTTP side — the paging, the caps and the refusals; what a page of results leaves in the
 catalog is `catalog/search`."""
 
-from fastapi import APIRouter, Depends
+import threading
+from collections.abc import Callable
+from typing import Literal
+
+from fastapi import APIRouter, Depends, Path
 from pydantic import BaseModel
 from sqlalchemy.orm import Session
 
@@ -26,8 +34,10 @@ from app.catalog import queries
 from app.catalog import search as catalog_search
 from app.db import get_db
 from app.models import User
+from app.ratelimit import RateLimiter
 from app.reader import queries as reader_queries
-from app.youtube import search_scrape
+from app.youtube import related_scrape, search_scrape
+from app.youtube.innertube import ScrapeError
 from app.youtube.client import YouTubeClient, YouTubeError
 from app.refusal import refused
 from app.routes._common import youtube_refusal
@@ -37,6 +47,115 @@ router = APIRouter(prefix="/api/search", tags=["search"])
 
 # search.list costs this many quota units per page.
 SEARCH_COST = 100
+# The scrape source reads at most this many pages for one search.
+SEARCH_MAX_PAGES = 12
+
+# A related list is filled to this many unseen videos, reading at most this many pages (about 20
+# videos each, measured) — the ones the user has seen are left out, so a page alone can fall short.
+RELATED_TARGET = 20
+RELATED_MAX_PAGES = 4
+# The locale YouTube fills a related list in: on the video's topic, or with what trends in Hungary.
+RELATED_REGIONS = {"topical": ("en", "US"), "hu": ("hu", "HU")}
+
+# A request that fails outright gives its slot back, but its shape is the user's (a cursor is a
+# free string): one built to fail would read YouTube for nothing, without end. So only this many
+# refunds an hour per user and list; past them a failure keeps its slot, as a success does.
+REFUNDS_PER_HOUR = 5
+_refunds = RateLimiter(max_events=REFUNDS_PER_HOUR, window_seconds=3600)
+# The API source's budget check and the search.list call it pays for run one at a time: checked
+# apart, searches sent together at the last 100 units all passed, and all spent. One process
+# serves the API (a single uvicorn worker, as `ratelimit` says), so a process lock holds them all.
+# A search waits this long for it, then is refused as busy: each waiter holds a request worker, and
+# a slow YouTube (30 s a call at worst) would otherwise park them all behind one another.
+_api_search = threading.Lock()
+API_SEARCH_WAIT_SECONDS = 10.0
+
+
+def _gather(
+    db: Session,
+    yt: YouTubeClient,
+    user: User,
+    *,
+    cursor: str | None,
+    max_pages: int,
+    fetch: Callable[[str | None], dict],
+    candidates_of: Callable[[list[dict], list[str]], list[dict]],
+    enough: Callable[[list[str]], bool],
+    scrape_failed: Callable[[Exception], None],
+    ingest_failed: Callable[[int, int], None],
+) -> tuple[list[str], str | None]:
+    """Page through a YouTube list into the catalog: fetch a page, keep its `candidates_of`, ingest
+    them, and go on from its continuation until `enough` or `max_pages`. Returns the kept ids and
+    the cursor to read further from. A page that fails after earlier ones gathered something ends
+    the list with what it has; a first page that fails raises the 422 refusal.
+
+    422, not 502: this is a JSON endpoint, and the FE treats 502/503/504 as a gateway dropout
+    (auto-retry + "connection lost" banner — lib/api/core RETRIABLE_GATEWAY), which hides the real
+    message and repeats the work. A Data API failure is logged where it happened; the scraper's is
+    not, and its text does not ride the answer — so `scrape_failed` says it. An ingest (DB writes +
+    enrich) fails the same way, guarded for any exception so a write or shape failure is a 422 with
+    a message too, not a bare 500; `ingest_failed` logs it inside the `except`, with the trace."""
+    collected: list[str] = []
+    next_cursor = cursor
+    for _ in range(max_pages):
+        try:
+            page = fetch(next_cursor)
+        except (YouTubeError, ScrapeError) as exc:
+            if collected:
+                break  # keep what earlier pages already gathered
+            if isinstance(exc, ScrapeError):
+                scrape_failed(exc)
+            raise youtube_refusal(exc) from exc
+        candidates = candidates_of(page["items"], collected)
+        try:
+            collected.extend(catalog_search.ingest_candidates(db, yt, user, candidates))
+        except Exception as exc:  # any ingest failure, not just YouTubeError
+            db.rollback()
+            ingest_failed(len(candidates), len(collected))
+            if collected:
+                break
+            raise youtube_refusal(exc) from exc
+        next_cursor = page["next_page_token"]
+        if enough(collected) or not next_cursor:
+            break
+    return collected, next_cursor
+
+
+def _give_back(claimed: int, feature: str, user_id: int) -> None:
+    """Refund the slot of a request that failed outright, while this user has refunds left."""
+    if _refunds.allow(f"{feature}:{user_id}"):
+        quota.refund_action(claimed)
+        return
+    log.info(
+        "a failed request kept its slot: this hour's refunds are spent",
+        extra=logs.event(f"{feature}.refund_refused", per_hour=REFUNDS_PER_HOUR),
+    )
+
+
+def _budget_spent() -> None:
+    """Say that a live search was refused because the shared YouTube budget cannot pay for it."""
+    log.info(
+        "a live search was refused: the shared YouTube budget is spent",
+        extra=logs.event("search.refused", reason="shared_quota", cost=SEARCH_COST),
+    )
+
+
+def _busy() -> None:
+    """Say that a live search was refused because other API searches held the line too long."""
+    log.info(
+        "a live search was refused: other searches held the line",
+        extra=logs.event("search.refused", reason="busy", waited=API_SEARCH_WAIT_SECONDS),
+    )
+
+
+def _cap_spent(feature: str, limit: int) -> None:
+    """Say that a request was refused for the per-user daily cap. The 429 is in the request line
+    either way; which of the caps produced it is not, and they are answered by different things —
+    this one by tomorrow, the shared budget by the admin."""
+    log.info(
+        "a request was refused: the per-user daily cap is spent",
+        extra=logs.event(f"{feature}.refused", reason="daily_cap", limit=limit),
+    )
 
 
 @router.get("/youtube")
@@ -59,109 +178,122 @@ def search_youtube(
     # Source: "scrape" (InnerTube, no API quota) or "api" (search.list, 100 units/page).
     source = sysconfig.get_str(db, "search_source")
 
-    # Per-user daily cap applies to both sources (anti-abuse); with scrape it's the only guard
-    # since the search itself spends no quota.
+    # Every request takes a slot of the per-user daily cap, a cursor's too (anti-abuse); with
+    # scrape it's the only guard since the search itself spends no quota. A limit of 0 turns live
+    # search off on this instance — a refusal tomorrow will not change.
     daily_limit = sysconfig.get_int(db, "search_daily_limit_per_user")
-    used = quota.actions_today(db, user.id, quota.QuotaAction.VIDEOS_SEARCH)
-    if used >= daily_limit:
-        # The 429 is in the request line either way; which of the two caps produced it is not, and
-        # they are answered by different things — this one by tomorrow, the other by the admin.
-        log.info(
-            "a live search was refused: the per-user daily cap is spent",
-            extra=logs.event("search.refused", reason="daily_cap", limit=daily_limit),
-        )
-        raise refused(429, "search_daily_limit", limit=daily_limit)
-    # The 100-unit budget pre-check only applies to the API source.
+    if daily_limit <= 0:
+        raise refused(409, "search_disabled")
+    # The 100-unit budget pre-check only applies to the API source. It is asked again under the
+    # lock, right before the call (`fetch`); this one refuses before a slot is taken.
     if source == "api" and not quota.can_spend(db, SEARCH_COST):
-        log.info(
-            "a live search was refused: the shared YouTube budget is spent",
-            extra=logs.event("search.refused", reason="shared_quota", cost=SEARCH_COST),
-        )
+        _budget_spent()
         raise refused(429, "quota_spent")
+    # The check and the take are one step (`quota.claim_action`), whichever the source: checked
+    # first and logged after, requests sent together at the last free slot all passed — and with
+    # the API source each of them spent 100 units. A search that fails outright gives it back below.
+    claimed = quota.claim_action(user.id, quota.QuotaAction.VIDEOS_SEARCH, daily_limit)
+    if claimed is None:
+        _cap_spent("search", daily_limit)
+        raise refused(429, "search_daily_limit", limit=daily_limit)
 
     # Channels this user has blocked are dropped before ingestion — never shown, never stored.
     blocked = queries.blocked_ids(db, user)
 
-    collected: list[str] = []
-    next_cursor = cursor
-    MAX_PAGES = 12  # scrape safety bound
-    with YouTubeClient(db, user) as yt:
-        for _ in range(MAX_PAGES):
-            try:
-                if source == "api":
-                    with quota.attribute(user.id, quota.QuotaAction.VIDEOS_SEARCH):
-                        page = yt.search_videos(term, page_token=next_cursor)
-                else:
-                    page = search_scrape.search(term, continuation=next_cursor)
-            except (YouTubeError, search_scrape.ScrapeError) as exc:
-                if collected:
-                    break  # keep what we already gathered
-                # 422, not 502: this is a JSON endpoint, and the FE treats 502/503/504 as a gateway
-                # dropout (auto-retry + "connection lost" banner — lib/api/core RETRIABLE_GATEWAY),
-                # which hides the real message and burns a second quota-costed search. Upstream failure
-                # on a JSON endpoint = 422 + message; reserve 5xx for the raw byte-proxies. (B-core-1)
-                # A Data API failure is logged where it happened; the scraper's is not, and its text
-                # no longer rides the answer — so it is said here.
-                if isinstance(exc, search_scrape.ScrapeError):
-                    log.warning(
-                        "the YouTube search scrape failed",
-                        extra=logs.event("search.scrape_failed", error=str(exc)),
-                    )
-                raise youtube_refusal(exc) from exc
+    # Whether search.list was paid for: YouTube charges a call it refuses too, so once the client
+    # recorded its units — or the request went out and no answer came back — the search has spent
+    # the shared budget. One that never reached YouTube (no connection, no token) has not.
+    charged = False
 
-            # Drop currently-live/upcoming, channel-less, and blocked-channel results.
-            candidates = [
-                it
-                for it in page["items"]
-                if it["live_broadcast"] == "none"
-                and it["channel_id"]
-                and it["channel_id"] not in blocked
-            ]
-            # Ingest (DB writes + enrich) runs outside the page-fetch try above; guard it too so a
-            # write/shape failure degrades to a 422 with a message on this JSON endpoint instead of a
-            # bare 500 hidden behind the FE's generic "search failed" (same rationale as B-core-1).
-            try:
-                collected.extend(catalog_search.ingest_candidates(db, yt, user, candidates))
-            except Exception as exc:  # any ingest failure, not just YouTubeError
-                db.rollback()
-                # The term is NOT in this line. A search string is exactly what the privacy
-                # policy keeps out of the log unless the user turned Teljes/TRACE on for
-                # themselves, and `redact_text` does not cover it — it scrubs addresses and
-                # credential params, not what someone typed into the search box. What the
-                # ERROR line carries instead is the shape of the failure, which is what tells
-                # an ingest bug from an upstream one; the term follows at TRACE, and only
-                # under this user's own consent — not whenever the process runs at TRACE.
-                log.exception(
-                    "live search ingest failed",
-                    extra=logs.event(
-                        "search.live.ingest_failed",
-                        source=source,
-                        candidates=len(candidates),
-                        collected=len(collected),
-                    ),
-                )
-                if logs.consented_trace():
-                    log.log(
-                        logs.TRACE,
-                        "the term the ingest failed on",
-                        # log-pii-ok: only under this user's own Teljes consent
-                        extra=logs.event("search.live.term", term=term),
-                    )
-                if collected:
-                    break  # keep what earlier pages already gathered
-                raise youtube_refusal(exc) from exc
-            next_cursor = page["next_page_token"]
-            # The API source pages cost 100 units each — never auto-page; one page per request.
-            if source == "api" or len(collected) >= limit or not next_cursor:
-                break
+    def fetch(token: str | None) -> dict:
+        if source != "api":
+            return search_scrape.search(term, continuation=token)
+        if not _api_search.acquire(timeout=API_SEARCH_WAIT_SECONDS):
+            _busy()
+            raise refused(429, "search_busy")
+        try:
+            return paid_search(token)
+        finally:
+            _api_search.release()
 
-    # Count this as ONE per-user search against the daily cap. The scrape source spends no
-    # quota, so it never logs an event on its own; the API source already logged exactly one
-    # (via record_usage — it never auto-pages). Logging once here, after a search that produced
-    # at least one page (a total failure raises 502 above and never reaches this), keeps the cap
-    # counting user searches instead of internal continuation pages.
-    if source != "api":
-        quota.log_action(user.id, quota.QuotaAction.VIDEOS_SEARCH)
+    def paid_search(token: str | None) -> dict:
+        nonlocal charged
+        with quota.attribute(user.id, quota.QuotaAction.VIDEOS_SEARCH):
+            if not quota.can_spend(db, SEARCH_COST):
+                _budget_spent()
+                raise refused(429, "quota_spent")
+            spent = yt.spent
+            try:
+                page = yt.search_videos(term, page_token=token)
+            except YouTubeError as exc:
+                charged = yt.spent > spent or exc.sent
+                raise
+            charged = True
+            return page
+
+    def candidates_of(items: list[dict], _collected: list[str]) -> list[dict]:
+        # Drop currently-live/upcoming, channel-less, and blocked-channel results.
+        return [
+            it
+            for it in items
+            if it["live_broadcast"] == "none"
+            and it["channel_id"]
+            and it["channel_id"] not in blocked
+        ]
+
+    def scrape_failed(exc: Exception) -> None:
+        log.warning(
+            "the YouTube search scrape failed",
+            extra=logs.event("search.scrape_failed", error=str(exc)),
+        )
+
+    def ingest_failed(candidates: int, collected: int) -> None:
+        # The term is NOT in this line. A search string is exactly what the privacy policy keeps
+        # out of the log unless the user turned Teljes/TRACE on for themselves, and `redact_text`
+        # does not cover it — it scrubs addresses and credential params, not what someone typed
+        # into the search box. What the ERROR line carries instead is the shape of the failure,
+        # which is what tells an ingest bug from an upstream one; the term follows at TRACE, and
+        # only under this user's own consent — not whenever the process runs at TRACE.
+        log.exception(
+            "live search ingest failed",
+            extra=logs.event(
+                "search.live.ingest_failed",
+                source=source,
+                candidates=candidates,
+                collected=collected,
+            ),
+        )
+        if logs.consented_trace():
+            log.log(
+                logs.TRACE,
+                "the term the ingest failed on",
+                # log-pii-ok: only under this user's own Teljes consent
+                extra=logs.event("search.live.term", term=term),
+            )
+
+    try:
+        with YouTubeClient(db, user) as yt:
+            collected, next_cursor = _gather(
+                db,
+                yt,
+                user,
+                cursor=cursor,
+                # The API source pages cost 100 units each — never auto-page; one page per request.
+                max_pages=1 if source == "api" else SEARCH_MAX_PAGES,
+                fetch=fetch,
+                candidates_of=candidates_of,
+                enough=lambda got: len(got) >= limit,
+                scrape_failed=scrape_failed,
+                ingest_failed=ingest_failed,
+            )
+    except Exception:
+        # A search that showed the user nothing does not count against them, unless search.list
+        # was already paid for: then the slot is what stops a failing search (a bogus cursor gets
+        # a 400 every time) from spending the shared budget without end. A busy refusal paid
+        # nothing and gives it back.
+        if not charged:
+            _give_back(claimed, "search", user.id)
+        raise
 
     ordered = collected[:limit]
 
@@ -201,3 +333,78 @@ def clear_search(
     removed = catalog_search.clear(db, user, ids)
     db.commit()
     return {"ok": True, "removed": removed}
+
+
+@router.get("/related/{video_id}")
+def related_videos(
+    video_id: str = Path(max_length=16),
+    cursor: str | None = None,
+    region: Literal["topical", "hu"] = "topical",
+    user: User = Depends(current_user),
+    db: Session = Depends(get_db),
+) -> dict:
+    """The videos YouTube shows beside this one, materialised into the catalog like search results,
+    without the ones this user has watched or hidden here or in their Takeout history, and without
+    blocked channels. Pages on (`cursor`) until `RELATED_TARGET` are gathered; `next_cursor` reads
+    further. `region` picks the locale YouTube fills the list in (`RELATED_REGIONS`)."""
+    # Every request takes a slot, a cursor's too: each one scrapes pages and enriches what they
+    # hold, and a cursor is a free string — counting only first pages let a replayed cursor read
+    # without end. The check and the take are one step (`quota.claim_action`); a request that fails
+    # outright gives its slot back, a few an hour (`_give_back`). A limit of 0 turns related videos
+    # off on this instance.
+    daily_limit = sysconfig.get_int(db, "related_daily_limit_per_user")
+    if daily_limit <= 0:
+        raise refused(409, "related_disabled")
+    claimed = quota.claim_action(user.id, quota.QuotaAction.VIDEOS_RELATED, daily_limit)
+    if claimed is None:
+        _cap_spent("related", daily_limit)
+        raise refused(429, "related_daily_limit", limit=daily_limit)
+
+    hl, gl = RELATED_REGIONS[region]
+    blocked = queries.blocked_ids(db, user)
+
+    def candidates_of(items: list[dict], collected: list[str]) -> list[dict]:
+        ids = [it["id"] for it in items]
+        drop = reader_queries.seen_by(db, user, ids) | set(collected) | {video_id}
+        return [it for it in items if it["id"] not in drop and it["channel_id"] not in blocked]
+
+    def scrape_failed(exc: Exception) -> None:
+        log.warning(
+            "the related-videos scrape failed",
+            extra=logs.event("related.scrape_failed", error=str(exc)),
+        )
+
+    def ingest_failed(candidates: int, collected: int) -> None:
+        log.exception(
+            "a related list's ingest failed",
+            extra=logs.event("related.ingest_failed", candidates=candidates, collected=collected),
+        )
+
+    try:
+        with YouTubeClient(db, user) as yt:
+            collected, next_cursor = _gather(
+                db,
+                yt,
+                user,
+                cursor=cursor,
+                max_pages=RELATED_MAX_PAGES,
+                fetch=lambda token: related_scrape.related(
+                    video_id, continuation=token, hl=hl, gl=gl
+                ),
+                candidates_of=candidates_of,
+                enough=lambda got: len(got) >= RELATED_TARGET,
+                scrape_failed=scrape_failed,
+                ingest_failed=ingest_failed,
+            )
+    except Exception:
+        _give_back(claimed, "related", user.id)
+        raise
+
+    rows = reader_queries.feed_rows_by_ids(db, user, collected)
+    # A related list mixes channels the user follows with ones they never met: each row says which.
+    followed = queries.followed_among(db, user, {r["channel_id"] for r in rows})
+    return {
+        "items": [{**r, "followed": r["channel_id"] in followed} for r in rows],
+        "next_cursor": next_cursor,
+        "has_more": bool(next_cursor),
+    }

backend/app/scheduler.py

@@ -22,6 +22,7 @@ from app.plex.tmdb import run_people_enrich
 from app.plex.watch_sync import run_plex_watch_reconcile, run_plex_watch_sync
 from app.notifications import create_notification
 from app.runstatus import RunStatus, RunTrigger
+from app.reader import takeout
 from app.state import is_sync_paused
 from app.sync.autotag import run_autotag_all
 from app.sync.explore import purge_ephemeral
@@ -118,8 +119,9 @@ def _notify_done(
     summary: str | None,
     result: dict | None = None,
 ) -> None:
-    """Post a completion notice to the triggering admin's inbox (manual runs only). Best
-    effort — a notification failure must never mask the job's own outcome."""
+    """Post a completion notice to the inbox of who started the run: an admin's manual run, or a
+    user's ad-hoc run that asks for one. Best effort — a notification failure must never mask the
+    job's own outcome."""
     try:
         create_notification(
             db,
@@ -272,7 +274,6 @@ def scheduler_snapshot(db) -> dict:
     open_runs = _open_run_ids()
     recent = jobruns.recent_runs(db, [*JOB_INTERVALS, *AD_HOC_JOBS], RECENT_RUNS)
     open_runs |= _open_run_ids()
-    going_id = max((r for r, j in open_runs.items() if j == CHANNEL_BACKFILL), default=None)
     jobs = []
     for job_id, default_interval in JOB_INTERVALS.items():
         a = acts.get(job_id, {})
@@ -288,30 +289,11 @@ def scheduler_snapshot(db) -> dict:
         )
     # Work a user starts on one target: no interval, no "run now" (it needs its target), but its
     # runs are shown beside the jobs' — it spends the same quota. Its one in-memory guard is the
-    # target's own (sync/runner), so "running" is the runs this process has open: two channels can
-    # backfill at once, so a run still going counts even when a newer one already ended. That run then
-    # leads the row, so its channel, its "no outcome yet" and its dot are one run's, never the
-    # newer run's outcome under the older one's channel.
-    backfills = recent.get(CHANNEL_BACKFILL, [])
-    going = db.get(JobRun, going_id) if going_id is not None else None
-    if going is not None and going.status != RunStatus.RUNNING:
-        going = None  # its outcome is written; only the bookkeeping has not caught up
-    if going is not None:
-        backfills = [going, *(r for r in backfills if r.id != going.id)][:RECENT_RUNS]
-    shown = backfills[0] if backfills else None
-    ad_hoc = [
-        {
-            "id": CHANNEL_BACKFILL,
-            "running": going is not None,
-            "target": shown.target if shown else None,
-            "target_title": (
-                catalog_queries.channel_title(db, shown.target)
-                if shown is not None and shown.target
-                else None
-            ),
-            **_last_runs(backfills, open_runs),
-        }
-    ]
+    # target's own (sync/runner, the import's per-user lock), so "running" is the runs this process
+    # has open: two targets can run at once, so a run still going counts even when a newer one already
+    # ended. That run then leads the row, so its target, its "no outcome yet" and its dot are one
+    # run's, never the newer run's outcome under the older one's target.
+    ad_hoc = [_ad_hoc_row(db, job, recent.get(job, []), open_runs) for job in AD_HOC_JOBS]
     return {
         "running_here": running_here,
         "enabled": settings.scheduler_enabled,
@@ -333,6 +315,27 @@ def _seen(run, open_runs: dict[int, str]) -> RunStatus:
     return RunStatus(run.status)
 
 
+def _ad_hoc_row(db, job: str, runs: list[JobRun], open_runs: dict[int, str]) -> dict:
+    going_id = max((r for r, j in open_runs.items() if j == job), default=None)
+    going = db.get(JobRun, going_id) if going_id is not None else None
+    if going is not None and going.status != RunStatus.RUNNING:
+        going = None  # its outcome is written; only the bookkeeping has not caught up
+    if going is not None:
+        runs = [going, *(r for r in runs if r.id != going.id)][:RECENT_RUNS]
+    shown = runs[0] if runs else None
+    return {
+        "id": job,
+        "running": going is not None,
+        "target": shown.target if shown else None,
+        "target_title": (
+            ad_hoc_target_title(db, job, shown.target)
+            if shown is not None and shown.target
+            else None
+        ),
+        **_last_runs(runs, open_runs),
+    }
+
+
 def _last_runs(runs: list, open_runs: dict[int, str]) -> dict:
     """A row's run fields from its newest runs (newest first): how the last one went, and the few
     before it for the history dots."""
@@ -402,7 +405,12 @@ def _autotag_job() -> None:
 
 
 def _shorts_job() -> None:
-    _job("shorts", run_shorts)
+    def work(db):
+        # A video waiting in someone's Takeout import becomes markable the moment the probe has
+        # said it is not a Short, so the import's waiting rows are settled right after it.
+        return run_shorts(db) | takeout.apply_waiting(db)
+
+    _job("shorts", work)
 
 
 def _subscriptions_job() -> None:
@@ -608,48 +616,140 @@ def trigger_all(actor_id: int | None = None) -> list[str]:
 # one channel. It runs on the same scheduler as a recorded run (trigger `ad_hoc`), so what it did and
 # spent is in `job_runs` like any job's, not a background task only the log ever heard of.
 CHANNEL_BACKFILL = "channel_backfill"
+# A user's Google Takeout import (`reader.takeout`): its target is the user.
+TAKEOUT_IMPORT = "takeout_import"
 # Every ad-hoc job id: the page names each one (a frontend contract test reads this tuple).
-AD_HOC_JOBS = (CHANNEL_BACKFILL,)
+AD_HOC_JOBS = (CHANNEL_BACKFILL, TAKEOUT_IMPORT)
+# What each one's target is called on the admin's pages: a channel's title, an importing user's name.
+_TARGET_TITLES: dict[str, Callable[..., str | None]] = {
+    CHANNEL_BACKFILL: catalog_queries.channel_title,
+    TAKEOUT_IMPORT: takeout.target_title,
+}
 
 
-def run_ad_hoc(job: str, fn: Callable[[], dict], *, actor_id: int, target: str) -> None:
+def ad_hoc_target_title(db, job: str, target: str) -> str | None:
+    """The name of an ad-hoc run's target (None for a job with none, or a target since gone)."""
+    title = _TARGET_TITLES.get(job)
+    return title(db, target) if title is not None else None
+
+
+# The ad-hoc runs added and not yet recorded, `job@target`: between the two the target's newest
+# run row is still the previous run's, so a page watching it asks `ad_hoc_queued` too.
+_queued: set[str] = set()
+_queued_guard = threading.Lock()
+# What the scheduler is told to run is only the key: the request itself waits here, the newest one
+# per key, until a fire of that key takes it. A request made while its key's run is under way is so
+# kept for the run after it, which that run asks for as it ends (`_running`): the scheduler would
+# drop a second fire of an id whose run has not returned (one instance per job id).
+_pending: dict[str, tuple[Callable[[], dict], int, bool]] = {}
+_running: set[str] = set()
+# Two at once, for the moment between a run letting go of its key and its thread returning: a fire
+# then is what takes the next request, and is not dropped.
+_AD_HOC_INSTANCES = 2
+
+
+def ad_hoc_queued(job: str, target: str) -> bool:
+    """Whether a run of `job` on `target` is queued and its run row not written yet."""
+    with _queued_guard:
+        return f"{job}@{target}" in _queued
+
+
+def run_ad_hoc(
+    job: str, fn: Callable[[], dict], *, actor_id: int, target: str, notify: bool = False
+) -> None:
     """Run `fn` now as `actor_id`'s run of `job` on `target`, a one-shot job on the scheduler. Its
     own exclusion (if any) lives in `fn`, which says so in its result; one id per job and target, so
-    a second request before the first has even started is that same run."""
+    a second request before the first has even started is that same run, and one made while it runs
+    is the run after it. `notify` posts its outcome to the user's inbox, for work whose page does not
+    watch it run."""
     if job not in AD_HOC_JOBS:
         raise KeyError(job)  # an unnamed job would show on the page as its bare id
-    sched = _scheduler
-    if sched is None:
+    if _scheduler is None:
         # Only between shutdown and exit: start_scheduler runs at every API startup.
         raise RuntimeError("the scheduler is not running")
+    key = f"{job}@{target}"
+    with _queued_guard:
+        _pending[key] = (fn, actor_id, notify)
+        _queued.add(key)
+        running = key in _running
+    if running:
+        log.info("ad-hoc %s for %s runs again once the run under way ends", job, target)
+        return
+    _fire(job, target)
+
+
+def _fire(job: str, target: str) -> None:
+    """Add the one-shot that takes `job@target`'s pending request. Raises (the request dropped) only
+    when the scheduler cannot take it."""
+    key = f"{job}@{target}"
+    sched = _scheduler
     try:
+        if sched is None:
+            raise RuntimeError("the scheduler is not running")
         sched.add_job(
-            _ad_hoc,
+            _ad_hoc_fired,
             "date",
-            args=(job, fn, actor_id, target),
-            id=f"{job}@{target}",
+            args=(job, target),
+            id=key,
             misfire_grace_time=None,
+            max_instances=_AD_HOC_INSTANCES,
         )
-    # silent-ok: not a failure — the run for this target is already queued, and it does this work.
+    # silent-ok: not a failure — a fire of this key is already queued, and it takes this request.
     except ConflictingIdError:
         log.info("ad-hoc %s for %s already queued", job, target)
+    except Exception:
+        with _queued_guard:
+            _pending.pop(key, None)
+            _queued.discard(key)
+        raise
 
 
-def _ad_hoc(job: str, fn: Callable[[], dict], actor_id: int, target: str) -> None:
+def _ad_hoc_fired(job: str, target: str) -> None:
+    key = f"{job}@{target}"
+    with _queued_guard:
+        request = _pending.pop(key, None)
+        if request is None:
+            return  # an earlier fire took it
+        _running.add(key)
+    fn, actor_id, notify = request
+    try:
+        _ad_hoc(job, fn, actor_id, target, notify)
+    finally:
+        with _queued_guard:
+            _running.discard(key)
+            again = key in _pending
+        if again:
+            try:
+                _fire(job, target)
+            except Exception:
+                # The request made meanwhile is lost with the scheduler: shut down, nothing runs.
+                log.exception("ad-hoc %s for %s not run again", job, target)
+
+
+def _ad_hoc(
+    job: str, fn: Callable[[], dict], actor_id: int, target: str, notify: bool = False
+) -> None:
+    key = f"{job}@{target}"
     try:
         run_id = jobruns.start(job, RunTrigger.AD_HOC, actor_id, target)
     except Exception:
         # No row to tell it in: the database is what failed. The log is the record.
         log.exception("ad-hoc %s for %s not started: its run could not be recorded", job, target)
         return
+    finally:
+        # Recorded (or never will be): the run row now speaks for it — unless another request came
+        # meanwhile, which is still queued.
+        with _queued_guard:
+            if key not in _pending:
+                _queued.discard(key)
     _opened(run_id, job)
     try:
-        _ad_hoc_work(job, fn, run_id)
+        _ad_hoc_work(job, fn, run_id, actor_id if notify else None)
     finally:
         _closed(run_id)
 
 
-def _ad_hoc_work(job: str, fn: Callable[[], dict], run_id: int) -> None:
+def _ad_hoc_work(job: str, fn: Callable[[], dict], run_id: int, notify_id: int | None) -> None:
     with logs.bind(run_id=run_id):
         # As in _run: only the work is inside the try.
         try:
@@ -658,8 +758,23 @@ def _ad_hoc_work(job: str, fn: Callable[[], dict], run_id: int) -> None:
             log.exception("job %s failed", job, extra=logs.event("scheduler.job.failed", job=job))
             err, failure = _failure(exc)
             _close(run_id, RunStatus.ERROR, error=err, failure=failure)
+            # The user's notice names the failure, never the upstream's own text: that is an English
+            # diagnostic, for the admin's run row and the log, not a user's inbox.
+            _notify_ad_hoc(
+                notify_id, job, RunStatus.ERROR, None, failure or {"failure": "unexpected"}
+            )
             return
-        _finish_ok(job, run_id, result)
+        summary, data = _finish_ok(job, run_id, result)
+        _notify_ad_hoc(notify_id, job, RunStatus.OK, summary, data)
+
+
+def _notify_ad_hoc(
+    user_id: int | None, job: str, status: RunStatus, summary: str | None, result: dict | None
+) -> None:
+    if user_id is None:
+        return
+    with SessionLocal() as db:
+        _notify_done(db, user_id, job, status, summary, result)
 
 
 # Room for every job at once (a "run all") beside a few ad-hoc runs: APScheduler's default pool of
@@ -700,3 +815,7 @@ def shutdown_scheduler() -> None:
     if _scheduler is not None:
         _scheduler.shutdown(wait=False)
         _scheduler = None
+    with _queued_guard:
+        # What it held will not run.
+        _queued.clear()
+        _pending.clear()

backend/app/sync/explore.py

@@ -14,6 +14,7 @@ from sqlalchemy import delete, or_, select
 from sqlalchemy.orm import Session
 
 from app import logs, sysconfig
+from app.catalog import service as catalog_service
 from app.catalog.models import Channel, ExploredChannel, SearchFind, Subscription, Video
 from app.reader import queries as reader_queries
 from app.sync.videos import insert_stubs, stub_from_playlist_item, apply_video_details
@@ -66,11 +67,13 @@ def _enrich_new(db: Session, yt: YouTubeClient, video_ids: list[str]) -> int:
 def explore_ingest_page(
     db: Session, yt: YouTubeClient, channel: Channel, page_token: str | None = None
 ) -> dict:
-    """Ingest one page (up to 50, newest-first) of the channel's uploads, marking the new
-    videos via_explore, then enrich them. Returns the count ingested and the next page token
-    (None when the back-catalogue is exhausted) so the caller can offer "load more". Stateless
-    pagination: the caller round-trips the token, so the channel's own backfill cursor (used by
-    the organic deep backfill) is never touched."""
+    """Ingest one page (up to 50, newest-first) of the channel's uploads, marking the new videos
+    via_explore while the channel is an exploration's (`from_explore`): of a channel someone follows
+    they are its uploads like any other, and a flag there would never be cleared, since subscribing
+    clears only an explored channel's. Then enrich them. Returns the count ingested and the next
+    page token (None when the back-catalogue is exhausted) so the caller can offer "load more".
+    Stateless pagination: the caller round-trips the token, so the channel's own backfill cursor
+    (used by the organic deep backfill) is never touched."""
     if not channel.uploads_playlist_id:
         return {"ingested": 0, "next_page_token": None}
     try:
@@ -86,9 +89,10 @@ def explore_ingest_page(
     for item in data.get("items", []):
         stub = stub_from_playlist_item(item, channel.id)
         if stub is not None:
-            stub["via_explore"] = True
+            stub["via_explore"] = channel.from_explore
             stubs.append(stub)
     inserted = insert_stubs(db, stubs)
+    catalog_service.settle_explored(db, channel)
     self_enriched = _enrich_new(db, yt, [s["id"] for s in stubs])
     log.info(
         "explore ingest channel=%s ingested=%s enriched=%s", channel.id, inserted, self_enriched
@@ -105,7 +109,9 @@ def purge_unkept_explored(db: Session) -> dict:
       - a live ExploredChannel row (someone explored it within the grace window — channel-level), or
       - a video has per-user state: watched/in-progress (VideoState), in a playlist (PlaylistItem),
         or surfaced by the user's own search (SearchFind) — these survive even if the channel goes.
-    A channel row is dropped only once it holds no videos at all. 0 quota."""
+    A channel row is dropped only once it holds no videos at all, and not while someone's Takeout
+    history shows it watched (their "watched but not followed" list names it from that row).
+    0 quota."""
     grace_days = sysconfig.get_int(db, "explore_grace_days")
     cutoff = datetime.now(timezone.utc) - timedelta(days=grace_days)
 
@@ -135,7 +141,9 @@ def purge_unkept_explored(db: Session) -> dict:
 
     has_videos = select(Video.id).where(Video.channel_id == Channel.id).exists()
     channels_deleted = db.execute(
-        delete(Channel).where(Channel.id.in_(orphan_ids), ~has_videos)
+        delete(Channel).where(
+            Channel.id.in_(orphan_ids), ~has_videos, ~reader_queries.in_a_watch_history(Channel.id)
+        )
     ).rowcount
     db.commit()
     return {

backend/app/sync/subscriptions.py

@@ -8,6 +8,7 @@ from sqlalchemy.orm import Session
 
 from app import logs
 from app.models import User
+from app.catalog import service as catalog_service
 from app.catalog.models import Channel, Subscription
 from app.utils import normalize_description
 from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail
@@ -109,6 +110,21 @@ def apply_channel_details(db: Session, items: list[dict], asked: Iterable[str])
         channel.details_synced_at = now
 
 
+def describe_channels(db: Session, yt: YouTubeClient, channel_ids: list[str]) -> None:
+    """Make sure each given channel is in the catalog with its details: a channel it lacks is added
+    as a stub, then all of them are asked for (`channels.list`, 1 unit a 50). Flushes; the caller
+    commits."""
+    if not channel_ids:
+        return
+    have = set(db.execute(select(Channel.id).where(Channel.id.in_(channel_ids))).scalars())
+    for cid in dict.fromkeys(channel_ids):
+        if cid not in have:
+            db.add(Channel(id=cid))
+    db.flush()
+    apply_channel_details(db, yt.get_channels(channel_ids), channel_ids)
+    db.flush()
+
+
 def should_prune(fetched_count: int, existing_count: int) -> bool:
     """Whether an import may delete the local subscriptions missing from this fetch.
 
@@ -182,6 +198,10 @@ def import_subscriptions(db: Session, user: User) -> dict:
                 db.add(sub_row)
             sub_row.yt_subscription_id = sub.get("yt_subscription_id")
         db.flush()
+        # A follow made on youtube.com keeps an explored channel as a follow made here does.
+        explored = select(Channel).where(Channel.id.in_(fetched), Channel.from_explore.is_(True))
+        for channel in db.execute(explored).scalars():
+            catalog_service.settle_explored(db, channel)
 
         # Prune subscriptions removed on YouTube — but only when this fetch is trustworthy
         # (see `should_prune`).

backend/app/sync/videos.py

@@ -420,11 +420,15 @@ def run_shorts_classification(db: Session, limit: int | None = None) -> dict:
     return {"probed": probed, "shorts": shorts}
 
 
-def ensure_videos(db: Session, yt: YouTubeClient, video_ids: list[str]) -> None:
+def ensure_videos(
+    db: Session, yt: YouTubeClient, video_ids: list[str], *, explore: bool = False
+) -> None:
     """Make sure every given video id exists in the catalog, fetching + ingesting the
     missing ones (and a stub channel for any unknown channel). Deleted/private videos that
     the API won't return are simply left out. The catalog's write for a playlist sync that names
-    videos it has never seen; commits."""
+    videos it has never seen; commits. `explore` adds the new ones as an exploration's
+    (`via_explore`): visible to who explores or follows their channel, and reclaimed by the cleanup
+    once nobody keeps them."""
     if not video_ids:
         return
     have = set(db.execute(select(Video.id).where(Video.id.in_(video_ids))).scalars().all())
@@ -458,7 +462,12 @@ def ensure_videos(db: Session, yt: YouTubeClient, video_ids: list[str]) -> None:
         seen.add(vid)
         # published_at is NOT NULL (0068); apply_video_details re-fills a missing date, but seed a now()
         # fallback here too so the constructor never carries an explicit NULL.
-        v = Video(id=vid, channel_id=cid, published_at=parse_dt(sn.get("publishedAt")) or now)
+        v = Video(
+            id=vid,
+            channel_id=cid,
+            published_at=parse_dt(sn.get("publishedAt")) or now,
+            via_explore=explore,
+        )
         apply_video_details(v, it)
         v.enriched_at = now
         db.add(v)

backend/app/sysconfig.py

@@ -140,6 +140,15 @@ SPECS: tuple[ConfigSpec, ...] = (
         max=100_000,
         applies=LIVE,
     ),
+    ConfigSpec(
+        "related_daily_limit_per_user",
+        "int",
+        "quota",
+        "related_daily_limit_per_user",
+        min=0,
+        max=100_000,
+        applies=LIVE,
+    ),
     # Live-search source: "scrape" (InnerTube, zero API quota) or "api" (search.list, 100u/page).
     ConfigSpec("search_source", "choice", "quota", "search_source", applies=LIVE),
     ConfigSpec(

backend/app/userscope.py

@@ -3,8 +3,9 @@
 **Ownership.** A user-owned table says whose a row is in one column (`user_id`, or `to_user_id` for
 something handed to them). Every read and write names that column through `owned_by` and fetches a
 row by id through `owned_row`, never with an inline `Model.user_id == …`: the rule "a row owned by
-someone else is answered like a missing one" then lives here, and a contract test
-(`test_unit_fences.py`) holds every module to it, core included.
+someone else is answered like a missing one" then lives here. A contract test
+(`test_unit_fences.py`) holds every module, core included, to the filter's form; a rule this module
+cannot say (own or shared, built-in or own) is written out where it applies, with its reason.
 
 **Recipients.** "An active, non-suspended account" — the download share-by-email and the recipient
 picker filter on it. Kept here so the two forms (SQL clauses for a query, a Python check for a loaded

backend/app/worker.py

@@ -31,6 +31,7 @@ from functools import partial
 from sqlalchemy import text
 from sqlalchemy.orm import Session
 
+import app.tables  # noqa: F401 — every table, before the first query
 from app import logs
 from app import logsink
 from app import sysconfig

backend/app/youtube/client.py

@@ -30,12 +30,26 @@ class YouTubeError(Exception):
     message is prose, it changes under us, and it is deliberately the poorer of the two here.
 
     `path` is the API endpoint that ANSWERED with the failure (`playlistItems`); empty when no
-    endpoint did — the request never got through (`transport`) or the token step refused first."""
+    endpoint did — the request never got through (`transport`) or the token step refused first.
 
-    def __init__(self, message: str, *, reason: str = "", path: str = ""):
+    `sent` is a `transport` failure that came after the request may have gone out (a read timeout,
+    a dropped answer): nothing answered, but YouTube may have counted the call."""
+
+    def __init__(self, message: str, *, reason: str = "", path: str = "", sent: bool = False):
         super().__init__(message)
         self.reason = reason
         self.path = path
+        self.sent = sent
+
+
+# The transport failures that stop a request before it reaches YouTube: nothing went out.
+_UNSENT = (
+    httpx.ConnectError,
+    httpx.ConnectTimeout,
+    httpx.PoolTimeout,
+    httpx.ProxyError,
+    httpx.UnsupportedProtocol,
+)
 
 
 def _error_reason(resp: httpx.Response) -> str:
@@ -162,6 +176,9 @@ def best_thumbnail(thumbnails: dict | None) -> str | None:
 
 
 class YouTubeClient:
+    # The quota units this client has recorded: what its calls have spent of the shared budget.
+    spent: int = 0
+
     def __init__(self, db, user: User):
         self.db = db
         self.user = user
@@ -189,9 +206,16 @@ class YouTubeClient:
             return self._http.request(method, url, **kwargs)
         except httpx.HTTPError as exc:
             raise YouTubeError(
-                f"YouTube API unreachable ({exc.__class__.__name__}: {exc})", reason="transport"
+                f"YouTube API unreachable ({exc.__class__.__name__}: {exc})",
+                reason="transport",
+                sent=not isinstance(exc, _UNSENT),
             ) from exc
 
+    def _spend(self, units: int) -> None:
+        """Record what a call YouTube answered cost — a refused one too: it is charged all the same."""
+        quota.record_usage(units)
+        self.spent += units
+
     # --- auth ---
     def _access_token(self) -> str:
         tok = self.user.token
@@ -269,7 +293,7 @@ class YouTubeClient:
         else:
             headers["Authorization"] = f"Bearer {self._access_token()}"
         resp = self._send("GET", f"{API_BASE}/{path}", params=p, headers=headers)
-        quota.record_usage(cost)
+        self._spend(cost)
         if resp.status_code != 200:
             raise _upstream_failure("GET", path, resp, expected)
         return resp.json()
@@ -401,7 +425,7 @@ class YouTubeClient:
             params={"id": subscription_id},
             headers={"Authorization": f"Bearer {self._access_token()}"},
         )
-        quota.record_usage(50)
+        self._spend(50)
         if resp.status_code not in (200, 204):
             raise _upstream_failure("DELETE", "subscriptions", resp)
 
@@ -431,7 +455,7 @@ class YouTubeClient:
     def _write(self, method: str, path: str, *, params: dict, json: dict | None = None) -> dict:
         headers = {"Authorization": f"Bearer {self._access_token()}"}
         resp = self._send(method, f"{API_BASE}/{path}", params=params, json=json, headers=headers)
-        quota.record_usage(50)
+        self._spend(50)
         if resp.status_code not in (200, 204):
             raise _upstream_failure(method, path, resp)
         return resp.json() if resp.content else {}

backend/app/youtube/innertube.py

@@ -0,0 +1,160 @@
+"""The website's own InnerTube API (`youtubei/v1/*`), read the way the website reads it — no quota.
+
+The zero-quota search (`search_scrape`) and the related videos (`related_scrape`) both POST here with
+the public WEB key and a client version, read out of a live results page. This deliberately calls an
+undocumented endpoint: if YouTube changes the JSON shape or blocks the host, each caller says so as a
+`ScrapeError`, never as an empty answer.
+
+The SOCS=CAI consent-skip cookie is sent like the Shorts probe's, so a server-side request isn't
+bounced to consent.youtube.com."""
+
+import re
+
+import httpx
+
+from app import logs
+
+log = logs.get_logger(__name__)
+
+_API_URL = "https://www.youtube.com/youtubei/v1/"
+_RESULTS_URL = "https://www.youtube.com/results"
+# Public WEB InnerTube key + a recent client version. These are also embedded in every results
+# page; we fetch the live pair once and cache it, falling back to these constants.
+_FALLBACK_KEY = "AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8"  # gitleaks:allow -- the public WEB key, in every results page
+_FALLBACK_CLIENT_VERSION = "2.20240101.00.00"
+
+_HEADERS = {
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
+    # SOCS skips YouTube's cookie-consent interstitial (otherwise every server-side request is
+    # redirected to consent.youtube.com).
+    "Cookie": "SOCS=CAI",
+    "Accept-Language": "en-US,en;q=0.9",
+}
+
+_KEY_RE = re.compile(r'"INNERTUBE_API_KEY":"([^"]+)"')
+_VER_RE = re.compile(r'"INNERTUBE_CLIENT_VERSION":"([^"]+)"')
+
+# Cached (api_key, client_version) extracted from a results page; refreshed on demand.
+_config: tuple[str, str] | None = None
+
+
+class ScrapeError(Exception):
+    """An InnerTube read could not be completed (network, blocked, or unexpected shape)."""
+
+
+def runs_text(node: dict | None) -> str | None:
+    """The text of an InnerTube text node: a `simpleText`, or its `runs` joined."""
+    if not node:
+        return None
+    if node.get("simpleText"):
+        return node["simpleText"]
+    runs = node.get("runs") or []
+    if runs:
+        return "".join(r.get("text", "") for r in runs)
+    return None
+
+
+def scraped_video(
+    vid: str, channel_id: object, channel_title: object, title: object, thumbnails: object
+) -> dict:
+    """One scraped video in the shape `YouTubeClient.search_videos` answers, so a page of either
+    source is ingested the same way. `published_at` is None (the website shows only a relative
+    "3 years ago"; the enrich fills the real date) and `live_broadcast` "none" (live/upcoming are
+    judged after the enrich, by `live_status`). The largest thumbnail is the last."""
+    thumbs = thumbnails if isinstance(thumbnails, list) else []
+    last = thumbs[-1] if thumbs else None
+    return {
+        "id": vid,
+        "channel_id": channel_id if isinstance(channel_id, str) and channel_id else None,
+        "channel_title": channel_title if isinstance(channel_title, str) else None,
+        "title": title if isinstance(title, str) else None,
+        "published_at": None,
+        "thumbnail_url": last.get("url") if isinstance(last, dict) else None,
+        "live_broadcast": "none",
+    }
+
+
+def video_renderer(node: dict) -> dict | None:
+    """A `videoRenderer` (search) or `compactVideoRenderer` (the older related column) flattened —
+    the two draw a video the same way. None without a video id; the channel is None when the byline
+    names none."""
+    vid = node.get("videoId")
+    if not isinstance(vid, str) or not vid:
+        return None
+    owner = node.get("ownerText") or node.get("longBylineText") or {}
+    runs = owner.get("runs") if isinstance(owner, dict) else None
+    first = runs[0] if isinstance(runs, list) and runs and isinstance(runs[0], dict) else {}
+    endpoint = (first.get("navigationEndpoint") or {}).get("browseEndpoint") or {}
+    return scraped_video(
+        vid,
+        endpoint.get("browseId"),
+        first.get("text"),
+        runs_text(node.get("title")),
+        (node.get("thumbnail") or {}).get("thumbnails"),
+    )
+
+
+def continuation_token(node: object) -> str | None:
+    """The "load more" cursor of a `continuationItemRenderer`, or None when `node` is not one."""
+    if not isinstance(node, dict):
+        return None
+    command = (node.get("continuationEndpoint") or {}).get("continuationCommand") or {}
+    token = command.get("token")
+    return token if isinstance(token, str) and token else None
+
+
+def _get_config(client: httpx.Client) -> tuple[str, str]:
+    """The current InnerTube (api_key, client_version), cached after the first fetch that FOUND them.
+
+    The fallback pair is never cached: it used to be, so one failed fetch (or a consent page with no
+    keys on it) at the wrong moment pinned every later search to a hard-coded key and version for
+    the life of the process — and when those went stale, search returned nothing, with no error
+    anywhere. Now the next call simply tries the live pair again."""
+    global _config
+    if _config is not None:
+        return _config
+    try:
+        html = client.get(_RESULTS_URL, params={"search_query": "music"}).text
+    except httpx.HTTPError as exc:
+        log.warning(
+            "the InnerTube config could not be fetched; falling back to the pinned key",
+            extra=logs.event("youtube.scrape.config_fallback", reason=type(exc).__name__),
+        )
+        return (_FALLBACK_KEY, _FALLBACK_CLIENT_VERSION)
+    key = _KEY_RE.search(html)
+    ver = _VER_RE.search(html)
+    if key and ver:
+        _config = (key.group(1), ver.group(1))
+        return _config
+    log.warning(
+        "the results page carried no InnerTube config; falling back to the pinned key",
+        extra=logs.event("youtube.scrape.config_fallback", reason="not_on_page"),
+    )
+    return (
+        key.group(1) if key else _FALLBACK_KEY,
+        ver.group(1) if ver else _FALLBACK_CLIENT_VERSION,
+    )
+
+
+def post(endpoint: str, body: dict, *, hl: str = "en", gl: str = "US") -> dict:
+    """POST `body` to `youtubei/v1/<endpoint>` as the WEB client in the `hl`/`gl` locale and return
+    the decoded JSON object. A network failure, an HTTP error or a body that is not a JSON object (a
+    block, consent or captcha page) raises `ScrapeError`; the caller judges the shape."""
+    with httpx.Client(timeout=20.0, headers=_HEADERS) as client:
+        api_key, client_version = _get_config(client)
+        context = {
+            "client": {"clientName": "WEB", "clientVersion": client_version, "hl": hl, "gl": gl}
+        }
+        try:
+            resp = client.post(
+                _API_URL + endpoint, params={"key": api_key}, json={"context": context, **body}
+            )
+            resp.raise_for_status()
+            data = resp.json()
+        except httpx.HTTPError as exc:
+            raise ScrapeError(f"InnerTube request failed: {exc}") from exc
+        except ValueError as exc:  # non-JSON body (blocked / consent / captcha)
+            raise ScrapeError("InnerTube returned a non-JSON response") from exc
+    if not isinstance(data, dict):
+        raise ScrapeError("InnerTube returned an unexpected shape")
+    return data

backend/app/youtube/related_scrape.py

@@ -0,0 +1,141 @@
+"""A video's related videos, read from the watch page's InnerTube `next` endpoint — no quota.
+
+The Data API has no related-videos call any more, so this is the only source: the same list the
+website shows beside a playing video. A page holds about 20 videos and a continuation token for the
+next 20. The returned page has the search scrape's shape (``{"items": [...], "next_page_token"}``)
+so the catalog ingests it the same way (`catalog.search.ingest_candidates`).
+
+YouTube draws each item as a `lockupViewModel` now; the older `compactVideoRenderer` is read too, so
+a rollback on its side does not empty the list. An item that names no single channel — a video by
+several channels draws an avatar stack, not one avatar — is left out: the catalog row needs its
+channel up front, and the stack does not say which one uploaded it.
+
+`hl`/`gl` pick the locale YouTube fills the list in: ``en``/``US`` keeps it on the video's topic,
+``hu``/``HU`` mixes in what is trending in Hungary (measured)."""
+
+from app.youtube.innertube import (
+    ScrapeError,
+    continuation_token,
+    post,
+    scraped_video,
+    video_renderer,
+)
+
+__all__ = ["ScrapeError", "related"]
+
+# Where a first page and a continuation page put their item list.
+_FIRST = (
+    "contents",
+    "twoColumnWatchNextResults",
+    "secondaryResults",
+    "secondaryResults",
+    "results",
+)
+_APPENDS = ("appendContinuationItemsAction", "reloadContinuationItemsCommand")
+
+
+def _at(node: object, *path: str | int) -> object:
+    """The value under `path` in nested dicts/lists, or None where the path stops."""
+    for key in path:
+        if isinstance(node, dict) and isinstance(key, str):
+            node = node.get(key)
+        elif isinstance(node, list) and isinstance(key, int) and -len(node) <= key < len(node):
+            node = node[key]
+        else:
+            return None
+    return node
+
+
+def _lockup(node: dict) -> dict | None:
+    if node.get("contentType") != "LOCKUP_CONTENT_TYPE_VIDEO" or not node.get("contentId"):
+        return None
+    meta = _at(node, "metadata", "lockupMetadataViewModel")
+    channel_id = _at(
+        meta,
+        "image",
+        "decoratedAvatarViewModel",
+        "rendererContext",
+        "commandContext",
+        "onTap",
+        "innertubeCommand",
+        "browseEndpoint",
+        "browseId",
+    )
+    channel_title = _at(
+        meta, "metadata", "contentMetadataViewModel", "metadataRows", 0, "metadataParts", 0
+    )
+    return scraped_video(
+        node["contentId"],
+        channel_id,
+        _at(channel_title, "text", "content"),
+        _at(meta, "title", "content"),
+        _at(node, "contentImage", "thumbnailViewModel", "image", "sources"),
+    )
+
+
+def _unavailable(data: dict) -> bool:
+    """Whether the watch page says the video is not available: its main column draws a
+    `backgroundPromoRenderer` ("This video isn't available anymore") and there is no related column.
+    YouTube's own word for "nothing related" — the one empty page that is not a shape we lost."""
+    sections = _at(data, "contents", "twoColumnWatchNextResults", "results", "results", "contents")
+    for section in sections if isinstance(sections, list) else []:
+        inner = _at(section, "itemSectionRenderer", "contents")
+        for node in inner if isinstance(inner, list) else []:
+            if isinstance(node, dict) and "backgroundPromoRenderer" in node:
+                return True
+    return False
+
+
+def _items(data: dict) -> list | None:
+    """The page's item list (first page or continuation), or None when it has neither frame."""
+    first = _at(data, *_FIRST)
+    if isinstance(first, list):
+        return first
+    if _unavailable(data):
+        return []
+    for endpoint in data.get("onResponseReceivedEndpoints") or []:
+        for key in _APPENDS:
+            items = _at(endpoint, key, "continuationItems")
+            if isinstance(items, list):
+                return items
+    return None
+
+
+def related(
+    video_id: str, continuation: str | None = None, *, hl: str = "en", gl: str = "US"
+) -> dict:
+    """One page of `video_id`'s related videos (`continuation` = the token a previous page gave).
+
+    A video YouTube says it no longer has is an empty page. An answer of a shape this does not know
+    — no related column, or one whose entries are all renderers it cannot read — is a `ScrapeError`:
+    not an empty page, which would read as "nothing related"."""
+    body = {"continuation": continuation} if continuation else {"videoId": video_id}
+    data = post("next", body, hl=hl, gl=gl)
+    items = _items(data)
+    if items is None:
+        raise ScrapeError("InnerTube returned no related list")
+    out: list[dict] = []
+    seen: set[str] = set()
+    token = None
+    known = 0  # entries drawn by a renderer read here (a video's, or the cursor)
+    for entry in items:
+        if not isinstance(entry, dict):
+            continue
+        if "lockupViewModel" in entry:
+            item = _lockup(entry["lockupViewModel"])
+        elif "compactVideoRenderer" in entry:
+            item = video_renderer(entry["compactVideoRenderer"])
+        else:
+            found = continuation_token(entry.get("continuationItemRenderer"))
+            known += found is not None
+            token = token or found
+            continue
+        known += 1
+        # A video by several channels draws an avatar stack, not one: no channel, left out.
+        if item and item["channel_id"] and item["id"] not in seen:
+            seen.add(item["id"])
+            out.append(item)
+    # A list with entries but none of them one we read: YouTube draws its items some new way.
+    if items and not known:
+        raise ScrapeError("InnerTube's related list holds no renderer this reads")
+    return {"items": out, "next_page_token": token}

backend/app/youtube/search_scrape.py

@@ -6,9 +6,9 @@ calls — and read the result list straight out of the JSON. The returned video
 through the normal enrich path (videos.list, 1 unit / 50), exactly as the official search does,
 so the search itself is effectively free and only the (cheap, shared) enrichment is charged.
 
-The SOCS=CAI consent-skip cookie is used like the Shorts probe so a server-side request isn't
-bounced to consent.youtube.com. This deliberately scrapes an undocumented endpoint: if YouTube
-changes the JSON shape or blocks the host, an admin can flip ``search_source`` back to ``api``.
+The request itself (the live key, the consent cookie) is `innertube`'s. This deliberately scrapes an
+undocumented endpoint: if YouTube changes the JSON shape or blocks the host, an admin can flip
+``search_source`` back to ``api``.
 
 The returned page mirrors :meth:`YouTubeClient.search_videos` exactly
 (``{"items": [...], "next_page_token": str | None}``) so the route is source-agnostic — the
@@ -17,110 +17,16 @@ only difference is the cursor is an InnerTube *continuation token*, not a Data A
 the real date is filled by the videos.list enrich. ``live_broadcast`` is reported as ``"none"``
 and live/upcoming filtering happens post-enrich via ``live_status`` (same as the API path)."""
 
-import re
+from app.youtube.innertube import ScrapeError, continuation_token, post, video_renderer
 
-import httpx
+__all__ = ["ScrapeError", "search"]
 
-from app import logs
-
-log = logs.get_logger(__name__)
-
-_SEARCH_URL = "https://www.youtube.com/youtubei/v1/search"
-_RESULTS_URL = "https://www.youtube.com/results"
-# Public WEB InnerTube key + a recent client version. These are also embedded in every results
-# page; we fetch the live pair once and cache it, falling back to these constants.
-_FALLBACK_KEY = "AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8"
-_FALLBACK_CLIENT_VERSION = "2.20240101.00.00"
 # sp / params value selecting the "type: video" search filter (base64 protobuf EgIQAQ==).
 _TYPE_VIDEO = "EgIQAQ=="
 
-_HEADERS = {
-    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
-    # SOCS skips YouTube's cookie-consent interstitial (otherwise every server-side request is
-    # redirected to consent.youtube.com).
-    "Cookie": "SOCS=CAI",
-    "Accept-Language": "en-US,en;q=0.9",
-}
-
 # The top-level keys a first page (contents) and a continuation page (onResponseReceivedCommands)
 # put their result list under.
 _RESULT_FRAMES = {"contents", "onResponseReceivedCommands"}
-_KEY_RE = re.compile(r'"INNERTUBE_API_KEY":"([^"]+)"')
-_VER_RE = re.compile(r'"INNERTUBE_CLIENT_VERSION":"([^"]+)"')
-
-# Cached (api_key, client_version) extracted from a results page; refreshed on demand.
-_config: tuple[str, str] | None = None
-
-
-class ScrapeError(Exception):
-    """A scrape-based search could not be completed (network, blocked, or unexpected shape)."""
-
-
-def _runs_text(node: dict | None) -> str | None:
-    if not node:
-        return None
-    if node.get("simpleText"):
-        return node["simpleText"]
-    runs = node.get("runs") or []
-    if runs:
-        return "".join(r.get("text", "") for r in runs)
-    return None
-
-
-def _get_config(client: httpx.Client) -> tuple[str, str]:
-    """The current InnerTube (api_key, client_version), cached after the first fetch that FOUND them.
-
-    The fallback pair is never cached: it used to be, so one failed fetch (or a consent page with no
-    keys on it) at the wrong moment pinned every later search to a hard-coded key and version for
-    the life of the process — and when those went stale, search returned nothing, with no error
-    anywhere. Now the next search simply tries the live pair again."""
-    global _config
-    if _config is not None:
-        return _config
-    try:
-        html = client.get(_RESULTS_URL, params={"search_query": "music"}).text
-    except httpx.HTTPError as exc:
-        log.warning(
-            "the InnerTube config could not be fetched; falling back to the pinned key",
-            extra=logs.event("youtube.scrape.config_fallback", reason=type(exc).__name__),
-        )
-        return (_FALLBACK_KEY, _FALLBACK_CLIENT_VERSION)
-    key = _KEY_RE.search(html)
-    ver = _VER_RE.search(html)
-    if key and ver:
-        _config = (key.group(1), ver.group(1))
-        return _config
-    log.warning(
-        "the results page carried no InnerTube config; falling back to the pinned key",
-        extra=logs.event("youtube.scrape.config_fallback", reason="not_on_page"),
-    )
-    return (
-        key.group(1) if key else _FALLBACK_KEY,
-        ver.group(1) if ver else _FALLBACK_CLIENT_VERSION,
-    )
-
-
-def _video_item(vr: dict) -> dict | None:
-    """Flatten a YouTube `videoRenderer` into the same shape as YouTubeClient.search_videos."""
-    vid = vr.get("videoId")
-    if not vid:
-        return None
-    owner = vr.get("ownerText") or vr.get("longBylineText") or {}
-    runs = owner.get("runs") or []
-    channel_id = channel_title = None
-    if runs:
-        channel_title = runs[0].get("text")
-        channel_id = runs[0].get("navigationEndpoint", {}).get("browseEndpoint", {}).get("browseId")
-    thumbs = (vr.get("thumbnail") or {}).get("thumbnails") or []
-    return {
-        "id": vid,
-        "channel_id": channel_id,
-        "channel_title": channel_title,
-        "title": _runs_text(vr.get("title")),
-        "published_at": None,  # scrape exposes only a relative time; enrich fills the real date
-        "thumbnail_url": thumbs[-1]["url"] if thumbs else None,
-        "live_broadcast": "none",  # live/upcoming filtered post-enrich via live_status
-    }
 
 
 def _collect_videos(obj, out: list[dict], seen: set[str]) -> None:
@@ -130,7 +36,7 @@ def _collect_videos(obj, out: list[dict], seen: set[str]) -> None:
     if isinstance(obj, dict):
         vr = obj.get("videoRenderer")
         if isinstance(vr, dict):
-            item = _video_item(vr)
+            item = video_renderer(vr)
             if item and item["id"] not in seen:
                 seen.add(item["id"])
                 out.append(item)
@@ -144,11 +50,9 @@ def _collect_videos(obj, out: list[dict], seen: set[str]) -> None:
 def _find_continuation(obj) -> str | None:
     """First continuationItemRenderer token (the 'load more' cursor), or None at the end."""
     if isinstance(obj, dict):
-        cir = obj.get("continuationItemRenderer")
-        if isinstance(cir, dict):
-            tok = cir.get("continuationEndpoint", {}).get("continuationCommand", {}).get("token")
-            if tok:
-                return tok
+        tok = continuation_token(obj.get("continuationItemRenderer"))
+        if tok:
+            return tok
         for v in obj.values():
             tok = _find_continuation(v)
             if tok:
@@ -167,34 +71,14 @@ def search(q: str, continuation: str | None = None) -> dict:
     `continuation` is the InnerTube continuation token from a previous page (None for the first
     page). Returns ``{"items": [...], "next_page_token": str | None}`` — the same shape as
     YouTubeClient.search_videos so the route handles both sources identically."""
-    with httpx.Client(timeout=20.0, headers=_HEADERS) as client:
-        api_key, client_version = _get_config(client)
-        context = {
-            "client": {
-                "clientName": "WEB",
-                "clientVersion": client_version,
-                "hl": "en",
-                "gl": "US",
-            }
-        }
-        body: dict = {"context": context}
-        if continuation:
-            body["continuation"] = continuation
-        else:
-            body["query"] = q
-            body["params"] = _TYPE_VIDEO
-        try:
-            resp = client.post(_SEARCH_URL, params={"key": api_key}, json=body)
-            resp.raise_for_status()
-            data = resp.json()
-        except httpx.HTTPError as exc:
-            raise ScrapeError(f"InnerTube request failed: {exc}") from exc
-        except ValueError as exc:  # non-JSON body (blocked / consent / captcha)
-            raise ScrapeError("InnerTube returned a non-JSON response") from exc
+    body: dict = (
+        {"continuation": continuation} if continuation else {"query": q, "params": _TYPE_VIDEO}
+    )
+    data = post("search", body)
 
     # Even a search with no results carries its result list's frame. Without one this is not "no
     # results" but a shape we do not know (YouTube moved, or a block page in JSON) — say so.
-    if not isinstance(data, dict) or not _RESULT_FRAMES & data.keys():
+    if not _RESULT_FRAMES & data.keys():
         raise ScrapeError("InnerTube returned an unexpected shape")
     items: list[dict] = []
     _collect_videos(data, items, set())

backend/requirements.lock

@@ -5,7 +5,8 @@
 # scans it, blocking. A security fix, or a yt-dlp release YouTube's changes force, may be ONE pin
 # bumped in place instead of a full regeneration, which would move every other package too — then
 # `pip install -r requirements.lock && pip check` must pass. In place so far: urllib3 2.7.0 -> 2.8.0 (PYSEC-2026-4175/4176/4177); yt-dlp 2026.7.4 -> 2026.8.19
-# (it marks the clients that need a stream PO token, so the downloader stops fetching into a 403).
+# (it marks the clients that need a stream PO token, so the downloader stops fetching into a 403);
+# certifi 2026.7.22 -> 2026.10.19 (GHSA-6q48-6c7w-9gw2); brotli 1.2.0 -> 1.2.1 (PYSEC-2026-4322).
 APScheduler==3.11.3
 Authlib==1.7.2
 Mako==1.4.1
@@ -20,8 +21,8 @@ anyio==4.14.2
 argon2-cffi-bindings==25.1.0
 argon2-cffi==25.1.0
 bgutil-ytdlp-pot-provider==1.3.1
-brotli==1.2.0
-certifi==2026.7.22
+brotli==1.2.1
+certifi==2026.10.19
 cffi==2.1.1
 charset-normalizer==3.5.1
 click==8.4.2

backend/tests/conftest.py

@@ -105,6 +105,11 @@ def pytest_configure(config):
         )
 
 
+def _write_xact(db) -> str | None:
+    """The id of the open transaction if it has written anything, else None (a read never takes one)."""
+    return db.scalar(text("SELECT pg_current_xact_id_if_assigned()::text"))
+
+
 def _request_dep(db, unfinished: list[str]):
     """`get_db` for a test request: the test's own session, plus the check a shared session hides.
 
@@ -114,15 +119,16 @@ def _request_dep(db, unfinished: list[str]):
     and the suite stays green over a handler that forgot to commit (deep review, debt burn). So a
     request that ends normally with a write it did not commit — flushed, or still only staged — is
     recorded, and the test fails on it at teardown. Only the ROOT transaction's end clears it: a
-    savepoint released inside the handler commits nothing. The test's own staged rows are flushed before the request starts — they are the
-    test's business, not the handler's. A request that raises is rolled back, as production's
+    savepoint released inside the handler commits nothing. What the test itself staged or wrote is
+    committed before the request starts — it is the test's business, not the handler's, and a
+    request then begins with no write open, so any write left at its end is the handler's. A request that raises is rolled back, as production's
     `close()` rolls it back: without that, what a failing handler flushed but never committed stayed
     readable to the rest of the test — the one state production cannot be in (an issue created
     on the tracker before a failed upload was kept only by an explicit commit, and nothing saw it
     missing)."""
 
     def dep(conn: HTTPConnection):
-        db.flush()
+        db.commit()
         state = {"dirty": False}
 
         def flushed(session, ctx):
@@ -138,7 +144,12 @@ def _request_dep(db, unfinished: list[str]):
         try:
             yield db
             staged = db.new or db.deleted or any(db.is_modified(o) for o in db.dirty)
-            if state["dirty"] or staged:
+            # Core DML (`db.execute(update(...))`) fires no flush and stages nothing; the database
+            # itself says whether a write is open — the request began with none, and a transaction
+            # that wrote and was not committed holds an id (see `release_read_connection`). A row
+            # lock (`FOR UPDATE`) takes one too: the handler ends what it locked.
+            core_dml = _write_xact(db) is not None
+            if state["dirty"] or staged or core_dml:
                 unfinished.append(f"{conn.scope.get('method', 'WS')} {conn.url.path}")
         except Exception:
             db.rollback()
@@ -220,7 +231,7 @@ def _configured(monkeypatch):
 @pytest.fixture
 def client(db, override_dep, _configured):
     """A TestClient wired to the test DB that goes through the REAL auth/session stack — no
-    `current_user` override — so login, the wallet gate and epoch revoke are exercised end to end.
+    `current_user` override — so login, the one-account session and epoch revoke are exercised end to end.
     Starlette's SessionMiddleware round-trips the signed cookie, so a login in one call authenticates
     the next on this same instance. No lifespan (the scheduler / HLS sweep stay off)."""
     override_dep(get_db, db.info["request"])
@@ -316,3 +327,15 @@ def _reset_log_erasures():
     from app import logs
 
     logs._erased.clear()
+
+
+@pytest.fixture(autouse=True)
+def _reset_ad_hoc_queue():
+    """The scheduler's ad-hoc queue is module-level too: a request a test left pending would read as
+    a later test's user's import still queued (ids restart from 1)."""
+    from app import scheduler
+
+    with scheduler._queued_guard:
+        scheduler._queued.clear()
+        scheduler._pending.clear()
+        scheduler._running.clear()

backend/tests/fixtures/innertube_next_continuation.json

@@ -0,0 +1,527 @@
+{
+ "onResponseReceivedEndpoints": [
+  {
+   "appendContinuationItemsAction": {
+    "continuationItems": [
+     {
+      "lockupViewModel": {
+       "contentImage": {
+        "thumbnailViewModel": {
+         "image": {
+          "sources": [
+           {
+            "url": "https://i.ytimg.com/vi/_HfdncCbMOE/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLARvB39pkdsiuUUQTsZjkhvbKrfcQ",
+            "width": 168,
+            "height": 94
+           },
+           {
+            "url": "https://i.ytimg.com/vi/_HfdncCbMOE/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLB15aZU5-tzwYCEbNTnZK3UX6ZCzQ",
+            "width": 336,
+            "height": 188
+           }
+          ]
+         },
+         "overlays": [
+          {
+           "thumbnailBottomOverlayViewModel": {
+            "badges": [
+             {
+              "thumbnailBadgeViewModel": {
+               "text": "1:08:18",
+               "badgeStyle": "THUMBNAIL_OVERLAY_BADGE_STYLE_DEFAULT",
+               "animationActivationTargetId": "_HfdncCbMOE",
+               "animatedText": "Now playing",
+               "animationActivationEntitySelectorType": "THUMBNAIL_BADGE_ANIMATION_ENTITY_SELECTOR_TYPE_PLAYER_STATE"
+              }
+             }
+            ]
+           }
+          }
+         ]
+        }
+       },
+       "metadata": {
+        "lockupMetadataViewModel": {
+         "title": {
+          "content": "MIT 6.S191 (2025): Large Language Models (Liquid AI)"
+         },
+         "image": {
+          "decoratedAvatarViewModel": {
+           "avatar": {
+            "avatarViewModel": {
+             "image": {
+              "sources": [
+               {
+                "url": "https://yt3.ggpht.com/ytc/AIdro_k517G0Dt8QgsF86GKzYkG3Rs8jSGCiYGJw04lx4_1KXQ=s68-c-k-c0x00ffffff-no-rj",
+                "width": 68,
+                "height": 68
+               }
+              ]
+             },
+             "avatarImageSize": "AVATAR_SIZE_M"
+            }
+           },
+           "a11yLabel": "Go to channel Alexander Amini",
+           "rendererContext": {
+            "commandContext": {
+             "onTap": {
+              "innertubeCommand": {
+               "browseEndpoint": {
+                "browseId": "UCtslD4DGH6PKyG_1gFAX7sg"
+               }
+              }
+             }
+            }
+           }
+          }
+         },
+         "metadata": {
+          "contentMetadataViewModel": {
+           "metadataRows": [
+            {
+             "metadataParts": [
+              {
+               "text": {
+                "content": "Alexander Amini"
+               }
+              }
+             ]
+            },
+            {
+             "metadataParts": [
+              {
+               "text": {
+                "content": "62K"
+               },
+               "accessibilityLabel": "62 thousand views",
+               "leadingIcon": {
+                "name": "PLAY_ARROW_OUTLINED",
+                "height": 12,
+                "width": 12,
+                "marginEndToken": {
+                 "spacing": 2
+                }
+               }
+              },
+              {
+               "text": {
+                "content": "1y ago"
+               },
+               "accessibilityLabel": "1 year ago"
+              }
+             ]
+            }
+           ],
+           "delimiter": " "
+          }
+         }
+        }
+       },
+       "contentId": "_HfdncCbMOE",
+       "contentType": "LOCKUP_CONTENT_TYPE_VIDEO",
+       "themedPalette": {
+        "lightThemePalette": {
+         "baseBackground": 4294964720,
+         "raisedBackground": 4294964720,
+         "additiveBackground": 566850118,
+         "textPrimary": 4281800969,
+         "textSecondary": 4285876787,
+         "outline": 860759827,
+         "keyLight": 4292983720,
+         "collectionStack": 4291337092,
+         "invertedBackground": 4287578649,
+         "textPrimaryInverse": 4294964720,
+         "textDisabled": 2135828243,
+         "dropShadow": 432632390,
+         "cardOutline": 180974150,
+         "overlayBackground": 3426162953,
+         "overlayTextPrimary": 4294964720,
+         "solidWash": 4283311891,
+         "solidRim": 4281800969,
+         "overlaySolidWash": 4294699488,
+         "overlaySolidRim": 4294964720,
+         "tonalWash": 2583688688,
+         "tonalRim": 202310912,
+         "overlayTonalWash": 436205040,
+         "overlayTonalRim": 436205040,
+         "outlineRim": 1280190227,
+         "overlayOutlineRim": 871618504,
+         "solidWashInverse": 4294170568,
+         "solidRimInverse": 2147215840,
+         "tonalWashInverse": 654308848,
+         "tonalRimInverse": 654043616,
+         "outlineRimInverse": 654308848,
+         "solidBackground": 4283311891,
+         "solidBackgroundInverse": 4291337092,
+         "tonalBackground": 868785028,
+         "tonalBackgroundInverse": 435410888,
+         "outlineInverse": 872147424,
+         "directionalLightStopPrimary": 4294699488,
+         "directionalLightStopSecondary": 4294964720,
+         "primary": 4279174400,
+         "filledStateLayer": 4281800969,
+         "tonalTouchLight": 4294964720,
+         "tonalHoverLight": 3439061472,
+         "tonalStateLayer": 420414720,
+         "overlayDisabledContainer": 871618504,
+         "overlayTonalTouchLight": 1291577824,
+         "overlayTonalHoverLight": 1291577824,
+         "overlayTonalStateLayer": 436205040,
+         "disabledContainer": 420414720,
+         "filledTouchLight": 4285876787,
+         "filledHoverLight": 4284627235,
+         "onOverlayDisabledContainer": 2583688688,
+         "overlayTonalContainer": 2583423456,
+         "overlayOutline": 872412656,
+         "onFilledContainer": 4294964720,
+         "filledContainer": 4280027906,
+         "tonalContainer": 641145097,
+         "overlayPrimary": 4294964720,
+         "onOverlayFilledContainer": 4279174400,
+         "overlayFilledTouchLight": 4294964720,
+         "overlayFilledHoverLight": 4294699488,
+         "overlayFilledStateLayer": 4292983720,
+         "overlayFilledContainer": 4294170568,
+         "onDisabledContainer": 1712260352,
+         "temporaryOutline": 860759827
+        },
+        "darkThemePalette": {
+         "baseBackground": 4280027906,
+         "raisedBackground": 4281800969,
+         "additiveBackground": 734622278,
+         "textPrimary": 4294699488,
+         "textSecondary": 4292983720,
+         "outline": 1072945096,
+         "keyLight": 4284627235,
+         "collectionStack": 4285876787,
+         "invertedBackground": 4293107837,
+         "textPrimaryInverse": 4280027906,
+         "textDisabled": 2146686920,
+         "dropShadow": 432632390,
+         "cardOutline": 180974150,
+         "overlayBackground": 3426162953,
+         "overlayTextPrimary": 4294964720,
+         "solidWash": 4294699488,
+         "solidRim": 4294964720,
+         "overlaySolidWash": 4294699488,
+         "overlaySolidRim": 4294964720,
+         "tonalWash": 217307080,
+         "tonalRim": 218101232,
+         "overlayTonalWash": 436205040,
+         "overlayTonalRim": 436205040,
+         "outlineRim": 871618504,
+         "overlayOutlineRim": 871618504,
+         "solidWashInverse": 4285876787,
+         "solidRimInverse": 871618504,
+         "tonalWashInverse": 872412656,
+         "tonalRimInverse": 870431656,
+         "outlineRimInverse": 1291843056,
+         "solidBackground": 4291337092,
+         "solidBackgroundInverse": 4283311891,
+         "tonalBackground": 435410888,
+         "tonalBackgroundInverse": 868785028,
+         "outlineInverse": 860759827,
+         "directionalLightStopPrimary": 4281800969,
+         "directionalLightStopSecondary": 4280027906,
+         "primary": 4294964720,
+         "filledStateLayer": 4292983720,
+         "tonalTouchLight": 1073474016,
+         "tonalHoverLight": 654043616,
+         "tonalStateLayer": 436205040,
+         "overlayDisabledContainer": 871618504,
+         "overlayTonalTouchLight": 1291577824,
+         "overlayTonalHoverLight": 1291577824,
+         "overlayTonalStateLayer": 436205040,
+         "disabledContainer": 436205040,
+         "filledTouchLight": 4294964720,
+         "filledHoverLight": 4294699488,
+         "onOverlayDisabledContainer": 2583688688,
+         "overlayTonalContainer": 2583423456,
+         "overlayOutline": 872412656,
+         "onFilledContainer": 4279174400,
+         "filledContainer": 4294170568,
+         "tonalContainer": 435410888,
+         "overlayPrimary": 4294964720,
+         "onOverlayFilledContainer": 4279174400,
+         "overlayFilledTouchLight": 4294964720,
+         "overlayFilledHoverLight": 4294699488,
+         "overlayFilledStateLayer": 4292983720,
+         "overlayFilledContainer": 4294170568,
+         "onDisabledContainer": 1728050672,
+         "temporaryOutline": 871618504
+        }
+       }
+      }
+     },
+     {
+      "lockupViewModel": {
+       "contentImage": {
+        "thumbnailViewModel": {
+         "image": {
+          "sources": [
+           {
+            "url": "https://i.ytimg.com/vi/P7Hkh2zOGQ0/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLAHInNtMierSAgtrGl5rLSKc-vdzQ",
+            "width": 168,
+            "height": 94
+           },
+           {
+            "url": "https://i.ytimg.com/vi/P7Hkh2zOGQ0/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLDXrm5g-rdjRIqqK309ImTdqpw4CA",
+            "width": 336,
+            "height": 188
+           }
+          ]
+         },
+         "overlays": [
+          {
+           "thumbnailBottomOverlayViewModel": {
+            "badges": [
+             {
+              "thumbnailBadgeViewModel": {
+               "text": "1:08:19",
+               "badgeStyle": "THUMBNAIL_OVERLAY_BADGE_STYLE_DEFAULT",
+               "animationActivationTargetId": "P7Hkh2zOGQ0",
+               "animatedText": "Now playing",
+               "animationActivationEntitySelectorType": "THUMBNAIL_BADGE_ANIMATION_ENTITY_SELECTOR_TYPE_PLAYER_STATE"
+              }
+             }
+            ]
+           }
+          }
+         ]
+        }
+       },
+       "metadata": {
+        "lockupMetadataViewModel": {
+         "title": {
+          "content": "MIT 6.S191 (2024): Google - Generative AI for Media"
+         },
+         "image": {
+          "decoratedAvatarViewModel": {
+           "avatar": {
+            "avatarViewModel": {
+             "image": {
+              "sources": [
+               {
+                "url": "https://yt3.ggpht.com/ytc/AIdro_k517G0Dt8QgsF86GKzYkG3Rs8jSGCiYGJw04lx4_1KXQ=s68-c-k-c0x00ffffff-no-rj",
+                "width": 68,
+                "height": 68
+               }
+              ]
+             },
+             "avatarImageSize": "AVATAR_SIZE_M"
+            }
+           },
+           "a11yLabel": "Go to channel Alexander Amini",
+           "rendererContext": {
+            "commandContext": {
+             "onTap": {
+              "innertubeCommand": {
+               "browseEndpoint": {
+                "browseId": "UCtslD4DGH6PKyG_1gFAX7sg"
+               }
+              }
+             }
+            }
+           }
+          }
+         },
+         "metadata": {
+          "contentMetadataViewModel": {
+           "metadataRows": [
+            {
+             "metadataParts": [
+              {
+               "text": {
+                "content": "Alexander Amini"
+               }
+              }
+             ]
+            },
+            {
+             "metadataParts": [
+              {
+               "text": {
+                "content": "49K"
+               },
+               "accessibilityLabel": "49 thousand views",
+               "leadingIcon": {
+                "name": "PLAY_ARROW_OUTLINED",
+                "height": 12,
+                "width": 12,
+                "marginEndToken": {
+                 "spacing": 2
+                }
+               }
+              },
+              {
+               "text": {
+                "content": "2y ago"
+               },
+               "accessibilityLabel": "2 years ago"
+              }
+             ]
+            }
+           ],
+           "delimiter": " "
+          }
+         }
+        }
+       },
+       "contentId": "P7Hkh2zOGQ0",
+       "contentType": "LOCKUP_CONTENT_TYPE_VIDEO",
+       "themedPalette": {
+        "lightThemePalette": {
+         "baseBackground": 4293981951,
+         "raisedBackground": 4293981951,
+         "additiveBackground": 558201801,
+         "textPrimary": 4278655801,
+         "textSecondary": 4280831104,
+         "outline": 856631890,
+         "keyLight": 4288855783,
+         "collectionStack": 4286094803,
+         "invertedBackground": 4279847311,
+         "textPrimaryInverse": 4293981951,
+         "textDisabled": 2131700306,
+         "dropShadow": 423984073,
+         "cardOutline": 172325833,
+         "overlayBackground": 3423017785,
+         "overlayTextPrimary": 4293981951,
+         "solidWash": 4279183954,
+         "solidRim": 4278655801,
+         "overlaySolidWash": 4292864764,
+         "overlaySolidRim": 4293981951,
+         "tonalWash": 2582705919,
+         "tonalRim": 201328143,
+         "overlayTonalWash": 435222271,
+         "overlayTonalRim": 435222271,
+         "outlineRim": 1276062290,
+         "overlayOutlineRim": 868669942,
+         "solidWashInverse": 4291222006,
+         "solidRimInverse": 2145381116,
+         "tonalWashInverse": 653326079,
+         "tonalRimInverse": 652208892,
+         "outlineRimInverse": 653326079,
+         "solidBackground": 4279183954,
+         "solidBackgroundInverse": 4286094803,
+         "tonalBackground": 863542739,
+         "tonalBackgroundInverse": 432462326,
+         "outlineInverse": 870312700,
+         "directionalLightStopPrimary": 4292864764,
+         "directionalLightStopSecondary": 4293981951,
+         "primary": 4278191631,
+         "filledStateLayer": 4278655801,
+         "tonalTouchLight": 4293981951,
+         "tonalHoverLight": 3437226748,
+         "tonalStateLayer": 419431951,
+         "overlayDisabledContainer": 868669942,
+         "overlayTonalTouchLight": 1289743100,
+         "overlayTonalHoverLight": 1289743100,
+         "overlayTonalStateLayer": 435222271,
+         "disabledContainer": 419431951,
+         "filledTouchLight": 4280831104,
+         "filledHoverLight": 4280040297,
+         "onOverlayDisabledContainer": 2582705919,
+         "overlayTonalContainer": 2581588732,
+         "overlayOutline": 871429887,
+         "onFilledContainer": 4293981951,
+         "filledContainer": 4278324509,
+         "tonalContainer": 637999929,
+         "overlayPrimary": 4293981951,
+         "onOverlayFilledContainer": 4278191631,
+         "overlayFilledTouchLight": 4293981951,
+         "overlayFilledHoverLight": 4292864764,
+         "overlayFilledStateLayer": 4288855783,
+         "overlayFilledContainer": 4291222006,
+         "onDisabledContainer": 1711277583,
+         "temporaryOutline": 856631890
+        },
+        "darkThemePalette": {
+         "baseBackground": 4278324509,
+         "raisedBackground": 4278655801,
+         "additiveBackground": 725973961,
+         "textPrimary": 4292864764,
+         "textSecondary": 4288855783,
+         "outline": 1069996534,
+         "keyLight": 4280040297,
+         "collectionStack": 4280831104,
+         "invertedBackground": 4286359267,
+         "textPrimaryInverse": 4278324509,
+         "textDisabled": 2143738358,
+         "dropShadow": 423984073,
+         "cardOutline": 172325833,
+         "overlayBackground": 3423017785,
+         "overlayTextPrimary": 4293981951,
+         "solidWash": 4292864764,
+         "solidRim": 4293981951,
+         "overlaySolidWash": 4292864764,
+         "overlaySolidRim": 4293981951,
+         "tonalWash": 214358518,
+         "tonalRim": 217118463,
+         "overlayTonalWash": 435222271,
+         "overlayTonalRim": 435222271,
+         "outlineRim": 868669942,
+         "overlayOutlineRim": 868669942,
+         "solidWashInverse": 4280831104,
+         "solidRimInverse": 868669942,
+         "tonalWashInverse": 871429887,
+         "tonalRimInverse": 866303719,
+         "outlineRimInverse": 1290860287,
+         "solidBackground": 4286094803,
+         "solidBackgroundInverse": 4279183954,
+         "tonalBackground": 432462326,
+         "tonalBackgroundInverse": 863542739,
+         "outlineInverse": 856631890,
+         "directionalLightStopPrimary": 4278655801,
+         "directionalLightStopSecondary": 4278324509,
+         "primary": 4293981951,
+         "filledStateLayer": 4288855783,
+         "tonalTouchLight": 1071639292,
+         "tonalHoverLight": 652208892,
+         "tonalStateLayer": 435222271,
+         "overlayDisabledContainer": 868669942,
+         "overlayTonalTouchLight": 1289743100,
+         "overlayTonalHoverLight": 1289743100,
+         "overlayTonalStateLayer": 435222271,
+         "disabledContainer": 435222271,
+         "filledTouchLight": 4293981951,
+         "filledHoverLight": 4292864764,
+         "onOverlayDisabledContainer": 2582705919,
+         "overlayTonalContainer": 2581588732,
+         "overlayOutline": 871429887,
+         "onFilledContainer": 4278191631,
+         "filledContainer": 4291222006,
+         "tonalContainer": 432462326,
+         "overlayPrimary": 4293981951,
+         "onOverlayFilledContainer": 4278191631,
+         "overlayFilledTouchLight": 4293981951,
+         "overlayFilledHoverLight": 4292864764,
+         "overlayFilledStateLayer": 4288855783,
+         "overlayFilledContainer": 4291222006,
+         "onDisabledContainer": 1727067903,
+         "temporaryOutline": 868669942
+        }
+       }
+      }
+     },
+     {
+      "continuationItemRenderer": {
+       "trigger": "CONTINUATION_TRIGGER_ON_ITEM_SHOWN",
+       "continuationEndpoint": {
+        "commandMetadata": {
+         "webCommandMetadata": {
+          "sendPost": true,
+          "apiUrl": "/youtubei/v1/next"
+         }
+        },
+        "continuationCommand": {
+         "token": "CCgSDRILU2RUWkFNREtyTlkYACqPDE1nb0lodVQ4eG9EdXVLSUYyczZMNnd6NENBcjFDQW9EOGo0QUNnN0NQZ3NJbk1XcnZzbWo2Y2UyQVFvRDhqNEFDZzNDUGdvSW90N0czWnVCMy1KSENnUHlQZ0FLRGNJLUNnaTVfOGVUbjV5cTJSd0tBX0ktQUFvT3dqNExDSkQydWJfNXp2ZXZfUUVLQV9JLUFBb053ajRLQ1BXWjVQakt2cjZWU2dvRDhqNEFDZzNDUGdvSW9aTDd1OUtGdF85NkNnUHlQZ0FLRGNJLUNnaUh3N2FLMzd6YzVIZ0tBX0ktQUFvT3dqNExDUE94aHRQTWtfVzB1UUVLQV9JLUFBb093ajRMQ092eTBzaXNsSWZScGdFS0FfSS1BQW9Pd2o0TENKS1FxdHpBeE5lZHJBRUtBX0ktQUFvTndqNEtDTFdteWE2cHNPMkJPZ29EOGo0QUNnM0NQZ29JanYtdy1wQ3k1YVozQ2dQeVBnQUtEc0ktQ3dqMmtmVF9tYlB1XzRvQkNnUHlQZ0FLRHNJLUN3aVpuYWZjc0tQVmpZUUJDZ1B5UGdBS0RjSS1DZ2lxM3BxM3FyX2s0U0VLQV9JLUFBb053ajRLQ0phNXU5eVJvTmJOS0FvRDhqNEFDZzNDUGdvSXFMeWR3T1NhNGJkV0NnUHlQZ0FLRGNJLUNnakh3X2ZucktlSDdXUUtBX0ktQUFvTndqNEtDUG4xak1mcXg4QzJNQW9EOGo0QUNnM0NQZ29JeWRpdXV0VDQ3UHNhQ2dQeVBnQUtEc0ktQ3dqaDRleUUzTFAzdV93QkNnUHlQZ0FLRGNJLUNnaU5zcmptOXBENTJEOEtBX0ktQUFvTndqNEtDTExudjR5ZXg3U1dTUW9EOGo0QUNnM0NQZ29JNnZHODhaR1JpTWNnQ2dQeVBnQUtEc0ktQ3dpRF85S3Z5WS1hbThJQkNnUHlQZ0FLRHNJLUN3alppNmJYbWR6YWxZb0JDZ1B5UGdBS0RzSS1Dd2lkemJfS2dPdWN0YUFCQ2dQeVBnQUtEY0ktQ2dqMXktMjYyb2JQeFdNS0FfSS1BQW9Pd2o0TENPLUE4SWVaMUlESXJRRUtBX0ktQUFvTndqNEtDUHlZNDVhUzhkeWpJZ29EOGo0QUNnM0NQZ29Jb191aWtfSE05ZHdTQ2dQeVBnQUtEc0ktQ3dqaTBiemlrdnpBbkxvQkNnUHlQZ0FLRHNJLUN3aW05cXI1eXNtTWlzOEJDZ1B5UGdBS0RjSS1DZ2k3M2V5NXlzQ1pzMG9LQV9JLUFBb093ajRMQ0xmT281WFU1X2JPbFFFS0FfSS1BQW9Od2o0S0NLTEVwYXlXOTliWkdnb0Q4ajRBQ2c3Q1Bnc0k2dWZSMnRPM3hLem9BUW9EOGo0QUNnN0NQZ3NJN2ZlZXZQRHV2WlRXQVFvRDhqNEFDZzNDUGdvSXFibS0zYXVBdDVWcUNnUHlQZ0FLRGNJLUNnaUc1UHpHZ082NG9nVVNLQUFDQkFZSUNnd09FQklVRmhnYUhCNGdJaVFtS0Nvc0xqQXlORFk0T2p3LVFFSkVSa2hLVEU0YUJBZ0FFQUVhQkFnQ0VBTWFCQWdFRUFVYUJBZ0dFQWNhQkFnSUVBa2FCQWdLRUFzYUJBZ01FQTBhQkFnT0VBOGFCQWdRRUJFYUJBZ1NFQk1hQkFnVUVCVWFCQWdXRUJjYUJBZ1lFQmthQkFnYUVCc2FCQWdjRUIwYUJBZ2VFQjhhQkFnZ0VDRWFCQWdpRUNNYUJBZ2tFQ1VhQkFnbUVDY2FCQWdvRUNrYUJBZ3FFQ3NhQkFnc0VDMGFCQWd1RUM4YUJBZ3dFREVhQkFneUVETWFCQWcwRURVYUJBZzJFRGNhQkFnNEVEa2FCQWc2RURzYUJBZzhFRDBhQkFnLUVEOGFCQWhBRUVFYUJBaENFRU1hQkFoRUVFVWFCQWhHRUVjYUJBaElFRWthQkFoS0VFc2FCQWhNRUUwYUJBaE9FRThxS0FBQ0JBWUlDZ3dPRUJJVUZoZ2FIQjRnSWlRbUtDb3NMakF5TkRZNE9qdy1RRUpFUmtoS1RFNGoPd2F0Y2gtbmV4dC1mZWVk",
+         "request": "CONTINUATION_REQUEST_TYPE_WATCH_NEXT"
+        }
+       }
+      }
+     }
+    ]
+   }
+  }
+ ]
+}

backend/tests/fixtures/innertube_next_first.json

@@ -0,0 +1,805 @@
+{
+ "contents": {
+  "twoColumnWatchNextResults": {
+   "secondaryResults": {
+    "secondaryResults": {
+     "results": [
+      {
+       "lockupViewModel": {
+        "contentImage": {
+         "thumbnailViewModel": {
+          "image": {
+           "sources": [
+            {
+             "url": "https://i.ytimg.com/vi/to-lHJfK4pw/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLBKP_oFChvoqvwVrKAy_PG9bn2_AA",
+             "width": 168,
+             "height": 94
+            },
+            {
+             "url": "https://i.ytimg.com/vi/to-lHJfK4pw/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLDyH3Z8DJhXMmFJ5eKEWMufk7ql9g",
+             "width": 336,
+             "height": 188
+            }
+           ]
+          },
+          "overlays": [
+           {
+            "thumbnailBottomOverlayViewModel": {
+             "badges": [
+              {
+               "thumbnailBadgeViewModel": {
+                "text": "1:02:00",
+                "badgeStyle": "THUMBNAIL_OVERLAY_BADGE_STYLE_DEFAULT",
+                "animationActivationTargetId": "to-lHJfK4pw",
+                "animatedText": "Now playing",
+                "animationActivationEntitySelectorType": "THUMBNAIL_BADGE_ANIMATION_ENTITY_SELECTOR_TYPE_PLAYER_STATE"
+               }
+              }
+             ]
+            }
+           }
+          ]
+         }
+        },
+        "metadata": {
+         "lockupMetadataViewModel": {
+          "title": {
+           "content": "MIT 6.S191 (2025): Reinforcement Learning"
+          },
+          "image": {
+           "decoratedAvatarViewModel": {
+            "avatar": {
+             "avatarViewModel": {
+              "image": {
+               "sources": [
+                {
+                 "url": "https://yt3.ggpht.com/ytc/AIdro_k517G0Dt8QgsF86GKzYkG3Rs8jSGCiYGJw04lx4_1KXQ=s68-c-k-c0x00ffffff-no-rj",
+                 "width": 68,
+                 "height": 68
+                }
+               ]
+              },
+              "avatarImageSize": "AVATAR_SIZE_M"
+             }
+            },
+            "a11yLabel": "Go to channel Alexander Amini",
+            "rendererContext": {
+             "commandContext": {
+              "onTap": {
+               "innertubeCommand": {
+                "browseEndpoint": {
+                 "browseId": "UCtslD4DGH6PKyG_1gFAX7sg"
+                }
+               }
+              }
+             }
+            }
+           }
+          },
+          "metadata": {
+           "contentMetadataViewModel": {
+            "metadataRows": [
+             {
+              "metadataParts": [
+               {
+                "text": {
+                 "content": "Alexander Amini"
+                }
+               }
+              ]
+             },
+             {
+              "metadataParts": [
+               {
+                "text": {
+                 "content": "128K"
+                },
+                "accessibilityLabel": "128 thousand views",
+                "leadingIcon": {
+                 "name": "PLAY_ARROW_OUTLINED",
+                 "height": 12,
+                 "width": 12,
+                 "marginEndToken": {
+                  "spacing": 2
+                 }
+                }
+               },
+               {
+                "text": {
+                 "content": "1y ago"
+                },
+                "accessibilityLabel": "1 year ago"
+               }
+              ]
+             }
+            ],
+            "delimiter": " "
+           }
+          }
+         }
+        },
+        "contentId": "to-lHJfK4pw",
+        "contentType": "LOCKUP_CONTENT_TYPE_VIDEO",
+        "themedPalette": {
+         "lightThemePalette": {
+          "baseBackground": 4293981439,
+          "raisedBackground": 4293981439,
+          "additiveBackground": 558196425,
+          "textPrimary": 4278850870,
+          "textSecondary": 4282009454,
+          "outline": 857088843,
+          "keyLight": 4289509597,
+          "collectionStack": 4287338944,
+          "invertedBackground": 4279842703,
+          "textPrimaryInverse": 4293981439,
+          "textDisabled": 2132157259,
+          "dropShadow": 423978697,
+          "cardOutline": 172320457,
+          "overlayBackground": 3423212854,
+          "overlayTextPrimary": 4293981439,
+          "solidWash": 4279640907,
+          "solidRim": 4278850870,
+          "overlaySolidWash": 4292929530,
+          "overlaySolidRim": 4293981439,
+          "tonalWash": 2582705407,
+          "tonalRim": 201327631,
+          "overlayTonalWash": 435221759,
+          "overlayTonalRim": 435221759,
+          "outlineRim": 1276519243,
+          "overlayOutlineRim": 868930802,
+          "solidWashInverse": 4291482866,
+          "solidRimInverse": 2145445882,
+          "tonalWashInverse": 653325567,
+          "tonalRimInverse": 652273658,
+          "outlineRimInverse": 653325567,
+          "solidBackground": 4279640907,
+          "solidBackgroundInverse": 4287338944,
+          "tonalBackground": 864786880,
+          "tonalBackgroundInverse": 432723186,
+          "outlineInverse": 870377466,
+          "directionalLightStopPrimary": 4292929530,
+          "directionalLightStopSecondary": 4293981439,
+          "primary": 4278191119,
+          "filledStateLayer": 4278850870,
+          "tonalTouchLight": 4293981439,
+          "tonalHoverLight": 3437291514,
+          "tonalStateLayer": 419431439,
+          "overlayDisabledContainer": 868930802,
+          "overlayTonalTouchLight": 1289807866,
+          "overlayTonalHoverLight": 1289807866,
+          "overlayTonalStateLayer": 435221759,
+          "disabledContainer": 419431439,
+          "filledTouchLight": 4282009454,
+          "filledHoverLight": 4280825181,
+          "onOverlayDisabledContainer": 2582705407,
+          "overlayTonalContainer": 2581653498,
+          "overlayOutline": 871429375,
+          "onFilledContainer": 4293981439,
+          "filledContainer": 4278323484,
+          "tonalContainer": 638194998,
+          "overlayPrimary": 4293981439,
+          "onOverlayFilledContainer": 4278191119,
+          "overlayFilledTouchLight": 4293981439,
+          "overlayFilledHoverLight": 4292929530,
+          "overlayFilledStateLayer": 4289509597,
+          "overlayFilledContainer": 4291482866,
+          "onDisabledContainer": 1711277071,
+          "temporaryOutline": 857088843
+         },
+         "darkThemePalette": {
+          "baseBackground": 4278323484,
+          "raisedBackground": 4278850870,
+          "additiveBackground": 725968585,
+          "textPrimary": 4292929530,
+          "textSecondary": 4289509597,
+          "outline": 1070257394,
+          "keyLight": 4280825181,
+          "collectionStack": 4282009454,
+          "invertedBackground": 4286355427,
+          "textPrimaryInverse": 4278323484,
+          "textDisabled": 2143999218,
+          "dropShadow": 423978697,
+          "cardOutline": 172320457,
+          "overlayBackground": 3423212854,
+          "overlayTextPrimary": 4293981439,
+          "solidWash": 4292929530,
+          "solidRim": 4293981439,
+          "overlaySolidWash": 4292929530,
+          "overlaySolidRim": 4293981439,
+          "tonalWash": 214619378,
+          "tonalRim": 217117951,
+          "overlayTonalWash": 435221759,
+          "overlayTonalRim": 435221759,
+          "outlineRim": 868930802,
+          "overlayOutlineRim": 868930802,
+          "solidWashInverse": 4282009454,
+          "solidRimInverse": 868930802,
+          "tonalWashInverse": 871429375,
+          "tonalRimInverse": 866957533,
+          "outlineRimInverse": 1290859775,
+          "solidBackground": 4287338944,
+          "solidBackgroundInverse": 4279640907,
+          "tonalBackground": 432723186,
+          "tonalBackgroundInverse": 864786880,
+          "outlineInverse": 857088843,
+          "directionalLightStopPrimary": 4278850870,
+          "directionalLightStopSecondary": 4278323484,
+          "primary": 4293981439,
+          "filledStateLayer": 4289509597,
+          "tonalTouchLight": 1071704058,
+          "tonalHoverLight": 652273658,
+          "tonalStateLayer": 435221759,
+          "overlayDisabledContainer": 868930802,
+          "overlayTonalTouchLight": 1289807866,
+          "overlayTonalHoverLight": 1289807866,
+          "overlayTonalStateLayer": 435221759,
+          "disabledContainer": 435221759,
+          "filledTouchLight": 4293981439,
+          "filledHoverLight": 4292929530,
+          "onOverlayDisabledContainer": 2582705407,
+          "overlayTonalContainer": 2581653498,
+          "overlayOutline": 871429375,
+          "onFilledContainer": 4278191119,
+          "filledContainer": 4291482866,
+          "tonalContainer": 432723186,
+          "overlayPrimary": 4293981439,
+          "onOverlayFilledContainer": 4278191119,
+          "overlayFilledTouchLight": 4293981439,
+          "overlayFilledHoverLight": 4292929530,
+          "overlayFilledStateLayer": 4289509597,
+          "overlayFilledContainer": 4291482866,
+          "onDisabledContainer": 1727067391,
+          "temporaryOutline": 868930802
+         }
+        }
+       }
+      },
+      {
+       "lockupViewModel": {
+        "contentImage": {
+         "thumbnailViewModel": {
+          "image": {
+           "sources": [
+            {
+             "url": "https://i.ytimg.com/vi/R8V8CbuxryI/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLDqXMjrpgDyQKHL2lWrxczwxKoefw",
+             "width": 168,
+             "height": 94
+            },
+            {
+             "url": "https://i.ytimg.com/vi/R8V8CbuxryI/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLB7JX9Z238s6DGqIS5-fhcEIkUxKw",
+             "width": 336,
+             "height": 188
+            }
+           ]
+          },
+          "overlays": [
+           {
+            "thumbnailBottomOverlayViewModel": {
+             "badges": [
+              {
+               "thumbnailBadgeViewModel": {
+                "text": "49:36",
+                "badgeStyle": "THUMBNAIL_OVERLAY_BADGE_STYLE_DEFAULT",
+                "animationActivationTargetId": "R8V8CbuxryI",
+                "animatedText": "Now playing",
+                "animationActivationEntitySelectorType": "THUMBNAIL_BADGE_ANIMATION_ENTITY_SELECTOR_TYPE_PLAYER_STATE"
+               }
+              }
+             ]
+            }
+           }
+          ]
+         }
+        },
+        "metadata": {
+         "lockupMetadataViewModel": {
+          "title": {
+           "content": "MIT 6.S191: Deep Generative Modeling"
+          },
+          "image": {
+           "decoratedAvatarViewModel": {
+            "avatar": {
+             "avatarViewModel": {
+              "image": {
+               "sources": [
+                {
+                 "url": "https://yt3.ggpht.com/ytc/AIdro_k517G0Dt8QgsF86GKzYkG3Rs8jSGCiYGJw04lx4_1KXQ=s68-c-k-c0x00ffffff-no-rj",
+                 "width": 68,
+                 "height": 68
+                }
+               ]
+              },
+              "avatarImageSize": "AVATAR_SIZE_M"
+             }
+            },
+            "a11yLabel": "Go to channel Alexander Amini",
+            "rendererContext": {
+             "commandContext": {
+              "onTap": {
+               "innertubeCommand": {
+                "browseEndpoint": {
+                 "browseId": "UCtslD4DGH6PKyG_1gFAX7sg"
+                }
+               }
+              }
+             }
+            }
+           }
+          },
+          "metadata": {
+           "contentMetadataViewModel": {
+            "metadataRows": [
+             {
+              "metadataParts": [
+               {
+                "text": {
+                 "content": "Alexander Amini"
+                }
+               }
+              ]
+             },
+             {
+              "metadataParts": [
+               {
+                "text": {
+                 "content": "38K"
+                },
+                "accessibilityLabel": "38 thousand views",
+                "leadingIcon": {
+                 "name": "PLAY_ARROW_OUTLINED",
+                 "height": 12,
+                 "width": 12,
+                 "marginEndToken": {
+                  "spacing": 2
+                 }
+                }
+               },
+               {
+                "text": {
+                 "content": "6mo ago"
+                },
+                "accessibilityLabel": "6 months ago"
+               }
+              ]
+             }
+            ],
+            "delimiter": " "
+           }
+          }
+         }
+        },
+        "contentId": "R8V8CbuxryI",
+        "contentType": "LOCKUP_CONTENT_TYPE_VIDEO",
+        "themedPalette": {
+         "lightThemePalette": {
+          "baseBackground": 4294965488,
+          "raisedBackground": 4294965488,
+          "additiveBackground": 566856261,
+          "textPrimary": 4281737226,
+          "textSecondary": 4285420858,
+          "outline": 860565782,
+          "keyLight": 4292724140,
+          "collectionStack": 4290815883,
+          "invertedBackground": 4287583769,
+          "textPrimaryInverse": 4294965488,
+          "textDisabled": 2135634198,
+          "dropShadow": 432638533,
+          "cardOutline": 180980293,
+          "overlayBackground": 3426099210,
+          "overlayTextPrimary": 4294965488,
+          "solidWash": 4283117846,
+          "solidRim": 4281737226,
+          "overlaySolidWash": 4294635232,
+          "overlaySolidRim": 4294965488,
+          "tonalWash": 2583689456,
+          "tonalRim": 202311680,
+          "overlayTonalWash": 436205808,
+          "overlayTonalRim": 436205808,
+          "outlineRim": 1279996182,
+          "overlayOutlineRim": 871554762,
+          "solidWashInverse": 4294106826,
+          "solidRimInverse": 2147151584,
+          "tonalWashInverse": 654309616,
+          "tonalRimInverse": 653979360,
+          "outlineRimInverse": 654309616,
+          "solidBackground": 4283117846,
+          "solidBackgroundInverse": 4290815883,
+          "tonalBackground": 868263819,
+          "tonalBackgroundInverse": 435347146,
+          "outlineInverse": 872083168,
+          "directionalLightStopPrimary": 4294635232,
+          "directionalLightStopSecondary": 4294965488,
+          "primary": 4279175168,
+          "filledStateLayer": 4281737226,
+          "tonalTouchLight": 4294965488,
+          "tonalHoverLight": 3438997216,
+          "tonalStateLayer": 420415488,
+          "overlayDisabledContainer": 871554762,
+          "overlayTonalTouchLight": 1291513568,
+          "overlayTonalHoverLight": 1291513568,
+          "overlayTonalStateLayer": 436205808,
+          "disabledContainer": 420415488,
+          "filledTouchLight": 4285420858,
+          "filledHoverLight": 4284302120,
+          "onOverlayDisabledContainer": 2583689456,
+          "overlayTonalContainer": 2583359200,
+          "overlayOutline": 872413424,
+          "onFilledContainer": 4294965488,
+          "filledContainer": 4280029186,
+          "tonalContainer": 641081354,
+          "overlayPrimary": 4294965488,
+          "onOverlayFilledContainer": 4279175168,
+          "overlayFilledTouchLight": 4294965488,
+          "overlayFilledHoverLight": 4294635232,
+          "overlayFilledStateLayer": 4292724140,
+          "overlayFilledContainer": 4294106826,
+          "onDisabledContainer": 1712261120,
+          "temporaryOutline": 860565782
+         },
+         "darkThemePalette": {
+          "baseBackground": 4280029186,
+          "raisedBackground": 4281737226,
+          "additiveBackground": 734628421,
+          "textPrimary": 4294635232,
+          "textSecondary": 4292724140,
+          "outline": 1072881354,
+          "keyLight": 4284302120,
+          "collectionStack": 4285420858,
+          "invertedBackground": 4293112700,
+          "textPrimaryInverse": 4280029186,
+          "textDisabled": 2146623178,
+          "dropShadow": 432638533,
+          "cardOutline": 180980293,
+          "overlayBackground": 3426099210,
+          "overlayTextPrimary": 4294965488,
+          "solidWash": 4294635232,
+          "solidRim": 4294965488,
+          "overlaySolidWash": 4294635232,
+          "overlaySolidRim": 4294965488,
+          "tonalWash": 217243338,
+          "tonalRim": 218102000,
+          "overlayTonalWash": 436205808,
+          "overlayTonalRim": 436205808,
+          "outlineRim": 871554762,
+          "overlayOutlineRim": 871554762,
+          "solidWashInverse": 4285420858,
+          "solidRimInverse": 871554762,
+          "tonalWashInverse": 872413424,
+          "tonalRimInverse": 870172076,
+          "outlineRimInverse": 1291843824,
+          "solidBackground": 4290815883,
+          "solidBackgroundInverse": 4283117846,
+          "tonalBackground": 435347146,
+          "tonalBackgroundInverse": 868263819,
+          "outlineInverse": 860565782,
+          "directionalLightStopPrimary": 4281737226,
+          "directionalLightStopSecondary": 4280029186,
+          "primary": 4294965488,
+          "filledStateLayer": 4292724140,
+          "tonalTouchLight": 1073409760,
+          "tonalHoverLight": 653979360,
+          "tonalStateLayer": 436205808,
+          "overlayDisabledContainer": 871554762,
+          "overlayTonalTouchLight": 1291513568,
+          "overlayTonalHoverLight": 1291513568,
+          "overlayTonalStateLayer": 436205808,
+          "disabledContainer": 436205808,
+          "filledTouchLight": 4294965488,
+          "filledHoverLight": 4294635232,
+          "onOverlayDisabledContainer": 2583689456,
+          "overlayTonalContainer": 2583359200,
+          "overlayOutline": 872413424,
+          "onFilledContainer": 4279175168,
+          "filledContainer": 4294106826,
+          "tonalContainer": 435347146,
+          "overlayPrimary": 4294965488,
+          "onOverlayFilledContainer": 4279175168,
+          "overlayFilledTouchLight": 4294965488,
+          "overlayFilledHoverLight": 4294635232,
+          "overlayFilledStateLayer": 4292724140,
+          "overlayFilledContainer": 4294106826,
+          "onDisabledContainer": 1728051440,
+          "temporaryOutline": 871554762
+         }
+        }
+       }
+      },
+      {
+       "lockupViewModel": {
+        "contentImage": {
+         "thumbnailViewModel": {
+          "image": {
+           "sources": [
+            {
+             "url": "https://i.ytimg.com/vi/HLKo4fJx_7k/hqdefault.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLAQcvMQEX1ny9Z8qu2mIMogpEMaVA",
+             "width": 168,
+             "height": 94
+            },
+            {
+             "url": "https://i.ytimg.com/vi/HLKo4fJx_7k/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLCkoPSkHifs5ifpWXXcb96XG_Ly1w",
+             "width": 336,
+             "height": 188
+            }
+           ]
+          },
+          "overlays": [
+           {
+            "thumbnailBottomOverlayViewModel": {
+             "badges": [
+              {
+               "thumbnailBadgeViewModel": {
+                "text": "47:36",
+                "badgeStyle": "THUMBNAIL_OVERLAY_BADGE_STYLE_DEFAULT",
+                "animationActivationTargetId": "HLKo4fJx_7k",
+                "animatedText": "Now playing",
+                "animationActivationEntitySelectorType": "THUMBNAIL_BADGE_ANIMATION_ENTITY_SELECTOR_TYPE_PLAYER_STATE"
+               }
+              }
+             ]
+            }
+           }
+          ]
+         }
+        },
+        "metadata": {
+         "lockupMetadataViewModel": {
+          "title": {
+           "content": "MIT 6.S191 (2025): Language Models and New Frontiers"
+          },
+          "image": {
+           "decoratedAvatarViewModel": {
+            "avatar": {
+             "avatarViewModel": {
+              "image": {
+               "sources": [
+                {
+                 "url": "https://yt3.ggpht.com/ytc/AIdro_k517G0Dt8QgsF86GKzYkG3Rs8jSGCiYGJw04lx4_1KXQ=s68-c-k-c0x00ffffff-no-rj",
+                 "width": 68,
+                 "height": 68
+                }
+               ]
+              },
+              "avatarImageSize": "AVATAR_SIZE_M"
+             }
+            },
+            "a11yLabel": "Go to channel Alexander Amini",
+            "rendererContext": {
+             "commandContext": {
+              "onTap": {
+               "innertubeCommand": {
+                "browseEndpoint": {
+                 "browseId": "UCtslD4DGH6PKyG_1gFAX7sg"
+                }
+               }
+              }
+             }
+            }
+           }
+          },
+          "metadata": {
+           "contentMetadataViewModel": {
+            "metadataRows": [
+             {
+              "metadataParts": [
+               {
+                "text": {
+                 "content": "Alexander Amini"
+                }
+               }
+              ]
+             },
+             {
+              "metadataParts": [
+               {
+                "text": {
+                 "content": "63K"
+                },
+                "accessibilityLabel": "63 thousand views",
+                "leadingIcon": {
+                 "name": "PLAY_ARROW_OUTLINED",
+                 "height": 12,
+                 "width": 12,
+                 "marginEndToken": {
+                  "spacing": 2
+                 }
+                }
+               },
+               {
+                "text": {
+                 "content": "1y ago"
+                },
+                "accessibilityLabel": "1 year ago"
+               }
+              ]
+             }
+            ],
+            "delimiter": " "
+           }
+          }
+         }
+        },
+        "contentId": "HLKo4fJx_7k",
+        "contentType": "LOCKUP_CONTENT_TYPE_VIDEO",
+        "themedPalette": {
+         "lightThemePalette": {
+          "baseBackground": 4294964208,
+          "raisedBackground": 4294964208,
+          "additiveBackground": 566847045,
+          "textPrimary": 4281734410,
+          "textSecondary": 4285417018,
+          "outline": 860561942,
+          "keyLight": 4292720812,
+          "collectionStack": 4290812043,
+          "invertedBackground": 4287575065,
+          "textPrimaryInverse": 4294964208,
+          "textDisabled": 2135630358,
+          "dropShadow": 432629317,
+          "cardOutline": 180971077,
+          "overlayBackground": 3426096394,
+          "overlayTextPrimary": 4294964208,
+          "solidWash": 4283114006,
+          "solidRim": 4281734410,
+          "overlaySolidWash": 4294633184,
+          "overlaySolidRim": 4294964208,
+          "tonalWash": 2583688176,
+          "tonalRim": 202310656,
+          "overlayTonalWash": 436204528,
+          "overlayTonalRim": 436204528,
+          "outlineRim": 1279992342,
+          "overlayOutlineRim": 871551946,
+          "solidWashInverse": 4294104010,
+          "solidRimInverse": 2147149536,
+          "tonalWashInverse": 654308336,
+          "tonalRimInverse": 653977312,
+          "outlineRimInverse": 654308336,
+          "solidBackground": 4283114006,
+          "solidBackgroundInverse": 4290812043,
+          "tonalBackground": 868259979,
+          "tonalBackgroundInverse": 435344330,
+          "outlineInverse": 872081120,
+          "directionalLightStopPrimary": 4294633184,
+          "directionalLightStopSecondary": 4294964208,
+          "primary": 4279174144,
+          "filledStateLayer": 4281734410,
+          "tonalTouchLight": 4294964208,
+          "tonalHoverLight": 3438995168,
+          "tonalStateLayer": 420414464,
+          "overlayDisabledContainer": 871551946,
+          "overlayTonalTouchLight": 1291511520,
+          "overlayTonalHoverLight": 1291511520,
+          "overlayTonalStateLayer": 436204528,
+          "disabledContainer": 420414464,
+          "filledTouchLight": 4285417018,
+          "filledHoverLight": 4284298280,
+          "onOverlayDisabledContainer": 2583688176,
+          "overlayTonalContainer": 2583357152,
+          "overlayOutline": 872412144,
+          "onFilledContainer": 4294964208,
+          "filledContainer": 4280027394,
+          "tonalContainer": 641078538,
+          "overlayPrimary": 4294964208,
+          "onOverlayFilledContainer": 4279174144,
+          "overlayFilledTouchLight": 4294964208,
+          "overlayFilledHoverLight": 4294633184,
+          "overlayFilledStateLayer": 4292720812,
+          "overlayFilledContainer": 4294104010,
+          "onDisabledContainer": 1712260096,
+          "temporaryOutline": 860561942
+         },
+         "darkThemePalette": {
+          "baseBackground": 4280027394,
+          "raisedBackground": 4281734410,
+          "additiveBackground": 734619205,
+          "textPrimary": 4294633184,
+          "textSecondary": 4292720812,
+          "outline": 1072878538,
+          "keyLight": 4284298280,
+          "collectionStack": 4285417018,
+          "invertedBackground": 4293105020,
+          "textPrimaryInverse": 4280027394,
+          "textDisabled": 2146620362,
+          "dropShadow": 432629317,
+          "cardOutline": 180971077,
+          "overlayBackground": 3426096394,
+          "overlayTextPrimary": 4294964208,
+          "solidWash": 4294633184,
+          "solidRim": 4294964208,
+          "overlaySolidWash": 4294633184,
+          "overlaySolidRim": 4294964208,
+          "tonalWash": 217240522,
+          "tonalRim": 218100720,
+          "overlayTonalWash": 436204528,
+          "overlayTonalRim": 436204528,
+          "outlineRim": 871551946,
+          "overlayOutlineRim": 871551946,
+          "solidWashInverse": 4285417018,
+          "solidRimInverse": 871551946,
+          "tonalWashInverse": 872412144,
+          "tonalRimInverse": 870168748,
+          "outlineRimInverse": 1291842544,
+          "solidBackground": 4290812043,
+          "solidBackgroundInverse": 4283114006,
+          "tonalBackground": 435344330,
+          "tonalBackgroundInverse": 868259979,
+          "outlineInverse": 860561942,
+          "directionalLightStopPrimary": 4281734410,
+          "directionalLightStopSecondary": 4280027394,
+          "primary": 4294964208,
+          "filledStateLayer": 4292720812,
+          "tonalTouchLight": 1073407712,
+          "tonalHoverLight": 653977312,
+          "tonalStateLayer": 436204528,
+          "overlayDisabledContainer": 871551946,
+          "overlayTonalTouchLight": 1291511520,
+          "overlayTonalHoverLight": 1291511520,
+          "overlayTonalStateLayer": 436204528,
+          "disabledContainer": 436204528,
+          "filledTouchLight": 4294964208,
+          "filledHoverLight": 4294633184,
+          "onOverlayDisabledContainer": 2583688176,
+          "overlayTonalContainer": 2583357152,
+          "overlayOutline": 872412144,
+          "onFilledContainer": 4279174144,
+          "filledContainer": 4294104010,
+          "tonalContainer": 435344330,
+          "overlayPrimary": 4294964208,
+          "onOverlayFilledContainer": 4279174144,
+          "overlayFilledTouchLight": 4294964208,
+          "overlayFilledHoverLight": 4294633184,
+          "overlayFilledStateLayer": 4292720812,
+          "overlayFilledContainer": 4294104010,
+          "onDisabledContainer": 1728050160,
+          "temporaryOutline": 871551946
+         }
+        }
+       }
+      },
+      {
+       "continuationItemRenderer": {
+        "trigger": "CONTINUATION_TRIGGER_ON_ITEM_SHOWN",
+        "continuationEndpoint": {
+         "commandMetadata": {
+          "webCommandMetadata": {
+           "sendPost": true,
+           "apiUrl": "/youtubei/v1/next"
+          }
+         },
+         "continuationCommand": {
+          "token": "CBQSDRILU2RUWkFNREtyTlkYACqXBk1nb0l5ZGl1dXRUNDdQc2EyczZMNnd5LUJBcTdCQW9EOGo0QUNnN0NQZ3NJbk1XcnZzbWo2Y2UyQVFvRDhqNEFDZzNDUGdvSW90N0czWnVCMy1KSENnUHlQZ0FLRGNJLUNnaTVfOGVUbjV5cTJSd0tBX0ktQUFvT3dqNExDSkQydWJfNXp2ZXZfUUVLQV9JLUFBb053ajRLQ1BXWjVQakt2cjZWU2dvRDhqNEFDZzNDUGdvSW9aTDd1OUtGdF85NkNnUHlQZ0FLRGNJLUNnaUh3N2FLMzd6YzVIZ0tBX0ktQUFvT3dqNExDUE94aHRQTWtfVzB1UUVLQV9JLUFBb093ajRMQ092eTBzaXNsSWZScGdFS0FfSS1BQW9Pd2o0TENKS1FxdHpBeE5lZHJBRUtBX0ktQUFvTndqNEtDTFdteWE2cHNPMkJPZ29EOGo0QUNnM0NQZ29JanYtdy1wQ3k1YVozQ2dQeVBnQUtEc0ktQ3dqMmtmVF9tYlB1XzRvQkNnUHlQZ0FLRHNJLUN3aVpuYWZjc0tQVmpZUUJDZ1B5UGdBS0RjSS1DZ2lxM3BxM3FyX2s0U0VLQV9JLUFBb053ajRLQ0phNXU5eVJvTmJOS0FvRDhqNEFDZzNDUGdvSXFMeWR3T1NhNGJkV0NnUHlQZ0FLRGNJLUNnakh3X2ZucktlSDdXUUtBX0ktQUFvTndqNEtDUG4xak1mcXg4QzJNQW9EOGo0QUNnM0NQZ29JeWRpdXV0VDQ3UHNhRWhRQUFnUUdDQW9NRGhBU0ZCWVlHaHdlSUNJa0pob0VDQUFRQVJvRUNBSVFBeG9FQ0FRUUJSb0VDQVlRQnhvRUNBZ1FDUm9FQ0FvUUN4b0VDQXdRRFJvRUNBNFFEeG9FQ0JBUUVSb0VDQklRRXhvRUNCUVFGUm9FQ0JZUUZ4b0VDQmdRR1JvRUNCb1FHeG9FQ0J3UUhSb0VDQjRRSHhvRUNDQVFJUm9FQ0NJUUl4b0VDQ1FRSlJvRUNDWVFKeW9VQUFJRUJnZ0tEQTRRRWhRV0dCb2NIaUFpSkNZag93YXRjaC1uZXh0LWZlZWQ%3D",
+          "request": "CONTINUATION_REQUEST_TYPE_WATCH_NEXT"
+         }
+        },
+        "button": {
+         "buttonRenderer": {
+          "style": "STYLE_SUGGESTIVE",
+          "size": "SIZE_DEFAULT",
+          "isDisabled": false,
+          "text": {
+           "runs": [
+            {
+             "text": "Show more"
+            }
+           ]
+          },
+          "command": {
+           "commandMetadata": {
+            "webCommandMetadata": {
+             "sendPost": true,
+             "apiUrl": "/youtubei/v1/next"
+            }
+           },
+           "continuationCommand": {
+            "token": "CBQSDRILU2RUWkFNREtyTlkYACqXBk1nb0l5ZGl1dXRUNDdQc2EyczZMNnd5LUJBcTdCQW9EOGo0QUNnN0NQZ3NJbk1XcnZzbWo2Y2UyQVFvRDhqNEFDZzNDUGdvSW90N0czWnVCMy1KSENnUHlQZ0FLRGNJLUNnaTVfOGVUbjV5cTJSd0tBX0ktQUFvT3dqNExDSkQydWJfNXp2ZXZfUUVLQV9JLUFBb053ajRLQ1BXWjVQakt2cjZWU2dvRDhqNEFDZzNDUGdvSW9aTDd1OUtGdF85NkNnUHlQZ0FLRGNJLUNnaUh3N2FLMzd6YzVIZ0tBX0ktQUFvT3dqNExDUE94aHRQTWtfVzB1UUVLQV9JLUFBb093ajRMQ092eTBzaXNsSWZScGdFS0FfSS1BQW9Pd2o0TENKS1FxdHpBeE5lZHJBRUtBX0ktQUFvTndqNEtDTFdteWE2cHNPMkJPZ29EOGo0QUNnM0NQZ29JanYtdy1wQ3k1YVozQ2dQeVBnQUtEc0ktQ3dqMmtmVF9tYlB1XzRvQkNnUHlQZ0FLRHNJLUN3aVpuYWZjc0tQVmpZUUJDZ1B5UGdBS0RjSS1DZ2lxM3BxM3FyX2s0U0VLQV9JLUFBb053ajRLQ0phNXU5eVJvTmJOS0FvRDhqNEFDZzNDUGdvSXFMeWR3T1NhNGJkV0NnUHlQZ0FLRGNJLUNnakh3X2ZucktlSDdXUUtBX0ktQUFvTndqNEtDUG4xak1mcXg4QzJNQW9EOGo0QUNnM0NQZ29JeWRpdXV0VDQ3UHNhRWhRQUFnUUdDQW9NRGhBU0ZCWVlHaHdlSUNJa0pob0VDQUFRQVJvRUNBSVFBeG9FQ0FRUUJSb0VDQVlRQnhvRUNBZ1FDUm9FQ0FvUUN4b0VDQXdRRFJvRUNBNFFEeG9FQ0JBUUVSb0VDQklRRXhvRUNCUVFGUm9FQ0JZUUZ4b0VDQmdRR1JvRUNCb1FHeG9FQ0J3UUhSb0VDQjRRSHhvRUNDQVFJUm9FQ0NJUUl4b0VDQ1FRSlJvRUNDWVFKeW9VQUFJRUJnZ0tEQTRRRWhRV0dCb2NIaUFpSkNZag93YXRjaC1uZXh0LWZlZWQ%3D",
+            "request": "CONTINUATION_REQUEST_TYPE_WATCH_NEXT"
+           }
+          }
+         }
+        }
+       }
+      }
+     ]
+    }
+   }
+  }
+ }
+}

backend/tests/test_activity_reads.py

@@ -159,6 +159,15 @@ def test_the_run_list_filters_pages_and_names_the_channel_and_the_log_lines(db,
     assert facets["total"] == 4
 
 
+def test_the_run_list_names_the_user_a_takeout_import_ran_for(db, make_user, as_user):
+    admin = make_user(role="admin")
+    importer = make_user("importer@example.com")
+    imported = _run(db, "takeout_import", "ok", importer.id, T0, f"user:{importer.id}", "ad_hoc")
+    db.commit()
+    (row,) = as_user(admin).get("/api/admin/runs").json()["items"]
+    assert (row["id"], row["target_title"]) == (imported, "importer@example.com")
+
+
 def test_the_quota_breakdown_covers_its_range_and_one_account(db, make_user, as_user):
     admin = make_user(role="admin")
     now = datetime.now(timezone.utc)

backend/tests/test_auth_session.py

@@ -1,21 +1,25 @@
 """Auth / session security core (TEST-1), DB-lane.
 
-The wallet impersonation gate, epoch-based session revocation, single-use tokens and the
-enumeration-safe uniform 401 are the security mag, and had zero tests (only `_oauth_redirect_uri` was
-covered). Starlette's SessionMiddleware round-trips the signed cookie under TestClient, so the whole
-browser-less flow is exercisable. Uses the real auth stack via the `client` fixture (NO current_user
-override) so login, the wallet gate and epoch revoke are the code under test.
+One account per browser session, epoch-based session revocation, single-use tokens and the
+enumeration-safe uniform 401 are the security mag. Starlette's SessionMiddleware round-trips the
+signed cookie under TestClient, so the whole browser-less flow is exercisable. Uses the real auth
+stack via the `client` fixture (NO current_user override) so login, the session and epoch revoke are
+the code under test.
 
 Note: `/api/me` is the bootstrap probe — it returns 200 `{authenticated: False}` when logged out, not a
 401. A revoked session is therefore checked on a PROTECTED endpoint (`/api/feed`), which does 401.
 """
 
+import base64
+import json
 from datetime import timedelta
+from pathlib import Path
 
 import pytest
 from fastapi.testclient import TestClient
+from itsdangerous import TimestampSigner
 
-from app.auth import ACTIVE_ACCOUNT_HEADER
+from app.config import settings
 from app.main import app
 
 _PW = "correct horse battery"
@@ -46,74 +50,48 @@ def test_wrong_password_and_unknown_email_give_a_byte_identical_401(client, make
     assert wrong_pw.json() == unknown.json()
 
 
-def test_a_non_wallet_account_header_falls_back_to_the_default(client, make_user):
-    # The X-Siftlode-Account header may only re-target an account already signed into THIS browser
-    # (in the session wallet). A header naming someone else must be ignored, not honored — otherwise
-    # it's an impersonation gate hole.
-    make_user(email="alice2@example.com", password=_PW)
-    mallory = make_user(email="mallory@example.com", password=_PW)
-    assert _login(client, "alice2@example.com", _PW).status_code == 200
-    me = client.get("/api/me", headers={ACTIVE_ACCOUNT_HEADER: str(mallory.id)})
-    assert me.status_code == 200
-    assert me.json()["email"] == "alice2@example.com"  # still Alice, not Mallory
+def _cookie(payload: dict) -> str:
+    """A session cookie signed the way SessionMiddleware signs one — for a cookie an older build wrote."""
+    data = base64.b64encode(json.dumps(payload).encode())
+    return TimestampSigner(str(settings.secret_key)).sign(data).decode()
 
 
-def test_a_tab_pinned_to_an_account_signed_out_elsewhere_acts_as_nobody(client, make_user):
-    # Two accounts in one browser: a tab pinned to Alice, Bob the default. Alice signs out in another
-    # tab. The pinned tab used to go on acting as the default, Bob — its player checkpoints and pref
-    # saves landed in his account, and its own Sign out signed HIM out.
-    alice = make_user(email="alice3@example.com", password=_PW)
+def test_signing_in_as_another_account_replaces_the_first(client, make_user):
+    # One account per browser session: a second sign-in is a fresh session, not a second account
+    # beside the first — so signing out leaves nobody signed in here.
+    make_user(email="alice3@example.com", password=_PW)
     make_user(email="bob3@example.com", password=_PW)
     assert _login(client, "alice3@example.com", _PW).status_code == 200
-    assert _login(client, "bob3@example.com", _PW).status_code == 200  # Bob is now the default
-    tab = {ACTIVE_ACCOUNT_HEADER: str(alice.id)}
-    assert client.get("/api/feed", headers=tab).status_code == 200
-    assert client.post("/auth/logout", headers=tab).status_code == 200  # Alice signs out
+    assert _login(client, "bob3@example.com", _PW).status_code == 200
+    assert client.get("/api/me").json()["email"] == "bob3@example.com"
 
-    assert client.get("/api/feed", headers=tab).status_code == 401
-    assert client.post("/auth/logout", headers=tab).status_code == 200
-    assert client.get("/api/feed").status_code == 200  # Bob is still signed in, the session intact
-    # The bootstrap probe still answers as the default: how the stale tab learns who it is now.
-    assert client.get("/api/me", headers=tab).json()["email"] == "bob3@example.com"
+    assert client.post("/auth/logout").status_code == 200
+    assert client.get("/api/feed").status_code == 401
+    assert client.get("/api/me").json() == {"authenticated": False}
 
 
-@pytest.mark.parametrize("ended", ["revoked", "suspended"])
-def test_a_tab_pinned_to_an_account_whose_session_ended_comes_back_as_the_default(
-    client, make_user, db, ended
-):
-    # A tab pinned to Alice, Bob the default. Alice's session then ends while she is still in this
-    # browser's wallet ("log out everywhere" on another device, or an admin suspends her). The tab's
-    # bootstrap probe used to answer "signed out": the landing page with Bob still signed in, and no way
-    # off it but closing the tab (signing in as anyone else there kept the tab pinned to Alice).
+def test_no_header_or_parameter_picks_another_account(client, make_user):
+    # Alice signed in here before Bob: nothing a request carries makes it act as her any more.
     alice = make_user(email="alice4@example.com", password=_PW)
     make_user(email="bob4@example.com", password=_PW)
     assert _login(client, "alice4@example.com", _PW).status_code == 200
     assert _login(client, "bob4@example.com", _PW).status_code == 200
-    tab = {ACTIVE_ACCOUNT_HEADER: str(alice.id)}
-    assert client.get("/api/feed", headers=tab).status_code == 200
-    if ended == "revoked":
-        alice.session_epoch = (alice.session_epoch or 0) + 1
-    else:
-        alice.is_suspended = True
-    db.commit()
+    me = client.get(f"/api/me?account={alice.id}", headers={"X-Siftlode-Account": str(alice.id)})
+    assert me.json()["email"] == "bob4@example.com"
 
-    assert client.get("/api/me", headers=tab).json()["email"] == "bob4@example.com"
-    assert client.get("/api/feed", headers=tab).status_code == 401  # still nobody, never Bob
-    assert client.get("/api/feed").status_code == 200  # Bob's session is intact
+
+@pytest.mark.parametrize("path", ["/api/me/accounts", "/api/me/switch"])
+def test_there_is_no_account_list_or_switch(client, make_user, path):
+    make_user(email="alice5@example.com", password=_PW)
+    assert _login(client, "alice5@example.com", _PW).status_code == 200
+    r = client.get(path) if path.endswith("accounts") else client.post(path, json={"user_id": 1})
+    assert r.status_code in (404, 405)
 
 
 @pytest.mark.parametrize("ended", ["revoked", "suspended", "deactivated"])
-def test_switching_to_an_account_whose_session_ended_refuses_it_and_drops_it_from_the_list(
-    client, make_user, db, monkeypatch, ended
-):
-    # The account switcher lists this browser's wallet. A revoked account was dropped from it on a refused
-    # switch, but a suspended one stayed, refused on every click; a deactivated one was not refused at all —
-    # made the default, the next request found it inactive and signed every account here out.
-    monkeypatch.setattr("app.routes.me.is_allowed", lambda _db, _email: True)  # both hold an invite
-    alice = make_user(email="alice5@example.com", password=_PW)
-    make_user(email="bob5@example.com", password=_PW)
-    assert _login(client, "alice5@example.com", _PW).status_code == 200
-    assert _login(client, "bob5@example.com", _PW).status_code == 200
+def test_a_session_whose_account_ended_signs_the_browser_out(client, make_user, db, ended):
+    alice = make_user(email="alice6@example.com", password=_PW)
+    assert _login(client, "alice6@example.com", _PW).status_code == 200
     if ended == "revoked":
         alice.session_epoch = (alice.session_epoch or 0) + 1
     elif ended == "suspended":
@@ -121,10 +99,111 @@ def test_switching_to_an_account_whose_session_ended_refuses_it_and_drops_it_fro
     else:
         alice.is_active = False
     db.commit()
+    assert client.get("/api/feed").status_code == 401
+    assert "user_id" not in _session(client)  # the refusal ended the session, not just this request
+    assert client.get("/api/me").json() == {"authenticated": False}
 
-    assert client.post("/api/me/switch", json={"user_id": alice.id}).status_code == 403
-    assert [a["email"] for a in client.get("/api/me/accounts").json()] == ["bob5@example.com"]
-    assert client.get("/api/feed").status_code == 200  # Bob is still signed in
+
+def test_a_cookie_from_the_multi_account_build_signs_in_again(client, make_user):
+    # The old session held a wallet and one epoch per account. It is not read: whoever holds one
+    # signs in once more.
+    alice = make_user(email="alice7@example.com", password=_PW)
+    old = {"user_id": alice.id, "account_ids": [alice.id], "epochs": {str(alice.id): 0}}
+    client.cookies.set("session", _cookie(old))
+    assert client.get("/api/feed").status_code == 401
+    assert client.get("/api/me").json() == {"authenticated": False}
+
+
+def test_deleting_the_account_signs_the_browser_out(client, make_user):
+    make_user(email="alice8@example.com", password=_PW)
+    assert _login(client, "alice8@example.com", _PW).status_code == 200
+    assert client.delete("/api/me/account").status_code == 200
+    # The cookie names nobody now: the probe answers signed out without looking the account up.
+    assert "user_id" not in _session(client)
+
+
+def _session(client) -> dict:
+    """The decoded payload of the client's session cookie."""
+    raw = client.cookies.get("session")
+    if not raw:
+        return {}
+    data = TimestampSigner(str(settings.secret_key)).unsign(raw.encode())
+    return json.loads(base64.b64decode(data))
+
+
+@pytest.mark.parametrize("signed_in_as", ["starter", "someone_else"])
+def test_a_google_link_lands_only_on_the_account_still_signed_in(db, make_user, signed_in_as):
+    # The link marker names who started it; a sign-in as someone else before the callback must not
+    # get that Google identity attached to the account that started it from the other person's session.
+    from starlette.requests import Request
+
+    from app.routes.auth import _complete_link
+
+    starter, other = make_user(), make_user()
+    session = {"user_id": starter.id if signed_in_as == "starter" else other.id, "epoch": 0}
+    request = Request({"type": "http", "session": session, "headers": []})
+    resp = _complete_link(request, db, starter.id, True, "sub-123", {"email": ""}, {"scope": ""})
+    db.refresh(starter)
+    if signed_in_as == "starter":
+        assert resp.headers["location"] == "/?link=ok"
+        assert starter.google_sub == "sub-123"
+    else:
+        assert resp.headers["location"] == "/?link=error"
+        assert starter.google_sub is None
+
+
+def test_no_app_code_reads_a_second_account():
+    # The multi-account machinery is gone; a module reading its session keys or its header again
+    # would bring back a session that names more than one account.
+    app_dir = Path(__file__).parent.parent / "app"
+    hits = [
+        f"{path.relative_to(app_dir)}: {word}"
+        for path in app_dir.rglob("*.py")
+        for word in ("account_ids", '"epochs"', "x-siftlode-account", 'query_params.get("account")')
+        if word in path.read_text(encoding="utf-8").lower()
+    ]
+    assert hits == []
+
+
+def test_a_tab_built_for_the_account_signed_in_before_is_refused(client, make_user):
+    # Two tabs of one browser: one signed in as someone else, so the cookie is Bob's under both. The
+    # tab still showing Alice says so with every request; acting as Bob there would put her player
+    # positions, pref saves and Sign out on his account. Refused, and the session stays Bob's.
+    alice = make_user(email="alice5@example.com", password=_PW)
+    bob = make_user(email="bob5@example.com", password=_PW)
+    assert _login(client, "alice5@example.com", _PW).status_code == 200
+    assert _login(client, "bob5@example.com", _PW).status_code == 200
+
+    stale = client.get("/api/feed", headers={"X-Siftlode-Expect-Account": str(alice.id)})
+    assert stale.status_code == 401
+    assert stale.json()["detail"]["reason"] == "account_changed"
+    assert (
+        client.get("/api/feed", headers={"X-Siftlode-Expect-Account": str(bob.id)}).status_code
+        == 200
+    )
+    assert client.get("/api/feed").status_code == 200
+    # Nor may its Sign out end Bob's session.
+    assert (
+        client.post(
+            "/auth/logout", headers={"X-Siftlode-Expect-Account": str(alice.id)}
+        ).status_code
+        == 401
+    )
+    # The probe answers who is signed in now, so the stale tab can reload as Bob.
+    probe = client.get("/api/me", headers={"X-Siftlode-Expect-Account": str(alice.id)})
+    assert probe.json()["email"] == "bob5@example.com"
+
+
+def test_setting_a_password_keeps_the_caller_signed_in(client, make_user):
+    # The change ends every OTHER session (the epoch moves); the caller is stamped with the new one.
+    make_user(email="dave@example.com", password=_PW)
+    assert _login(client, "dave@example.com", _PW).status_code == 200
+    r = client.post(
+        "/auth/set-password",
+        json={"password": "a brand new passphrase", "current_password": _PW},
+    )
+    assert r.status_code == 200
+    assert client.get("/api/feed").status_code == 200
 
 
 def test_logout_others_revokes_the_other_session_but_keeps_the_caller(client, make_user):

backend/tests/test_autotag_orphans.py

@@ -8,6 +8,8 @@ drops an unknown id, so the view silently showed the whole feed under its old na
 from sqlalchemy import select
 
 from app.catalog.models import Channel
+from app.db import SessionLocal
+from app.sync.autotag import run_autotag_all
 from app.reader.models import ChannelTag, SavedView, Tag
 from app.reader import service as reader_service
 
@@ -66,3 +68,15 @@ def test_a_view_kept_tag_list_larger_than_a_bind_parameter_limit_still_cleans_up
     db.commit()
     assert reader_service.prune_orphan_system_tags(db) == 1
     db.commit()
+
+
+def test_the_job_commits_its_cleanup(db):
+    """The prune only flushes; the job ends its own transaction. Read from a session of its own,
+    the orphan is gone only once the job committed — the job's session would see its own delete
+    either way."""
+    orphan = Tag(user_id=None, name="Cooking", category="topic")
+    db.add(orphan)
+    db.commit()
+    assert run_autotag_all(db, only_missing=True)["orphan_tags_removed"] == 1
+    with SessionLocal() as fresh:
+        assert fresh.execute(select(Tag).where(Tag.name == "Cooking")).first() is None

backend/tests/test_body_limit.py

@@ -23,6 +23,7 @@ from app import bodylimit, debugsession, sysconfig
 from app.config import settings
 from app.models import SystemConfig
 from app.bodylimit import DEFAULT_MAX_BODY_BYTES, BodyLimitMiddleware
+from app.reader import takeout
 from app.routes import diagnostics as diagnostics_routes
 
 SMALL = b"x" * 16
@@ -221,6 +222,7 @@ def test_the_default_ceiling_clears_the_largest_body_a_default_route_takes():
 CAPPED_BODY_SITES = {
     ("diagnostics.py", "_parsed_events", "capped_body", "BODY_MAX_BYTES"),
     ("debug_sessions.py", "_image_body", "capped_body", "debugsession.SCREENSHOT_MAX_BYTES"),
+    ("feed.py", "_takeout_body", "capped_body", "takeout.MAX_BYTES"),
     ("files.py", "spool_upload", "capped_stream_to", "cap"),
 }
 
@@ -368,6 +370,9 @@ def test_every_body_reading_site_is_known_and_fits_under_its_ceiling():
     # The upload site's limit is the sysconfig knob, which is bounded BY the edge ceiling rather
     # than being a constant to compare against — that pairing is its own test below.
     assert ("POST", "/api/files/upload") in bodylimit.ROUTE_MAX_BODY_BYTES
+    # A Takeout file is larger than the default: without its ceiling every real history is a 413.
+    assert takeout.MAX_BYTES > DEFAULT_MAX_BODY_BYTES
+    assert bodylimit.ROUTE_MAX_BODY_BYTES[("POST", "/api/takeout")] == takeout.MAX_BYTES
 
 
 def test_release_read_connection_ends_the_transaction_it_was_called_for(db):

backend/tests/test_catalog_service.py

@@ -12,6 +12,7 @@ from datetime import datetime, timedelta, timezone
 from sqlalchemy import select
 
 from app.catalog import queries, search, service
+from app.db import SessionLocal
 from app.catalog.models import (
     BlockedChannel,
     Channel,
@@ -22,6 +23,8 @@ from app.catalog.models import (
 )
 from app.reader import queries as reader_queries
 from app.reader import service as reader_service
+from app.sync import explore as sync_explore
+from app.sync import subscriptions as sync_subscriptions
 
 
 def _channel(db, cid="UCc", **kw):
@@ -141,6 +144,89 @@ def test_subscribing_keeps_an_explored_channel_and_its_videos(db, make_user):
     assert queries.subscription(db, user, "UCc") is None
 
 
+def test_exploring_a_channel_someone_follows_brings_its_uploads_in_as_the_catalogs_own(
+    db, make_user, monkeypatch
+):
+    follower, explorer = make_user(), make_user()
+    ch = _channel(db, uploads_playlist_id="UUc")
+    db.add(Subscription(user_id=follower.id, channel_id="UCc"))
+    db.commit()
+    service.explore(db, explorer, ch)
+    db.commit()
+
+    class _Yt:
+        def get_playlist_items(self, playlist_id, page_token):
+            item = {"contentDetails": {"videoId": "v1", "videoPublishedAt": "2026-05-01T00:00:00Z"}}
+            return {"items": [item]}
+
+    monkeypatch.setattr(sync_explore, "_enrich_new", lambda session, yt, ids: 0)
+    sync_explore.explore_ingest_page(db, _Yt(), ch)
+    db.commit()
+    db.expire_all()
+    # The channel is followed, so not the exploration's: a video flagged as one would stay flagged
+    # (subscribing clears only an explored channel's) and its watched marks would wait for good.
+    assert (ch.from_explore, db.get(Video, "v1").via_explore) == (False, False)
+
+
+def test_a_follow_pulled_in_from_youtube_makes_an_explored_channel_the_catalogs_own(
+    db, make_user, monkeypatch
+):
+    explorer, follower = make_user(), make_user()
+    ch = _channel(db, details_synced_at=datetime.now(timezone.utc))
+    service.explore(db, explorer, ch)
+    db.commit()
+    _video(db, "v1", via_explore=True)
+
+    class _Yt:
+        def __init__(self, session, user):
+            pass
+
+        def __enter__(self):
+            return self
+
+        def __exit__(self, *exc):
+            return False
+
+        def iter_subscriptions(self):
+            yield {"channel_id": "UCc", "title": "UCc", "yt_subscription_id": "sub-UCc"}
+
+        def get_channels(self, ids):
+            return []
+
+    monkeypatch.setattr(sync_subscriptions, "YouTubeClient", _Yt)
+    sync_subscriptions.import_subscriptions(db, follower)
+    db.commit()
+    db.expire_all()
+    # Followed on youtube.com is followed all the same: left flagged, the channel's later videos
+    # came in as an exploration's, hidden from the Library and their watched marks waiting for good.
+    assert (db.get(Channel, "UCc").from_explore, db.get(Video, "v1").via_explore) == (False, False)
+
+
+def test_a_follow_landing_while_an_explored_page_comes_in_leaves_none_of_it_flagged(
+    db, make_user, monkeypatch
+):
+    explorer, follower = make_user(), make_user()
+    ch = _channel(db, uploads_playlist_id="UUc")
+    service.explore(db, explorer, ch)
+    db.commit()
+
+    class _Yt:
+        def get_playlist_items(self, playlist_id, page_token):
+            # Someone subscribes, in a request of their own, while YouTube answers this one.
+            with SessionLocal() as other:
+                service.record_subscription(other, follower, other.get(Channel, "UCc"), "sub-UCc")
+                other.commit()
+            item = {"contentDetails": {"videoId": "v1", "videoPublishedAt": "2026-05-01T00:00:00Z"}}
+            return {"items": [item]}
+
+    monkeypatch.setattr(sync_explore, "_enrich_new", lambda session, yt, ids: 0)
+    sync_explore.explore_ingest_page(db, _Yt(), ch)
+    db.commit()
+    db.expire_all()
+    # The subscribe cleared what was flagged before it; a page that lands after it must not be.
+    assert (db.get(Channel, "UCc").from_explore, db.get(Video, "v1").via_explore) == (False, False)
+
+
 def test_a_block_is_once_and_ends_the_exploration(db, make_user):
     user = make_user()
     ch = _channel(db)

backend/tests/test_discover.py

@@ -1105,10 +1105,16 @@ def test_two_overlapping_reorders_never_share_a_slot(db, as_user):
 
         worker = threading.Thread(target=second_tab)
         worker.start()
-        # Commit A only once B is blocked on a row lock, so B's read is the stale snapshot.
+        # Commit A only once B is blocked on a row lock, so B's read is the stale snapshot. pg_locks is
+        # the whole server's: only a wait in this database is B's (a parallel worker's is not).
         waiting = 0
         for _ in range(200):
-            waiting = probe.scalar(text("SELECT count(*) FROM pg_locks WHERE NOT granted"))
+            waiting = probe.scalar(
+                text(
+                    "SELECT count(*) FROM pg_locks WHERE NOT granted AND pid IN"
+                    " (SELECT pid FROM pg_stat_activity WHERE datname = current_database())"
+                )
+            )
             probe.rollback()
             if waiting:
                 break

backend/tests/test_entry_points.py

@@ -0,0 +1,69 @@
+"""Every process the app runs as sees every table before its first query.
+
+The units declare their tables in their own `models` modules, and a core model names a unit's by
+string (`User.subscriptions` → `Subscription`). A fresh interpreter that never imported the unit's
+module fails to configure its mappers on its first query: the download child once did exactly that,
+and every job failed, while this suite stayed green — it imports `app.tables` in-process, which hides
+the gap. So each entry point is imported here in a clean interpreter, and must register the whole
+schema on its own.
+"""
+
+import ast
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+import app.tables  # noqa: F401
+from app.db import Base
+
+BACKEND = Path(__file__).parent.parent
+APP = BACKEND / "app"
+
+# Each `python -m` module the deployment starts, and the API's app module.
+ENTRY_POINTS = ("app.main", "app.worker", "app.downloads.runner")
+
+# The table list goes out on one marked line: the API's logging writes to the same stdout, and a
+# line it logs while importing is not the probe's answer.
+_MARK = "entry-point-tables:"
+_PROBE = (
+    "import importlib, json, sys\n"
+    "importlib.import_module(sys.argv[1])\n"
+    "from sqlalchemy.orm import configure_mappers\n"
+    "from app.db import Base\n"
+    "configure_mappers()\n"
+    f"print({_MARK!r} + json.dumps(sorted(Base.metadata.tables)))\n"
+)
+
+
+def test_each_entry_point_registers_the_whole_schema_on_its_own():
+    whole = sorted(Base.metadata.tables)
+    for module in ENTRY_POINTS:
+        r = subprocess.run(
+            [sys.executable, "-c", _PROBE, module],
+            cwd=BACKEND,
+            capture_output=True,
+            text=True,
+            timeout=120,
+            check=False,
+        )
+        assert r.returncode == 0, (module, r.stderr[-2000:])
+        answer = [ln for ln in r.stdout.splitlines() if ln.startswith(_MARK)]
+        assert len(answer) == 1, (module, r.stdout[-2000:])
+        assert json.loads(answer[0].removeprefix(_MARK)) == whole, module
+
+
+def test_every_module_that_runs_as_a_process_is_listed():
+    mains = set()
+    for path in APP.rglob("*.py"):
+        tree = ast.parse(path.read_text(encoding="utf-8"))
+        for node in tree.body:
+            if (
+                isinstance(node, ast.If)
+                and isinstance(node.test, ast.Compare)
+                and isinstance(node.test.left, ast.Name)
+                and node.test.left.id == "__name__"
+            ):
+                rel = path.relative_to(BACKEND).with_suffix("").as_posix()
+                mains.add(rel.replace("/", "."))
+    assert mains <= set(ENTRY_POINTS), mains - set(ENTRY_POINTS)

backend/tests/test_files_api.py

@@ -124,6 +124,9 @@ def test_only_an_own_folder_is_renamed_and_only_by_its_user(db, make_user, as_us
     assert clash.json()["detail"]["reason"] == "file_folder_name_taken"
     no = c.patch(f"/api/files/folders/{src.id}", json={"name": "YouTube 2"})
     assert no.status_code == 409 and no.json()["detail"]["reason"] == "file_folder_not_own"
+    # Which folder is answered before what the name is: a blank one is still "not yours to name".
+    blank = c.patch(f"/api/files/folders/{src.id}", json={"name": " "})
+    assert blank.status_code == 409 and blank.json()["detail"]["reason"] == "file_folder_not_own"
     stranger = as_user(make_user()).patch(f"/api/files/folders/{own.id}", json={"name": "X"})
     assert stranger.status_code == 404
     db.refresh(own)

backend/tests/test_google_callback.py

@@ -7,6 +7,7 @@ the refused sign-in that doubles as an access request, and the link flow's confl
 """
 
 import pytest
+from authlib.integrations.starlette_client import OAuthError
 from sqlalchemy import select
 from starlette.responses import RedirectResponse
 
@@ -19,15 +20,20 @@ _PW = "correct horse battery"
 
 class _FakeGoogle:
     """authlib's client, reduced to the two calls the routes make: the redirect out to Google and
-    the token exchange when Google redirects back."""
+    the token exchange when Google redirects back. Like authlib, it honours only the newest flow's
+    state: starting a flow retires every earlier one."""
 
     def __init__(self, userinfo: dict):
         self.userinfo = userinfo
+        self.asked: dict = {}
 
-    async def authorize_redirect(self, _request, _redirect_uri, **_kw):
+    async def authorize_redirect(self, _request, _redirect_uri, **kw):
+        self.asked = kw
         return RedirectResponse(url="/fake-google")
 
-    async def authorize_access_token(self, _request):
+    async def authorize_access_token(self, request):
+        if request.query_params.get("state") != self.asked.get("state"):
+            raise OAuthError(error="mismatching_state")
         return {"userinfo": self.userinfo, "access_token": "at", "scope": "openid email profile"}
 
 
@@ -44,9 +50,13 @@ def google(monkeypatch):
     return fake
 
 
-def _sign_in(client, google, **userinfo):
+def _sign_in(client, google, state: str | None = None, **userinfo):
+    """Google redirecting back: by default with the state of the flow started last."""
     google.userinfo = {"email_verified": True, **userinfo}
-    return client.get("/auth/callback", follow_redirects=False)
+    state = state or google.asked.get("state")
+    return client.get(
+        "/auth/callback", params={"state": state} if state else {}, follow_redirects=False
+    )
 
 
 def _rows(db, model, column, value):
@@ -218,12 +228,20 @@ def test_a_refused_sign_in_records_a_request_in_the_reported_language(
     assert _rows(db, User, User.email, "new@example.com") == []
 
 
-def _start_link(client, email: str):
+def _password_login(client, email: str):
     assert (
         client.post("/auth/password-login", json={"email": email, "password": _PW}).status_code
         == 200
     )
-    assert client.get("/auth/link", follow_redirects=False).headers["location"] == "/fake-google"
+
+
+def _connect(client, access: str):
+    return client.get(f"/auth/google/connect?access={access}", follow_redirects=False)
+
+
+def _start_link(client, email: str):
+    _password_login(client, email)
+    assert _connect(client, "identity").headers["location"] == "/fake-google"
 
 
 def test_linking_an_identity_another_account_holds_is_a_conflict(client, db, google, make_user):
@@ -258,3 +276,151 @@ def test_linking_a_free_identity_attaches_it(client, db, google, make_user):
     assert r.headers["location"] == "/?link=ok"
     db.refresh(me)
     assert (me.google_sub, me.google_email) == ("sub-free", "me3.google@example.com")
+
+
+def test_a_connect_abandoned_at_google_leaves_the_next_sign_in_a_sign_in(
+    client, db, google, make_user, monkeypatch
+):
+    # A connect left at Google's screen (Back, a closed tab) never reaches the callback that pops
+    # its marker. "Sign in as someone else" afterwards must sign that person in, not link their
+    # Google identity onto the account still signed in here.
+    me = make_user(email="me4@example.com")
+    make_user(email="them4@example.com", google_sub="sub-them4", password=None)
+    monkeypatch.setattr(settings, "allowed_emails", "them4@example.com")
+    _start_link(client, "me4@example.com")
+    assert client.get("/auth/login", follow_redirects=False).headers["location"] == "/fake-google"
+
+    r = _sign_in(client, google, sub="sub-them4", email="them4@example.com")
+
+    assert r.headers["location"] == "/"
+    assert client.get("/api/me").json()["email"] == "them4@example.com"
+    db.refresh(me)
+    assert me.google_sub is None
+
+
+def test_a_link_whose_session_was_revoked_meanwhile_does_not_finish(client, db, google, make_user):
+    # "Log out everywhere" (or a password change) between the start and the callback: the cookie
+    # still names the account, but its session no longer holds, so it may not attach an identity.
+    me = make_user(email="me5@example.com")
+    _start_link(client, "me5@example.com")
+    me.session_epoch = (me.session_epoch or 0) + 1
+    db.commit()
+
+    r = _sign_in(client, google, sub="sub-late", email="late@example.com")
+
+    assert r.headers["location"] == "/?link=error"
+    db.refresh(me)
+    assert me.google_sub is None
+
+
+def test_an_older_flow_coming_back_leaves_the_connect_in_flight_a_link(
+    client, db, google, make_user
+):
+    # Two round-trips from one browser: an older one (another tab's, or Back to Google's screen)
+    # returns after a connect started. Its state is refused, and it must not spend the connect's
+    # marker: the connect itself, coming back, would then land as a plain sign-in.
+    me = make_user(email="me6@example.com")
+    _start_link(client, "me6@example.com")
+
+    older = _sign_in(client, google, state="an-older-flow", sub="sub-6", email="g6@example.com")
+    assert older.headers["location"] == "/?oauth=cancelled"
+    r = _sign_in(client, google, sub="sub-6", email="g6@example.com")
+
+    assert r.headers["location"] == "/?link=ok"
+    db.refresh(me)
+    assert me.google_sub == "sub-6"
+
+
+def test_a_connect_from_a_tab_showing_another_account_starts_nothing(client, google, make_user):
+    # A navigation carries no headers, so the tab names the account it was built for in the query.
+    # One still showing the account signed in before must not start a link onto the one signed in
+    # now: it lands on the app, which boots as that one and says why.
+    me = make_user(email="me7@example.com")
+    _password_login(client, "me7@example.com")
+
+    r = client.get(
+        f"/auth/google/connect?access=identity&expect={me.id + 1}", follow_redirects=False
+    )
+
+    assert r.headers["location"] == auth_routes.ACCOUNT_CHANGED_LANDING == "/?account=changed"
+    assert google.asked == {}
+    assert client.get("/api/me").json()["email"] == "me7@example.com"
+
+
+def test_a_connect_naming_the_signed_in_account_goes_to_google(client, google, make_user):
+    me = make_user(email="me8@example.com")
+    _password_login(client, "me8@example.com")
+
+    r = client.get(f"/auth/google/connect?access=identity&expect={me.id}", follow_redirects=False)
+
+    assert r.headers["location"] == "/fake-google"
+    assert google.asked["state"]
+
+
+@pytest.mark.parametrize(
+    ("access", "scope", "prompt"),
+    [
+        ("identity", "openid email profile", "select_account"),
+        ("read", auth_routes.READ_SCOPES, "consent"),
+        ("write", auth_routes.WRITE_SCOPES, "consent"),
+    ],
+)
+def test_one_connect_route_asks_google_for_what_the_access_needs(
+    client, google, make_user, access, scope, prompt
+):
+    make_user(email=f"ask-{access}@example.com")
+    _password_login(client, f"ask-{access}@example.com")
+
+    assert _connect(client, access).headers["location"] == "/fake-google"
+
+    assert (google.asked["scope"], google.asked["prompt"]) == (scope, prompt)
+    assert google.asked["access_type"] == "offline"
+
+
+@pytest.mark.parametrize("access", ["read", "write"])
+def test_a_youtube_grant_attaches_to_the_password_account_and_returns_to_the_app(
+    client, db, google, make_user, access
+):
+    me = make_user(email=f"grant-{access}@example.com")
+    _password_login(client, f"grant-{access}@example.com")
+    _connect(client, access)
+
+    r = _sign_in(client, google, sub=f"sub-grant-{access}", email=f"g-{access}@example.com")
+
+    # Back to the app, not the "linked" notice: the onboarding wizard picks up where it was.
+    assert r.headers["location"] == "/"
+    db.refresh(me)
+    assert me.google_sub == f"sub-grant-{access}"
+    assert _rows(db, User, User.google_sub, f"sub-grant-{access}") == [me]
+
+
+@pytest.mark.parametrize("access", ["", "admin", "READ"])
+def test_an_unknown_access_starts_no_connect(client, google, make_user, access):
+    make_user(email="odd@example.com")
+    _password_login(client, "odd@example.com")
+
+    assert _connect(client, access).status_code == 422
+    assert google.asked == {}
+
+
+def test_a_connect_needs_a_signed_in_account(client, google):
+    assert _connect(client, "identity").status_code == 401
+    assert google.asked == {}
+
+
+@pytest.mark.parametrize("access", ["identity", "read", "write"])
+def test_without_google_configured_a_connect_says_it_failed(client, make_user, monkeypatch, access):
+    monkeypatch.setattr(auth_routes, "google_oauth", lambda _db: None)
+    make_user(email=f"nog-{access}@example.com")
+    _password_login(client, f"nog-{access}@example.com")
+
+    assert _connect(client, access).headers["location"] == "/?link=error"
+
+
+@pytest.mark.parametrize("path", ["/auth/link", "/auth/upgrade", "/auth/upgrade?access=write"])
+def test_the_old_connect_routes_are_gone(client, google, make_user, path):
+    make_user(email="old@example.com")
+    _password_login(client, "old@example.com")
+
+    assert client.get(path, follow_redirects=False).status_code == 404
+    assert google.asked == {}

backend/tests/test_me_preferences.py

@@ -0,0 +1,74 @@
+"""PUT /api/me/preferences merges the keys it is sent into the stored ones, DB-lane.
+
+A client saves its preferences a key at a time, and several at once on the boot that moves this
+browser's copies to the account. Each merge must start from the row as stored when it runs, not as
+the request first read it — two merges over the same earlier read keep only the last one's keys.
+"""
+
+from fastapi import Depends, Request
+from sqlalchemy import event, update
+from sqlalchemy.engine import Engine
+from sqlalchemy.orm import Session
+
+from app.auth import current_user
+from app.db import SessionLocal, get_db
+from app.models import User
+
+_PW = "correct horse battery"
+
+
+def test_a_save_keeps_a_key_another_save_stored_after_this_one_read_the_account(
+    client, db, make_user, override_dep
+):
+    user = make_user(email="prefs@example.com", password=_PW)
+    assert (
+        client.post("/auth/password-login", json={"email": "prefs@example.com", "password": _PW})
+    ).status_code == 200
+
+    def read_then_another_save(request: Request, session: Session = Depends(get_db)) -> User:
+        # The account as this request reads it, then a second save committing in the meantime.
+        caller = current_user(request, session)
+        with SessionLocal() as other:
+            other.execute(
+                update(User)
+                .where(User.id == caller.id)
+                .values(preferences={"seenVersion": "1.2.3"})
+            )
+            other.commit()
+        return caller
+
+    override_dep(current_user, read_then_another_save)
+    r = client.put("/api/me/preferences", json={"startOnHome": False})
+
+    assert r.status_code == 200
+    db.expire_all()
+    assert db.get(User, user.id).preferences == {"seenVersion": "1.2.3", "startOnHome": False}
+
+
+def test_the_merge_reads_the_account_under_a_lock_that_orders_merges_and_nothing_else(
+    client, make_user
+):
+    # The re-read above is half of it: two merges that both read before either commits would still
+    # keep only the last, unless the read takes a lock the other merge waits for. FOR NO KEY UPDATE
+    # is that lock, and the weakest one: a plain FOR UPDATE would also wait behind any transaction
+    # that only added a row referring to the account (a playlist pull, then minutes of YouTube).
+    make_user(email="prefs-lock@example.com", password=_PW)
+    assert (
+        client.post(
+            "/auth/password-login", json={"email": "prefs-lock@example.com", "password": _PW}
+        )
+    ).status_code == 200
+    sent: list[str] = []
+
+    def record(_conn, _cursor, statement, *_rest):
+        sent.append(statement)
+
+    event.listen(Engine, "before_cursor_execute", record)
+    try:
+        assert client.put("/api/me/preferences", json={"startOnHome": False}).status_code == 200
+    finally:
+        event.remove(Engine, "before_cursor_execute", record)
+
+    locks = [s for s in sent if "FROM users" in s and " FOR " in s]
+    assert locks, sent
+    assert all(s.rstrip().endswith("FOR NO KEY UPDATE") for s in locks), locks

backend/tests/test_me_search_source.py

@@ -1,5 +1,6 @@
 """`/api/me` names the live-search source, so the search button's tooltip can say what a search costs
-under the source that actually runs (the default website source spends no API quota)."""
+under the source that actually runs (the default website source spends no API quota), and says
+whether live search and related videos are on at all."""
 
 from app import sysconfig
 
@@ -24,3 +25,24 @@ def test_me_names_the_running_search_source(client, db, make_user):
     sysconfig.set_value(db, "search_source", "api")
     db.commit()
     assert _me(client)["search_source"] == "api"
+
+
+def test_me_says_whether_live_search_and_related_videos_are_on(client, db, make_user):
+    # A daily limit of 0 turns the feature off; the UI leaves it out instead of offering a refusal.
+    user = make_user(password=_PW)
+    login = client.post("/auth/password-login", json={"email": user.email, "password": _PW})
+    assert login.status_code == 200
+
+    me = _me(client)
+    assert (me["search_enabled"], me["related_enabled"]) == (True, True)
+
+    sysconfig.set_value(db, "search_daily_limit_per_user", 0)
+    db.commit()
+    me = _me(client)
+    assert (me["search_enabled"], me["related_enabled"]) == (False, True)
+
+    sysconfig.set_value(db, "search_daily_limit_per_user", 5)
+    sysconfig.set_value(db, "related_daily_limit_per_user", 0)
+    db.commit()
+    me = _me(client)
+    assert (me["search_enabled"], me["related_enabled"]) == (True, False)

backend/tests/test_migration_0112.py

@@ -0,0 +1,90 @@
+"""Migration 0112's data step, DB-lane: a user who already has two Watch later lists keeps the oldest,
+with the other's videos moved in after its own (one it already held is not added twice) and a Home
+pin of the other moved onto it, and the one-per-user index then refuses a second list.
+
+The schema half is checked by test_migrations.py; a video or a pin lost on the way is invisible there.
+"""
+
+from datetime import datetime, timezone
+
+import pytest
+from alembic import command
+from alembic.config import Config
+from sqlalchemy import text
+from sqlalchemy.exc import IntegrityError
+
+from app.catalog.models import Channel, Video
+from app.db import engine
+
+
+def _lists(conn, user_id: int, n: int) -> list[int]:
+    values = ", ".join(f"(:u, 'Watch later', 'watch_later', {i})" for i in range(n))
+    return list(
+        conn.execute(
+            text(
+                f"INSERT INTO playlists (user_id, name, kind, position) VALUES {values} RETURNING id"
+            ),
+            {"u": user_id},
+        ).scalars()
+    )
+
+
+def _pin(conn, user_id: int, playlist_id: int, position: int) -> None:
+    conn.execute(
+        text("INSERT INTO comfort_pins (user_id, playlist_id, position) VALUES (:u, :p, :pos)"),
+        {"u": user_id, "p": playlist_id, "pos": position},
+    )
+
+
+def test_a_second_watch_later_is_folded_into_the_first(db, make_user):
+    user, other = make_user(), make_user()
+    db.add(Channel(id="UCm", title="M"))
+    now = datetime.now(timezone.utc)
+    for v in ("v1", "v2", "v3"):
+        db.add(Video(id=v, channel_id="UCm", published_at=now, duration_seconds=60))
+    db.commit()
+    cfg = Config("alembic.ini")
+    command.downgrade(cfg, "0111_audit_request_id")
+    try:
+        with engine.begin() as conn:
+            first, second, third = _lists(conn, user.id, 3)
+            theirs, their_second = _lists(conn, other.id, 2)
+            conn.execute(
+                text(
+                    "INSERT INTO playlist_items (playlist_id, video_id, position) VALUES"
+                    " (:a, 'v1', 0), (:b, 'v2', 0), (:b, 'v1', 1), (:c, 'v3', 0)"
+                ),
+                {"a": first, "b": second, "c": theirs},
+            )
+            # The user pinned two of the copies, the kept one neither: the first on the shelf moves.
+            _pin(conn, user.id, third, 0)
+            _pin(conn, user.id, second, 1)
+            # The other pinned both: the kept one's pin stays, and is the only one.
+            _pin(conn, other.id, their_second, 0)
+            _pin(conn, other.id, theirs, 1)
+    finally:
+        command.upgrade(cfg, "head")
+
+    with engine.connect() as conn:
+        lists = conn.execute(
+            text("SELECT id, user_id FROM playlists WHERE kind = 'watch_later' ORDER BY id")
+        ).all()
+        assert [tuple(r) for r in lists] == [(first, user.id), (theirs, other.id)]
+        items = conn.execute(
+            text(
+                "SELECT playlist_id, video_id, position FROM playlist_items ORDER BY playlist_id, position"
+            )
+        ).all()
+        assert [tuple(r) for r in items] == [(first, "v1", 0), (first, "v2", 1), (theirs, "v3", 0)]
+        pins = conn.execute(
+            text("SELECT user_id, playlist_id, position FROM comfort_pins ORDER BY user_id")
+        ).all()
+        assert [tuple(r) for r in pins] == [(user.id, first, 0), (other.id, theirs, 1)]
+    with pytest.raises(IntegrityError), engine.begin() as conn:
+        conn.execute(
+            text(
+                "INSERT INTO playlists (user_id, name, kind, position)"
+                " VALUES (:u, 'Watch later', 'watch_later', 2)"
+            ),
+            {"u": user.id},
+        )

backend/tests/test_migration_0116.py

@@ -0,0 +1,50 @@
+"""Migration 0116's data step, DB-lane: an explored channel someone already follows becomes the
+catalog's own, and no video stays flagged as an exploration's on a channel that is not one.
+
+The schema half is checked by test_migrations.py; a flag left on is invisible there.
+"""
+
+from alembic import command
+from alembic.config import Config
+from sqlalchemy import text
+
+from app.db import engine
+
+
+def test_followed_explored_channels_and_their_videos_are_the_catalogs_own(db, make_user):
+    follower = make_user().id
+    cfg = Config("alembic.ini")
+    command.downgrade(cfg, "0115_takeout_channels")
+    try:
+        with engine.begin() as conn:
+
+            def channel(cid, explored):
+                conn.execute(
+                    text("INSERT INTO channels (id, from_explore) VALUES (:c, :e)"),
+                    {"c": cid, "e": explored},
+                )
+                conn.execute(
+                    text(
+                        "INSERT INTO videos (id, channel_id, published_at, via_explore)"
+                        " VALUES (:v, :c, now(), true)"
+                    ),
+                    {"v": f"v-{cid}", "c": cid},
+                )
+
+            channel("UCfollowed", True)  # explored, then followed on youtube.com
+            channel("UCexplored", True)  # explored, nobody follows it: stays the exploration's
+            channel("UCorganic", False)  # a video flagged after a subscribe had cleared the channel
+            conn.execute(
+                text("INSERT INTO subscriptions (user_id, channel_id) VALUES (:u, 'UCfollowed')"),
+                {"u": follower},
+            )
+    finally:
+        command.upgrade(cfg, "head")
+
+    with engine.connect() as conn:
+        channels = dict(conn.execute(text("SELECT id, from_explore FROM channels")).tuples().all())
+        videos = dict(
+            conn.execute(text("SELECT channel_id, via_explore FROM videos")).tuples().all()
+        )
+    assert channels == {"UCfollowed": False, "UCexplored": True, "UCorganic": False}
+    assert videos == {"UCfollowed": False, "UCexplored": True, "UCorganic": False}

backend/tests/test_override_dep_convention.py

@@ -82,3 +82,42 @@ def test_a_request_that_raises_leaves_nothing_it_did_not_commit(db):
         request.throw(RuntimeError("the handler failed"))
     assert db.scalar(select(User).where(User.email == "rolled-back-probe@example.com")) is None
     assert db.info["unfinished"] == []  # a failing request is rolled back, not reported
+
+
+def test_the_commit_check_sees_a_core_dml_write(db, make_user):
+    """`db.execute(update(...))` flushes nothing and stages nothing, so the session's own bookkeeping
+    never sees it: a handler whose only write is Core DML and that forgot its commit passed. The
+    database's transaction id is what tells — also when the test itself had a write open before the
+    request, which the handler's write would otherwise share. A committed one must still pass."""
+    import pytest
+    from sqlalchemy import update
+    from starlette.requests import HTTPConnection
+
+    from app.models import User
+
+    user = make_user()
+
+    def drive(commit: bool, *, test_wrote: bool = False) -> None:
+        if test_wrote:  # the test's own write, open and uncommitted, as the request starts
+            db.execute(update(User).where(User.id == user.id).values(display_name="test"))
+        conn = HTTPConnection(
+            {"type": "http", "method": "POST", "path": "/probe", "headers": [], "query_string": b""}
+        )
+        request = db.info["request"](conn)
+        session = next(request)
+        session.execute(update(User).where(User.id == user.id).values(display_name="dml"))
+        if commit:
+            session.commit()
+        with pytest.raises(StopIteration):
+            next(request)
+
+    drive(commit=True)
+    assert db.info["unfinished"] == []
+    drive(commit=False)
+    assert db.info["unfinished"] == ["POST /probe"]
+    db.info["unfinished"].clear()  # recorded as it should be — not this test's failure
+    drive(commit=True, test_wrote=True)
+    assert db.info["unfinished"] == []
+    drive(commit=False, test_wrote=True)
+    assert db.info["unfinished"] == ["POST /probe"]
+    db.info["unfinished"].clear()

backend/tests/test_owner_scoping.py

@@ -10,7 +10,10 @@ from datetime import datetime, timedelta, timezone
 from sqlalchemy import func, select
 
 from app import debugsession, notifications, quota
+from app.catalog.models import Channel, Video
+from app.downloads.models import DownloadJob, DownloadProfile
 from app.models import DebugScreenshot, DebugSession, Notification, QuotaEvent
+from app.reader.models import Playlist, PlaylistItem, VideoState
 from app.routes import tokens as tokens_routes
 from app.routes.auth import RESET_TTL, _issue_token
 
@@ -72,9 +75,9 @@ def test_trimming_read_notifications_trims_only_mine(db, make_user):
 
 def test_todays_actions_count_only_mine(db, make_user):
     me, other = make_user(), make_user()
-    quota.log_action(me.id, quota.QuotaAction.VIDEOS_SEARCH)
-    quota.log_action(other.id, quota.QuotaAction.VIDEOS_SEARCH)
-    quota.log_action(other.id, quota.QuotaAction.VIDEOS_SEARCH)
+    quota.claim_action(me.id, quota.QuotaAction.VIDEOS_SEARCH, 5)
+    quota.claim_action(other.id, quota.QuotaAction.VIDEOS_SEARCH, 5)
+    quota.claim_action(other.id, quota.QuotaAction.VIDEOS_SEARCH, 5)
     assert quota.actions_today(db, me.id, quota.QuotaAction.VIDEOS_SEARCH) == 1
 
 
@@ -133,3 +136,113 @@ def test_screenshot_storage_counts_only_mine(db, make_user):
         )
     db.commit()
     assert debugsession.stored_screenshot_bytes(db, me.id) == 10
+
+
+# --- the units' owner filters, through their routes --------------------------------------------
+# The same rule one layer up: a second account acting on the SAME video, list or profile id must
+# reach only its own row. Each test holds the first account's row unchanged afterwards.
+
+
+def _videos(db, *ids):
+    db.add(Channel(id="UCo", title="O"))
+    db.flush()
+    for vid in ids:
+        db.add(
+            Video(
+                id=vid,
+                channel_id="UCo",
+                published_at=datetime.now(timezone.utc),
+                duration_seconds=3600,
+            )
+        )
+    db.commit()
+
+
+def _video_state(db, user, vid="v1"):
+    db.expire_all()
+    row = db.execute(
+        select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == vid)
+    ).scalar_one_or_none()
+    return None if row is None else (row.status, row.position_seconds)
+
+
+def test_unmarking_a_video_leaves_another_users_state(db, make_user, as_user):
+    me, other = make_user(), make_user()
+    _videos(db, "v1")
+    db.add(VideoState(user_id=other.id, video_id="v1", status="watched"))
+    db.commit()
+    r = as_user(me).post("/api/videos/v1/state", json={"status": "new"})
+    assert r.status_code == 200, r.text
+    assert _video_state(db, other) == ("watched", 0)
+
+
+def test_clearing_a_video_leaves_another_users_state(db, make_user, as_user):
+    me, other = make_user(), make_user()
+    _videos(db, "v1")
+    db.add(VideoState(user_id=other.id, video_id="v1", status="hidden", position_seconds=90))
+    db.commit()
+    assert as_user(me).delete("/api/videos/v1/state").status_code == 200
+    assert _video_state(db, other) == ("hidden", 90)
+
+
+def test_a_resume_point_is_set_and_cleared_on_my_own_row_only(db, make_user, as_user):
+    me, other = make_user(), make_user()
+    _videos(db, "v1")
+    db.add(VideoState(user_id=other.id, video_id="v1", status="hidden", position_seconds=90))
+    db.commit()
+    client = as_user(me)
+    body = {"position_seconds": 600, "duration_seconds": 3600}
+    assert client.post("/api/videos/v1/progress", json=body).status_code == 200
+    assert (_video_state(db, me), _video_state(db, other)) == (("new", 600), ("hidden", 90))
+    body = {"position_seconds": 0, "duration_seconds": 3600}
+    assert client.post("/api/videos/v1/progress", json=body).status_code == 200
+    assert (_video_state(db, me), _video_state(db, other)) == (None, ("hidden", 90))
+
+
+def _watch_later_ids(db, user):
+    db.expire_all()
+    return sorted(
+        db.execute(
+            select(PlaylistItem.video_id)
+            .join(Playlist, Playlist.id == PlaylistItem.playlist_id)
+            .where(Playlist.user_id == user.id, Playlist.kind == "watch_later")
+        ).scalars()
+    )
+
+
+def test_watch_later_adds_and_removes_use_my_own_list(db, make_user, as_user):
+    me, other = make_user(), make_user()
+    _videos(db, "v1", "v2")
+    theirs = as_user(other).post("/api/playlists/watch-later", json={"video_id": "v1"}).json()
+    client = as_user(me)
+    mine = client.post("/api/playlists/watch-later", json={"video_id": "v2"}).json()
+    assert mine["playlist_id"] != theirs["playlist_id"]
+    # Their saved video is not on my list, so removing it from mine leaves theirs alone.
+    assert client.delete("/api/playlists/watch-later/v1").status_code == 200
+    assert (_watch_later_ids(db, me), _watch_later_ids(db, other)) == (["v2"], ["v1"])
+
+
+def test_another_users_private_profile_is_refused_like_a_missing_one(db, make_user, as_user):
+    me, other = make_user(), make_user()
+    theirs = DownloadProfile(user_id=other.id, name="Theirs", spec={"format": "best"})
+    builtin = DownloadProfile(user_id=None, name="Best", spec={"format": "best"}, is_builtin=True)
+    db.add_all([theirs, builtin])
+    db.commit()
+    client = as_user(me)
+
+    def enqueue(profile_id):
+        return client.post(
+            "/api/downloads", json={"source": "dQw4w9WgXcQ", "profile_id": profile_id}
+        )
+
+    missing = enqueue(theirs.id + builtin.id + 1)
+    refused = enqueue(theirs.id)
+    assert (refused.status_code, refused.json()) == (404, missing.json())
+    assert refused.json()["detail"] == {"code": "not_found", "reason": "profile"}
+    db.expire_all()
+    assert db.execute(select(DownloadJob)).first() is None
+    # A built-in is everyone's to use.
+    r = enqueue(builtin.id)
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    assert db.execute(select(DownloadJob.profile_id)).scalars().all() == [builtin.id]

backend/tests/test_playlist_sync_claim.py

@@ -89,9 +89,13 @@ def test_a_claim_is_released_when_its_block_raises(db, make_user):
     except RuntimeError:
         pass
     # Asked of pg_locks, not by claiming again: a lock leaked on a pooled connection would be granted
-    # again to that same connection (advisory locks are re-entrant per session).
+    # again to that same connection (advisory locks are re-entrant per session). pg_locks is the whole
+    # server's, so only this database's: a parallel worker's own claim on the same id counted here.
     held = db.scalar(
-        text("SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND objid = :id"),
+        text(
+            "SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND objid = :id"
+            " AND database = (SELECT oid FROM pg_database WHERE datname = current_database())"
+        ),
         {"id": pl.id},
     )
     assert held == 0

backend/tests/test_plex_image_sizes.py

@@ -48,7 +48,7 @@ def _setup(db, make_user, monkeypatch, tmp_path) -> TestClient:
     )
     db.commit()
     monkeypatch.setattr(app_state, "_configured_cache", True)
-    monkeypatch.setattr(playback, "resolved_user_id", lambda request: (user.id, True))
+    monkeypatch.setattr(playback, "session_user_id", lambda request: user.id)
     monkeypatch.setattr(playback, "PlexClient", _FakePlex)
     monkeypatch.setattr(imgcache, "CACHE_DIR", tmp_path)
     _FakePlex.calls = []

backend/tests/test_plex_service.py

@@ -12,7 +12,7 @@ from datetime import datetime, timezone
 
 from sqlalchemy import select
 
-from app.plex import library, queries, service
+from app.plex import library, queries, rollup, service
 from app.plex.models import PlexCollection, PlexItem, PlexLibrary, PlexShow, PlexState
 from app.plex.query import LibraryFilters
 
@@ -71,9 +71,9 @@ def test_a_users_watch_state_is_theirs_alone(db, make_user):
     _state(db, owner, it, status="watched")
     assert queries.state(db, owner, it.id).status == "watched"
     assert queries.state(db, stranger, it.id) is None
-    assert queries.states(db, stranger, [it.id]) == {}
-    assert set(queries.states(db, owner, [it.id])) == {it.id}
-    assert queries.states(db, owner, []) == {}
+    assert rollup.item_state_map(db, stranger, [it.id]) == {}
+    assert set(rollup.item_state_map(db, owner, [it.id])) == {it.id}
+    assert rollup.item_state_map(db, owner, []) == {}
 
 
 def test_the_grid_shows_the_callers_state_and_hides_only_what_they_hid(db, make_user):
@@ -421,3 +421,32 @@ def test_only_the_owners_own_link_pushes_their_watch_state(db, make_user):
     db.commit()
     assert watch_sync.link_for_push(db, owner.id).id == link.id
     assert watch_sync.link_for_push(db, other.id) is None
+
+
+def test_a_first_write_that_lost_the_race_updates_the_winners_row(db, make_user):
+    """The first checkpoint of an item and the `final` flush beside it both read "no state"; the one
+    that writes second must land on the row the first made, not fail on `uq_plex_user_item`."""
+    user = make_user()
+    it = _movie(db, _lib(db), "m-race")
+    _state(db, user, it, position_seconds=10)  # the request that won, committed
+    st = service.checkpoint(db, user, it.id, None, 42, NOW, dirty=True)  # read "none" before it
+    assert (st.position_seconds, st.synced_to_plex) == (42, False)
+    st = service.set_status(db, user, it.id, None, "hidden")
+    assert st.status == "hidden"
+    db.commit()
+    rows = db.scalars(select(PlexState).where(PlexState.item_id == it.id)).all()
+    assert [(r.position_seconds, r.status) for r in rows] == [(42, "hidden")]
+
+
+def test_a_season_marked_beside_a_first_checkpoint_updates_its_row(db, make_user, monkeypatch):
+    """Marking a season reads every episode's state once; a first checkpoint that made one row since
+    must be updated, not collided with on `uq_plex_user_item`."""
+    user = make_user()
+    lib = _lib(db)
+    ep = _movie(db, lib, "e-race")
+    _state(db, user, ep, position_seconds=10)  # the checkpoint that won, committed
+    monkeypatch.setattr(rollup, "item_state_map", lambda *a, **k: {})  # read before it committed
+    service.bulk_state(db, user.id, [ep], watched=True)
+    db.commit()
+    rows = db.scalars(select(PlexState).where(PlexState.item_id == ep.id)).all()
+    assert [(r.status, r.position_seconds) for r in rows] == [("watched", 0)]

backend/tests/test_plex_subtitle_auth.py

@@ -32,7 +32,7 @@ def test_a_suspended_account_cannot_drive_the_subtitle_extractor(
     monkeypatch.setattr(app_state, "_configured_cache", True)
     # A validly SIGNED cookie for that account — which is all the old gate looked at.
     for mod in (auth, playback):
-        monkeypatch.setattr(mod, "resolved_user_id", lambda request: (user.id, True))
+        monkeypatch.setattr(mod, "session_user_id", lambda request: user.id)
     monkeypatch.setattr(subtitles, "CACHE_DIR", tmp_path)
     pool = _NoPool()
     monkeypatch.setattr(subtitles, "_WIN_POOL", pool)

backend/tests/test_plex_watch_sync.py

@@ -14,6 +14,7 @@ from types import SimpleNamespace
 from sqlalchemy import select
 
 from app.models import User
+from app.plex import queries
 from app.plex.models import PlexItem, PlexLibrary, PlexState
 from app.plex.watch_sync import _pull_apply, _reconcile_inprogress_movies
 from app.routes.plex.playback import (
@@ -385,3 +386,30 @@ def test_should_push_resume_periodic_is_throttled():
     )
     # Once the window elapses, the next tick pushes again.
     assert _should_push_resume(*key, now + timedelta(seconds=_PUSH_THROTTLE_S), final=False) is True
+
+
+def test_a_pull_that_lost_the_race_keeps_the_local_change_it_met(db, monkeypatch):
+    """The pull read "no state", and a first local checkpoint made one before it wrote: that row is
+    still waiting for its push, so the pull leaves it (the dirty guard), and fails on nothing."""
+    item_id, user_id = _seed_item_and_user(db)
+    now = datetime.now(timezone.utc)
+    db.add(PlexState(user_id=user_id, item_id=item_id, position_seconds=300, synced_to_plex=False))
+    db.commit()
+    real = queries.state
+    stale = [True]
+    monkeypatch.setattr(
+        queries, "state", lambda d, u, i: None if stale and stale.pop() else real(d, u, i)
+    )
+    assert _pull_apply(db, user_id, item_id, "new", 120, plex_ts=now) is None
+    db.commit()
+    rows = db.scalars(select(PlexState).where(PlexState.item_id == item_id)).all()
+    assert [(r.position_seconds, r.synced_to_plex) for r in rows] == [(300, False)]
+
+
+def test_a_first_pull_makes_a_row_in_step_with_plex(db):
+    item_id, user_id = _seed_item_and_user(db)
+    now = datetime.now(timezone.utc)
+    assert _pull_apply(db, user_id, item_id, "new", 120, plex_ts=now) == "in_progress"
+    db.commit()
+    rows = db.scalars(select(PlexState).where(PlexState.item_id == item_id)).all()
+    assert [(r.position_seconds, r.synced_to_plex) for r in rows] == [(120, True)]

backend/tests/test_quota_ownership.py

@@ -44,15 +44,15 @@ def test_reads_see_separate_session_spend_mid_transaction(db):
     assert after - before == 7
 
 
-def test_log_action_survives_caller_rollback(db):
+def test_a_claimed_slot_survives_caller_rollback(db):
     user = User(email="searcher@example.com")
     db.add(user)
     db.commit()  # a committed user to attribute the (free) action to
     db.add(User(email="pending@example.com"))  # uncommitted caller work
-    quota.log_action(user.id, quota.QuotaAction.VIDEOS_SEARCH)
+    assert quota.claim_action(user.id, quota.QuotaAction.VIDEOS_SEARCH, 5) is not None
     db.rollback()
 
-    # The zero-cost action event survived (per-user daily caps must still count it) ...
+    # The zero-cost claim event survived (per-user daily caps must still count it) ...
     events = db.execute(select(QuotaEvent)).scalars().all()
     assert len(events) == 1
     assert events[0].user_id == user.id

backend/tests/test_reader_service.py

@@ -308,3 +308,36 @@ def test_a_pin_is_one_per_thing_and_only_its_owners(db, make_user):
     pins.remove(db, pins.own_pin(db, user, pin.id))
     db.commit()
     assert db.execute(select(ComfortPin)).first() is None
+
+
+def test_a_watch_later_save_that_lost_the_race_uses_the_winners_list(db, make_user, monkeypatch):
+    """The first two saves at once both read "no Watch later"; the second must land in the list the
+    first made, not make one of its own."""
+    user = make_user()
+    won = service.watch_later(db, user)
+    db.commit()
+    real = queries.watch_later
+    stale = [True]  # this request's first read happened before the winner committed
+
+    def read(d, u):
+        return None if stale and stale.pop() else real(d, u)
+
+    monkeypatch.setattr(queries, "watch_later", read)
+    assert service.watch_later(db, user).id == won.id
+    db.commit()
+    assert [p.id for p in queries.playlists_for(db, user) if p.kind == "watch_later"] == [won.id]
+
+
+def test_a_first_resume_point_beside_the_watched_mark_lands_on_its_row(db, make_user, monkeypatch):
+    """The first checkpoint of a video and the end-of-playback mark both read "no row"; the
+    checkpoint that writes second must update the mark's row, not fail on `uq_user_video`."""
+    user = make_user()
+    _videos(db, "v1")
+    service.mark(db, user, "v1", "watched")  # the mark that won, committed
+    db.commit()
+    monkeypatch.setattr(queries, "video_state", lambda *a, **k: None)  # read before it committed
+    service.set_progress(db, user, "v1", 90)
+    db.commit()
+    db.expire_all()
+    rows = db.execute(select(VideoState)).scalars().all()
+    assert [(r.status, r.position_seconds) for r in rows] == [("watched", 90)]

backend/tests/test_related_route.py

@@ -0,0 +1,428 @@
+"""`GET /api/search/related/{video_id}` — a video's related videos, without what the user has seen.
+
+The scrape is replaced by pages the test hands out, and YouTube's videos.list by an offline enrich
+(30-minute videos, so no Shorts probe goes out). What is held: the list leaves out what this user
+watched or hid — here or in their Takeout history, unless taken back to "new" since — blocked
+channels and the video itself; it pages on until it has enough; it lands in the catalog the way
+search finds do; and a failure, or the day's cap, says so instead of answering an empty list."""
+
+import threading
+from datetime import datetime, timedelta, timezone
+
+import pytest
+from sqlalchemy import func, select, text
+
+from app import quota, sysconfig
+from app.catalog.models import BlockedChannel, Channel, Subscription, Video
+from app.models import QuotaEvent
+from app.reader.models import TakeoutItem, TakeoutKind, VideoState, VideoUnmark
+from app.routes import search as search_routes
+from app.youtube.client import YouTubeError
+from app.youtube.innertube import ScrapeError
+
+T0 = datetime(2026, 10, 1, tzinfo=timezone.utc)
+SOURCE = "srcvideo001"
+
+
+def _vid(n: int) -> str:
+    return f"rel{n:08d}"
+
+
+def _item(n: int, channel: str = "UCrelated01") -> dict:
+    return {
+        "id": _vid(n),
+        "channel_id": channel,
+        "channel_title": f"Chan {channel}",
+        "title": f"Related {n}",
+        "published_at": None,
+        "thumbnail_url": None,
+        "live_broadcast": "none",
+    }
+
+
+class _FakeYT:
+    def __init__(self, *_a, **_kw):
+        pass
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        return False
+
+    def get_channels(self, ids):
+        return []
+
+    def get_videos(self, ids):
+        return [
+            {
+                "id": i,
+                "snippet": {"title": f"T {i}", "publishedAt": "2025-01-01T00:00:00Z"},
+                "contentDetails": {"duration": "PT30M"},
+                "statistics": {},
+                "status": {},
+            }
+            for i in ids
+        ]
+
+
+@pytest.fixture
+def pages(monkeypatch, db):
+    """The scrape answers with the pages in `state["pages"]`, one per call, recording each call."""
+    sysconfig.set_value(db, "shorts_probe_max_seconds", 60)
+    state: dict = {"pages": [], "calls": []}
+
+    def related(video_id, continuation=None, *, hl="en", gl="US"):
+        state["calls"].append((video_id, continuation, hl, gl))
+        page = state["pages"].pop(0)
+        if isinstance(page, Exception):
+            raise page
+        return page
+
+    monkeypatch.setattr(search_routes.related_scrape, "related", related)
+    monkeypatch.setattr(search_routes, "YouTubeClient", _FakeYT)
+    return state
+
+
+def _page(items: list[dict], token: str | None = "NEXT") -> dict:
+    return {"items": items, "next_page_token": token}
+
+
+def _ids(resp) -> list[str]:
+    assert resp.status_code == 200, resp.text
+    return [it["id"] for it in resp.json()["items"]]
+
+
+def _known(db, *ns: int, channel: str = "UCrelated01") -> None:
+    """These related videos already in the catalog (a state needs the row)."""
+    if db.get(Channel, channel) is None:
+        db.add(Channel(id=channel, title="Chan"))
+    for n in ns:
+        db.add(Video(id=_vid(n), channel_id=channel, title=f"Related {n}", published_at=T0))
+    db.commit()
+
+
+def test_a_related_list_comes_into_the_catalog_as_search_finds_do(db, make_user, as_user, pages):
+    user = make_user()
+    pages["pages"] = [_page([_item(n) for n in range(1, 21)])]
+    resp = as_user(user).get(f"/api/search/related/{SOURCE}")
+    assert _ids(resp) == [_vid(n) for n in range(1, 21)]
+    assert resp.json()["next_cursor"] == "NEXT" and resp.json()["has_more"] is True
+    video = db.get(Video, _vid(1))
+    # Flagged like a search find: hidden from the shared Library, reclaimed when nobody keeps it.
+    assert video.via_search is True and video.enriched_at is not None
+    assert pages["calls"] == [(SOURCE, None, "en", "US")]
+
+
+def test_what_the_user_watched_or_hid_here_is_left_out(db, make_user, as_user, pages):
+    user = make_user()
+    other = make_user()
+    _known(db, 1, 2, 3, 4)
+    db.add_all(
+        [
+            VideoState(user_id=user.id, video_id=_vid(1), status="watched", watched_at=T0),
+            VideoState(user_id=user.id, video_id=_vid(2), status="hidden"),
+            # In progress is not "seen": it stays, to be finished.
+            VideoState(user_id=user.id, video_id=_vid(3), status="new", position_seconds=120),
+            # Someone else's watch is theirs.
+            VideoState(user_id=other.id, video_id=_vid(4), status="watched", watched_at=T0),
+        ]
+    )
+    db.commit()
+    pages["pages"] = [_page([_item(n) for n in range(1, 6)], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(3), _vid(4), _vid(5)]
+
+
+def test_what_the_takeout_history_shows_watched_is_left_out_unless_taken_back(
+    db, make_user, as_user, pages
+):
+    user = make_user()
+    other = make_user()
+    _known(db, 2)
+    db.add_all(
+        [
+            # Not in the catalog at all: the history alone names it.
+            TakeoutItem(user_id=user.id, kind=TakeoutKind.WATCHED, video_id=_vid(1), at=T0),
+            # Watched on YouTube, then taken back to "new" here: it may come again.
+            TakeoutItem(user_id=user.id, kind=TakeoutKind.WATCHED, video_id=_vid(2), at=T0),
+            VideoUnmark(user_id=user.id, video_id=_vid(2), at=T0 + timedelta(days=1)),
+            # Only in Watch later: not watched.
+            TakeoutItem(user_id=user.id, kind=TakeoutKind.WATCH_LATER, video_id=_vid(3), at=T0),
+            # Another user's history.
+            TakeoutItem(user_id=other.id, kind=TakeoutKind.WATCHED, video_id=_vid(4), at=T0),
+        ]
+    )
+    db.commit()
+    pages["pages"] = [_page([_item(n) for n in range(1, 5)], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(2), _vid(3), _vid(4)]
+    # Left out before the ingest: what the user has seen is not stored for a list that hides it.
+    assert db.get(Video, _vid(1)) is None
+
+
+def test_an_unmark_older_than_the_history_row_does_not_bring_it_back(db, make_user, as_user, pages):
+    user = make_user()
+    _known(db, 1)
+    db.add_all(
+        [
+            VideoUnmark(user_id=user.id, video_id=_vid(1), at=T0 - timedelta(days=1)),
+            TakeoutItem(user_id=user.id, kind=TakeoutKind.WATCHED, video_id=_vid(1), at=T0),
+        ]
+    )
+    db.commit()
+    pages["pages"] = [_page([_item(1), _item(2)], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(2)]
+
+
+def test_another_users_unmark_does_not_bring_it_back(db, make_user, as_user, pages):
+    user = make_user()
+    other = make_user()
+    _known(db, 1)
+    db.add_all(
+        [
+            TakeoutItem(user_id=user.id, kind=TakeoutKind.WATCHED, video_id=_vid(1), at=T0),
+            VideoUnmark(user_id=other.id, video_id=_vid(1), at=T0 + timedelta(days=1)),
+        ]
+    )
+    db.commit()
+    pages["pages"] = [_page([_item(1), _item(2)], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(2)]
+
+
+def test_blocked_channels_and_the_video_itself_are_left_out_and_never_stored(
+    db, make_user, as_user, pages
+):
+    user = make_user()
+    db.add(Channel(id="UCblocked01", title="Blocked"))
+    db.commit()
+    db.add(BlockedChannel(user_id=user.id, channel_id="UCblocked01"))
+    db.commit()
+    own = {**_item(9), "id": SOURCE}
+    pages["pages"] = [_page([_item(1), _item(2, "UCblocked01"), own], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(1)]
+    assert db.get(Video, _vid(2)) is None
+
+
+def test_it_pages_on_until_enough_unseen_videos_are_gathered(db, make_user, as_user, pages):
+    user = make_user()
+    _known(db, *range(1, 16))
+    for n in range(1, 16):
+        db.add(VideoState(user_id=user.id, video_id=_vid(n), status="watched", watched_at=T0))
+    db.commit()
+    pages["pages"] = [
+        _page([_item(n) for n in range(1, 21)], token="page2"),  # 15 seen, 5 new
+        _page([_item(n) for n in range(21, 41)], token="page3"),  # 20 new: 25 now
+        _page([_item(n) for n in range(41, 61)], token="page4"),  # never asked
+    ]
+    ids = _ids(resp := as_user(user).get(f"/api/search/related/{SOURCE}"))
+    assert ids == [_vid(n) for n in range(16, 41)]
+    assert [c[1] for c in pages["calls"]] == [None, "page2"]
+    assert resp.json()["next_cursor"] == "page3"
+
+
+def test_a_repeat_on_a_later_page_counts_once(db, make_user, as_user, pages):
+    user = make_user()
+    pages["pages"] = [
+        _page([_item(n) for n in range(1, 11)], token="page2"),
+        _page([_item(n) for n in range(5, 25)], token=None),
+    ]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [
+        _vid(n) for n in range(1, 25)
+    ]
+
+
+def test_it_reads_at_most_four_pages(db, make_user, as_user, pages):
+    user = make_user()
+    pages["pages"] = [_page([_item(n)], token=f"page{n + 1}") for n in range(1, 7)]
+    ids = _ids(resp := as_user(user).get(f"/api/search/related/{SOURCE}"))
+    assert ids == [_vid(n) for n in range(1, 5)]
+    assert len(pages["calls"]) == 4 and resp.json()["next_cursor"] == "page5"
+
+
+def test_the_cursor_and_the_hungarian_locale_reach_the_scrape(db, make_user, as_user, pages):
+    user = make_user()
+    pages["pages"] = [_page([_item(1)], token=None)]
+    resp = as_user(user).get(f"/api/search/related/{SOURCE}?cursor=C1&region=hu")
+    assert _ids(resp) == [_vid(1)]
+    assert pages["calls"] == [(SOURCE, "C1", "hu", "HU")]
+    assert resp.json()["has_more"] is False
+
+
+def test_an_unknown_region_is_refused(db, make_user, as_user, pages):
+    user = make_user()
+    assert as_user(user).get(f"/api/search/related/{SOURCE}?region=xx").status_code == 422
+    assert pages["calls"] == []
+
+
+def test_a_failed_scrape_says_youtube_is_unavailable(db, make_user, as_user, pages):
+    user = make_user()
+    pages["pages"] = [ScrapeError("blocked")]
+    resp = as_user(user).get(f"/api/search/related/{SOURCE}")
+    assert resp.status_code == 422
+    assert resp.json()["detail"] == {"code": "youtube", "reason": "unavailable"}
+
+
+def test_a_failure_on_a_later_page_keeps_what_came_before(db, make_user, as_user, pages):
+    user = make_user()
+    pages["pages"] = [_page([_item(1), _item(2)], token="page2"), ScrapeError("blocked")]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(1), _vid(2)]
+
+
+def test_one_list_is_one_event_against_the_days_cap(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "related_daily_limit_per_user", 1)
+    pages["pages"] = [
+        _page([_item(1)], token="page2"),
+        _page([_item(2)], token=None),
+    ]
+    client = as_user(user)
+    assert client.get(f"/api/search/related/{SOURCE}").status_code == 200
+    events = db.scalars(
+        select(QuotaEvent).where(
+            QuotaEvent.user_id == user.id, QuotaEvent.action == quota.QuotaAction.VIDEOS_RELATED
+        )
+    ).all()
+    assert len(events) == 1
+    refused = client.get(f"/api/search/related/{SOURCE}")
+    assert refused.status_code == 429
+    assert refused.json()["detail"] == {
+        "code": "refused",
+        "reason": "related_daily_limit",
+        "limit": 1,
+    }
+    assert len(pages["calls"]) == 2  # the refused one asked YouTube nothing
+
+
+def test_reading_further_takes_a_slot_too(db, make_user, as_user, pages):
+    # Each request scrapes and enriches; a cursor is a free string, so one that cost nothing let a
+    # replayed cursor read without end.
+    user = make_user()
+    sysconfig.set_value(db, "related_daily_limit_per_user", 1)
+    pages["pages"] = [_page([_item(1)], token=None), _page([_item(2)], token=None)]
+    client = as_user(user)
+    assert client.get(f"/api/search/related/{SOURCE}").status_code == 200
+    refused = client.get(f"/api/search/related/{SOURCE}?cursor=more")
+    assert refused.status_code == 429
+    assert refused.json()["detail"]["reason"] == "related_daily_limit"
+    assert len(pages["calls"]) == 1
+
+
+def test_a_list_that_failed_gives_its_slot_back(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "related_daily_limit_per_user", 1)
+    pages["pages"] = [ScrapeError("blocked"), _page([_item(1)], token=None)]
+    client = as_user(user)
+    assert client.get(f"/api/search/related/{SOURCE}").status_code == 422
+    assert quota.actions_today(db, user.id, quota.QuotaAction.VIDEOS_RELATED) == 0
+    assert _ids(client.get(f"/api/search/related/{SOURCE}")) == [_vid(1)]
+
+
+def test_failures_past_the_hours_refunds_keep_their_slot(db, make_user, as_user, pages):
+    # A cursor is a free string: a request built to fail would otherwise read YouTube without end.
+    user = make_user()
+    sysconfig.set_value(db, "related_daily_limit_per_user", 50)
+    n = search_routes.REFUNDS_PER_HOUR
+    pages["pages"] = [ScrapeError("bad cursor") for _ in range(n + 1)]
+    client = as_user(user)
+    for _ in range(n + 1):
+        assert client.get(f"/api/search/related/{SOURCE}?cursor=bogus").status_code == 422
+    assert quota.actions_today(db, user.id, quota.QuotaAction.VIDEOS_RELATED) == 1
+
+
+def test_a_zero_limit_turns_related_videos_off(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "related_daily_limit_per_user", 0)
+    resp = as_user(user).get(f"/api/search/related/{SOURCE}")
+    assert resp.status_code == 409
+    assert resp.json()["detail"] == {"code": "refused", "reason": "related_disabled"}
+    assert pages["calls"] == []
+
+
+def test_the_last_slot_is_taken_under_a_lock(db, make_user):
+    # Checked first and logged after seconds of scraping, requests sent together at the last free
+    # slot all passed. The claim waits for one already taking a slot for this user and action.
+    user = make_user()
+    action = quota.QuotaAction.VIDEOS_RELATED
+    db.execute(select(func.pg_advisory_xact_lock(func.hashtext(action), user.id)))
+    got: list[int | None] = []
+    t = threading.Thread(target=lambda: got.append(quota.claim_action(user.id, action, 1)))
+    t.start()
+    t.join(timeout=1)
+    assert t.is_alive()  # it waits for the one holding the lock
+    # ...and counts only once it has the lock: the one holding it takes the last slot meanwhile.
+    db.add(QuotaEvent(user_id=user.id, action=action, units=0))
+    db.commit()
+    t.join(timeout=10)
+    assert got == [None]
+
+
+def _videos_list_fails(monkeypatch, *, on_call: int = 1, reason: str = "quotaExceeded") -> None:
+    """YouTube's videos.list refuses from its `on_call`-th call on."""
+    calls = {"n": 0}
+    real = _FakeYT.get_videos
+
+    def get_videos(self, ids):
+        calls["n"] += 1
+        if calls["n"] >= on_call:
+            raise YouTubeError("videos -> 403", reason=reason, path="videos")
+        return real(self, ids)
+
+    monkeypatch.setattr(_FakeYT, "get_videos", get_videos)
+
+
+def test_a_lookup_that_fails_says_why_not_no_videos(db, make_user, as_user, pages, monkeypatch):
+    # Nothing it found can be kept unenriched; an empty list would read as "nothing related".
+    user = make_user()
+    _videos_list_fails(monkeypatch)
+    pages["pages"] = [_page([_item(1)], token=None)]
+    resp = as_user(user).get(f"/api/search/related/{SOURCE}")
+    assert resp.status_code == 422
+    assert resp.json()["detail"] == {"code": "youtube", "reason": "quota"}
+    assert quota.actions_today(db, user.id, quota.QuotaAction.VIDEOS_RELATED) == 0
+
+
+def test_a_lookup_failing_on_a_later_page_keeps_what_came_before(
+    db, make_user, as_user, pages, monkeypatch
+):
+    user = make_user()
+    _videos_list_fails(monkeypatch, on_call=2)
+    pages["pages"] = [_page([_item(1)], token="page2"), _page([_item(2)], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(1)]
+
+
+def test_an_ingest_that_breaks_the_session_is_rolled_back(
+    db, make_user, as_user, pages, monkeypatch
+):
+    # A database error leaves the session unusable until it is rolled back: the rows of the pages
+    # before could not be read for the answer.
+    user = make_user()
+    real = search_routes.catalog_search.ingest_candidates
+    calls = {"n": 0}
+
+    def ingest(db_, yt, user_, candidates):
+        calls["n"] += 1
+        if calls["n"] == 2:
+            db_.execute(text("SELECT 1/0"))
+        return real(db_, yt, user_, candidates)
+
+    monkeypatch.setattr(search_routes.catalog_search, "ingest_candidates", ingest)
+    pages["pages"] = [_page([_item(1)], token="page2"), _page([_item(2)], token=None)]
+    assert _ids(as_user(user).get(f"/api/search/related/{SOURCE}")) == [_vid(1)]
+
+
+def test_each_row_says_whether_the_user_follows_its_channel(db, make_user, as_user, pages):
+    user = make_user()
+    stranger = make_user()
+    _known(db, 1, channel="UCfollowed1")
+    _known(db, 2, channel="UCstranger1")
+    db.add(Subscription(user_id=user.id, channel_id="UCfollowed1"))
+    db.add(Subscription(user_id=stranger.id, channel_id="UCstranger1"))
+    db.commit()
+    pages["pages"] = [
+        _page([_item(1, "UCfollowed1"), _item(2, "UCstranger1"), _item(3, "UCnobody01")], None)
+    ]
+    rows = as_user(user).get(f"/api/search/related/{SOURCE}").json()["items"]
+    assert {r["id"]: r["followed"] for r in rows} == {
+        _vid(1): True,
+        _vid(2): False,  # another user's follow is not this one's
+        _vid(3): False,
+    }

backend/tests/test_related_scrape.py

@@ -0,0 +1,232 @@
+"""youtube/related_scrape — a video's related videos from the InnerTube `next` endpoint.
+
+The two pages under `fixtures/` are real answers (2026-10-24), trimmed to three items and the
+continuation: they hold the `lockupViewModel` shape YouTube draws today. Everything here breaks
+silently when YouTube moves — a parser that stops finding videos, a cursor that stops paging — so a
+shape it does not know must say so, never read as "nothing related". The network is an httpx
+MockTransport."""
+
+import copy
+import json
+from pathlib import Path
+
+import httpx
+import pytest
+
+from app.youtube import innertube, related_scrape
+
+FIXTURES = Path(__file__).parent / "fixtures"
+FIRST = json.loads((FIXTURES / "innertube_next_first.json").read_text(encoding="utf-8"))
+CONTINUATION = json.loads(
+    (FIXTURES / "innertube_next_continuation.json").read_text(encoding="utf-8")
+)
+RESULTS_HTML = (
+    '<script>{"INNERTUBE_API_KEY":"live-key","INNERTUBE_CLIENT_VERSION":"2.20261001"}</script>'
+)
+
+
+def _results(page: dict) -> list:
+    return page["contents"]["twoColumnWatchNextResults"]["secondaryResults"]["secondaryResults"][
+        "results"
+    ]
+
+
+@pytest.fixture
+def net(monkeypatch):
+    """Route every httpx.Client the InnerTube module opens through a handler the test sets."""
+    calls: list[httpx.Request] = []
+    state: dict = {"handler": None, "calls": calls}
+
+    def transport(request: httpx.Request) -> httpx.Response:
+        calls.append(request)
+        if request.url.path == "/results":
+            return httpx.Response(200, text=RESULTS_HTML)
+        return state["handler"](request)
+
+    real = httpx.Client
+    monkeypatch.setattr(
+        innertube.httpx,
+        "Client",
+        lambda **kw: real(transport=httpx.MockTransport(transport), **kw),
+    )
+    monkeypatch.setattr(innertube, "_config", None)
+    return state
+
+
+def _answer(net, page, status=200):
+    net["handler"] = lambda request: (
+        httpx.Response(status, text=page)
+        if isinstance(page, str)
+        else httpx.Response(status, json=page)
+    )
+
+
+def test_a_first_page_gives_its_videos_with_their_channel_and_the_next_cursor(net):
+    _answer(net, FIRST)
+    page = related_scrape.related("SdTZAMDKrNY")
+    assert [it["id"] for it in page["items"]] == ["to-lHJfK4pw", "R8V8CbuxryI", "HLKo4fJx_7k"]
+    first = page["items"][0]
+    assert first["channel_id"] == "UCtslD4DGH6PKyG_1gFAX7sg"
+    assert first["channel_title"] == "Alexander Amini"
+    assert first["title"] == "MIT 6.S191 (2025): Reinforcement Learning"
+    assert first["thumbnail_url"].startswith("https://i.ytimg.com/vi/to-lHJfK4pw/")
+    assert first["published_at"] is None and first["live_broadcast"] == "none"
+    assert page["next_page_token"]
+    body = json.loads(net["calls"][-1].content)
+    assert body["videoId"] == "SdTZAMDKrNY" and "continuation" not in body
+    assert net["calls"][-1].url.path == "/youtubei/v1/next"
+
+
+def test_the_largest_thumbnail_is_taken(net):
+    _answer(net, FIRST)
+    item = related_scrape.related("SdTZAMDKrNY")["items"][0]
+    sources = _results(FIRST)[0]["lockupViewModel"]["contentImage"]["thumbnailViewModel"]["image"][
+        "sources"
+    ]
+    assert item["thumbnail_url"] == sources[-1]["url"]
+
+
+def test_a_continuation_page_reads_its_videos_and_sends_only_the_cursor(net):
+    _answer(net, CONTINUATION)
+    page = related_scrape.related("SdTZAMDKrNY", continuation="TOKEN", hl="hu", gl="HU")
+    assert len(page["items"]) == 2
+    assert all(it["channel_id"].startswith("UC") for it in page["items"])
+    assert page["next_page_token"]
+    body = json.loads(net["calls"][-1].content)
+    assert body["continuation"] == "TOKEN" and "videoId" not in body
+    assert body["context"]["client"]["hl"] == "hu" and body["context"]["client"]["gl"] == "HU"
+
+
+def test_the_locale_defaults_to_the_topical_one(net):
+    _answer(net, FIRST)
+    related_scrape.related("SdTZAMDKrNY")
+    client = json.loads(net["calls"][-1].content)["context"]["client"]
+    assert (client["hl"], client["gl"]) == ("en", "US")
+
+
+def test_a_video_by_several_channels_is_left_out(net):
+    page = copy.deepcopy(FIRST)
+    meta = _results(page)[1]["lockupViewModel"]["metadata"]["lockupMetadataViewModel"]
+    meta["image"] = {"avatarStackViewModel": {"avatars": []}}
+    _answer(net, page)
+    ids = [it["id"] for it in related_scrape.related("SdTZAMDKrNY")["items"]]
+    assert ids == ["to-lHJfK4pw", "HLKo4fJx_7k"]
+
+
+def test_what_is_not_a_video_is_left_out_and_a_repeat_counts_once(net):
+    page = copy.deepcopy(FIRST)
+    results = _results(page)
+    playlist = copy.deepcopy(results[0])
+    playlist["lockupViewModel"]["contentType"] = "LOCKUP_CONTENT_TYPE_PLAYLIST"
+    playlist["lockupViewModel"]["contentId"] = "PLxyz"
+    results.insert(0, playlist)
+    results.insert(2, copy.deepcopy(results[1]))
+    results.insert(0, {"adSlotRenderer": {}})
+    _answer(net, page)
+    ids = [it["id"] for it in related_scrape.related("SdTZAMDKrNY")["items"]]
+    assert ids == ["to-lHJfK4pw", "R8V8CbuxryI", "HLKo4fJx_7k"]
+
+
+def test_the_previous_compact_renderer_is_still_read(net):
+    page = copy.deepcopy(FIRST)
+    _results(page).insert(
+        0,
+        {
+            "compactVideoRenderer": {
+                "videoId": "oldshape001",
+                "title": {"simpleText": "Old shape"},
+                "longBylineText": {
+                    "runs": [
+                        {
+                            "text": "Old Chan",
+                            "navigationEndpoint": {"browseEndpoint": {"browseId": "UColdchan"}},
+                        }
+                    ]
+                },
+                "thumbnail": {"thumbnails": [{"url": "s.jpg"}, {"url": "b.jpg"}]},
+            }
+        },
+    )
+    _answer(net, page)
+    first = related_scrape.related("SdTZAMDKrNY")["items"][0]
+    assert first == {
+        "id": "oldshape001",
+        "channel_id": "UColdchan",
+        "channel_title": "Old Chan",
+        "title": "Old shape",
+        "published_at": None,
+        "thumbnail_url": "b.jpg",
+        "live_broadcast": "none",
+    }
+
+
+def test_the_last_page_has_no_cursor(net):
+    page = copy.deepcopy(FIRST)
+    _results(page).pop()  # the continuationItemRenderer
+    _answer(net, page)
+    assert related_scrape.related("SdTZAMDKrNY")["next_page_token"] is None
+
+
+def test_a_video_youtube_no_longer_has_is_an_empty_page(net):
+    _answer(
+        net,
+        {
+            "contents": {
+                "twoColumnWatchNextResults": {
+                    "results": {
+                        "results": {
+                            "contents": [
+                                {
+                                    "itemSectionRenderer": {
+                                        "contents": [{"backgroundPromoRenderer": {}}]
+                                    }
+                                }
+                            ]
+                        }
+                    }
+                }
+            }
+        },
+    )
+    assert related_scrape.related("aaaaaaaaaaa") == {"items": [], "next_page_token": None}
+
+
+def test_a_watch_frame_with_no_related_column_and_no_word_why_is_an_error(net):
+    # The frame alone, its related column moved somewhere this does not look: a lost shape, not
+    # "nothing related".
+    page = copy.deepcopy(FIRST)
+    watch = page["contents"]["twoColumnWatchNextResults"]
+    watch["sideColumn"] = watch.pop("secondaryResults")
+    _answer(net, page)
+    with pytest.raises(related_scrape.ScrapeError):
+        related_scrape.related("SdTZAMDKrNY")
+
+
+def test_a_related_list_drawn_by_renderers_this_does_not_read_is_an_error(net):
+    page = copy.deepcopy(FIRST)
+    results = _results(page)
+    results[:] = [{"videoTileViewModel": entry} for entry in results]
+    _answer(net, page)
+    with pytest.raises(related_scrape.ScrapeError):
+        related_scrape.related("SdTZAMDKrNY")
+
+
+@pytest.mark.parametrize(
+    "answer",
+    [
+        {"responseContext": {}},  # no watch page frame at all
+        {"onResponseReceivedEndpoints": [{"somethingElse": {}}]},  # a continuation we do not know
+        "<html>consent</html>",  # not JSON
+        [1, 2],  # JSON, not an object
+    ],
+)
+def test_an_answer_of_an_unknown_shape_is_an_error_not_an_empty_list(net, answer):
+    _answer(net, answer)
+    with pytest.raises(related_scrape.ScrapeError):
+        related_scrape.related("SdTZAMDKrNY")
+
+
+def test_a_refused_request_is_an_error_whatever_its_body_says(net):
+    _answer(net, FIRST, status=429)
+    with pytest.raises(related_scrape.ScrapeError):
+        related_scrape.related("SdTZAMDKrNY")

backend/tests/test_route_commits.py

@@ -0,0 +1,400 @@
+"""Writes driven through HTTP, each read back: the handler called its unit and committed.
+
+A route's helper only flushes; the handler commits. The `db` fixture fails a test whose request left
+a write uncommitted, so every test here drives one successful write through its route and then
+reads the result back. The read-back holds that the handler called its unit; it runs in the
+request's own session, so it sees a write that was never committed — the commit is the fixture's
+to hold (a test that must see it committed reads from a session of its own). The services' own
+tests hold what each write does; these hold that the route finishes it. Nothing here reaches
+YouTube or Plex: the YouTube client is a stub and no user has a Plex link. Needs the DB.
+"""
+
+from datetime import datetime, timedelta, timezone
+
+import pytest
+from sqlalchemy import select
+
+from app import quota
+from app.auth import require_write_scope
+from app.catalog.models import Channel, ExploredChannel, SearchFind, Subscription, Video
+from app.downloads.models import DownloadJob, DownloadProfile, DownloadShare
+from app.plex.models import PlexItem, PlexLibrary, PlexSeason, PlexShow, PlexState
+from app.db import SessionLocal
+from app.models import Notification
+from app.notifications import READ_SOFT_CAP
+from app.reader.models import SavedView, Tag, VideoState
+from app.routes import channels
+
+NOW = datetime.now(timezone.utc)
+
+
+def _video(db, vid="v1", **kw):
+    if db.get(Channel, "UCr") is None:
+        db.add(Channel(id="UCr", title="R"))
+        db.flush()
+    db.add(Video(id=vid, channel_id="UCr", published_at=NOW, duration_seconds=60, **kw))
+    db.commit()
+
+
+# --- tags ------------------------------------------------------------------------------------
+
+
+def test_a_tag_made_through_the_route_is_listed_back(db, make_user, as_user):
+    user = make_user()
+    client = as_user(user)
+    r = client.post("/api/tags", json={"name": "Science"})
+    assert r.status_code == 200, r.text
+    assert [t["name"] for t in client.get("/api/tags").json()] == ["Science"]
+
+
+def test_a_tag_renamed_through_the_route_is_listed_back(db, make_user, as_user):
+    user = make_user()
+    tag = Tag(user_id=user.id, name="Old")
+    db.add(tag)
+    db.commit()
+    client = as_user(user)
+    assert client.patch(f"/api/tags/{tag.id}", json={"name": "New"}).status_code == 200
+    assert [t["name"] for t in client.get("/api/tags").json()] == ["New"]
+
+
+def test_a_tag_deleted_through_the_route_is_gone(db, make_user, as_user):
+    user = make_user()
+    tag = Tag(user_id=user.id, name="Gone")
+    db.add(tag)
+    db.commit()
+    client = as_user(user)
+    assert client.delete(f"/api/tags/{tag.id}").json() == {"deleted": tag.id}
+    assert client.get("/api/tags").json() == []
+
+
+# --- watch state ------------------------------------------------------------------------------
+
+
+def _state(db, user, vid="v1"):
+    db.expire_all()
+    return db.execute(
+        select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == vid)
+    ).scalar_one_or_none()
+
+
+def test_a_video_marked_watched_through_the_route_stays_watched(db, make_user, as_user):
+    user = make_user()
+    _video(db)
+    r = as_user(user).post("/api/videos/v1/state", json={"status": "watched"})
+    assert r.status_code == 200, r.text
+    assert _state(db, user).status == "watched"
+
+
+def test_a_video_unmarked_through_the_route_stays_unmarked(db, make_user, as_user):
+    # "new" is its own branch of the handler, with its own commit.
+    user = make_user()
+    _video(db)
+    db.add(VideoState(user_id=user.id, video_id="v1", status="watched"))
+    db.commit()
+    r = as_user(user).post("/api/videos/v1/state", json={"status": "new"})
+    assert r.status_code == 200, r.text
+    assert _state(db, user) is None
+
+
+def test_a_video_state_cleared_through_the_route_is_gone(db, make_user, as_user):
+    user = make_user()
+    _video(db)
+    db.add(VideoState(user_id=user.id, video_id="v1", status="hidden", position_seconds=90))
+    db.commit()
+    assert as_user(user).delete("/api/videos/v1/state").status_code == 200
+    assert _state(db, user) is None
+
+
+# --- saved views ------------------------------------------------------------------------------
+
+
+def test_views_reordered_through_the_route_list_back_in_that_order(db, make_user, as_user):
+    user = make_user()
+    a = SavedView(user_id=user.id, source="feed", name="A", filters={}, position=0)
+    b = SavedView(user_id=user.id, source="feed", name="B", filters={}, position=1)
+    db.add_all([a, b])
+    db.commit()
+    client = as_user(user)
+    assert client.post("/api/saved-views/reorder", json={"ids": [b.id, a.id]}).status_code == 200
+    listed = client.get("/api/saved-views").json()
+    assert [(v["name"], v["position"]) for v in listed] == [("B", 0), ("A", 1)]
+
+
+# --- download profiles and shares -------------------------------------------------------------
+
+_SPEC = {"format": "best"}
+
+
+def _profile(db, user, name="Mine"):
+    p = DownloadProfile(user_id=user.id, name=name, spec=_SPEC, is_builtin=False, sort_order=101)
+    db.add(p)
+    db.commit()
+    return p
+
+
+def _profile_names(client):
+    return [p["name"] for p in client.get("/api/downloads/profiles").json()]
+
+
+def test_a_profile_made_through_the_route_is_listed_back(db, make_user, as_user):
+    user = make_user()
+    client = as_user(user)
+    r = client.post("/api/downloads/profiles", json={"name": "Phone", "spec": _SPEC})
+    assert r.status_code == 200, r.text
+    assert _profile_names(client) == ["Phone"]
+
+
+def test_a_profile_renamed_through_the_route_is_listed_back(db, make_user, as_user):
+    user = make_user()
+    p = _profile(db, user)
+    client = as_user(user)
+    r = client.patch(f"/api/downloads/profiles/{p.id}", json={"name": "Renamed"})
+    assert r.status_code == 200, r.text
+    assert _profile_names(client) == ["Renamed"]
+
+
+def test_a_profile_deleted_through_the_route_is_gone(db, make_user, as_user):
+    user = make_user()
+    p = _profile(db, user)
+    client = as_user(user)
+    assert client.delete(f"/api/downloads/profiles/{p.id}").json() == {"deleted": p.id}
+    assert _profile_names(client) == []
+
+
+def test_a_share_taken_back_through_the_route_is_gone(db, make_user, as_user):
+    owner, friend = make_user(), make_user()
+    job = DownloadJob(
+        user_id=owner.id, source_kind="youtube", source_ref="dQw4w9WgXcQ", profile_snapshot={}
+    )
+    db.add(job)
+    db.flush()
+    db.add(DownloadShare(job_id=job.id, from_user_id=owner.id, to_user_id=friend.id))
+    db.commit()
+    r = as_user(owner).delete(f"/api/downloads/{job.id}/share/{friend.email}")
+    assert r.json() == {"ok": True}
+    db.expire_all()
+    assert db.execute(select(DownloadShare)).first() is None
+
+
+# --- channels ---------------------------------------------------------------------------------
+
+
+class _YouTube:
+    """The YouTube client, answering every write the channel routes make."""
+
+    def __init__(self, db, user):
+        pass
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        return False
+
+    def insert_subscription(self, channel_id):
+        return "yt-sub-1"
+
+    def delete_subscription(self, sub_id):
+        return None
+
+    def get_channels(self, ids):
+        return []
+
+
+@pytest.fixture
+def youtube(monkeypatch):
+    monkeypatch.setattr(channels, "YouTubeClient", _YouTube)
+    monkeypatch.setattr(quota, "backfill_reserve_reached", lambda: False)
+
+
+def _channel(db, cid="UCc", **kw):
+    ch = Channel(id=cid, title="C", details_synced_at=NOW, **kw)
+    db.add(ch)
+    db.commit()
+    return ch
+
+
+def _subscription(db, user, cid="UCc"):
+    db.expire_all()
+    return db.execute(
+        select(Subscription).where(Subscription.user_id == user.id, Subscription.channel_id == cid)
+    ).scalar_one_or_none()
+
+
+def test_a_subscribe_through_the_route_is_recorded(db, make_user, as_user, override_dep, youtube):
+    user = make_user()
+    _channel(db)
+    client = as_user(user)
+    override_dep(require_write_scope, lambda: user)
+    assert client.post("/api/channels/UCc/subscribe").json() == {"ok": True, "subscribed": "UCc"}
+    assert _subscription(db, user).yt_subscription_id == "yt-sub-1"
+
+
+def test_an_explore_through_the_route_is_recorded(db, make_user, as_user, youtube, monkeypatch):
+    user = make_user()
+    _channel(db)
+    # The ingest pages YouTube; what the route itself records is the exploration.
+    monkeypatch.setattr(channels, "details_stale", lambda ch, now: False)
+    monkeypatch.setattr(
+        channels,
+        "explore_ingest_page",
+        lambda db, yt, ch, token: {"ingested": 0, "next_page_token": None},
+    )
+    r = as_user(user).post("/api/channels/UCc/explore")
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    assert db.execute(select(ExploredChannel.user_id)).scalars().all() == [user.id]
+    assert db.get(Channel, "UCc").from_explore is True
+
+
+def test_an_unsubscribe_through_the_route_drops_the_subscription(
+    db, make_user, as_user, override_dep, youtube
+):
+    user = make_user()
+    _channel(db)
+    db.add(Subscription(user_id=user.id, channel_id="UCc", yt_subscription_id="yt-sub-1"))
+    db.commit()
+    client = as_user(user)
+    override_dep(require_write_scope, lambda: user)
+    r = client.delete("/api/channels/UCc/subscription")
+    assert r.json() == {"ok": True, "unsubscribed": "UCc"}
+    assert _subscription(db, user) is None
+
+
+def test_a_backfill_reset_through_the_route_is_kept(db, make_user, as_user, monkeypatch):
+    admin, reader = make_user(role="admin"), make_user()
+    _channel(db, backfill_done=True, backfill_cursor="page-9", recent_synced_at=NOW)
+    db.add(Subscription(user_id=reader.id, channel_id="UCc", deep_requested=False))
+    db.commit()
+    # The recent pull afterwards reaches YouTube; the reset is the route's own write.
+    monkeypatch.setattr(channels, "run_recent_backfill", lambda *a, **kw: None)
+    r = as_user(admin).post("/api/channels/UCc/reset-backfill")
+    assert r.status_code == 200, r.text
+    db.expire_all()
+    ch = db.get(Channel, "UCc")
+    assert (ch.backfill_done, ch.backfill_cursor, ch.recent_synced_at) == (False, None, None)
+    assert _subscription(db, reader).deep_requested is True
+
+
+# --- live search ------------------------------------------------------------------------------
+
+
+def test_search_results_cleared_through_the_route_are_gone(db, make_user, as_user):
+    user = make_user()
+    _video(db, "s1", via_search=True)
+    db.add(SearchFind(user_id=user.id, video_id="s1"))
+    db.commit()
+    r = as_user(user).post("/api/search/clear", json={"video_ids": ["s1"]})
+    assert r.json() == {"ok": True, "removed": 1}
+    db.expire_all()
+    assert db.get(Video, "s1") is None
+    assert db.execute(select(SearchFind)).first() is None
+
+
+def test_clearing_leaves_a_video_the_caller_never_found(db, make_user, as_user):
+    # A related list ingests its rows with no find: they are in someone's list right now, and an id
+    # any user could name must not delete them from under it.
+    user = make_user()
+    _video(db, "s1", via_search=True)
+    _video(db, "s2", via_search=True)
+    db.add(SearchFind(user_id=user.id, video_id="s1"))
+    db.commit()
+    r = as_user(user).post("/api/search/clear", json={"video_ids": ["s1", "s2"]})
+    assert r.json() == {"ok": True, "removed": 1}
+    db.expire_all()
+    assert db.get(Video, "s1") is None and db.get(Video, "s2") is not None
+    # Naming only ids it never found removes nothing.
+    r = as_user(user).post("/api/search/clear", json={"video_ids": ["s2"]})
+    assert r.json() == {"ok": True, "removed": 0}
+
+
+# --- Plex show and season marks ---------------------------------------------------------------
+
+
+def _show(db):
+    lib = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
+    db.add(lib)
+    db.flush()
+    sh = PlexShow(rating_key="sh1", library_id=lib.id, title="Show")
+    db.add(sh)
+    db.flush()
+    se = PlexSeason(rating_key="se1", show_id=sh.id, season_number=1)
+    db.add(se)
+    db.flush()
+    for n in (1, 2):
+        db.add(
+            PlexItem(
+                rating_key=f"e{n}",
+                library_id=lib.id,
+                kind="episode",
+                title=f"E{n}",
+                show_id=sh.id,
+                season_id=se.id,
+                season_number=1,
+                episode_number=n,
+            )
+        )
+    db.commit()
+
+
+def _plex_marks(db, user):
+    db.expire_all()
+    return sorted(
+        db.execute(
+            select(PlexItem.rating_key, PlexState.status)
+            .join(PlexState, PlexState.item_id == PlexItem.id)
+            .where(PlexState.user_id == user.id)
+        ).all()
+    )
+
+
+def test_a_show_marked_watched_through_the_route_stays_watched(db, make_user, as_user):
+    user = make_user()
+    _show(db)
+    r = as_user(user).post("/api/plex/show/sh1/state", json={"watched": True})
+    assert r.json() == {"ok": True, "changed": 2, "watched": True}
+    assert _plex_marks(db, user) == [("e1", "watched"), ("e2", "watched")]
+
+
+def test_a_season_marked_watched_through_the_route_stays_watched(db, make_user, as_user):
+    user = make_user()
+    _show(db)
+    r = as_user(user).post("/api/plex/season/se1/state", json={"watched": True})
+    assert r.json() == {"ok": True, "changed": 2, "watched": True}
+    assert _plex_marks(db, user) == [("e1", "watched"), ("e2", "watched")]
+
+
+def _read_past_the_cap(db, user) -> list[Notification]:
+    """READ_SOFT_CAP read notices, oldest first, and one unread notice after them."""
+    base = datetime(2024, 1, 1, tzinfo=timezone.utc)
+    rows = [
+        Notification(
+            user_id=user.id,
+            type="t",
+            title=f"n{i}",
+            read=i < READ_SOFT_CAP,
+            created_at=base + timedelta(minutes=i),
+        )
+        for i in range(READ_SOFT_CAP + 1)
+    ]
+    db.add_all(rows)
+    db.commit()
+    return rows
+
+
+def _committed_notices(user) -> tuple[int, bool]:
+    """What another session sees: how many of the user's notices are left, and whether the oldest is."""
+    with SessionLocal() as s:
+        ids = s.scalars(select(Notification.title).where(Notification.user_id == user.id)).all()
+    return len(ids), "n0" in ids
+
+
+@pytest.mark.parametrize("path", ["{id}/read", "read_all"])
+def test_reading_a_notice_past_the_cap_commits_the_trim(db, make_user, as_user, path):
+    """Marking a notice read makes READ_SOFT_CAP + 1 read ones: the handler commits the read flag and
+    the trim of the oldest together, and the trim counts the notice just read."""
+    user = make_user()
+    rows = _read_past_the_cap(db, user)
+    r = as_user(user).post("/api/me/notifications/" + path.format(id=rows[-1].id))
+    assert r.json() == {"ok": True}
+    assert _committed_notices(user) == (READ_SOFT_CAP, False)

backend/tests/test_scheduler.py

@@ -6,6 +6,7 @@ through (at the top of _job): holding _activity_lock across the check-and-set me
 caller can hold a job's "running" claim at a time, and it's claimable again once released."""
 
 import logging
+import threading
 import time
 from datetime import datetime, timedelta, timezone
 
@@ -342,6 +343,29 @@ def test_an_ad_hoc_run_is_the_users_run_of_its_target_with_its_spend_and_failure
     assert (failed.status, failed.error) == ("error", "quota exceeded")
 
 
+def _ad_hoc_row(db, job: str) -> dict:
+    rows = scheduler.scheduler_snapshot(db)["ad_hoc"]
+    assert [r["id"] for r in rows] == list(scheduler.AD_HOC_JOBS)  # every one has its row
+    return next(r for r in rows if r["id"] == job)
+
+
+def test_a_takeout_import_has_its_row_named_by_the_importing_user(db, make_user):
+    user = make_user("importer@example.com")
+    user.display_name = "Ida Importer"
+    db.commit()
+    assert _ad_hoc_row(db, "takeout_import")["recent"] == []
+    scheduler._ad_hoc("takeout_import", lambda: {"watched": 2}, user.id, f"user:{user.id}")
+
+    row = _ad_hoc_row(db, "takeout_import")
+    assert (row["running"], row["target"], row["target_title"], row["status"]) == (
+        False,
+        f"user:{user.id}",
+        "Ida Importer",
+        "ok",
+    )
+    assert _ad_hoc_row(db, "channel_backfill")["recent"] == []
+
+
 def test_the_page_shows_the_ad_hoc_runs_beside_the_jobs_with_their_channel(db, make_user):
     user = make_user("shown@example.com")
     db.add(Channel(id="UC_shown", title="BBC Earth"))
@@ -349,7 +373,7 @@ def test_the_page_shows_the_ad_hoc_runs_beside_the_jobs_with_their_channel(db, m
     scheduler._ad_hoc("channel_backfill", lambda: {"skipped": "at capacity"}, user.id, "UC_none")
     scheduler._ad_hoc("channel_backfill", lambda: {"videos_added": 69}, user.id, "UC_shown")
 
-    (row,) = scheduler.scheduler_snapshot(db)["ad_hoc"]
+    row = _ad_hoc_row(db, "channel_backfill")
     assert (row["id"], row["running"], row["target"], row["target_title"]) == (
         "channel_backfill",
         False,
@@ -367,12 +391,12 @@ def test_the_page_shows_the_ad_hoc_runs_beside_the_jobs_with_their_channel(db, m
     seen = []
     scheduler._ad_hoc(
         "channel_backfill",
-        lambda: seen.append(scheduler.scheduler_snapshot(db)["ad_hoc"][0]["running"]) or {},
+        lambda: seen.append(_ad_hoc_row(db, "channel_backfill")["running"]) or {},
         user.id,
         "UC_shown",
     )
     assert seen == [True]
-    assert scheduler.scheduler_snapshot(db)["ad_hoc"][0]["running"] is False
+    assert _ad_hoc_row(db, "channel_backfill")["running"] is False
 
 
 def _finished(db, job) -> list[JobRun]:
@@ -397,9 +421,46 @@ def test_a_second_ad_hoc_request_before_the_first_starts_is_that_same_run(
         "channel_backfill@UC_x",
         "channel_backfill@UC_y",
     ]
+    # Queued, so its target's newest run row is not this run's yet: a page watching it can tell.
+    assert scheduler.ad_hoc_queued(scheduler.CHANNEL_BACKFILL, "UC_x")
+    assert not scheduler.ad_hoc_queued(scheduler.CHANNEL_BACKFILL, "UC_z")
     live_scheduler.remove_all_jobs()  # never run: a run finishing after the test would land in the next
 
 
+def test_a_request_made_while_its_run_is_under_way_is_the_run_after_it(
+    db, make_user, live_scheduler
+):
+    """The scheduler drops a second fire of an id whose run has not returned: the second Takeout file
+    of a pair, uploaded while the first one's import ran, was never imported, and its key stayed
+    queued until a restart (the page spun on "Processing…" for good)."""
+    user = make_user("again@example.com")
+    first_runs, release = threading.Event(), threading.Event()
+    calls = []
+
+    def first():
+        first_runs.set()
+        assert release.wait(10)
+        calls.append("first")
+        return {"n": 1}
+
+    def second():
+        calls.append("second")
+        return {"n": 2}
+
+    scheduler.run_ad_hoc(scheduler.CHANNEL_BACKFILL, first, actor_id=user.id, target="UC_again")
+    assert first_runs.wait(10)
+    scheduler.run_ad_hoc(scheduler.CHANNEL_BACKFILL, second, actor_id=user.id, target="UC_again")
+    assert scheduler.ad_hoc_queued(scheduler.CHANNEL_BACKFILL, "UC_again")
+    release.set()
+
+    deadline = time.monotonic() + 10
+    while time.monotonic() < deadline and len(_finished(db, "channel_backfill")) < 2:
+        time.sleep(0.05)
+    assert calls == ["first", "second"]
+    assert sorted(r.result["n"] for r in _finished(db, "channel_backfill")) == [1, 2]
+    assert not scheduler.ad_hoc_queued(scheduler.CHANNEL_BACKFILL, "UC_again")
+
+
 def test_a_manual_run_leaves_no_actor_behind_on_its_thread(db, make_user):
     """The scheduler's pool threads are reused: an actor a manual run left set made the next
     scheduled run on that thread an admin's "manual" run, with a notice in their inbox."""
@@ -475,7 +536,7 @@ def test_a_run_whose_outcome_cannot_be_written_is_not_called_failed(db, make_use
     scheduler._ad_hoc("channel_backfill", lambda: {"videos_added": 1}, user.id, "UC_blip")
     assert calls == ["ok"]
     # Its row is still `running`, but nothing here is: the page does not call it going.
-    (row,) = scheduler.scheduler_snapshot(db)["ad_hoc"]
+    row = _ad_hoc_row(db, "channel_backfill")
     assert (row["running"], row["status"], row["recent"][0]["status"]) == (
         False,
         "interrupted",
@@ -529,7 +590,7 @@ def test_an_ad_hoc_run_still_going_shows_although_a_newer_one_ended(db, make_use
         return {}
 
     scheduler._ad_hoc("channel_backfill", long_one, user.id, "UC_long")
-    (row,) = rows
+    (row,) = [r for r in rows if r["id"] == "channel_backfill"]
     assert (row["running"], row["target"], row["target_title"]) == (True, "UC_long", "Long one")
     # The outcome beside it is the long one's (none yet), never the short one's skip.
     assert (row["status"], row["result"], row["last_error"]) == (None, None, None)

backend/tests/test_search_cap.py

@@ -0,0 +1,361 @@
+"""`GET /api/search/youtube` — the per-user daily cap of the live search, both sources.
+
+The scrape and search.list are replaced by pages the test hands out, and YouTube's videos.list by an
+offline enrich. What is held: every request takes a slot, a cursor's too; a search that failed
+outright gives its slot back; a zero limit turns live search off, which says so instead of "try
+again tomorrow"; a lookup that fails says why instead of answering "no results"; only a few
+failures an hour get their slot back; and the API source — 100 units a page — reads one page per
+request, is refused before the call when the shared budget cannot pay for it (checked again under
+one lock with the call, whose wait is bounded), counts one slot per search though search.list logs
+its own event, and keeps the slot of a search search.list was already paid for — but not of one
+that never reached YouTube."""
+
+import threading
+
+import pytest
+
+from app import quota, sysconfig
+from app.routes import search as search_routes
+from app.youtube.client import YouTubeError
+from app.youtube.innertube import ScrapeError
+
+ACTION = quota.QuotaAction.VIDEOS_SEARCH
+
+
+def _item(n: int) -> dict:
+    return {
+        "id": f"srch{n:07d}",
+        "channel_id": "UCsearch001",
+        "channel_title": "Chan",
+        "title": f"Found {n}",
+        "published_at": None,
+        "thumbnail_url": None,
+        "live_broadcast": "none",
+    }
+
+
+class _FakeYT:
+    fail_videos = False
+
+    def __init__(self, *_a, **_kw):
+        self.spent = 0
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        return False
+
+    def get_channels(self, ids):
+        return []
+
+    # search.list: the pages the test hands out, each logged as the real client does (100 units,
+    # under the action the route attributes it to) — a refusal YouTube answered too, but not a
+    # failure no endpoint answered (no connection, no token).
+    api_pages: list = []
+    api_calls = 0
+
+    def search_videos(self, q, page_token=None, order="relevance"):
+        _FakeYT.api_calls += 1
+        page = _FakeYT.api_pages.pop(0)
+        if not (isinstance(page, YouTubeError) and not page.path):
+            quota.record_usage(100)
+            self.spent += 100
+        if isinstance(page, Exception):
+            raise page
+        return page
+
+    def get_videos(self, ids):
+        if _FakeYT.fail_videos:
+            raise YouTubeError("videos -> 403", reason="quotaExceeded", path="videos")
+        return [
+            {
+                "id": i,
+                "snippet": {"title": f"T {i}", "publishedAt": "2025-01-01T00:00:00Z"},
+                "contentDetails": {"duration": "PT30M"},
+                "statistics": {},
+                "status": {},
+            }
+            for i in ids
+        ]
+
+
+@pytest.fixture
+def pages(monkeypatch, db):
+    """The scrape answers with the pages in `state["pages"]`, one per call."""
+    sysconfig.set_value(db, "shorts_probe_max_seconds", 60)
+    sysconfig.set_value(db, "search_source", "scrape")
+    state: dict = {"pages": [], "calls": 0}
+
+    def search(term, continuation=None):
+        state["calls"] += 1
+        page = state["pages"].pop(0)
+        if isinstance(page, Exception):
+            raise page
+        return page
+
+    monkeypatch.setattr(search_routes.search_scrape, "search", search)
+    monkeypatch.setattr(search_routes, "YouTubeClient", _FakeYT)
+    monkeypatch.setattr(_FakeYT, "fail_videos", False)
+    monkeypatch.setattr(_FakeYT, "api_pages", [])
+    monkeypatch.setattr(_FakeYT, "api_calls", 0)
+    return state
+
+
+def _page(n: int) -> dict:
+    return {"items": [_item(n)], "next_page_token": None}
+
+
+def test_every_request_takes_a_slot_a_cursors_too(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "search_daily_limit_per_user", 1)
+    pages["pages"] = [_page(1), _page(2)]
+    client = as_user(user)
+    assert client.get("/api/search/youtube?q=cats").status_code == 200
+    refused = client.get("/api/search/youtube?q=cats&cursor=more")
+    assert refused.status_code == 429
+    assert refused.json()["detail"]["reason"] == "search_daily_limit"
+    assert pages["calls"] == 1
+
+
+def test_a_search_that_failed_gives_its_slot_back(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "search_daily_limit_per_user", 1)
+    pages["pages"] = [ScrapeError("blocked"), _page(1)]
+    client = as_user(user)
+    assert client.get("/api/search/youtube?q=cats").status_code == 422
+    assert quota.actions_today(db, user.id, ACTION) == 0
+    assert client.get("/api/search/youtube?q=cats").status_code == 200
+    assert quota.actions_today(db, user.id, ACTION) == 1
+
+
+def test_a_zero_limit_turns_live_search_off(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "search_daily_limit_per_user", 0)
+    resp = as_user(user).get("/api/search/youtube?q=cats")
+    assert resp.status_code == 409
+    assert resp.json()["detail"] == {"code": "refused", "reason": "search_disabled"}
+    assert pages["calls"] == 0
+
+
+def test_a_zero_limit_turns_the_api_source_off_too(db, make_user, as_user, pages):
+    user = make_user()
+    sysconfig.set_value(db, "search_source", "api")
+    sysconfig.set_value(db, "search_daily_limit_per_user", 0)
+    resp = as_user(user).get("/api/search/youtube?q=cats")
+    assert resp.status_code == 409
+    assert resp.json()["detail"]["reason"] == "search_disabled"
+
+
+def test_a_lookup_that_fails_says_why_not_no_results(db, make_user, as_user, pages, monkeypatch):
+    user = make_user()
+    monkeypatch.setattr(_FakeYT, "fail_videos", True)
+    pages["pages"] = [_page(1)]
+    resp = as_user(user).get("/api/search/youtube?q=cats")
+    assert resp.status_code == 422
+    assert resp.json()["detail"] == {"code": "youtube", "reason": "quota"}
+    assert quota.actions_today(db, user.id, ACTION) == 0
+
+
+def _api(db) -> None:
+    sysconfig.set_value(db, "search_source", "api")
+    sysconfig.set_value(db, "quota_daily_budget", 10000)
+
+
+def test_the_api_source_reads_one_page_per_request(db, make_user, as_user, pages):
+    # Each page is 100 units: a short page is answered as it is, with its cursor, never paged on.
+    _api(db)
+    _FakeYT.api_pages = [{"items": [_item(1)], "next_page_token": "nextpage"}, _page(2)]
+    resp = as_user(make_user()).get("/api/search/youtube?q=cats&limit=20")
+    assert resp.status_code == 200
+    assert _FakeYT.api_calls == 1
+    assert resp.json()["next_cursor"] == "nextpage"
+
+
+def test_the_api_source_counts_one_slot_per_search(db, make_user, as_user, pages):
+    # search.list logs its own event under the action (record_usage): accounting, not a second slot.
+    user = make_user()
+    _api(db)
+    sysconfig.set_value(db, "search_daily_limit_per_user", 2)
+    _FakeYT.api_pages = [_page(1), _page(2), _page(3)]
+    client = as_user(user)
+    assert client.get("/api/search/youtube?q=cats").status_code == 200
+    assert quota.actions_today(db, user.id, ACTION) == 1
+    assert client.get("/api/search/youtube?q=dogs").status_code == 200
+    refused = client.get("/api/search/youtube?q=birds")
+    assert refused.status_code == 429
+    assert refused.json()["detail"]["reason"] == "search_daily_limit"
+    assert _FakeYT.api_calls == 2
+
+
+def test_the_api_source_takes_its_slot_under_the_lock(db, make_user, as_user, pages, monkeypatch):
+    user = make_user()
+    _api(db)
+    claimed: list = []
+    real = quota.claim_action
+
+    def claim(uid, action, limit):
+        claimed.append(action)
+        return real(uid, action, limit)
+
+    monkeypatch.setattr(search_routes.quota, "claim_action", claim)
+    _FakeYT.api_pages = [_page(1)]
+    assert as_user(user).get("/api/search/youtube?q=cats").status_code == 200
+    assert claimed == [ACTION]
+
+
+def test_the_api_source_is_refused_before_the_call_when_the_budget_cannot_pay(
+    db, make_user, as_user, pages
+):
+    user = make_user()
+    _api(db)
+    sysconfig.set_value(db, "quota_daily_budget", 50)
+    _FakeYT.api_pages = [_page(1)]
+    resp = as_user(user).get("/api/search/youtube?q=cats")
+    assert resp.status_code == 429
+    assert resp.json()["detail"]["reason"] == "quota_spent"
+    assert _FakeYT.api_calls == 0
+    assert quota.actions_today(db, user.id, ACTION) == 0
+
+
+def test_an_api_search_that_search_list_refused_keeps_its_slot(db, make_user, as_user, pages):
+    # YouTube charges a refused call too (the client records the units first), so a search that
+    # reached search.list has spent the shared budget: refunding it let a bogus cursor spend it all.
+    user = make_user()
+    _api(db)
+    sysconfig.set_value(db, "search_daily_limit_per_user", 1)
+    _FakeYT.api_pages = [
+        YouTubeError("search -> 400", reason="invalidPageToken", path="search"),
+        _page(1),
+    ]
+    client = as_user(user)
+    assert client.get("/api/search/youtube?q=cats&cursor=bogus").status_code == 422
+    assert quota.actions_today(db, user.id, ACTION) == 1
+    refused = client.get("/api/search/youtube?q=cats")
+    assert refused.status_code == 429
+    assert refused.json()["detail"]["reason"] == "search_daily_limit"
+    assert _FakeYT.api_calls == 1
+
+
+def test_an_api_search_the_budget_refused_at_the_call_gives_its_slot_back(
+    db, make_user, as_user, pages, monkeypatch
+):
+    user = make_user()
+    _api(db)
+    answers = iter([True, False])  # the pre-check passes; the budget is gone by the call
+    monkeypatch.setattr(search_routes.quota, "can_spend", lambda _db, _u: next(answers))
+    _FakeYT.api_pages = [_page(1)]
+    resp = as_user(user).get("/api/search/youtube?q=cats")
+    assert resp.status_code == 429
+    assert resp.json()["detail"]["reason"] == "quota_spent"
+    assert _FakeYT.api_calls == 0
+    assert quota.actions_today(db, user.id, ACTION) == 0
+
+
+def test_the_budget_check_and_the_call_it_pays_for_hold_one_lock(
+    db, make_user, as_user, pages, monkeypatch
+):
+    # Checked apart, searches sent together at the last 100 units all passed, and all spent: the
+    # check and the call sit inside ONE hold of the lock, not two.
+    user = make_user()
+    _api(db)
+    events: list[str] = []
+    real_check = quota.can_spend
+    real_call = _FakeYT.search_videos
+
+    class _Recorded:
+        def __init__(self):
+            self._lock = threading.Lock()
+
+        def acquire(self, timeout=-1):
+            events.append("acquire")
+            return self._lock.acquire(timeout=timeout)
+
+        def release(self):
+            events.append("release")
+            self._lock.release()
+
+    def can_spend(db_, units):
+        events.append("check")
+        return real_check(db_, units)
+
+    def search_videos(self, q, page_token=None, order="relevance"):
+        events.append("call")
+        return real_call(self, q, page_token=page_token, order=order)
+
+    monkeypatch.setattr(search_routes, "_api_search", _Recorded())
+    monkeypatch.setattr(search_routes.quota, "can_spend", can_spend)
+    monkeypatch.setattr(_FakeYT, "search_videos", search_videos)
+    _FakeYT.api_pages = [_page(1)]
+    assert as_user(user).get("/api/search/youtube?q=cats").status_code == 200
+    assert events == ["check", "acquire", "check", "call", "release"]
+
+
+def test_a_search_kept_waiting_for_the_lock_is_refused_as_busy_and_keeps_nothing(
+    db, make_user, as_user, pages, monkeypatch
+):
+    # Each waiter holds a request worker: a slow YouTube must not park them all behind one another.
+    user = make_user()
+    _api(db)
+    monkeypatch.setattr(search_routes, "API_SEARCH_WAIT_SECONDS", 0.05)
+    _FakeYT.api_pages = [_page(1)]
+    # Held by another search for a while: a wait without its bound would get it, and be answered.
+    assert search_routes._api_search.acquire(timeout=1)
+    later = threading.Timer(2.0, search_routes._api_search.release)
+    later.start()
+    try:
+        resp = as_user(user).get("/api/search/youtube?q=cats")
+    finally:
+        later.join()
+    assert resp.status_code == 429
+    assert resp.json()["detail"]["reason"] == "search_busy"
+    assert _FakeYT.api_calls == 0
+    assert quota.actions_today(db, user.id, ACTION) == 0
+
+
+@pytest.mark.parametrize(
+    "failure",
+    [
+        YouTubeError("YouTube API unreachable (ConnectError)", reason="transport"),
+        YouTubeError("User has no stored OAuth token"),
+    ],
+    ids=["no-connection", "no-token"],
+)
+def test_an_api_search_that_never_reached_youtube_gives_its_slot_back(
+    db, make_user, as_user, pages, failure
+):
+    user = make_user()
+    _api(db)
+    sysconfig.set_value(db, "search_daily_limit_per_user", 1)
+    _FakeYT.api_pages = [failure, _page(1)]
+    client = as_user(user)
+    assert client.get("/api/search/youtube?q=cats").status_code == 422
+    assert quota.actions_today(db, user.id, ACTION) == 0
+    assert client.get("/api/search/youtube?q=cats").status_code == 200
+
+
+def test_an_api_search_whose_answer_never_came_keeps_its_slot(db, make_user, as_user, pages):
+    # The request went out (a read timeout): YouTube may have counted it.
+    user = make_user()
+    _api(db)
+    sysconfig.set_value(db, "search_daily_limit_per_user", 1)
+    _FakeYT.api_pages = [
+        YouTubeError("YouTube API unreachable (ReadTimeout)", reason="transport", sent=True),
+        _page(1),
+    ]
+    client = as_user(user)
+    assert client.get("/api/search/youtube?q=cats").status_code == 422
+    assert quota.actions_today(db, user.id, ACTION) == 1
+    assert client.get("/api/search/youtube?q=cats").status_code == 429
+
+
+def test_failures_past_the_hours_refunds_keep_their_slot(db, make_user, as_user, pages):
+    # A cursor is a free string: a request built to fail would otherwise read YouTube without end.
+    user = make_user()
+    sysconfig.set_value(db, "search_daily_limit_per_user", 50)
+    n = search_routes.REFUNDS_PER_HOUR
+    pages["pages"] = [ScrapeError("bad cursor") for _ in range(n + 1)]
+    client = as_user(user)
+    for _ in range(n + 1):
+        assert client.get("/api/search/youtube?q=cats&cursor=bogus").status_code == 422
+    assert quota.actions_today(db, user.id, ACTION) == 1

backend/tests/test_search_scrape.py

@@ -9,7 +9,7 @@ import json
 import httpx
 import pytest
 
-from app.youtube import search_scrape
+from app.youtube import innertube, search_scrape
 
 RESULTS_HTML = (
     '<script>{"INNERTUBE_API_KEY":"live-key","INNERTUBE_CLIENT_VERSION":"2.20261001"}</script>'
@@ -79,11 +79,11 @@ def net(monkeypatch):
 
     real = httpx.Client
     monkeypatch.setattr(
-        search_scrape.httpx,
+        innertube.httpx,
         "Client",
         lambda **kw: real(transport=httpx.MockTransport(transport), **kw),
     )
-    monkeypatch.setattr(search_scrape, "_config", None)
+    monkeypatch.setattr(innertube, "_config", None)
     state["calls"] = calls
     return state
 
@@ -171,7 +171,7 @@ def test_a_failed_key_fetch_falls_back_for_now_and_tries_again_next_time(net, re
     with no error anywhere."""
     net["handler"] = _serve(results_html=results_html)
     search_scrape.search("a")
-    assert net["calls"][-1].url.params["key"] == search_scrape._FALLBACK_KEY
+    assert net["calls"][-1].url.params["key"] == innertube._FALLBACK_KEY
 
     net["handler"] = _serve()
     search_scrape.search("b")

backend/tests/test_search_scrape_ingest.py

@@ -13,10 +13,13 @@ a placeholder never becomes the video's permanent date.
 
 from datetime import datetime, timezone
 
+import pytest
+
 from app import sysconfig
 from app.models import User
 from app.catalog.models import Video
 from app.catalog.search import ingest_candidates
+from app.youtube.client import YouTubeError
 
 
 class _FakeYT:
@@ -97,6 +100,25 @@ def test_scrape_ingest_enrich_miss_keeps_nonnull_placeholder(db):
     assert c.enriched_at is None
 
 
+class _FailingYT(_FakeYT):
+    def get_videos(self, ids):
+        raise YouTubeError("videos -> 403", reason="quotaExceeded", path="videos")
+
+
+def test_a_videos_list_failure_raises_and_keeps_the_stubs(db):
+    # Swallowed, it answered as a page with nothing on it — "no results" for a search whose lookup
+    # never ran. The stubs stay: the FK targets, backfilled later.
+    user = User(email="searcher3@example.com")
+    db.add(user)
+    db.commit()
+
+    with pytest.raises(YouTubeError):
+        ingest_candidates(db, _FailingYT({}), user, [_candidate("vid_ddddddd", "UC_chan_d")])
+
+    db.rollback()  # what the caller does: the stubs were committed before the lookup
+    assert db.get(Video, "vid_ddddddd") is not None
+
+
 def test_search_youtube_route_returns_422_when_ingest_fails(db, monkeypatch, override_dep):
     # Route-level guard (B-core-1): an ingest failure with nothing yet collected must surface as a 422
     # + message on this JSON endpoint (the FE hides 5xx behind a generic "connection lost" + auto-retry,

backend/tests/test_unit_fences.py

@@ -44,6 +44,9 @@ UNIT_MODELS = {
         "PlaylistItem",
         "SavedView",
         "ComfortPin",
+        "TakeoutItem",
+        "TakeoutChannelDismissal",
+        "VideoUnmark",
         "Tag",
         "ChannelTag",
     },
@@ -360,6 +363,13 @@ UNIT_COMMITS = {
         "sync/playlists.py:pull_user_playlists",
         "sync/playlists.py:merge_playlist",
         "sync/playlists.py:repull_playlist",
+        # The Takeout import is a recorded run with its own session: the watched part lands before
+        # the Watch later part asks YouTube (which commits the videos it fetched).
+        "reader/takeout.py:run_import",
+        # Run inside the Shorts job, on its session: each user's waiting rows land on their own.
+        "reader/takeout.py:apply_waiting",
+        # Inside either of those: the lengths YouTube gave land before the channels are asked for.
+        "reader/takeout.py:_channel_list",
     },
     # The catalog's ingest jobs commit per page or batch, a YouTube call between each: a page
     # fetched and stored is not lost when the next call fails, and an enrich that fails rolls back
@@ -408,7 +418,38 @@ UNIT_COMMITS = {
         "plex/watch_sync.py:run_plex_watch_reconcile",
     },
 }
+# Where core code may commit, each with why. Everything else in core flushes and leaves the commit
+# to its caller, as a unit's code does; a route's own helpers are held by test_route_conventions.
+CORE_COMMITS = {
+    # Quota spend and a background run's row are durable whatever the caller does next: each opens
+    # a session of its own.
+    "quota.py",
+    "jobruns.py",
+    # The scheduler's jobs own their session.
+    "scheduler.py",
+    # The instance's one settings row: created on first read, and the setup steps that finish it.
+    "state.py",
+    # A token's last-used stamp lands ahead of the route: a read-only route never commits, and a
+    # write route's rollback must not take it along. The expiry warning is a scheduled job.
+    "apitokens.py:_token_caller",
+    "apitokens.py:warn_expiring",
+    # An account's erasure is one operation shared by the self-service and the admin delete; the
+    # Google revoke and the confirmation run after it, in the background.
+    "auth.py:purge_user",
+    # The undelivered-email notice opens its own session (its callers are often background tasks).
+    "email.py:_alert_admins",
+    # One notice, committed unless its caller batches (`commit=False`).
+    "notifications.py:create_notification",
+    # A refreshed (or dead) YouTube grant must persist even when the call that needed it fails and
+    # its caller rolls back.
+    "youtube/client.py:_access_token",
+    "youtube/client.py:_drop_dead_grant",
+}
 SQL_WRITES = {"insert", "update", "delete", "pg_insert"}
+ALL_MODEL_NAMES = set(UNIT_MODEL_NAMES) | CORE_MODELS
+# The column methods that compare an owner column as a filter does (`Model.user_id.in_(ids)`).
+# `.is_(None)` is not one: it asks for the rows nobody owns (the system's own tags).
+OWNER_METHODS = {"in_", "not_in", "notin_", "__eq__", "__ne__"}
 
 
 def _package_unit(rel: str) -> str | None:
@@ -440,6 +481,64 @@ def _imported_models(tree: ast.AST) -> dict[str, str]:
     return out
 
 
+def _model_aliases(tree: ast.AST) -> dict[str, str]:
+    """Every other name a module binds to a model, by that name: an import under another name, and
+    `x = aliased(Model)` — an alias is the same table, and its owner column the same filter."""
+    out = {}
+    for node in ast.walk(tree):
+        if isinstance(node, ast.ImportFrom) and "models" in (node.module or ""):
+            for alias in node.names:
+                if alias.name in ALL_MODEL_NAMES:
+                    out[alias.asname or alias.name] = alias.name
+    for node in ast.walk(tree):
+        if (
+            isinstance(node, ast.Assign)
+            and isinstance(node.value, ast.Call)
+            and _called_name(node.value) == "aliased"
+            and node.value.args
+            and isinstance(node.value.args[0], ast.Name)
+        ):
+            source = node.value.args[0].id
+            model = out.get(source, source if source in ALL_MODEL_NAMES else None)
+            for target in node.targets:
+                if model and isinstance(target, ast.Name):
+                    out[target.id] = model
+    return out
+
+
+def _called_name(call: ast.Call) -> str | None:
+    func = call.func
+    return func.id if isinstance(func, ast.Name) else getattr(func, "attr", None)
+
+
+def _sql_write_names(tree: ast.AST) -> tuple[set[str], set[str]]:
+    """The names a module calls SQLAlchemy's statement writes by — `insert`/`update`/`delete` as
+    imported, under any alias — and the names it binds SQLAlchemy's modules to (`sa.update(...)`)."""
+    functions, modules = set(SQL_WRITES), set()
+    for node in ast.walk(tree):
+        if isinstance(node, ast.ImportFrom) and (node.module or "").split(".")[0] == "sqlalchemy":
+            for alias in node.names:
+                if alias.name in {"insert", "update", "delete"}:
+                    functions.add(alias.asname or alias.name)
+                elif alias.name in {"dialects", "postgresql", "sql"}:
+                    modules.add(alias.asname or alias.name)
+        elif isinstance(node, ast.Import):
+            for alias in node.names:
+                if alias.name.split(".")[0] == "sqlalchemy":
+                    modules.add(alias.asname or alias.name.split(".")[0])
+    return functions, modules
+
+
+def _written_table(call: ast.Call) -> str | None:
+    """The name of what a statement write's first argument names: `Model`, or `Model.__table__`."""
+    if not call.args:
+        return None
+    arg = call.args[0]
+    if isinstance(arg, ast.Attribute) and arg.attr == "__table__":
+        arg = arg.value
+    return arg.id if isinstance(arg, ast.Name) else None
+
+
 def _enclosing_functions(tree: ast.AST) -> dict[ast.AST, str]:
     out = {}
     for fn in ast.walk(tree):
@@ -461,49 +560,67 @@ def package_violations(rel: str, source: str) -> list[str]:
             continue
         if theirs not in READS.get(mine or "", set()):
             found.append(f"0: {mine or 'core'} imports {theirs}'s {name}")
+    write_functions, sql_modules = _sql_write_names(tree)
     for node in ast.walk(tree):
-        if not isinstance(node, ast.Call) or not isinstance(node.func, ast.Name):
+        if not isinstance(node, ast.Call):
             continue
+        func = node.func
         written = None
-        if node.func.id in models:
-            written = models[node.func.id]
-        elif node.func.id in SQL_WRITES and node.args and isinstance(node.args[0], ast.Name):
-            written = models.get(node.args[0].id)
+        if isinstance(func, ast.Name) and func.id in models:
+            written = models[func.id]
+        elif (isinstance(func, ast.Name) and func.id in write_functions) or (
+            isinstance(func, ast.Attribute)
+            and func.attr in {"insert", "update", "delete"}
+            and isinstance(_root(func), ast.Name)
+            and _root(func).id in sql_modules
+        ):
+            written = models.get(_written_table(node) or "")
         if written is None:
             continue
         theirs = UNIT_MODEL_NAMES[written]
         if theirs != mine:
             found.append(f"{node.lineno}: {mine or 'core'} writes {theirs}'s {written}")
     found += _owner_filters(tree)
-    if mine is not None:
-        functions = _enclosing_functions(tree)
-        allowed = UNIT_COMMITS.get(mine, set())
-        for node in ast.walk(tree):
-            if (
-                isinstance(node, ast.Call)
-                and isinstance(node.func, ast.Attribute)
-                and node.func.attr == "commit"
-                and rel not in allowed
-                and f"{rel}:{functions.get(node)}" not in allowed
-            ):
-                found.append(f"{node.lineno}: commits in {functions.get(node)}")
+    functions = _enclosing_functions(tree)
+    allowed = UNIT_COMMITS.get(mine, set()) if mine is not None else CORE_COMMITS
+    for node in ast.walk(tree):
+        if (
+            isinstance(node, ast.Call)
+            and isinstance(node.func, ast.Attribute)
+            and node.func.attr == "commit"
+            and rel not in allowed
+            and f"{rel}:{functions.get(node)}" not in allowed
+        ):
+            found.append(f"{node.lineno}: commits in {functions.get(node)}")
     return sorted(found, key=lambda v: int(v.split(":")[0]))
 
 
 def _owner_filters(tree: ast.AST) -> list[str]:
     """A comparison on a table's owner column written inline (`Model.user_id == …`), not through
     `userscope.owned_by`."""
+    names = {name: name for name in ALL_MODEL_NAMES} | _model_aliases(tree)
+
+    def owner_column(node: ast.expr) -> bool:
+        return (
+            isinstance(node, ast.Attribute)
+            and isinstance(node.value, ast.Name)
+            and node.value.id in names
+            and node.attr in OWNER_COLUMNS
+        )
+
     found = []
     for node in ast.walk(tree):
         if isinstance(node, ast.Compare):
             for side in (node.left, *node.comparators):
-                if (
-                    isinstance(side, ast.Attribute)
-                    and isinstance(side.value, ast.Name)
-                    and side.value.id in set(UNIT_MODEL_NAMES) | CORE_MODELS
-                    and side.attr in OWNER_COLUMNS
-                ):
+                if owner_column(side):
                     found.append(f"{node.lineno}: filters {side.value.id}.{side.attr} inline")
+        elif isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute):
+            if node.func.attr in OWNER_METHODS and owner_column(node.func.value):
+                found.append(f"{node.lineno}: filters an owner column with {node.func.attr}")
+            elif node.func.attr == "filter_by" and any(
+                k.arg in OWNER_COLUMNS for k in node.keywords
+            ):
+                found.append(f"{node.lineno}: filters an owner column with filter_by")
     return found
 
 
@@ -524,11 +641,32 @@ def test_every_unit_package_exists():
     for prefixes in UNIT_PACKAGES.values():
         for p in prefixes:
             assert (APP / p).exists(), p
-    for commits in UNIT_COMMITS.values():
+    for commits in (*UNIT_COMMITS.values(), CORE_COMMITS):
         for entry in commits:
             assert (APP / entry.split(":")[0]).exists(), entry
 
 
+def test_every_listed_commit_is_still_there():
+    """An entry names a commit that exists: one left behind by a rename, or by a function that no
+    longer commits, would go on allowing a commit to whatever takes its name next."""
+    stale = []
+    for commits in (*UNIT_COMMITS.values(), CORE_COMMITS):
+        for entry in commits:
+            rel, _, name = entry.partition(":")
+            tree = ast.parse((APP / rel).read_text(encoding="utf-8"))
+            functions = _enclosing_functions(tree)
+            committing = {
+                functions.get(node)
+                for node in ast.walk(tree)
+                if isinstance(node, ast.Call)
+                and isinstance(node.func, ast.Attribute)
+                and node.func.attr == "commit"
+            }
+            if (name not in committing) if name else not committing:
+                stale.append(entry)
+    assert stale == []
+
+
 PACKAGE_CASES = {
     # (module path, source): violations
     ("downloads/x.py", "from app.models import Video"): 0,  # capture reads catalog
@@ -544,6 +682,22 @@ PACKAGE_CASES = {
     ("downloads/x.py", "q = DownloadJob.user_id == 1"): 1,
     ("downloads/x.py", "q = owned_by(DownloadJob.user_id, 1)"): 0,
     ("quota.py", "q = QuotaEvent.user_id == 1"): 1,  # core too
+    ("downloads/x.py", "from app.downloads.models import DownloadJob as J\nq = J.user_id == 1"): 1,
+    ("downloads/x.py", "st = aliased(VideoState)\nq = st.user_id == 1"): 1,
+    ("downloads/x.py", "q = DownloadJob.user_id.in_([1])"): 1,
+    ("downloads/x.py", "q = select(DownloadJob).filter_by(user_id=1)"): 1,
+    ("downloads/x.py", "q = Tag.user_id.is_(None)"): 0,  # the rows nobody owns
+    (
+        "downloads/x.py",
+        "from sqlalchemy import update as up\nfrom app.models import Video\nup(Video)",
+    ): 1,
+    (
+        "downloads/x.py",
+        "import sqlalchemy as sa\nfrom app.models import Video\nsa.update(Video)",
+    ): 1,
+    ("downloads/x.py", "from app.models import Video\nupdate(Video.__table__)"): 1,
+    ("notifications.py", "def f(db):\n    db.commit()"): 1,  # core flushes too
+    ("state.py", "def f(db):\n    db.commit()"): 0,
     ("downloads/x.py", "def f(db):\n    db.commit()"): 1,
     ("downloads/gc.py", "def f(db):\n    db.commit()"): 0,
     ("downloads/upload.py", "def ingest_upload(db):\n    db.commit()"): 0,

backend/tests/test_watched_channels.py

@@ -0,0 +1,538 @@
+"""The channels a user's Takeout history shows them watching but not following: which channels make
+the list (five videos of five minutes or more, none they follow, block, put out or own), what the
+import asks YouTube for it and when, and the channel's watched videos made ready to play."""
+
+from datetime import datetime, timedelta, timezone
+
+import pytest
+from sqlalchemy import select, update
+
+from app import quota
+from app.catalog import service as catalog_service
+from app.catalog.models import BlockedChannel, Channel, ExploredChannel, Subscription, Video
+from app.db import SessionLocal
+from app.reader import queries, service, takeout
+from app.reader.models import TakeoutChannelDismissal, TakeoutItem, TakeoutKind
+from app.routes import feed as feed_routes
+from app.sync import explore as sync_explore
+from app.sync import videos as sync_videos
+from app.youtube.client import YouTubeError
+
+T0 = datetime(2026, 5, 1, 12, 0, tzinfo=timezone.utc)
+LONG = 600
+SHORT = 120
+
+
+def _ch(n: int) -> str:
+    return f"UC{n:022d}"
+
+
+def _vid(channel: int, n: int) -> str:
+    return f"c{channel:02d}v{n:07d}"
+
+
+def _watched(db, user, channel: int, lengths: list[int | None], *, at=T0, asked=True):
+    """Stage `len(lengths)` watched videos of a channel, the newest first, each with its length
+    already asked (None: YouTube no longer has it) — or not asked yet."""
+    service.stage_takeout(
+        db,
+        user,
+        TakeoutKind.WATCHED,
+        {_vid(channel, i): (at - timedelta(days=i), _ch(channel)) for i in range(len(lengths))},
+    )
+    db.flush()
+    if asked:
+        ids = [_vid(channel, i) for i in range(len(lengths))]
+        service.record_takeout_lengths(
+            db, user, ids, {v: s for v, s in zip(ids, lengths, strict=True) if s is not None}
+        )
+    db.commit()
+
+
+def _channel(db, n: int, title: str | None = None):
+    db.add(Channel(id=_ch(n), title=title or f"Channel {n}", details_synced_at=T0))
+    db.commit()
+
+
+def _listed(db, user) -> list[tuple[str, int]]:
+    return [(ch.id, n) for ch, n, _ in queries.watched_channels(db, user)]
+
+
+# --- which channels make the list --------------------------------------------------------------
+
+
+def test_a_channel_makes_the_list_with_five_watched_videos_of_five_minutes_or_more(db, make_user):
+    user = make_user("list@example.com")
+    for n in (1, 2, 3):
+        _channel(db, n)
+    _watched(db, user, 1, [LONG] * 5 + [SHORT] * 3)  # five long, three short: on it with 5
+    _watched(db, user, 2, [LONG] * 4 + [SHORT] * 6)  # ten watched, only four long: off it
+    _watched(db, user, 3, [299, 300, 300, 300, 300, 300])  # five at exactly five minutes: on it
+    # A tie: channel 1's counted videos were watched later (channel 3's newest is under five minutes).
+    assert _listed(db, user) == [(_ch(1), 5), (_ch(3), 5)]
+
+
+def test_a_video_youtube_no_longer_has_does_not_count(db, make_user):
+    user = make_user("gone@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 4 + [None] * 3)
+    assert _listed(db, user) == []
+
+
+def test_the_most_watched_lead_and_a_tie_goes_to_the_one_watched_more_lately(db, make_user):
+    user = make_user("order@example.com")
+    for n in (1, 2, 3):
+        _channel(db, n)
+    _watched(db, user, 1, [LONG] * 6, at=T0 - timedelta(days=100))
+    _watched(db, user, 2, [LONG] * 9, at=T0 - timedelta(days=300))
+    _watched(db, user, 3, [LONG] * 6, at=T0)
+    assert _listed(db, user) == [(_ch(2), 9), (_ch(3), 6), (_ch(1), 6)]
+    last = {ch.id: when for ch, _, when in queries.watched_channels(db, user)}
+    assert last[_ch(3)] == T0  # the last watched of the counted ones
+
+
+def test_a_channel_the_user_follows_blocked_put_out_or_owns_is_not_on_the_list(db, make_user):
+    user = make_user("off@example.com", yt_channel_id=_ch(4))
+    for n in (1, 2, 3, 4, 5):
+        _channel(db, n)
+        _watched(db, user, n, [LONG] * 5)
+    db.add(Subscription(user_id=user.id, channel_id=_ch(1)))
+    db.add(BlockedChannel(user_id=user.id, channel_id=_ch(2)))
+    db.commit()
+    service.dismiss_watched_channel(db, user, _ch(3))
+    db.commit()
+    assert _listed(db, user) == [(_ch(5), 5)]
+
+
+def test_one_users_follows_and_dismissals_leave_anothers_list_alone(db, make_user):
+    me, other = make_user("me@example.com"), make_user("other@example.com")
+    _channel(db, 1)
+    for user in (me, other):
+        _watched(db, user, 1, [LONG] * 5)
+    db.add(Subscription(user_id=other.id, channel_id=_ch(1)))
+    db.commit()
+    service.dismiss_watched_channel(db, other, _ch(1))
+    db.commit()
+    assert _listed(db, me) == [(_ch(1), 5)]
+    assert _listed(db, other) == []
+
+
+def test_a_channel_whose_lengths_are_not_all_known_is_pending_until_they_are(db, make_user):
+    user = make_user("pending@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 6, asked=False)
+    _watched(db, user, 2, [LONG] * 4, asked=False)  # four watched: it can never make the list
+    assert queries.watched_channels_pending(db, user) == 1
+    assert _listed(db, user) == []
+    ids = [_vid(1, i) for i in range(6)]
+    service.record_takeout_lengths(db, user, ids, dict.fromkeys(ids, LONG))
+    db.commit()
+    assert queries.watched_channels_pending(db, user) == 0
+    assert _listed(db, user) == [(_ch(1), 6)]
+
+
+def test_a_new_upload_of_the_history_keeps_the_lengths_already_asked(db, make_user):
+    user = make_user("again@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 5)
+    _watched(db, user, 1, [LONG] * 5, at=T0 + timedelta(days=1), asked=False)
+    assert _listed(db, user) == [(_ch(1), 5)]
+    assert queries.takeout_lengths_to_ask(db, user) == []
+
+
+# --- what the import asks YouTube for ----------------------------------------------------------
+
+
+class FakeYouTube:
+    """Stands in for `YouTubeClient`: answers `videos.list` with the lengths it was given and
+    `channels.list` with a title per channel, and records what it was asked."""
+
+    def __init__(self, lengths: dict[str, int] | None = None, *, fail: bool = False):
+        self.lengths = lengths or {}
+        self.fail = fail
+        self.fail_channels = False
+        self.videos_asked: list[list[str]] = []
+        self.channels_asked: list[list[str]] = []
+        self.actions: list[tuple[int | None, str]] = []
+
+    def __call__(self, db, user):
+        return self
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        return False
+
+    def get_videos(self, ids):
+        self.videos_asked.append(list(ids))
+        self.actions.append((quota._actor_id.get(), quota._action.get()))
+        if self.fail:
+            raise YouTubeError("refused", reason="quotaExceeded", path="videos")
+        return [
+            {"id": v, "contentDetails": {"duration": f"PT{self.lengths[v]}S"}}
+            for v in ids
+            if v in self.lengths
+        ]
+
+    def get_channels(self, ids):
+        self.channels_asked.append(list(ids))
+        self.actions.append((quota._actor_id.get(), quota._action.get()))
+        if self.fail_channels:
+            raise YouTubeError("refused", reason="backendError", path="channels")
+        return [{"id": c, "snippet": {"title": f"Named {c[-2:]}"}} for c in ids]
+
+
+@pytest.fixture
+def youtube(monkeypatch, db, make_user):
+    fetcher = make_user("fetcher@example.com")
+    fake = FakeYouTube()
+    monkeypatch.setattr(takeout, "YouTubeClient", fake)
+    monkeypatch.setattr(takeout, "get_service_user", lambda session: fetcher)
+    monkeypatch.setattr(takeout.quota, "backfill_reserve_reached", lambda: False)
+    return fake
+
+
+def test_the_import_asks_the_lengths_the_list_needs_and_names_its_channels(db, make_user, youtube):
+    user = make_user("import@example.com")
+    _watched(db, user, 1, [None] * 7, asked=False)  # the most watched channel
+    _watched(db, user, 2, [None] * 5, asked=False)
+    _watched(db, user, 3, [None] * 4, asked=False)  # too few to ever make it: not asked about
+    youtube.lengths = {_vid(1, i): LONG for i in range(6)} | {_vid(2, i): SHORT for i in range(5)}
+
+    result = takeout.run_import(user.id)
+
+    asked = [_vid(1, i) for i in range(7)] + [_vid(2, i) for i in range(5)]
+    assert youtube.videos_asked == [asked]  # the top channel first
+    assert result["lengths_asked"] == 12
+    # Only channel 1 made it (six long; its seventh is gone, channel 2's are short): it is named.
+    assert youtube.channels_asked == [[_ch(1)]]
+    assert result["channels_described"] == 1
+    assert {
+        (user.id, quota.QuotaAction.VIDEOS_LOOKUP),
+        (user.id, quota.QuotaAction.CHANNELS_DISCOVER),
+    } == set(youtube.actions)
+    db.expire_all()
+    assert [(ch.title, n) for ch, n, _ in queries.watched_channels(db, user)] == [("Named 01", 6)]
+    assert queries.watched_channels_pending(db, user) == 0
+    # Asked once: a second run asks nothing again, the gone video included.
+    youtube.videos_asked.clear()
+    youtube.channels_asked.clear()
+    assert "lengths_asked" not in takeout.run_import(user.id)
+    assert youtube.videos_asked == [] and youtube.channels_asked == []
+
+
+def test_the_lookups_stop_at_the_users_daily_share_and_the_rest_wait(
+    db, make_user, youtube, monkeypatch
+):
+    user = make_user("share@example.com")
+    _watched(db, user, 1, [None] * 8, asked=False)
+    _watched(db, user, 2, [None] * 6, asked=False)
+    monkeypatch.setattr(takeout, "_lengths", takeout.RateLimiter(10, 3600))
+    takeout.run_import(user.id)
+    assert youtube.videos_asked == [[_vid(1, i) for i in range(8)] + [_vid(2, 0), _vid(2, 1)]]
+    db.expire_all()
+    assert queries.watched_channels_pending(db, user) == 1  # channel 2 still has some to ask
+    youtube.videos_asked.clear()
+    takeout.run_import(user.id)
+    assert youtube.videos_asked == []  # the share is spent for today
+
+
+@pytest.mark.parametrize("why", ["no_fetcher", "reserve"])
+def test_nothing_is_asked_without_a_connected_account_or_past_the_reserve(
+    db, make_user, youtube, monkeypatch, why
+):
+    user = make_user(f"{why}@example.com")
+    _watched(db, user, 1, [None] * 5, asked=False)
+    if why == "no_fetcher":
+        monkeypatch.setattr(takeout, "get_service_user", lambda session: None)
+    else:
+        monkeypatch.setattr(takeout.quota, "backfill_reserve_reached", lambda: True)
+    assert "lengths_asked" not in takeout.run_import(user.id)
+    assert youtube.videos_asked == []
+    assert queries.watched_channels_pending(db, user) == 1
+
+
+def test_a_youtube_failure_asking_lengths_fails_neither_the_import_nor_its_marks(
+    db, make_user, youtube
+):
+    user = make_user("fail@example.com")
+    _watched(db, user, 1, [None] * 5, asked=False)
+    youtube.fail = True
+    result = takeout.run_import(user.id)
+    assert "lengths_asked" not in result
+    assert queries.takeout_lengths_to_ask(db, user) == [_vid(1, i) for i in range(5)]
+    db.expire_all()
+    rows = db.execute(select(TakeoutItem).where(TakeoutItem.user_id == user.id)).scalars()
+    assert all(r.looked_up_at is None for r in rows)  # left to ask again
+
+
+def test_the_shorts_job_asks_what_an_import_could_not(db, make_user, youtube):
+    user = make_user("later@example.com")
+    _watched(db, user, 1, [None] * 5, asked=False)
+    youtube.lengths = {_vid(1, i): LONG for i in range(5)}
+    assert takeout.apply_waiting(db)["takeout_lengths_asked"] == 5
+    db.expire_all()
+    assert [n for _, n, _ in queries.watched_channels(db, user)] == [5]
+
+
+@pytest.mark.parametrize("why", ["follows", "blocked", "dismissed", "own"])
+def test_a_channel_off_the_list_is_not_asked_about_nor_looked_at_again(
+    db, make_user, youtube, monkeypatch, why
+):
+    user = make_user(f"off-{why}@example.com", yt_channel_id=_ch(1) if why == "own" else None)
+    _channel(db, 1)
+    _watched(db, user, 1, [None] * 5, asked=False)
+    if why == "follows":
+        db.add(Subscription(user_id=user.id, channel_id=_ch(1)))
+    elif why == "blocked":
+        db.add(BlockedChannel(user_id=user.id, channel_id=_ch(1)))
+    elif why == "dismissed":
+        service.dismiss_watched_channel(db, user, _ch(1))
+    db.commit()
+    runs = []
+    monkeypatch.setattr(takeout, "_channel_list", lambda session, u: runs.append(u.id) or {})
+    assert takeout.apply_waiting(db) == {}
+    # Its rows are never asked about, so they would stay "to ask" for good: the Shorts job must not
+    # take the user up again for them on every run.
+    assert runs == []
+    assert youtube.videos_asked == []
+
+
+def test_naming_the_channels_is_tried_again_and_meanwhile_the_list_says_it_is_not_complete(
+    db, make_user, youtube
+):
+    user = make_user("unnamed@example.com")
+    _watched(db, user, 1, [None] * 5, asked=False)
+    youtube.lengths = {_vid(1, i): LONG for i in range(5)}
+    youtube.fail_channels = True
+    takeout.run_import(user.id)
+    db.expire_all()
+    # Every length is known, the channel could not be named: not on the list, and not complete.
+    assert queries.takeout_lengths_to_ask(db, user) == []
+    assert _listed(db, user) == []
+    assert queries.watched_channels_pending(db, user) == 1
+    youtube.fail_channels = False
+    takeout.apply_waiting(db)
+    db.expire_all()
+    assert [(ch.title, n) for ch, n, _ in queries.watched_channels(db, user)] == [("Named 01", 5)]
+    assert queries.watched_channels_pending(db, user) == 0
+
+
+def test_a_listed_channel_still_to_be_described_is_not_counted_as_one_to_come(db, make_user):
+    user = make_user("stub@example.com")
+    # A stub the catalog made when one of its videos came in: a title, no details yet.
+    db.add(Channel(id=_ch(1), title="From a video"))
+    db.commit()
+    _watched(db, user, 1, [LONG] * 5)
+    assert queries.watched_channels_to_describe(db, user) == [_ch(1)]
+    assert _listed(db, user) == [(_ch(1), 5)]
+    # It is on the list already: "the list may grow" would promise one more that cannot come.
+    assert queries.watched_channels_pending(db, user) == 0
+
+
+def test_the_lengths_leave_the_watch_later_lookups_their_own_share(
+    db, make_user, youtube, monkeypatch
+):
+    user = make_user("shares@example.com")
+    monkeypatch.setattr(takeout, "_lookups", takeout.RateLimiter(10, 3600))
+    monkeypatch.setattr(takeout, "_lengths", takeout.RateLimiter(10, 3600))
+    _watched(db, user, 1, [None] * 15, asked=False)
+    takeout.run_import(user.id)  # the history: its lengths spend all they may today
+    youtube.videos_asked.clear()
+    service.stage_takeout(db, user, TakeoutKind.WATCH_LATER, {"later000001": (T0, None)})
+    db.commit()
+    takeout.run_import(user.id)  # then the export's Watch later file
+    assert ["later000001"] in youtube.videos_asked
+
+
+# --- the channel's watched videos, ready to play ------------------------------------------------
+
+
+def _fake_ensure(calls):
+    def ensure(session, yt, ids, *, explore=False):
+        calls.append((list(ids), explore))
+        for vid in ids:
+            if vid.endswith("4"):
+                continue  # gone from YouTube since its length was asked
+            session.add(Video(id=vid, channel_id=_ch(1), published_at=T0, via_explore=explore))
+        session.commit()
+
+    return ensure
+
+
+def test_a_channels_watched_videos_come_ready_to_play_and_explored(
+    db, make_user, as_user, monkeypatch
+):
+    user = make_user("play@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG, SHORT, LONG, None, LONG, LONG, LONG])
+    db.add(Video(id=_vid(1, 0), channel_id=_ch(1), published_at=T0))  # already in the catalog
+    db.commit()
+    calls = []
+    monkeypatch.setattr(feed_routes, "ensure_videos", _fake_ensure(calls))
+    monkeypatch.setattr(feed_routes.quota, "backfill_reserve_reached", lambda: False)
+
+    resp = as_user(user).post(f"/api/takeout/channels/{_ch(1)}/videos")
+
+    assert resp.status_code == 200, resp.text
+    body = resp.json()
+    # Only the counted ones, newest first; the short one and the gone ones are left out.
+    assert calls == [([_vid(1, 2), _vid(1, 4), _vid(1, 5), _vid(1, 6)], True)]
+    assert [it["id"] for it in body["items"]] == [_vid(1, 0), _vid(1, 2), _vid(1, 5), _vid(1, 6)]
+    assert body["items"][0]["takeout_watched_at"].startswith("2026-05-01")
+    assert body["gone"] == 2  # one YouTube had lost before, one since
+    assert body["ok"] is True
+    db.expire_all()
+    assert db.execute(
+        select(ExploredChannel).where(
+            ExploredChannel.user_id == user.id, ExploredChannel.channel_id == _ch(1)
+        )
+    ).scalar_one()
+    assert (
+        db.get(Channel, _ch(1)).from_explore is True
+    )  # nobody follows it: the cleanup's to reclaim
+
+
+def test_a_followed_channels_watched_videos_come_in_as_the_catalogs_own(
+    db, make_user, as_user, monkeypatch
+):
+    user, follower = make_user("viewer@example.com"), make_user("follower@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 5)
+    db.add(Subscription(user_id=follower.id, channel_id=_ch(1)))
+    db.commit()
+    calls = []
+    monkeypatch.setattr(feed_routes, "ensure_videos", _fake_ensure(calls))
+    monkeypatch.setattr(feed_routes.quota, "backfill_reserve_reached", lambda: False)
+    assert as_user(user).post(f"/api/takeout/channels/{_ch(1)}/videos").status_code == 200
+    # Someone follows it, so it is no exploration's to reclaim: a video flagged as one would never
+    # be cleared again (subscribing clears only an explored channel's) and never marked watched.
+    assert calls == [([_vid(1, i) for i in range(5)], False)]
+    db.expire_all()
+    assert db.get(Channel, _ch(1)).from_explore is False
+
+
+def test_a_follow_landing_while_the_videos_come_in_leaves_none_of_them_flagged(
+    db, make_user, as_user, monkeypatch
+):
+    user, follower = make_user("racer@example.com"), make_user("late@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 5)
+    fake = _fake_ensure(calls := [])
+
+    def ensure(session, yt, ids, *, explore=False):
+        # Someone subscribes, in a request of their own, while YouTube answers this one.
+        with SessionLocal() as other:
+            ch = other.get(Channel, _ch(1))
+            catalog_service.record_subscription(other, follower, ch, "sub-late")
+            other.commit()
+        fake(session, yt, ids, explore=explore)
+
+    monkeypatch.setattr(feed_routes, "ensure_videos", ensure)
+    monkeypatch.setattr(feed_routes.quota, "backfill_reserve_reached", lambda: False)
+    assert as_user(user).post(f"/api/takeout/channels/{_ch(1)}/videos").status_code == 200
+    assert calls[0][1] is True  # asked as the exploration's
+    db.expire_all()
+    flagged = db.scalars(select(Video.id).where(Video.via_explore.is_(True))).all()
+    assert (db.get(Channel, _ch(1)).from_explore, flagged) == (False, [])
+
+
+def test_the_cleanup_keeps_a_channel_a_watch_history_lists_and_reclaims_its_videos(db, make_user):
+    user = make_user("kept@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 5)
+    catalog_service.explore(db, user, db.get(Channel, _ch(1)))
+    db.add(Video(id=_vid(1, 0), channel_id=_ch(1), published_at=T0, via_explore=True))
+    db.add(Channel(id=_ch(2), from_explore=True))  # explored, in nobody's history
+    db.commit()
+    db.execute(update(ExploredChannel).values(created_at=T0 - timedelta(days=365)))
+    db.commit()
+    out = sync_explore.purge_unkept_explored(db)
+    # The unplayed video goes (opening the list's videos brings it back); the channel stays listed.
+    assert (out["videos_deleted"], out["channels_deleted"]) == (1, 1)
+    db.expire_all()
+    assert db.get(Channel, _ch(2)) is None
+    assert _listed(db, user) == [(_ch(1), 5)]
+
+
+def test_a_blocked_or_unknown_channels_videos_are_refused(db, make_user, as_user):
+    user = make_user("refused@example.com")
+    _channel(db, 1)
+    db.add(BlockedChannel(user_id=user.id, channel_id=_ch(1)))
+    db.commit()
+    client = as_user(user)
+    assert client.post(f"/api/takeout/channels/{_ch(1)}/videos").status_code == 403
+    assert client.post(f"/api/takeout/channels/{_ch(9)}/videos").status_code == 404
+
+
+def test_ensure_videos_adds_an_explorations_videos_as_its_own(db):
+    db.add(Channel(id=_ch(1), title="C"))
+    db.commit()
+
+    class YT:
+        def get_videos(self, ids):
+            return [{"id": v, "snippet": {"channelId": _ch(1)}} for v in ids]
+
+    sync_videos.ensure_videos(db, YT(), ["exploredvid"], explore=True)
+    sync_videos.ensure_videos(db, YT(), ["organicvid0"])
+    db.expire_all()
+    assert db.get(Video, "exploredvid").via_explore is True
+    assert db.get(Video, "organicvid0").via_explore is False
+
+
+# --- the routes -----------------------------------------------------------------------------------
+
+
+def test_the_list_route_says_what_it_holds_and_whether_a_history_was_uploaded(
+    db, make_user, as_user
+):
+    user = make_user("route@example.com")
+    client = as_user(user)
+    assert client.get("/api/takeout/channels").json() == {
+        "imported": False,
+        "pending": 0,
+        "channels": [],
+    }
+    _channel(db, 1, "Some channel")
+    _watched(db, user, 1, [LONG] * 5)
+    _watched(db, user, 2, [None] * 5, asked=False)
+    body = client.get("/api/takeout/channels").json()
+    assert body["imported"] is True and body["pending"] == 1
+    assert [(c["id"], c["title"], c["watched"]) for c in body["channels"]] == [
+        (_ch(1), "Some channel", 5)
+    ]
+    assert body["channels"][0]["last_watched_at"].startswith("2026-05-01")
+
+
+def test_a_channel_put_out_of_the_list_comes_back_when_restored(db, make_user, as_user):
+    user = make_user("dismiss@example.com")
+    _channel(db, 1)
+    _watched(db, user, 1, [LONG] * 5)
+    client = as_user(user)
+    url = f"/api/takeout/channels/{_ch(1)}/dismissal"
+    assert client.put(url).json() == {"ok": True, "channel_id": _ch(1)}
+    assert client.put(url).status_code == 200  # again: no error
+    assert client.get("/api/takeout/channels").json()["channels"] == []
+    assert client.delete(url).json() == {"ok": True, "channel_id": _ch(1)}
+    assert [c["id"] for c in client.get("/api/takeout/channels").json()["channels"]] == [_ch(1)]
+    db.expire_all()
+    assert db.execute(select(TakeoutChannelDismissal)).scalars().all() == []
+
+
+def test_only_a_channel_of_the_users_own_history_can_be_put_out_of_the_list(db, make_user, as_user):
+    user, other = make_user("mine@example.com"), make_user("theirs@example.com")
+    _channel(db, 1)
+    _watched(db, other, 1, [LONG] * 5)
+    client = as_user(user)
+    assert client.put(f"/api/takeout/channels/{_ch(1)}/dismissal").status_code == 404
+    assert client.put(f"/api/takeout/channels/{_ch(9)}/dismissal").status_code == 404
+    db.expire_all()
+    assert db.execute(select(TakeoutChannelDismissal)).scalars().all() == []
+
+
+def test_a_dismissal_of_an_overlong_channel_id_is_refused_before_the_database(
+    db, make_user, as_user
+):
+    user = make_user("long@example.com")
+    assert as_user(user).put(f"/api/takeout/channels/{'U' * 33}/dismissal").status_code == 422

backend/tests/test_youtube_refusals.py

@@ -193,3 +193,45 @@ def test_any_other_token_refusal_keeps_the_grant(monkeypatch):
         c._access_token()
     assert tok.scopes == "s"
     assert refusal_reason(caught.value) == "unavailable"
+
+
+@pytest.mark.parametrize(
+    ("failure", "sent"),
+    [
+        (httpx.ConnectError("refused"), False),
+        (httpx.ConnectTimeout("slow connect"), False),
+        (httpx.ProxyError("proxy down"), False),
+        (httpx.ReadTimeout("no answer"), True),
+        (httpx.RemoteProtocolError("dropped"), True),
+    ],
+)
+def test_a_transport_failure_says_whether_the_request_went_out(failure, sent):
+    """A search that never reached YouTube spent nothing and gets its slot back; one whose answer
+    never came may have been counted, and keeps it (`routes/search`)."""
+
+    def request(*_a, **_k):
+        raise failure
+
+    c = yt.YouTubeClient.__new__(yt.YouTubeClient)
+    c._http = SimpleNamespace(request=request)
+    with pytest.raises(YouTubeError) as caught:
+        c._send("GET", "https://example.invalid")
+    assert caught.value.reason == "transport"
+    assert caught.value.sent is sent
+
+
+def test_a_call_youtube_answered_is_spent_a_refused_one_too(monkeypatch):
+    resp = httpx.Response(
+        400,
+        json={"error": {"errors": [{"reason": "invalidPageToken"}]}},
+        request=httpx.Request("GET", f"{yt.API_BASE}/search"),
+    )
+    recorded: list[int] = []
+    c = yt.YouTubeClient.__new__(yt.YouTubeClient)
+    c.db, c.spent = None, 0
+    monkeypatch.setattr(yt.sysconfig, "get_str", lambda d, key: "a-key")
+    monkeypatch.setattr(yt.quota, "record_usage", recorded.append)
+    monkeypatch.setattr(c, "_send", lambda *a, **k: resp)
+    with pytest.raises(YouTubeError):
+        c._get("search", {}, cost=100)
+    assert (recorded, c.spent) == ([100], 100)

docs/DESIGN.md

@@ -225,7 +225,10 @@ Also in force:
   tag chip is `Chip`; a sortable row's grip is `DragHandle`. On the always-dark player chrome both
   `Button` and `IconButton` use the `overlay` variant (white, a white layer), whatever the theme; the Plex
   control row is IconButton's `player` rung (36, a 20px glyph, round, 44 on touch). A pressed toggle is
-  `pressed`: the solid accent on a `Button`, the accent glyph on an `IconButton`. A raw `<button>` (or a bare
+  `pressed`: the solid accent on a `Button`, the accent glyph on an `IconButton`, kept under the pointer
+  (no hover layer or hover colour over it). A mode cycle (the player's auto-advance and loop) is no
+  two-state switch: `lit` gives it the same look when it is not off, without `aria-pressed`, and its name
+  says the mode. A raw `<button>` (or a bare
   `FocusSafeButton`) outside the control primitives (eslint `BUTTON_PRIMITIVES`) is a lint error. A text-less button must have an accessible name, and the type enforces
   it. [WCAG 4.1.2]
 - **The chip family** (`Chip`, E28 S3b): rounded-full, never under 24 (A3 grows it to 44 on touch).
@@ -251,8 +254,13 @@ Also in force:
 - A button that starts slow work shows `loading` (a spinner, `aria-busy`) and can't be pressed twice.
   [Nielsen #1]
 - **A glyph means one thing.** Play (▷) starts playback, or resumes a paused transfer; a retry is the
-  refresh arrow (`RefreshCw`); a reset or revert is the counter-clockwise arrow (`RotateCcw`). A failed
-  download's Retry had worn the Play glyph, which promised a video. [Nielsen #4]
+  refresh arrow (`RefreshCw`), and so is doing a download again; a reset or revert is the
+  counter-clockwise arrow (`RotateCcw`), and so is playing from the beginning, on the card and in its
+  menu alike; clearing a video's progress and status is the eraser (`Eraser`). A failed download's Retry
+  had worn the Play glyph, which promised a video. The skip arrows (⏮ ⏭) are Previous /
+  Next; the auto-advance mode beside them wears list glyphs (`ListEnd` next, `ListStart` previous,
+  `Shuffle` random; off is `ListEnd`, not lit) — a mode in the skip arrows put two identical ⏭ on the
+  panel's glyph-only row. [Nielsen #4]
 
 ## Surfaces and page chrome — _Decided_ (batch B)
 
@@ -275,7 +283,9 @@ Also in force:
 - **Page chrome (B4):** a page's tabs and filters sit in ONE sticky band of the page colour, with the fade
   under it (below). Below ~600px of height the filter row folds into one "Filters · n" button. Every page has
   a visually hidden `h1`; the nav rail and the browser tab title are the visible identity. [WCAG 2.4.6]
-  Built as: the band is `PageShell`'s toolbar slot (`PageToolbar`), above the one scroller. The band
+  Built as: the band is `PageShell`'s toolbar slot (`PageToolbar`), above the one scroller; the floating
+  header over it is its header slot (`PageHeader`: the page's own pills, the palette's Jump-to pill where a
+  page has none, `PageHeaderDefault`). The band
   holds controls only: an intro paragraph scrolls with the content (in the band it ate three lines of a
   short window). The fold is `useFilterFold` + `FilterFoldButton`: the same row
   shown or hidden in place, never copied into a popover. The hidden `h1` is `PageShell`'s `title`; a
@@ -496,7 +506,10 @@ of a full-width panel and nothing on the right.
 ## Interaction rules — _Decided_
 
 - **Keyboard:** everything works by keyboard, in visual order, with no traps. A shortcut is a row in
-  `lib/keymap.ts`; list and grid navigation is `NavGrid`/`keyboardNav`. [WCAG 2.1.1, 2.4.3, 2.1.2]
+  `lib/keymap.ts`; list and grid navigation is `NavGrid`/`keyboardNav`. An open dialog or player
+  silences the page's list keys, but a list inside the topmost one walks while the focus is in it
+  (the player's related videos); the walk stays inside that layer, and its arrows there move the list,
+  not the video. [WCAG 2.1.1, 2.4.3, 2.1.2]
 - **Focus:** every control has a visible focus ring (the base ring in `index.css`; don't add a second). A
   sticky or fixed layer never covers the focused element. A list item the keyboard lands on rings 2 px
   outside itself where it has a gap round it (a card); a table row, a log line or a row flush in a
@@ -554,7 +567,8 @@ of a full-width panel and nothing on the right.
   row is not a card and keeps its one play button: started, that button continues, its "…" has Continue +
   "Play from the beginning", and its picture carries the cards' progress bar.
 - **The YouTube modal:** its chrome uses the `Button` rungs (44 on touch); Previous / Next live in the panel
-  (its transport row, or the shut panel's strip) and on Shift+← / →, with no strips beside the card. Close is
+  (its transport row: glyphs only — Previous, the count, Next, auto-advance, loop on one line, each named in
+  its tooltip, the modes with their current one — or the shut panel's strip) and on Shift+← / →, with no strips beside the card. Close is
   an icon at the top right of the panel (wide: its head, or its strip) and at the end of the title row
   under the video (stacked). Wide, the panel shows the description outright, clamped to three
   lines with a More (`ClampedText`; a Tab onto a link below the cut opens it), and collapses to a 2.75rem
@@ -562,6 +576,18 @@ of a full-width panel and nothing on the right.
   open. Stacked, the description opens from an ⓘ, because a hover popover never opens on a tap (measured
   live). The wheel: over the video it sets the volume; over a panel with something to scroll it scrolls
   only, never the volume, even at its end; over one with nothing to scroll it sets the volume. [WCAG 1.4.13]
+- **Related videos (owner's picks A, A, a, a):** on demand, never open by itself — the modal has no room to
+  spare. Wide, a "Related" tab beside the description in the side panel; stacked, a button beside the ⓘ
+  opens them in a drawer over the player. A row is the thumbnailed video row in its narrow form (a 6rem
+  picture, the relative time only, the "…" its one control — every toggle is a row of that menu), with
+  a check before the channel the user follows (named on hover). A click plays it in the same player
+  (Back returns to the opened video), and the list stays the opened video's. Played, it is the item it
+  now is (owner's pick D1 a): its title, its own watched / playlist / download controls, its progress
+  kept, marked watched at its end — and its end does not step the queue. What the user watched or hid
+  is never in it; a list read so far with nothing unseen says so above Show more. Over it, a two-way
+  switch: on topic or trending (the Hungarian chart, labelled just "Trending"), kept on the account. Turned off on the instance (a daily
+  limit of 0, owner's pick D2 a), there is no tab and no button; a refusal a retry cannot answer (the
+  day's cap, turned off, the shared budget) offers no Retry.
 - **Clip mode (both players, owner's picks A + P1 + a):** a scissors in the control row (Plex: the C5 row,
   before the ⋯; YouTube: the actions beside Download, and the docked bar) or `C` opens a clip panel IN the
   player; `[` / `]` mark the start / end at the playhead and open it when shut. The video being watched is the
@@ -793,3 +819,8 @@ of a full-width panel and nothing on the right.
 | 2026-10-17 | Text on glass (owner ★)                          | Light `--muted` 15% toward `--fg` in every scheme (≥ 6:1 on the bare page); the item count, the badges and the current Files row on opaque tints mixed into the surface                                                                                                                                                                                                                             | Only the two failing components; the item count's number in `--fg`                                                                                                                                     | Pixel-measured over every scheme × mode × backdrop, the count fell to 3.8:1 (dark too: thumbnails scroll under the toolbar's glass) and Channels' tabs to 4.1:1; an `--fg` count outshouts the filters beside it             |
 | 2026-10-18 | Scheduler run history (owner D4, E1, E2 ★)       | A row's last three runs as dots, newest first (interrupted: a hollow warn ring, set apart from skipped and failed), and a history button that opens the job's runs on the Activity page; a user's ad-hoc runs in their own block under the jobs, with no interval and no run button                                                                                                                 | An inline expandable history per row; a dark-grey interrupted dot; ad-hoc runs only on the Activity page                                                                                               | Three dots answer "did it run, did it work" at a glance; the whole history needs filters, which the Activity page has                                                                                                        |
 | 2026-10-18 | Activity page (owner D1, D2, G1–G3 ★)            | One page, "Napló" / "Activity": four tabs (Runs, Admin actions, Server log, Quota) under one filter bar (text, a period of 1 h · 24 h · 7 d · 30 d · 90 d defaulting to 7 days, an account), the view in the URL; runs and admin actions as dense rows that open in place as a log line does, filtered and paged on the server; a run's or a request's log lines one pivot away, as a history entry | One merged timeline; a page per source; a default period per tab; card rows; the audit log's client-side table                                                                                         | Each source keeps its own table and retention, read in one place; one bar keeps what you narrowed across tabs; about 1.8k runs a day need a log's density and a server-side filter                                           |
+| 2026-10-24 | Related videos (owner U1–U3 ★, D1 A, D3)         | On demand: a tab beside the description (wide), a drawer (stacked); the narrow thumbnailed row, its menu alone, a followed check; plays in place and the list stays the opened video's; an on-topic / Hungarian-trending switch kept on the account                                                                                                                                                 | Always shown; the end-of-video grid alone; a dialog; a big card or a text row; a list that follows the playing video                                                                                   | The modal is full on a small screen; a picture is what a choice among strangers rests on; a list that stays put is tried one by one, and spends no more of the day's cap                                                     |
+| 2026-10-25 | Related: one played; off (owner D1 a, D2 a)      | Played from the list, it is the player's item: its title, controls and progress, marked watched at its end, the queue left where it was; a daily limit of 0 takes the tab and the button away                                                                                                                                                                                                       | A linked video's in-place play (no controls, nothing kept); a tab that only shows a refusal                                                                                                            | What is watched should count as watched, wherever it was started from; a control that can only be refused is not offered                                                                                                     |
+| 2026-10-25 | Player panel glyphs (owner 1A, 2A, 3)            | In the side panel the queue row is glyph-only on one line, and Mark watched is a glyph toggle among the actions (✓ / ✓✓); the docked bar keeps its labels; the related switch at the default size, "Trending"                                                                                                                                                                                       | Labelled buttons wrapping onto three lines; glyphs on the docked bar too; Mark watched on its own line                                                                                                 | The panel is narrow and the list controls are the same few every time; each glyph keeps its full name and shortcut in its tooltip; the docked bar has the width for words                                                    |
+| 2026-10-25 | Auto-advance glyphs (owner E A, H a)             | Auto-advance wears list glyphs (`ListEnd` next and off, `ListStart` previous, `Shuffle` random), in the panel and the docked bar; the auto-advance and loop cycles are `lit`, not `pressed`                                                                                                                                                                                                         | The skip arrows for the mode; an X'd list for off; `aria-pressed` on a cycle                                                                                                                           | Beside Previous / Next the skip arrows read as a second Next; a cycle of three or four modes is no two-state switch, and its name already says which                                                                         |
+| 2026-10-25 | Restart and Reset glyphs (owner L A)             | A menu's Play from the beginning wears `RotateCcw`, as the card's own Restart does; Reset (clear progress and status) wears `Eraser`; Redownload wears `RefreshCw`, the retry arrow                                                                                                                                                                                                                 | The skip arrow ⏮ on the menu's Restart; `RotateCcw` on both Restart and Reset; `TimerReset` for Restart                                                                                               | ⏮ is Previous beside the player; one action wore two glyphs, and two actions in one menu would have worn one                                                                                                                |