Compare commits
50
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bf5ceec8a1 | ||
|
|
e788326247 | ||
|
|
38c71b7b4f | ||
|
|
cd5b26efa7 | ||
|
|
0892a1a400 | ||
|
|
b443ac0c84 | ||
|
|
ad05932217 | ||
|
|
6ae7825a14 | ||
|
|
cde5a98fd2 | ||
|
|
e83c5e2d97 | ||
|
|
6a0c1d377f | ||
|
|
4ef963b005 | ||
|
|
0b24f2d8fa | ||
|
|
d5867bacca | ||
|
|
63a0fecc8b | ||
|
|
fdf8076013 | ||
|
|
6287fc9cf8 | ||
|
|
3c6c23507b | ||
|
|
ef99f7b5c7 | ||
|
|
94418a087e | ||
|
|
28b77fed1e | ||
|
|
50764a916d | ||
|
|
f21cdfa156 | ||
|
|
d6dada7fe5 | ||
|
|
6645caa157 | ||
|
|
59ec4a28ba | ||
|
|
dc4f4f5143 | ||
|
|
909732b3c1 | ||
|
|
215f022fe3 | ||
|
|
6ea750a698 | ||
|
|
c5f121139b | ||
|
|
b04b2cbd62 | ||
|
|
3a5212cd8a | ||
|
|
b376373af5 | ||
|
|
8206384c91 | ||
|
|
e27656ee7e | ||
|
|
d73cdcee97 | ||
|
|
5ce292cd28 | ||
|
|
ad6ccbac05 | ||
|
|
531eb2eef1 | ||
|
|
2cf1b688d9 | ||
|
|
3b0e365e26 | ||
|
|
533571f572 | ||
|
|
4b6db47083 | ||
|
|
5229fd96be | ||
|
|
238e6f8a61 | ||
|
|
a35eece652 | ||
|
|
b703234c17 | ||
|
|
dbb4065e7b | ||
|
|
e82f87acb0 |
@@ -7,6 +7,48 @@ edit the release notes and re-run the generator.
|
||||
|
||||
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
|
||||
|
||||
## 0.109.2 — 2026-10-23
|
||||
|
||||
### Fixed
|
||||
|
||||
- Plex player: when the server restarts mid-episode (an update), playback carries on from the buffer and picks up where you are once it is back, instead of showing "Playback couldn't start" and stopping where the buffer ended.
|
||||
|
||||
## 0.109.1 — 2026-10-23
|
||||
|
||||
### New
|
||||
|
||||
- Development: the frontend is one folder per page (its page, rail, store and definition), with the layers held by lint and the shared /watch page held apart by a contract test; the header knows no page, and the feed and Plex each keep their own saved views.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Saved views: a view you applied and then changed is still active after a visit to another page, in the feed and in Plex.
|
||||
|
||||
## 0.109.0 — 2026-10-21
|
||||
|
||||
### New
|
||||
|
||||
- Accounts: a browser is signed in to one account at a time. The account menu's "Sign in as someone else" replaces it; the account list and switcher are gone. After this update everyone signs in once more.
|
||||
- Settings follow you: the Home start page, the channel manager and Plex playlist layouts, the discovery sorts and a dismissed onboarding or release banner are saved to your account, on every device. Performance mode stays per device.
|
||||
- Development: every key the browser stores declares its kind, held by a contract test; one Google connect route serves the sign-in and the YouTube access grants.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Accounts: a tab still showing another account than the one signed in now reloads as that one and says why; a failed "Sign in as someone else" says what happened.
|
||||
- A link whose path starts with // no longer opens the error screen.
|
||||
- A member following a link to an admin page lands on the feed, with the rail and the address bar in step.
|
||||
- Google: an unknown access request is refused instead of quietly asking for read access, and a YouTube grant on an instance without Google says so.
|
||||
|
||||
## 0.108.1 — 2026-10-20
|
||||
|
||||
### New
|
||||
|
||||
- Development: the backend is split into four units (downloads, reader, catalog, Plex), each owning its tables. A contract test holds what each one reads and writes, and that every query names whose rows it asks for.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Watch later: saving the first two videos at once no longer makes two lists. An existing second list is merged into the first, its videos and its comfort-shelf pin with it.
|
||||
- Playback: the first resume point of a video or a Plex item no longer fails when it is saved together with the watched mark, a whole season marked at once, or a Plex sync.
|
||||
|
||||
## 0.108.0 — 2026-10-18
|
||||
|
||||
### New
|
||||
|
||||
@@ -63,55 +63,73 @@ how it gets built. Concretely:
|
||||
## Architecture shape (don't fight it)
|
||||
|
||||
`routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
|
||||
`downloads/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`downloads/`, `reader/`, `discover/`, `catalog/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `ratelimit`, `bodylimit`, `progress`, `collation`,
|
||||
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
|
||||
Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
|
||||
entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
|
||||
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
|
||||
its menus, `Modal`/`Overlay`, `Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`, …). That
|
||||
boundary is one-way and **lint-enforced**, not just described: a platform component may not import a
|
||||
feature component or a domain API slice, and one layer down `lib/` may not import from `components/` at
|
||||
all, not even a type (`no-restricted-imports` in `eslint.config.js`, with its own test). A new generic
|
||||
component goes in `ui/`; see [`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
|
||||
Data via react-query (`qk` factory), per-account `localStorage`, and server prefs; API through
|
||||
`lib/api/*` slices over `core.req<T>`.
|
||||
The domain is four **units**, each owning its tables: Capture (`downloads/`), Reader (`reader/`, `discover/`, the playlist
|
||||
sync), Catalog (`catalog/`, the `sync/` ingests) and Plex (`plex/`). A unit's route does no database work: it reads through
|
||||
the unit's `queries`, writes through its `service` (which flushes) and commits. A unit's code reads another unit's tables
|
||||
only along a declared edge and writes them only through that unit's service; core reaches a unit's tables through its
|
||||
`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`) — as far as an AST sees: what it
|
||||
cannot (an ORM write on another unit's loaded row, SQL in `text()`, a commit by `begin()`) is review's to catch, listed in
|
||||
[`backend/app/README.md`](backend/app/README.md) with the conventions.
|
||||
Frontend, by folder under `src/`: `main.tsx` provider tree → `shell/` (`App`, header, nav rail, palette,
|
||||
boot; a shared `/watch` link boots from its own entry, `watch.html` → `watchMain.tsx` → `watch/`, held
|
||||
app-free by a contract test) → **`modules/<page>/`** (one folder per page: the page, its rail, its store, its tests,
|
||||
and its `module.ts` definition — label, icon, gate, the lazy page and rail, its store — collected in nav order by
|
||||
`features/navigation/moduleRegistry`, which also mounts every store at the root; a new page is its folder, its id in
|
||||
`lib/urlState` and its entry there, plus its panel in `lib/panelLayout` if it has a rail) →
|
||||
**`features/<name>/`** (what several pages share: video cards, filters, downloads, the player, navigation) → the
|
||||
**platform layer `components/ui/`** (domain-free primitives: the popover engine and its menus, `Modal`/`Overlay`,
|
||||
`Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`,
|
||||
…) → `lib/`. The direction is one-way and **lint-enforced**, not just described (`local/layers`, with
|
||||
its own test): an import only goes down; another module is reached only through its **`index.ts`** (its
|
||||
public face — the stores, mounted at the root and read app-wide, which `features/` may read too) or
|
||||
lazily by `import()`; a platform component may not import a domain API slice, and `lib/` nothing above
|
||||
it, not even a type. A lazily imported page, rail or dialog that a static import drags onto the boot
|
||||
path fails the build (`chunkGuard.ts`). A new generic component goes in `ui/`; see
|
||||
[`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
|
||||
Data via react-query (`qk` factory), the signed-in account's `localStorage` (`storage.ts`), and server
|
||||
prefs; API through `lib/api/*` slices over `core.req<T>`.
|
||||
|
||||
**Not wanted** (reviewed and rejected as over-engineering for this size): microservices, event bus,
|
||||
CQRS, a repository/DI framework, a router library, a generic CRUD-router factory or base serializer.
|
||||
|
||||
## Canonical patterns — when you have a choice, pick THIS
|
||||
|
||||
| Area | Use | Not |
|
||||
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; helpers `flush` (test-held list) | helpers that commit (only `quota` and `jobruns` own their session) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
|
||||
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
|
||||
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
|
||||
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
|
||||
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
|
||||
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
|
||||
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect | a bare `catch {}` or `notify()` with no error attached |
|
||||
| FE floating or full-screen layer | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers") | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]` |
|
||||
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
|
||||
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
|
||||
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
|
||||
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
|
||||
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback` | a render-phase `ref.current = x` — the React Compiler skips the component |
|
||||
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
|
||||
| FE popover / scroll-fade ref | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}` | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips |
|
||||
| FE async handler with cleanup | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
|
||||
| Area | Use | Not |
|
||||
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404`. A composite rule (own or shared, built-in or own) says so in its `queries` | an inline owner filter — `Model.user_id == …`, an alias's, `.in_()`, `filter_by(user_id=…)` (fence-held, every module) — or a `db.get` and a compare where `owned_row` fits |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; a unit's `service` and every helper `flush` (fence-held, every module) | a helper that commits — only a listed one (what owns its session, or a core step that must land whatever its caller does next), each with its reason: the fence's `UNIT_COMMITS` / `CORE_COMMITS`, a route's `COMMITTING_HELPERS` (`test_route_conventions`) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
| FE browser storage | a key in `storage.ts` with its kind (contract-held): a `pref` through `usePref` (the server's, follows the account); `device` / `view` / `cache` this browser's; `public` kept for the /watch page | a `"siftlode.x"` literal; a preference kept only in `localStorage` (it stays on one device) |
|
||||
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
|
||||
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
|
||||
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
|
||||
| FE a page's header content | the page renders it into the floating header itself (`PageHeader`, `ui/PageShell`) | a `page === …` branch in the shell's header, or a bridge carrying a page's state up to it |
|
||||
| FE module store | its definition's `Store` (`module.ts`), mounted at the root by `ModuleStores`; it reads no other module's store (contract-held) | a provider hand-nested in `main.tsx` |
|
||||
| FE saved views | the module's own store from `features/filters` (`createSavedViewsStore` + `useSavedViewsStore`) | one store switched by the active page — every visit elsewhere reset it |
|
||||
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
|
||||
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
|
||||
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect | a bare `catch {}` or `notify()` with no error attached |
|
||||
| FE floating or full-screen layer | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers") | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]` |
|
||||
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
|
||||
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
|
||||
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
|
||||
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
|
||||
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback` | a render-phase `ref.current = x` — the React Compiler skips the component |
|
||||
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
|
||||
| FE popover / scroll-fade ref | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}` | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips |
|
||||
| FE async handler with cleanup | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
|
||||
|
||||
Not repeated here, because a lint rule refuses the wrong one and its message names the right one:
|
||||
buttons (`ui/Button`, `IconButton`, `Pressable`, `Chip`, `DragHandle`), text fields and native
|
||||
|
||||
@@ -640,48 +640,6 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/channels.py": [
|
||||
{
|
||||
"code": "reportReturnType",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 11,
|
||||
"lineCount": 6
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 49,
|
||||
"endColumn": 56,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 38,
|
||||
"endColumn": 44,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 33,
|
||||
"endColumn": 39,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/discover.py": [
|
||||
{
|
||||
"code": "reportInvalidTypeForm",
|
||||
@@ -692,235 +650,7 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/playlists.py": [
|
||||
{
|
||||
"code": "reportOptionalOperand",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalOperand",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 13,
|
||||
"endColumn": 5,
|
||||
"lineCount": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 16,
|
||||
"endColumn": 5,
|
||||
"lineCount": 11
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 9
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 13,
|
||||
"endColumn": 5,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 12,
|
||||
"endColumn": 31,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 23,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 12,
|
||||
"endColumn": 34,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 12,
|
||||
"endColumn": 28,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 23,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/browse.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 61,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 61,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 13,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 13,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 15,
|
||||
"endColumn": 17,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 28,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 15,
|
||||
"endColumn": 17,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 28,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/collections.py": [
|
||||
{
|
||||
"code": "reportAssignmentType",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 39,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 76,
|
||||
"endColumn": 80,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 75,
|
||||
"endColumn": 79,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 80,
|
||||
"endColumn": 84,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
@@ -978,117 +708,7 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/discover.py": [
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 4,
|
||||
"endColumn": 63,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 18,
|
||||
"endColumn": 48,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 4,
|
||||
"endColumn": 63,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 18,
|
||||
"endColumn": 48,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/playback.py": [
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 6,
|
||||
"endColumn": 14,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 51,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 51,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/playlists.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 77,
|
||||
"endColumn": 83,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 77,
|
||||
"endColumn": 83,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 10,
|
||||
"endColumn": 18,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/scheduler.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 55,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
@@ -1106,42 +726,6 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/search.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 31,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 6,
|
||||
"endColumn": 14,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/sync.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 65,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 50,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/sync/explore.py": [
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""One Watch later list per user, held by the database
|
||||
|
||||
Revision ID: 0112_one_watch_later
|
||||
Revises: 0111_audit_request_id
|
||||
Create Date: 2026-10-20
|
||||
|
||||
Watch later was made on a user's first save by a read then an insert, so two first saves at once
|
||||
each made a list, and the reader then picked between them at random. A partial unique index holds
|
||||
"one per user" now. An instance that already has a second list keeps the oldest: the others' videos
|
||||
move into it (after its own, in their order; one it already holds is not added twice), a Home pin
|
||||
of one of them moves to it (unless it is pinned itself; of several, the first on the shelf), then
|
||||
they go — a pin left on them goes with them, as the kept list already holds one.
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0112_one_watch_later"
|
||||
down_revision: str | None = "0111_audit_request_id"
|
||||
branch_labels: str | None = None
|
||||
depends_on: str | None = None
|
||||
|
||||
_EXTRA = """
|
||||
SELECT p.id, k.keep_id
|
||||
FROM playlists p
|
||||
JOIN (
|
||||
SELECT user_id, min(id) AS keep_id FROM playlists WHERE kind = 'watch_later' GROUP BY user_id
|
||||
) k ON k.user_id = p.user_id
|
||||
WHERE p.kind = 'watch_later' AND p.id <> k.keep_id
|
||||
"""
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
f"""
|
||||
INSERT INTO playlist_items (playlist_id, video_id, position, added_at)
|
||||
SELECT e.keep_id, i.video_id,
|
||||
(SELECT coalesce(max(position), -1) FROM playlist_items WHERE playlist_id = e.keep_id)
|
||||
+ row_number() OVER (PARTITION BY e.keep_id ORDER BY i.playlist_id, i.position, i.id),
|
||||
i.added_at
|
||||
FROM playlist_items i
|
||||
JOIN ({_EXTRA}) e ON e.id = i.playlist_id
|
||||
ON CONFLICT ON CONSTRAINT uq_playlist_video DO NOTHING
|
||||
"""
|
||||
)
|
||||
op.execute(
|
||||
f"""
|
||||
UPDATE comfort_pins c SET playlist_id = m.keep_id
|
||||
FROM (
|
||||
SELECT DISTINCT ON (e.keep_id) p.id AS pin_id, e.keep_id
|
||||
FROM comfort_pins p
|
||||
JOIN ({_EXTRA}) e ON e.id = p.playlist_id
|
||||
WHERE NOT EXISTS (SELECT 1 FROM comfort_pins k WHERE k.playlist_id = e.keep_id)
|
||||
ORDER BY e.keep_id, p.position, p.id
|
||||
) m
|
||||
WHERE c.id = m.pin_id
|
||||
"""
|
||||
)
|
||||
op.execute(f"DELETE FROM playlists WHERE id IN (SELECT id FROM ({_EXTRA}) e)")
|
||||
op.create_index(
|
||||
"uq_playlist_watch_later",
|
||||
"playlists",
|
||||
["user_id"],
|
||||
unique=True,
|
||||
postgresql_where=sa.text("kind = 'watch_later'"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("uq_playlist_watch_later", table_name="playlists")
|
||||
+56
-16
@@ -1,9 +1,8 @@
|
||||
# Backend conventions
|
||||
|
||||
The rules below are already followed across `backend/app` — they lived only in a few module
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
|
||||
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
|
||||
I copy".
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them. When you add code,
|
||||
match these so the next reader doesn't have to guess "which pattern do I copy".
|
||||
|
||||
## Module layering
|
||||
|
||||
@@ -13,12 +12,13 @@ Three layers, by *who* runs the code — not by feature:
|
||||
response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
|
||||
Business logic that a request happens to trigger does **not** live here if anything else runs it too.
|
||||
|
||||
2. **`plex/`, `downloads/`, `sync/`, `issues/` — domain logic shared by more than one caller.** Anything the
|
||||
**scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
|
||||
route, the worker, and the scheduler share one definition. Examples from this codebase:
|
||||
`downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
|
||||
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline;
|
||||
`issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
|
||||
2. **The units and the other domain packages — logic shared by more than one caller.** A unit's
|
||||
database work (below), and anything the **scheduler or the worker** runs (not just a request), lives
|
||||
in a domain package, so the route, the worker, and the scheduler share one definition. Examples from
|
||||
this codebase: `downloads/service.py` owns enqueue + the job state transitions (routes call it);
|
||||
`plex/sync.py` is run by both the admin route and the scheduler; `downloads/pipeline.py` is the
|
||||
worker's child pipeline; `issues/bundle.py` is the one definition of a bug report's files, for the
|
||||
zip export and the tracker.
|
||||
|
||||
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's
|
||||
folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every
|
||||
@@ -34,19 +34,59 @@ Three layers, by *who* runs the code — not by feature:
|
||||
a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
|
||||
pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
|
||||
|
||||
## Units
|
||||
|
||||
The domain is four units, each owning its tables (`models.py` in its package):
|
||||
|
||||
| Unit | Package | Its routes |
|
||||
| --- | --- | --- |
|
||||
| Capture | `downloads/`, the worker | `downloads.py`, `files.py`, `public.py` |
|
||||
| Reader | `reader/`, `discover/`, `feedquery.py`, the playlist sync and tag suggestions in `sync/` | `feed.py`, `playlists.py`, `saved_views.py`, `discover.py`, `tags.py` |
|
||||
| Catalog | `catalog/`, the ingests in `sync/` | `channels.py`, `search.py`, `sync.py` |
|
||||
| Plex | `plex/` | `plex/` |
|
||||
|
||||
- **A unit's route does no database work.** It validates, reads through the unit's `queries.py`,
|
||||
writes through its `service.py`, serializes, and commits. It may name a unit's model as a type
|
||||
(under `TYPE_CHECKING`), never use one as a value, and it passes a kind (`"movie"`), not a model, to
|
||||
a query.
|
||||
- **A unit's code keeps to its edges.** It reads another unit's tables only along a declared edge
|
||||
(Capture and Reader read Catalog's and Plex's: the feed joins `Video` with `VideoState` in one
|
||||
query) and writes them only through the owner's service. Core code reaches a unit's tables through
|
||||
that unit's `queries`.
|
||||
- **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id,
|
||||
user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the
|
||||
same answer as a missing one). A unit's `queries.own_x` returns None and its route answers
|
||||
`not_found`; a core route calls `owned_or_404`. A rule `owned_row` cannot say — a job its owner
|
||||
shared, a built-in profile nobody owns, a folder read under `FOR SHARE` — is written out in the
|
||||
unit's own code, beside the reason.
|
||||
|
||||
All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's
|
||||
packages and edges (`UNIT_PACKAGES`, `READS`) and what may commit (`UNIT_COMMITS`, `CORE_COMMITS`)
|
||||
are listed there. It reads the code's AST, so it holds the forms written here: an owner column
|
||||
compared inline (a model's, an import alias's or an `aliased()` one's, by `==`, `!=`, `.in_()`, or a
|
||||
`filter_by(user_id=…)`), a model built or an `insert`/`update`/`delete` (under any import name, of a
|
||||
model or its `__table__`) across units, and a `.commit()` call off the lists all fail it. What an AST
|
||||
cannot tell is review's to catch: an ORM write on another unit's loaded row (an attribute, a
|
||||
`db.delete`), SQL in `text()`, a column or model reached another way (bound to a variable, through
|
||||
`__table__.c` or a module path), and a commit made by a `begin()` block or a `commit` handed in.
|
||||
|
||||
## Transaction ownership
|
||||
|
||||
This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
|
||||
(only `quota` and `jobruns` own their session)"** — so read it that way, not as an exception to it. The commit belongs
|
||||
to the **operation owner**; anything called *inside* another transaction only stages its change
|
||||
This follows CLAUDE.md's canonical rule — **the handler commits; a unit's `service` and every helper
|
||||
flush** — so read it that way, not as an exception to it. The commit belongs to the **operation
|
||||
owner**; anything called *inside* another transaction only stages its change
|
||||
(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
|
||||
`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
|
||||
mutation it records, so an audit entry can never exist without its change (or vice-versa).
|
||||
|
||||
- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
|
||||
whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
|
||||
a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
|
||||
`jobruns`' run rows — the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
|
||||
- The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it
|
||||
makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the
|
||||
request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, the
|
||||
scheduler's jobs, a unit's jobs (the worker's `jobdb` writes, the syncs and ingests that commit per
|
||||
page or batch so a later failure loses nothing already stored), and a few core steps that must land
|
||||
whatever their caller does next (a token's last-used stamp, a refreshed YouTube grant) — the fence's
|
||||
`UNIT_COMMITS` and `CORE_COMMITS` name each with its reason, `test_route_conventions` a route's own
|
||||
committing helpers. Everything else flushes.
|
||||
- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
|
||||
loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
|
||||
attribute — see `service.decr_ref_count`.
|
||||
|
||||
@@ -26,6 +26,7 @@ from fastapi import Depends, Request
|
||||
from sqlalchemy import delete, or_, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -223,7 +224,9 @@ def revoke_all(db: Session, user_id: int) -> int:
|
||||
change, "log out everywhere", a revoked untrusted password — ends its tokens too: one minted
|
||||
from a stolen session must not outlive the owner's answer to that theft. Returns how many."""
|
||||
gone = (
|
||||
db.execute(delete(ApiToken).where(ApiToken.user_id == user_id).returning(ApiToken.id))
|
||||
db.execute(
|
||||
delete(ApiToken).where(owned_by(ApiToken.user_id, user_id)).returning(ApiToken.id)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
@@ -110,10 +110,9 @@ def record(
|
||||
|
||||
def gc(db: Session, retention_days: int) -> int:
|
||||
"""Delete audit rows older than `retention_days`. Returns the number removed. A
|
||||
retention_days <= 0 disables pruning (keep forever)."""
|
||||
retention_days <= 0 disables pruning (keep forever). Flushes; the job commits."""
|
||||
if retention_days <= 0:
|
||||
return 0
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
|
||||
result = db.execute(delete(AuditLog).where(AuditLog.created_at < cutoff))
|
||||
db.commit()
|
||||
return result.rowcount or 0
|
||||
|
||||
+49
-89
@@ -4,6 +4,7 @@ from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.requests import HTTPConnection
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import debugsession
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -24,8 +25,6 @@ def operator_contact() -> str | None:
|
||||
return next(iter(sorted(settings.admin_email_set)), None)
|
||||
|
||||
|
||||
MAX_SESSION_ACCOUNTS = 6
|
||||
|
||||
# OAuth scope vocabulary — the atoms + the request bundles, kept together as core. has_*_scope check
|
||||
# the atoms against a user's stored grant; the *_SCOPES bundles are what the login/callback endpoints
|
||||
# request (imported by app/routes/auth.py).
|
||||
@@ -230,7 +229,7 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
|
||||
# request's access line, a queued row, a later background task would otherwise put rows back
|
||||
# under the id after this DELETE (logs.forget_user).
|
||||
logs.forget_user(user_id)
|
||||
db.execute(delete(LogEvent).where(LogEvent.user_id == user_id))
|
||||
db.execute(delete(LogEvent).where(owned_by(LogEvent.user_id, user_id)))
|
||||
# Every canonically-equal invite goes, not just the literal spelling: is_allowed matches on the
|
||||
# canonical form, so leaving a sibling row behind would keep granting access to a mailbox whose
|
||||
# account was just erased — and prod really does carry approved pairs for one mailbox.
|
||||
@@ -252,94 +251,63 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
|
||||
background.add_task(email_mod.send_account_deleted, email, lang, operator_contact())
|
||||
|
||||
|
||||
ACTIVE_ACCOUNT_HEADER = "x-siftlode-account"
|
||||
def session_user_id(conn: HTTPConnection) -> int | None:
|
||||
"""The account this browser session signed in as, read from the signed cookie alone (no DB).
|
||||
One account per session: signing in as another replaces it (`routes.auth._establish_session`)."""
|
||||
return conn.session.get("user_id")
|
||||
|
||||
|
||||
def resolved_user_id(conn: HTTPConnection, *, strict: bool = True) -> tuple[int | None, bool]:
|
||||
"""Which account this connection acts as, and whether that's the session's default account.
|
||||
Takes an HTTPConnection rather than a Request: it reads only the session and the headers.
|
||||
|
||||
The signed cookie holds the *wallet* (`account_ids`, every account signed into this browser)
|
||||
plus a default `user_id`. A caller may name its account with the X-Siftlode-Account header (or
|
||||
`?account=`), but ONLY an account in the wallet — so a tab can't impersonate an account that
|
||||
never authenticated here. A connection that names no account acts as the cookie default.
|
||||
|
||||
A connection naming an account NOT in the wallet acts as nobody (`None`) — typically a tab still
|
||||
pinned to an account that was signed out in another tab. It must not fall back to the default:
|
||||
that tab would go on acting as an account it never showed, its player checkpoints and pref saves
|
||||
landing there and its Sign out signing THAT account out. Only the bootstrap /api/me probe passes
|
||||
`strict=False` and gets the default, which is how such a tab learns who it now is (the frontend
|
||||
re-pins it and reloads). An account whose session ended while it stayed in the wallet (deleted,
|
||||
suspended, revoked) leaves the wallet when it is refused (`_session_user`), so a tab pinned to it
|
||||
takes the same way back.
|
||||
"""
|
||||
default_id = conn.session.get("user_id")
|
||||
wallet = conn.session.get("account_ids") or []
|
||||
# Header for normal XHR; ?account= for a context that can't set headers (a plain <a> file
|
||||
# download). Both are wallet-gated below.
|
||||
hdr = conn.headers.get(ACTIVE_ACCOUNT_HEADER) or conn.query_params.get("account")
|
||||
if hdr:
|
||||
try:
|
||||
hid = int(hdr)
|
||||
except ValueError:
|
||||
# Under `strict` this becomes a 401 and the tab authenticates as nobody, which reads
|
||||
# in the browser exactly like a session that expired. The value itself stays out —
|
||||
# it came from a request, and `wallet` says which accounts were actually on offer.
|
||||
log.debug(
|
||||
"the active-account header could not be read as an id",
|
||||
extra=logs.event("auth.account_header.invalid", accounts=len(wallet)),
|
||||
)
|
||||
hid = None
|
||||
if hid is not None and hid in wallet:
|
||||
return hid, hid == default_id
|
||||
if strict:
|
||||
return None, False
|
||||
return default_id, True
|
||||
def stamp_session(request: Request, user: User) -> None:
|
||||
"""Record the account's current `session_epoch` in the signed cookie, so `current_user` can later
|
||||
reject the cookie once the epoch moves (password reset/change, or "log out everywhere")."""
|
||||
request.session["epoch"] = user.session_epoch or 0
|
||||
|
||||
|
||||
def drop_from_wallet(session: dict, user_id: int) -> None:
|
||||
"""Take one account out of this browser's wallet (`account_ids`); the session default is untouched."""
|
||||
session["account_ids"] = [a for a in (session.get("account_ids") or []) if a != user_id]
|
||||
# The account a tab was built for, sent with every request it makes (`identityHeaders`, the
|
||||
# frontend). The cookie alone says who is asking, and one browser holds one account: a sign-in as
|
||||
# someone else in another tab replaces it under every open tab. A tab still showing the previous
|
||||
# account would otherwise act as the new one — its player checkpoints, pref saves, Sign out and
|
||||
# Delete account landing there — until it noticed. It never selects an account; a mismatch is
|
||||
# refused, and the tab reloads as whoever is signed in now.
|
||||
EXPECTED_ACCOUNT_HEADER = "x-siftlode-expect-account"
|
||||
|
||||
|
||||
def _end_account_session(request: Request, user_id: int, is_default: bool) -> None:
|
||||
"""An account whose session here is no longer valid. Only nuke the whole session when it is the
|
||||
SESSION DEFAULT; a per-tab account just leaves the wallet (other tabs' default may still be valid).
|
||||
Leaving is what lets a tab pinned to it recover: its next /api/me names an account no longer signed
|
||||
in here and gets the default. Left in, it answered "signed out" — the landing page, with the default
|
||||
still signed in, and no way off it but closing the tab."""
|
||||
if is_default:
|
||||
request.session.clear()
|
||||
else:
|
||||
drop_from_wallet(request.session, user_id)
|
||||
def built_for_another_account(user_id: int, expected: str | None) -> bool:
|
||||
"""Whether a tab naming `expected` is showing another account than the one signed in. No name is
|
||||
no claim (a caller from before this check, a script): only a mismatch is refused, and the
|
||||
session stays — it is the other tab's sign-in, and it holds. The one comparison for every way a
|
||||
tab names its account: this header on a request, `expect` on a navigation."""
|
||||
if expected is None or expected == str(user_id):
|
||||
return False
|
||||
log.info(
|
||||
"a request from a tab built for another account was refused",
|
||||
extra=logs.event("auth.session.account_changed", user_id=user_id),
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def current_user(request: Request, db: Session = Depends(get_db)) -> User:
|
||||
return _session_user(request, db, *resolved_user_id(request))
|
||||
user = _session_user(request, db, session_user_id(request))
|
||||
if built_for_another_account(user.id, request.headers.get(EXPECTED_ACCOUNT_HEADER)):
|
||||
raise refused(401, "account_changed")
|
||||
return user
|
||||
|
||||
|
||||
def _session_user(request: Request, db: Session, user_id: int | None, is_default: bool) -> User:
|
||||
def _session_user(request: Request, db: Session, user_id: int | None) -> User:
|
||||
if not user_id:
|
||||
raise refused(401, "not_authenticated")
|
||||
user = db.get(User, user_id)
|
||||
if user is None or not user.is_active or user.is_suspended:
|
||||
# Suspended/deactivated mid-session → end it so the block takes effect at once.
|
||||
_end_account_session(request, user_id, is_default)
|
||||
request.session.clear()
|
||||
raise refused(401, "not_authenticated")
|
||||
# SA4: reject a cookie whose recorded epoch is behind the account's current one (revoked by a
|
||||
# password reset/change or "log out everywhere"). A cookie with no recorded epoch (pre-SA4, or an
|
||||
# account signed in before this shipped) is treated as epoch 0 — so the FIRST bump revokes it too.
|
||||
epochs = request.session.get("epochs") or {}
|
||||
if int(epochs.get(str(user_id), 0)) != (user.session_epoch or 0):
|
||||
_end_account_session(request, user_id, is_default)
|
||||
# A cookie whose epoch is behind the account's (revoked by a password reset/change or "log out
|
||||
# everywhere") is refused — and so is one with no epoch at all, which only a build before the
|
||||
# one-account session wrote: its holder signs in once more.
|
||||
epoch = request.session.get("epoch")
|
||||
if epoch is None or int(epoch) != (user.session_epoch or 0):
|
||||
request.session.clear()
|
||||
raise refused(401, "not_authenticated")
|
||||
# Always keep the session default in the switchable list — covers sessions created before
|
||||
# multi-session existed (their account_ids was never seeded), so the switcher isn't empty.
|
||||
default_id = request.session.get("user_id")
|
||||
accounts = request.session.get("account_ids") or []
|
||||
if default_id and default_id not in accounts:
|
||||
accounts.append(default_id)
|
||||
request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
|
||||
# Attribute this request's log lines to the account. Here, not in the request middleware:
|
||||
# this is the one place a session becomes an account, and the middleware runs outside both the
|
||||
# session middleware and this task's context. `request.state` is shared across middleware layers
|
||||
@@ -357,25 +325,17 @@ def _session_user(request: Request, db: Session, user_id: int | None, is_default
|
||||
def optional_current_user(request: Request, db: Session = Depends(get_db)) -> User | None:
|
||||
"""Like `current_user` but returns None instead of raising 401 when there's no valid session.
|
||||
Used by the bootstrap /api/me probe so the logged-out landing doesn't log a failed request to
|
||||
the browser console (a non-2xx fetch is a console error no matter how the app handles it).
|
||||
The one lenient resolution (`strict=False`): a tab naming an account no longer signed in here gets
|
||||
the default account, which is how it learns who it now is — see `resolved_user_id`. That includes an
|
||||
account refused right here, which `_session_user` has just taken out of the wallet."""
|
||||
user_id, is_default = resolved_user_id(request, strict=False)
|
||||
the browser console (a non-2xx fetch is a console error no matter how the app handles it)."""
|
||||
user_id = session_user_id(request)
|
||||
if user_id is None:
|
||||
return None
|
||||
try:
|
||||
return _session_user(request, db, user_id, is_default)
|
||||
# silent-ok: this is the anonymous case as much as the refused one — every landing page hit
|
||||
# comes through here. The second attempt below is the handling, and it reports.
|
||||
return _session_user(request, db, user_id)
|
||||
except HTTPException:
|
||||
if is_default:
|
||||
return None
|
||||
try:
|
||||
return _session_user(request, db, *resolved_user_id(request, strict=False))
|
||||
except HTTPException:
|
||||
# Both the pinned account AND the default one were refused, so the tab is signed out
|
||||
# without being told why — and it got here from a session that named a real account.
|
||||
# A session that named an account, refused: the browser is signed out without being told
|
||||
# why, so say it here.
|
||||
log.info(
|
||||
"a session was resolved to nobody: neither the named nor the default account held",
|
||||
"a session was refused: its account no longer holds",
|
||||
extra=logs.event("auth.session.probe_rejected", user_id=user_id),
|
||||
)
|
||||
return None
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
"""Catalog's tables: the shared YouTube world every user reads — channels and their videos — and
|
||||
each user's footprint in it: what they subscribe to, block, explored or found by search. Only this
|
||||
unit writes them; another unit reads them along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
Computed,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.models import User
|
||||
|
||||
|
||||
class Channel(Base, TimestampMixin):
|
||||
"""A YouTube channel. Shared across all users (one channel's videos are the same
|
||||
for everyone), so its expensive metadata is fetched and stored only once."""
|
||||
|
||||
__tablename__ = "channels"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(32), primary_key=True) # "UC..." channel id
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
handle: Mapped[str | None] = mapped_column(String(255))
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
uploads_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
subscriber_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
video_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Channel "About" metadata (fetched with the same channels.list call, no extra quota):
|
||||
# lifetime view count, the channel's join date, banner image, and external links.
|
||||
total_view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
banner_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
external_links: Mapped[list | None] = mapped_column(JSON)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
country: Mapped[str | None] = mapped_column(String(8))
|
||||
# Creator keyword tags (brandingSettings.channel.keywords) — a single space-separated string,
|
||||
# multi-word tags quoted; the client parses it into chips for the About tab.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
# Sync bookkeeping.
|
||||
details_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
recent_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_rss_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# When the newest uploads were last read through the Data API because the RSS feed would not
|
||||
# answer (`sync/runner.py` `run_feed_fallback`): the once-per-`rss_fallback_hours` clock.
|
||||
feed_fallback_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
backfill_cursor: Mapped[str | None] = mapped_column(String(128))
|
||||
backfill_done: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this channel only entered the catalog because a live YouTube search
|
||||
# surfaced one of its videos (nobody subscribes to it / it has no organic upload sync).
|
||||
# Lets the Library hide search-only "noise" by default.
|
||||
from_search: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
# True when this channel exists only because a user EXPLORED it (opened its page without
|
||||
# subscribing) — an ephemeral, per-explorer addition. The cleanup job hard-deletes such a
|
||||
# channel (and its via_explore videos) once nobody keeps it (no subscription, no live
|
||||
# ExploredChannel row, no per-user state) after a grace period. Cleared when someone
|
||||
# subscribes (the "keep" signal promotes it to organic).
|
||||
from_explore: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
|
||||
videos: Mapped[list["Video"]] = relationship(back_populates="channel")
|
||||
subscriptions: Mapped[list["Subscription"]] = relationship(back_populates="channel")
|
||||
|
||||
|
||||
class Subscription(Base, TimestampMixin):
|
||||
"""Per-user link to a channel, with the user's own priority/visibility overrides."""
|
||||
|
||||
__tablename__ = "subscriptions"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# YouTube's subscription resource id, needed to unsubscribe via the API.
|
||||
yt_subscription_id: Mapped[str | None] = mapped_column(String(128))
|
||||
priority: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
hidden: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Per-user opt-in to full-history (deep) backfill of this channel. Default off so a
|
||||
# new user's unique channels don't trigger a big shared-quota burst; the deep
|
||||
# scheduler only picks up channels at least one user has requested.
|
||||
deep_requested: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
subscribed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
user: Mapped["User"] = relationship(back_populates="subscriptions")
|
||||
channel: Mapped["Channel"] = relationship(back_populates="subscriptions")
|
||||
|
||||
|
||||
# Video.live_status values that mean "currently a live/upcoming broadcast" — transient states
|
||||
# hidden from feeds/search and revisited by the refresh job until they settle into a VOD.
|
||||
LIVE_OR_UPCOMING = ("live", "upcoming")
|
||||
|
||||
|
||||
class Video(Base, TimestampMixin):
|
||||
"""A YouTube video. Shared across users; per-user state lives elsewhere."""
|
||||
|
||||
__tablename__ = "videos"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(16), primary_key=True) # video id
|
||||
# No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
|
||||
# (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
|
||||
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
|
||||
# sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
|
||||
# Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
|
||||
# Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
|
||||
channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
|
||||
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
# NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
|
||||
# forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
|
||||
# that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
|
||||
# PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
|
||||
# the real date); the ix_videos_published_at index stays for the `oldest` sort.
|
||||
published_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
|
||||
)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
# The feed's views / duration / title sorts each ride a migration-defined index in their exact
|
||||
# ORDER BY, one per direction (and per ICU collation for title): migration 0096, held to
|
||||
# `feedquery.order_by` by test_feed_sort_indexes.
|
||||
duration_seconds: Mapped[int | None] = mapped_column(Integer)
|
||||
view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
like_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
category_id: Mapped[int | None] = mapped_column(Integer)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
detected_language: Mapped[str | None] = mapped_column(String(16))
|
||||
# Creator-supplied keyword tags (snippet.tags joined by spaces) — fetched for free with the
|
||||
# snippet we already request. Indexed into search_vector so the feed search matches the
|
||||
# uploader's own keywords, not just words in the title.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
# Accumulated distinct live-search queries that surfaced this video (across all users) —
|
||||
# folds YouTube's own relevance judgement into the local search: a video YouTube returned
|
||||
# for "ti amo magyarul" becomes findable by that query locally even if its title lacks it.
|
||||
search_terms: Mapped[str | None] = mapped_column(Text)
|
||||
# No btree here or on the flag columns below: migration 0069 (PERF-3) dropped the never-picked
|
||||
# boolean btrees and replaced the useful ones with partial indexes over the target subset
|
||||
# (via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — migration-defined, like the
|
||||
# search_vector GIN.
|
||||
is_short: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Whether the youtube.com/shorts/<id> probe has run for this video.
|
||||
shorts_probed: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video first entered the catalog via a live YouTube search (not an
|
||||
# organic channel/playlist sync). Used to hide search-discovered "noise" from the
|
||||
# Library by default; left False by the organic ingest paths.
|
||||
via_search: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video entered the catalog because a user EXPLORED its (un-subscribed)
|
||||
# channel. Like via_search it's hidden from the shared Library by default, but it's
|
||||
# additionally PER-USER private: a via_explore video is only visible to users who
|
||||
# explored or subscribe to its channel (a gate in the feed query), so one user's
|
||||
# curiosity never floods everyone else's catalog. Left False by the organic ingest paths.
|
||||
via_explore: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# none | live | upcoming | premiere | was_live
|
||||
live_status: Mapped[str] = mapped_column(
|
||||
String(16), default="none", server_default="none", index=True
|
||||
)
|
||||
enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# videos.list?part=status fields (populated during enrichment); used by the
|
||||
# maintenance/validation job to decide whether a video is playable anywhere.
|
||||
embeddable: Mapped[bool | None] = mapped_column(Boolean)
|
||||
privacy_status: Mapped[str | None] = mapped_column(String(16)) # public|unlisted|private
|
||||
upload_status: Mapped[str | None] = mapped_column(
|
||||
String(16)
|
||||
) # processed|failed|rejected|deleted
|
||||
# Maintenance/validation lifecycle. last_checked_at drives the rolling re-validation
|
||||
# (least-recently-checked first); unavailable_since marks a video as currently
|
||||
# unplayable (hidden from the feed immediately, hard-deleted after a grace period);
|
||||
# unavailable_reason records why (deleted|private|paywalled|abandoned|stuck_live).
|
||||
# last_checked_at: no plain btree — migration 0069 (PERF-18) replaced it with the partial
|
||||
# ix_videos_recheck_order (last_checked_at NULLS FIRST WHERE unavailable_since IS NULL) that the
|
||||
# revalidate job's exact ORDER BY rides; unavailable_since is a partial WHERE IS NOT NULL (PERF-3).
|
||||
last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_reason: Mapped[str | None] = mapped_column(String(24))
|
||||
|
||||
# Weighted full-text search document (DB-generated, never written by the app): title (A) >
|
||||
# creator keywords + search queries (B) > truncated description (C). ts_rank honours the
|
||||
# weights so title matches outrank description matches. Backed by a GIN index (migration
|
||||
# 0032). The accent-insensitive `unaccent_simple` config comes from migration 0031.
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(keywords, '') || ' ' || "
|
||||
"coalesce(search_terms, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(description, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
channel: Mapped["Channel"] = relationship(back_populates="videos")
|
||||
|
||||
|
||||
class SearchFind(Base, TimestampMixin):
|
||||
"""Per-user record of a video the user surfaced via their own live YouTube search.
|
||||
|
||||
Lets "videos I searched for" appear in that user's own (Mine) feed and the Mine Source
|
||||
filter — distinct from the GLOBAL `Video.via_search` provenance, which marks a video as
|
||||
search-discovered by *anyone* and drives the shared Library's Source filter."""
|
||||
|
||||
__tablename__ = "search_finds"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_search_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
|
||||
class ExploredChannel(Base, TimestampMixin):
|
||||
"""Per-user record of a channel the user opened the page of WITHOUT subscribing.
|
||||
|
||||
Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
|
||||
user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
|
||||
explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
|
||||
purge: a from_explore channel with no live ExploredChannel row (and no subscription / no
|
||||
per-user state on its videos) is hard-deleted by the cleanup job. Subscribing is the "keep"
|
||||
signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
|
||||
(from TimestampMixin) is the grace clock; a re-visit refreshes it."""
|
||||
|
||||
__tablename__ = "explored_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_explored_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class BlockedChannel(Base, TimestampMixin):
|
||||
"""Per-user channel blocklist. Videos from a blocked channel are excluded from this user's
|
||||
live YouTube search (and not ingested on their behalf) and hidden from their feed / explore
|
||||
views — a personal "never show me this channel again" that doesn't affect other users."""
|
||||
|
||||
__tablename__ = "blocked_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_blocked_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
@@ -0,0 +1,296 @@
|
||||
"""Catalog's reads: channels and videos, and each user's footprint in them (subscriptions, blocks,
|
||||
explorations), for the HTTP layer and for the core code that asks about them.
|
||||
|
||||
A user's row is read through `userscope`, so someone else's is the same answer as a missing one.
|
||||
Catalog does not read another unit's tables: what it needs from Reader (a playlist's channels, a
|
||||
user's tag links) it asks `reader.queries`.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Select, and_, case, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
Subscription,
|
||||
Video,
|
||||
)
|
||||
from app.collation import collated
|
||||
from app.models import User
|
||||
from app.sync.explore import followed_channel
|
||||
from app.userscope import owned_by
|
||||
|
||||
# --- channels --------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def channel(db: Session, channel_id: str) -> Channel | None:
|
||||
return db.get(Channel, channel_id)
|
||||
|
||||
|
||||
def channel_title(db: Session, channel_id: str) -> str | None:
|
||||
return db.scalar(select(Channel.title).where(Channel.id == channel_id))
|
||||
|
||||
|
||||
def channel_titles(db: Session, ids: set[str]) -> dict[str, str]:
|
||||
"""Each channel's title by id; a channel with no title yet is left out."""
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(ids)))
|
||||
return {cid: title for cid, title in rows.tuples() if title is not None}
|
||||
|
||||
|
||||
def channel_aggregates(channel_ids: list[str]) -> Select:
|
||||
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
|
||||
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
|
||||
alone; a test holds that (test_channel_list_index_only)."""
|
||||
return (
|
||||
select(
|
||||
Video.channel_id,
|
||||
func.count(Video.id),
|
||||
func.max(Video.published_at),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.where(Video.channel_id.in_(channel_ids))
|
||||
.group_by(Video.channel_id)
|
||||
)
|
||||
|
||||
|
||||
def aggregates(db: Session, channel_ids: list[str]) -> list[tuple[str, int, datetime | None, int]]:
|
||||
"""`channel_aggregates`, run: (channel id, stored videos, last upload, total seconds)."""
|
||||
if not channel_ids:
|
||||
return []
|
||||
return list(db.execute(channel_aggregates(channel_ids)).tuples())
|
||||
|
||||
|
||||
def video_counts(db: Session, channel_id: str) -> tuple[int, int, int]:
|
||||
"""A channel's stored videos, its Shorts and its live/upcoming ones, in one grouped pass."""
|
||||
n_all, n_short, n_live = db.execute(
|
||||
select(
|
||||
func.count(Video.id),
|
||||
func.count(Video.id).filter(Video.is_short.is_(True)),
|
||||
func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
|
||||
).where(Video.channel_id == channel_id)
|
||||
).one()
|
||||
return int(n_all or 0), int(n_short or 0), int(n_live or 0)
|
||||
|
||||
|
||||
def deep_queued(db: Session, channel_ids: list[str]) -> set[str]:
|
||||
"""Which of these channels at least one user (any user) has asked full history for."""
|
||||
if not channel_ids:
|
||||
return set()
|
||||
return set(
|
||||
db.execute(
|
||||
select(Subscription.channel_id)
|
||||
.where(
|
||||
Subscription.channel_id.in_(channel_ids),
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.distinct()
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def followed(db: Session, channel_id: str) -> bool:
|
||||
"""Whether anyone subscribes to the channel."""
|
||||
return (
|
||||
db.execute(
|
||||
select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
# --- a user's subscriptions, blocks and explorations ----------------------------------------
|
||||
|
||||
|
||||
def subscriptions(db: Session, user: User, lang: str) -> list[tuple[Subscription, Channel]]:
|
||||
"""The user's subscriptions with their channels, highest priority first, then by title in the
|
||||
reader's alphabet."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Subscription, Channel)
|
||||
.join(Channel, Channel.id == Subscription.channel_id)
|
||||
.where(owned_by(Subscription.user_id, user))
|
||||
.order_by(Subscription.priority.desc(), collated(Channel.title, lang))
|
||||
).tuples()
|
||||
)
|
||||
|
||||
|
||||
def subscription(db: Session, user: User, channel_id: str) -> Subscription | None:
|
||||
return db.execute(
|
||||
select(Subscription).where(
|
||||
owned_by(Subscription.user_id, user), Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def subscribed_channels(user: User):
|
||||
"""The channel ids the user subscribes to, as a subquery."""
|
||||
return select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
|
||||
|
||||
|
||||
def exploration(db: Session, user: User, channel_id: str) -> ExploredChannel | None:
|
||||
return db.execute(
|
||||
select(ExploredChannel).where(
|
||||
owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def is_blocked(db: Session, user: User, channel_id: str) -> bool:
|
||||
return (
|
||||
db.execute(
|
||||
select(BlockedChannel.id).where(
|
||||
owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def blocked_ids(db: Session, user: User) -> set[str]:
|
||||
return set(
|
||||
db.scalars(select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user)))
|
||||
)
|
||||
|
||||
|
||||
def blocked_channels(db: Session, user: User, lang: str) -> list[Channel]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel)
|
||||
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
|
||||
.where(owned_by(BlockedChannel.user_id, user))
|
||||
.order_by(collated(Channel.title, lang))
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- sync progress ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def catalog_totals(db: Session, user: User) -> dict:
|
||||
"""The instance's catalog size and this user's subscription count (the sync status card)."""
|
||||
return {
|
||||
"subscriptions": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Subscription)
|
||||
.where(owned_by(Subscription.user_id, user))
|
||||
),
|
||||
"channels_total": db.scalar(select(func.count()).select_from(Channel)),
|
||||
"channels_backfilling": db.scalar(
|
||||
select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
|
||||
),
|
||||
"videos_total": db.scalar(select(func.count()).select_from(Video)),
|
||||
"pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def my_progress(db: Session, user: User) -> tuple[int, int, int, int]:
|
||||
"""Over the channels this user subscribes to: how many, how many have details, how many their
|
||||
recent uploads, how many their whole history."""
|
||||
total, details_synced, recent_synced, all_deep_done = db.execute(
|
||||
select(
|
||||
func.count(Channel.id),
|
||||
func.count(Channel.details_synced_at),
|
||||
func.count(Channel.recent_synced_at),
|
||||
func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
|
||||
)
|
||||
.select_from(Channel)
|
||||
.join(
|
||||
Subscription,
|
||||
and_(Subscription.channel_id == Channel.id, owned_by(Subscription.user_id, user)),
|
||||
)
|
||||
).one()
|
||||
return total or 0, details_synced or 0, recent_synced or 0, int(all_deep_done or 0)
|
||||
|
||||
|
||||
def my_deep_requested(db: Session, user: User) -> list[Channel]:
|
||||
"""The channels this user asked full history for."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Channel.id,
|
||||
owned_by(Subscription.user_id, user),
|
||||
Subscription.deep_requested.is_(True),
|
||||
),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def my_video_count(db: Session, user: User) -> int:
|
||||
"""The videos of the channels this user subscribes to and has not hidden."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(Video.id)).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
owned_by(Subscription.user_id, user),
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def video_total(db: Session) -> int:
|
||||
return db.scalar(select(func.count(Video.id))) or 0
|
||||
|
||||
|
||||
def deep_backlog(db: Session) -> list[Channel]:
|
||||
"""Every channel someone asked full history for that does not have it yet."""
|
||||
deep_requested = (
|
||||
select(Subscription.id)
|
||||
.where(Subscription.channel_id == Channel.id, Subscription.deep_requested.is_(True))
|
||||
.exists()
|
||||
)
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel).where(Channel.backfill_done.is_(False), deep_requested)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def work_queue(db: Session) -> dict:
|
||||
"""What the catalog's jobs still have queued, instance-wide (the scheduler card)."""
|
||||
return {
|
||||
# The same selection run_recent_backfill works through, or the card counts channels the
|
||||
# job will never touch and never reaches zero.
|
||||
"channels_recent_pending": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Channel)
|
||||
.where(Channel.recent_synced_at.is_(None), followed_channel())
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_shorts": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_live_refresh": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(
|
||||
or_(
|
||||
Video.live_status.in_(("live", "upcoming")),
|
||||
and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
|
||||
)
|
||||
)
|
||||
)
|
||||
or 0,
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Live YouTube search, the catalog's side: what a page of results leaves in the shared catalog.
|
||||
|
||||
A result becomes a catalog row so it renders with the normal feed cards and the in-app player and
|
||||
gains per-user state. The ingest runs in steps with a YouTube call between each, and each step's
|
||||
rows are committed before the next call: an enrich that fails rolls back to the stubs, which stay
|
||||
(the FK targets for what follows, backfilled later). Results are flagged (`videos.via_search`,
|
||||
`channels.from_search`) so the Library can hide this search-discovered noise; Shorts and
|
||||
live/upcoming videos are never kept. The HTTP layer (`routes/search.py`) runs the paging, the caps
|
||||
and the refusals.
|
||||
"""
|
||||
|
||||
from collections.abc import Sequence
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete, func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, quota, sysconfig
|
||||
from app.catalog import queries
|
||||
from app.catalog.models import LIVE_OR_UPCOMING, Channel, SearchFind, Subscription, Video
|
||||
from app.models import User
|
||||
from app.reader import queries as reader_queries
|
||||
from app.sync.subscriptions import apply_channel_details
|
||||
from app.sync.videos import apply_video_details, insert_stubs, parse_dt
|
||||
from app.userscope import owned_by
|
||||
from app.utils import like_escape
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def classify_shorts(db: Session, videos: Sequence[Video]) -> None:
|
||||
"""Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
|
||||
|
||||
Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
|
||||
synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
|
||||
duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
|
||||
quota). Undetermined probes are left for the scheduler to retry."""
|
||||
probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
|
||||
to_probe: list[Video] = []
|
||||
for v in videos:
|
||||
if v.shorts_probed:
|
||||
continue
|
||||
if v.enriched_at is None:
|
||||
# Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
|
||||
# so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
|
||||
# (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
|
||||
continue
|
||||
if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
|
||||
v.is_short = False
|
||||
v.shorts_probed = True
|
||||
else:
|
||||
to_probe.append(v)
|
||||
if to_probe:
|
||||
with make_client() as client:
|
||||
|
||||
def work(v: Video):
|
||||
return v, probe_is_short(client, v.id)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
for v, result in pool.map(work, to_probe):
|
||||
if result is None:
|
||||
continue
|
||||
v.is_short = result
|
||||
v.shorts_probed = True
|
||||
db.commit()
|
||||
|
||||
|
||||
def ingest_candidates(
|
||||
db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
|
||||
) -> list[str]:
|
||||
"""Materialise one page of search candidates into the catalog (channel stubs + enrich,
|
||||
video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
|
||||
video ids — non-Short, non-live — in the input relevance order."""
|
||||
ids = [it["id"] for it in candidates]
|
||||
if not ids:
|
||||
return []
|
||||
|
||||
# 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
|
||||
# then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
|
||||
channel_ids = list({it["channel_id"] for it in candidates})
|
||||
existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
|
||||
new_channel_ids = [c for c in channel_ids if c not in existing]
|
||||
if new_channel_ids:
|
||||
titles = {it["channel_id"]: it["channel_title"] for it in candidates}
|
||||
for cid in new_channel_ids:
|
||||
db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
|
||||
db.commit()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
|
||||
details = yt.get_channels(new_channel_ids)
|
||||
apply_channel_details(db, details, new_channel_ids)
|
||||
db.commit()
|
||||
except YouTubeError:
|
||||
# The stubs remain and the details can be backfilled later — but until they are, those
|
||||
# channels sit in the results with no thumbnail and no subscriber count, looking like
|
||||
# channels that have neither.
|
||||
log.warning(
|
||||
"the channels found by a search could not be enriched",
|
||||
extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
|
||||
)
|
||||
db.rollback() # details can be backfilled later; the stubs remain
|
||||
|
||||
# 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
|
||||
# NOTHING), so an organically-synced video stays organic.
|
||||
# published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
|
||||
# (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
|
||||
# enrich in step 3 overwrites it with the video's real snippet date.
|
||||
now = datetime.now(timezone.utc)
|
||||
insert_stubs(
|
||||
db,
|
||||
[
|
||||
{
|
||||
"id": it["id"],
|
||||
"channel_id": it["channel_id"],
|
||||
"title": it["title"],
|
||||
"published_at": parse_dt(it["published_at"]) or now,
|
||||
"thumbnail_url": it["thumbnail_url"],
|
||||
"via_search": True,
|
||||
}
|
||||
for it in candidates
|
||||
],
|
||||
)
|
||||
|
||||
# 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
|
||||
videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
|
||||
detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
|
||||
except YouTubeError:
|
||||
# Every candidate then stays unenriched, and an unenriched video cannot be judged live or
|
||||
# Short — so it is dropped. A working search returns nothing and reads as "no results".
|
||||
log.warning(
|
||||
"the videos found by a search could not be enriched",
|
||||
extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
|
||||
)
|
||||
detail_by_id = {}
|
||||
enriched_at = datetime.now(
|
||||
timezone.utc
|
||||
) # when THIS enrich ran (distinct from the stub placeholder)
|
||||
for v in videos:
|
||||
item = detail_by_id.get(v.id)
|
||||
if item is None:
|
||||
continue
|
||||
apply_video_details(v, item)
|
||||
v.enriched_at = enriched_at
|
||||
db.commit()
|
||||
|
||||
# 4) Confirm/deny Shorts (no quota) so none enter via search.
|
||||
classify_shorts(db, videos)
|
||||
|
||||
keep = {
|
||||
v.id
|
||||
for v in videos
|
||||
if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
|
||||
}
|
||||
return [vid for vid in ids if vid in keep]
|
||||
|
||||
|
||||
def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
|
||||
"""Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
|
||||
|
||||
"Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
|
||||
video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
|
||||
wildcard — either way the term was never folded in, and local search lost that relevance."""
|
||||
padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.id.in_(video_ids))
|
||||
.where(padded.notilike(f"% {like_escape(term)} %"))
|
||||
.values(
|
||||
search_terms=func.trim(
|
||||
func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def new_first(db: Session, user: User, ids: list[str]) -> list[str]:
|
||||
"""Reorder results so genuinely-new discoveries come first and things the user already has
|
||||
(a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
|
||||
to the back — keeping the relevance order within each group."""
|
||||
if not ids:
|
||||
return ids
|
||||
have = set(
|
||||
db.scalars(
|
||||
select(Video.id).where(
|
||||
Video.id.in_(ids), Video.channel_id.in_(queries.subscribed_channels(user))
|
||||
)
|
||||
)
|
||||
) | reader_queries.held_by(db, user, ids)
|
||||
return [i for i in ids if i not in have] + [i for i in ids if i in have]
|
||||
|
||||
|
||||
def record_finds(db: Session, user: User, video_ids: list[str], term: str) -> None:
|
||||
"""These results as THIS user's search finds (the Mine feed, the Source filter), and the term
|
||||
folded into each one's `search_terms` so local full-text search inherits YouTube's relevance.
|
||||
Idempotent."""
|
||||
db.execute(
|
||||
pg_insert(SearchFind)
|
||||
.values([{"user_id": user.id, "video_id": vid} for vid in video_ids])
|
||||
.on_conflict_do_nothing(index_elements=["user_id", "video_id"])
|
||||
)
|
||||
_fold_term(db, video_ids, term)
|
||||
|
||||
|
||||
def clear(db: Session, user: User, video_ids: list[str]) -> int:
|
||||
"""Discard these results "as if never added": drop this user's finds of them, then delete the
|
||||
ones nobody kept — a `via_search` video no reader holds (`reader.queries.kept_by_a_reader`),
|
||||
whose channel nobody subscribes to, and that no other user still has as a find. Returns how
|
||||
many videos went."""
|
||||
db.execute(
|
||||
delete(SearchFind).where(
|
||||
owned_by(SearchFind.user_id, user), SearchFind.video_id.in_(video_ids)
|
||||
)
|
||||
)
|
||||
subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
result = db.execute(
|
||||
delete(Video).where(
|
||||
Video.id.in_(video_ids),
|
||||
Video.via_search.is_(True),
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~subscribed,
|
||||
~other_find,
|
||||
)
|
||||
)
|
||||
return result.rowcount if isinstance(result, CursorResult) else 0
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Catalog's writes a request makes: a user's subscription settings, what they explore or block, a
|
||||
subscription recorded or dropped, a channel's sync markers reset.
|
||||
|
||||
Every function here flushes and leaves the commit to its caller (the handler). The YouTube side of a
|
||||
subscribe or an unsubscribe is the handler's (it needs the user's client and its refusals); what
|
||||
lands here is the local record of it. The ingest jobs (`sync/`, `catalog/search`) write the
|
||||
catalog's shared rows themselves.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete, update
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog import queries
|
||||
from app.catalog.models import BlockedChannel, Channel, ExploredChannel, Subscription, Video
|
||||
from app.models import User
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
def remember_own_channel(user: User, channel_id: str) -> None:
|
||||
"""The user's own YouTube channel, so discovery never offers it to them."""
|
||||
user.yt_channel_id = channel_id
|
||||
|
||||
|
||||
def update_subscription(
|
||||
db: Session,
|
||||
sub: Subscription,
|
||||
*,
|
||||
priority: int | None = None,
|
||||
hidden: bool | None = None,
|
||||
deep_requested: bool | None = None,
|
||||
) -> bool:
|
||||
"""Change the settings given (None leaves one as it is). Returns whether full history was just
|
||||
turned ON — the caller then pulls the channel now rather than at the scheduler's next tick."""
|
||||
turned_on = False
|
||||
if priority is not None:
|
||||
sub.priority = priority
|
||||
if hidden is not None:
|
||||
sub.hidden = hidden
|
||||
if deep_requested is not None:
|
||||
# Turning it off won't undo already-fetched videos; it stops further deep paging if nobody
|
||||
# else still wants it.
|
||||
turned_on = deep_requested and not sub.deep_requested
|
||||
sub.deep_requested = deep_requested
|
||||
db.flush()
|
||||
return turned_on
|
||||
|
||||
|
||||
def set_deep_all(db: Session, user: User, on: bool) -> int:
|
||||
"""Every one of the user's channels in (or out of) full-history backfill; how many rows moved."""
|
||||
result = db.execute(
|
||||
update(Subscription).where(owned_by(Subscription.user_id, user)).values(deep_requested=on)
|
||||
)
|
||||
return result.rowcount if isinstance(result, CursorResult) else 0
|
||||
|
||||
|
||||
def explore(db: Session, user: User, channel: Channel) -> None:
|
||||
"""Record (or refresh the grace clock of) the user's exploration of a channel, and flag the
|
||||
channel ephemeral unless someone follows it — the cleanup job's keep rule reads both."""
|
||||
existing = queries.exploration(db, user, channel.id)
|
||||
if existing is None:
|
||||
db.add(ExploredChannel(user_id=user.id, channel_id=channel.id))
|
||||
else:
|
||||
existing.created_at = datetime.now(timezone.utc)
|
||||
if not channel.from_explore and not queries.followed(db, channel.id):
|
||||
channel.from_explore = True
|
||||
db.flush()
|
||||
|
||||
|
||||
def reset_backfill(db: Session, channel: Channel) -> None:
|
||||
"""Fetch the channel again from scratch: its backfill markers cleared, and full history asked
|
||||
for on EVERY subscription to it — the deep scheduler gates on an EXISTS over those, so the admin
|
||||
who resets it need not subscribe themselves."""
|
||||
channel.backfill_done = False
|
||||
channel.backfill_cursor = None
|
||||
channel.recent_synced_at = None
|
||||
db.execute(
|
||||
update(Subscription)
|
||||
.where(Subscription.channel_id == channel.id)
|
||||
.values(deep_requested=True)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def record_subscription(db: Session, user: User, channel: Channel, yt_subscription_id: str) -> None:
|
||||
"""The subscription YouTube now holds, recorded here. Subscribing is the "keep" signal: an
|
||||
explored channel becomes an organic one, so the cleanup job leaves it alone and its videos join
|
||||
the shared Library like any subscribed channel's uploads."""
|
||||
db.add(
|
||||
Subscription(
|
||||
user_id=user.id,
|
||||
channel_id=channel.id,
|
||||
yt_subscription_id=yt_subscription_id or None,
|
||||
subscribed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
if channel.from_explore:
|
||||
channel.from_explore = False
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.channel_id == channel.id, Video.via_explore.is_(True))
|
||||
.values(via_explore=False)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def drop_subscription(db: Session, sub: Subscription) -> None:
|
||||
db.delete(sub)
|
||||
db.flush()
|
||||
|
||||
|
||||
def block(db: Session, user: User, channel_id: str) -> None:
|
||||
"""Block a channel for this user (idempotent), and end any exploration of it so the cleanup job
|
||||
can reclaim it."""
|
||||
if queries.is_blocked(db, user, channel_id):
|
||||
return
|
||||
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
|
||||
db.execute(
|
||||
delete(ExploredChannel).where(
|
||||
owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def unblock(db: Session, user: User, channel_id: str) -> None:
|
||||
db.execute(
|
||||
delete(BlockedChannel).where(
|
||||
owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
|
||||
)
|
||||
)
|
||||
@@ -26,6 +26,7 @@ from uuid import uuid4
|
||||
from sqlalchemy import and_, delete, func, not_, select, update
|
||||
from sqlalchemy.orm import Session, defer
|
||||
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app import diagnostics
|
||||
from app.models import DebugScreenshot, DebugSession, LogEvent, User
|
||||
|
||||
@@ -76,7 +77,9 @@ def open_session(db: Session, user: User) -> DebugSession | None:
|
||||
|
||||
def _unclosed(db: Session, user: User) -> DebugSession | None:
|
||||
return db.scalar(
|
||||
select(DebugSession).where(DebugSession.user_id == user.id, DebugSession.ended_at.is_(None))
|
||||
select(DebugSession).where(
|
||||
owned_by(DebugSession.user_id, user), DebugSession.ended_at.is_(None)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -126,12 +129,8 @@ def from_header(db: Session, user: User, value: str | None) -> DebugSession | No
|
||||
"""The session a request names, if it is the caller's own and still live; else None."""
|
||||
if not value or len(value) != _ID_LEN or not all(c in "0123456789abcdef" for c in value):
|
||||
return None
|
||||
session = db.get(DebugSession, value)
|
||||
if (
|
||||
session is None
|
||||
or session.user_id != user.id
|
||||
or not is_live(session, datetime.now(timezone.utc))
|
||||
):
|
||||
session = owned_row(db, DebugSession, value, user)
|
||||
if session is None or not is_live(session, datetime.now(timezone.utc)):
|
||||
return None
|
||||
return session
|
||||
|
||||
@@ -354,7 +353,7 @@ def stored_screenshot_bytes(db: Session, user_id: int) -> int:
|
||||
db.scalar(
|
||||
select(func.coalesce(func.sum(DebugScreenshot.size), 0))
|
||||
.join(DebugSession, DebugSession.id == DebugScreenshot.session_id)
|
||||
.where(DebugSession.user_id == user_id)
|
||||
.where(owned_by(DebugSession.user_id, user_id))
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
@@ -22,7 +22,8 @@ to the client, which has both the native payload and the translations.
|
||||
"""
|
||||
|
||||
from app.discover.budget import remaining_seconds
|
||||
from app.models import Channel, Playlist
|
||||
from app.catalog.models import Channel
|
||||
from app.reader.models import Playlist
|
||||
|
||||
# Plex artwork is a poster for the things you browse and a landscape still for the things you play.
|
||||
_POSTER_KINDS = ("movie", "show")
|
||||
|
||||
@@ -22,10 +22,14 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.discover import cards, queries
|
||||
from app.models import Channel, ComfortPin, PlexItem, PlexShow, Playlist, User, Video
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import ComfortPin, Playlist
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import SHOW_NEW, leaf_card, show_card
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.userscope import owned_by, owned_row
|
||||
|
||||
# kind -> the ComfortPin column that holds it. The client's vocabulary and the schema's, in one place.
|
||||
PIN_COLUMNS: dict[str, str] = {
|
||||
@@ -68,11 +72,11 @@ def resolve_ref(db: Session, user: User, kind: str, ref: str) -> tuple[str, obje
|
||||
# such pin target", which is the same answer this function gives for one that does not exist.
|
||||
except ValueError:
|
||||
return None
|
||||
pl = db.get(Playlist, pid)
|
||||
# A playlist is per-user, unlike the other four (which are shared catalog rows whose PRIVACY
|
||||
# lives in the state tables). Someone else's list must read as "no such thing", not as a 403 —
|
||||
# the same reason `owned_or_404` 404s rather than 403s.
|
||||
return (col, pl.id) if pl is not None and pl.user_id == user.id else None
|
||||
pl = owned_row(db, Playlist, pid, user)
|
||||
return (col, pl.id) if pl is not None else None
|
||||
|
||||
|
||||
def reorder(db: Session, user: User, ids: list[int]) -> None:
|
||||
@@ -92,7 +96,7 @@ def reorder(db: Session, user: User, ids: list[int]) -> None:
|
||||
# old slots — and one global lock order means two reorders cannot deadlock.
|
||||
locked = db.execute(
|
||||
select(ComfortPin)
|
||||
.where(ComfortPin.user_id == user.id)
|
||||
.where(owned_by(ComfortPin.user_id, user.id))
|
||||
.order_by(ComfortPin.id)
|
||||
.with_for_update()
|
||||
.execution_options(populate_existing=True)
|
||||
@@ -117,7 +121,7 @@ def list_pins(db: Session, user: User) -> list[ComfortPin]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(ComfortPin)
|
||||
.where(ComfortPin.user_id == user.id)
|
||||
.where(owned_by(ComfortPin.user_id, user.id))
|
||||
.order_by(ComfortPin.position.asc(), ComfortPin.id.asc())
|
||||
)
|
||||
.scalars()
|
||||
@@ -132,17 +136,44 @@ def next_position(db: Session, user: User) -> int:
|
||||
deletes the row, and every FK cascade silently removes a pin when the thing it points at goes
|
||||
away — so past the first hole a count under-counts and would drop the new pin into the MIDDLE of
|
||||
the shelf, or onto a position another pin already holds (nothing in the schema forbids that).
|
||||
Same idiom as `routes/playlists._next_playlist_position`."""
|
||||
Same idiom as a new playlist's place in the rail (`reader.service`)."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(ComfortPin.position), -1)).where(
|
||||
ComfortPin.user_id == user.id
|
||||
owned_by(ComfortPin.user_id, user.id)
|
||||
)
|
||||
)
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def own_pin(db: Session, user: User, pin_id: int) -> ComfortPin | None:
|
||||
return owned_row(db, ComfortPin, pin_id, user)
|
||||
|
||||
|
||||
def pin_of(db: Session, user: User, column: str, value: object) -> ComfortPin | None:
|
||||
"""The user's pin of this thing (`column`/`value` from `resolve_ref`), if it is on the shelf."""
|
||||
return db.execute(
|
||||
select(ComfortPin).where(
|
||||
owned_by(ComfortPin.user_id, user), getattr(ComfortPin, column) == value
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def add(db: Session, user: User, column: str, value: object) -> ComfortPin:
|
||||
"""A new pin at the end of the shelf. A second pin of the same thing (another tab, at the same
|
||||
moment) raises `IntegrityError` at the flush: the unique constraint is the guard."""
|
||||
pin = ComfortPin(user_id=user.id, position=next_position(db, user), **{column: value})
|
||||
db.add(pin)
|
||||
db.flush()
|
||||
return pin
|
||||
|
||||
|
||||
def remove(db: Session, pin: ComfortPin) -> None:
|
||||
db.delete(pin)
|
||||
db.flush()
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Batch:
|
||||
"""What the shelf resolves for every pinned channel and playlist in one go."""
|
||||
|
||||
@@ -6,7 +6,7 @@ Two rules shape this module:
|
||||
**Reuse the query layer, not the route.** The YouTube half calls `feedquery.filtered_query` /
|
||||
`sort_keys`, so the Home surface inherits the feed's own definitions of subscribed, blocked,
|
||||
explore-visible and in-progress for free, and cannot drift from them. The Plex half calls
|
||||
`plex.rollup` and the `plex.cards` serializers rather than `routes/plex/discover.py`: that route
|
||||
`plex.rollup` and the `plex.cards` serializers rather than `plex/discover.py`: that read-model
|
||||
narrows EVERY rail by the Plex sidebar's `LibraryFilters`, which is right for a surface that HAS a
|
||||
sidebar and wrong for a landing page that does not.
|
||||
|
||||
@@ -29,22 +29,17 @@ from app.feedquery import (
|
||||
serialize,
|
||||
sort_keys,
|
||||
)
|
||||
from app.models import (
|
||||
Channel,
|
||||
PlaylistItem,
|
||||
PlexItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import PlaylistItem, VideoState
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import display_title, leaf_card, movie_card, show_card
|
||||
from app.plex.query import apply_show_state_filter, content_libs
|
||||
from app.userscope import owned_by
|
||||
|
||||
# How many rows a single rail may carry — Home's rails and the Plex module's Discover rails alike
|
||||
# (`routes/plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
|
||||
# (`plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
|
||||
# is a rail. One definition, because a comment was all that held two copies in step.
|
||||
RAIL_LIMIT = 20
|
||||
|
||||
@@ -98,7 +93,7 @@ def yt_in_progress(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[tup
|
||||
q = (
|
||||
select(*feed_columns(user, st), st.progress_updated_at.label("progressed_at"))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.join(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.join(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(
|
||||
st.position_seconds > 0,
|
||||
st.status.notin_(("watched", "hidden")),
|
||||
@@ -141,7 +136,7 @@ def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
|
||||
row = db.execute(
|
||||
select(*feed_columns(user, st))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(Video.id == video_id)
|
||||
).first()
|
||||
return serialize(row) if row is not None else None
|
||||
@@ -184,7 +179,7 @@ def playlists_summary(
|
||||
select(*feed_columns(user, st), PlaylistItem.playlist_id.label("_playlist"))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(PlaylistItem.playlist_id.in_(playlist_ids))
|
||||
.order_by(PlaylistItem.playlist_id, PlaylistItem.position.asc())
|
||||
).all()
|
||||
@@ -209,7 +204,7 @@ def _unseen_movie_conds(uid: int):
|
||||
.select_from(PlexState)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
PlexState.user_id == uid,
|
||||
owned_by(PlexState.user_id, uid),
|
||||
or_(
|
||||
PlexState.status.in_(("watched", "hidden")),
|
||||
# Started counts as "not a surprise" too — see the note in `yt_surprise`.
|
||||
@@ -226,7 +221,7 @@ def plex_in_progress(
|
||||
) -> list[tuple[object, dict]]:
|
||||
"""In-progress movies + partially-watched shows as (progressed_at, card).
|
||||
|
||||
Mirrors `routes/plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
|
||||
Mirrors `plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
|
||||
NULL status still counts as started, so `notin_` alone would drop it), and the same shared
|
||||
`rollup.agg_subquery` + `apply_show_state_filter` for shows, so the two rails cannot disagree on
|
||||
what "in progress" means. What it does NOT inherit is that route's `LibraryFilters` narrowing:
|
||||
@@ -237,7 +232,9 @@ def plex_in_progress(
|
||||
if movie_libs:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_libs),
|
||||
@@ -331,7 +328,9 @@ def next_episode(db: Session, uid: int, show_id: int, *, lang: str) -> dict | No
|
||||
)
|
||||
base = (
|
||||
select(PlexItem, PlexState)
|
||||
.outerjoin(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.outerjoin(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
@@ -11,7 +11,8 @@ from sqlalchemy.orm import Session
|
||||
from app import sysconfig
|
||||
from app.discover import cards, queries
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.models import PlexShow, User
|
||||
from app.models import User
|
||||
from app.plex.models import PlexShow
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.rollup import NO_TS
|
||||
|
||||
@@ -75,7 +76,7 @@ def surprise(
|
||||
) -> list[dict]:
|
||||
"""The hero's draw pool: unseen things that fit the budget, from both sources under ONE seed.
|
||||
|
||||
Interleaved rather than concatenated — the same alternation `routes/plex/discover.py` uses for its
|
||||
Interleaved rather than concatenated — the same alternation `plex/discover.py` uses for its
|
||||
movie/show split — so a short pool never comes back all-YouTube or all-Plex just because one side
|
||||
happened to be queried first.
|
||||
|
||||
|
||||
@@ -328,9 +328,14 @@ def create_own(db: Session, user_id: int, name: str) -> FileFolder:
|
||||
return folder
|
||||
|
||||
|
||||
def renamable(folder: FileFolder) -> bool:
|
||||
"""Only a folder of the user's own making takes a name: a source folder's is its origin's."""
|
||||
return folder.kind == FolderKind.OWN
|
||||
|
||||
|
||||
def rename(db: Session, folder: FileFolder, name: str) -> None:
|
||||
"""Rename one of the user's own folders (a source folder's name is its origin's)."""
|
||||
if folder.kind != FolderKind.OWN:
|
||||
if not renamable(folder):
|
||||
raise FolderError("file_folder_not_own")
|
||||
folder.name = name
|
||||
try:
|
||||
|
||||
@@ -28,7 +28,8 @@ from app.downloads.jobdb import (
|
||||
)
|
||||
from app.downloads.stated import StatedFailure
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DISPLAY_TEXT_MAX, PlexItem
|
||||
from app.models import DISPLAY_TEXT_MAX
|
||||
from app.plex.models import PlexItem
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
from app.plex import lookups as plex_lookups
|
||||
from app.plex import paths as plex_paths
|
||||
|
||||
@@ -296,13 +296,11 @@ def own_link(db: Session, user: User, link_id: int) -> DownloadLink | None:
|
||||
if link is None:
|
||||
return None
|
||||
target = (
|
||||
db.get(DownloadJob, link.job_id)
|
||||
owned_row(db, DownloadJob, link.job_id, user)
|
||||
if link.job_id is not None
|
||||
else db.get(FileFolder, link.folder_id)
|
||||
else owned_row(db, FileFolder, link.folder_id, user)
|
||||
)
|
||||
if target is None or target.user_id != user.id:
|
||||
return None
|
||||
return link
|
||||
return link if target is not None else None
|
||||
|
||||
|
||||
def links_to(
|
||||
|
||||
@@ -16,6 +16,7 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import app.tables # noqa: F401 — a fresh interpreter: every table, before the first query
|
||||
from app import logs
|
||||
from app import logsink
|
||||
from app.config import settings
|
||||
|
||||
@@ -24,7 +24,9 @@ from app.downloads import credentials
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import folders, formats, mediakind, quota, storage
|
||||
from app.downloads.status import HOLDING, JobKind, JobStatus
|
||||
from app.models import DISPLAY_TEXT_MAX, Channel, PlexItem, PlexShow, User, Video
|
||||
from app.models import DISPLAY_TEXT_MAX, User
|
||||
from app.plex.models import PlexItem, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.downloads.models import (
|
||||
DownloadJob,
|
||||
DownloadProfile,
|
||||
|
||||
+117
-24
@@ -1,10 +1,11 @@
|
||||
"""Feed read-model: the pure query-building and row-serialization layer behind the feed.
|
||||
"""Feed read-model: Reader's query-building, paging and row serialization behind the feed.
|
||||
|
||||
Extracted from `routes/feed.py` so the read-model has one home instead of living inside the
|
||||
router module — the feed route handlers and the playlist-detail query share these helpers (the
|
||||
projection, the filter builder, and keyset paging). Nothing here defines a route or reaches for
|
||||
request state; it's all `(db, user, params) -> query`, and a malformed cursor raises a plain
|
||||
`ValueError` for the route handler to translate (the tsquery sanitizer lives in `app.utils`).
|
||||
One home for the read-model rather than the router module: the feed routes, the playlist-detail
|
||||
query and Home share these helpers (the projection, the filter builder, and keyset paging), and the
|
||||
feed's routes only validate and answer — a page, its counts and its facets are built here. Nothing
|
||||
here defines a route or reaches for request state; it's all `(db, user, params) -> query`, and a
|
||||
malformed cursor raises `BadCursor` for the route handler to translate (the tsquery sanitizer lives
|
||||
in `app.utils`).
|
||||
"""
|
||||
|
||||
import base64
|
||||
@@ -16,22 +17,19 @@ from sqlalchemy import BigInteger, Select, and_, case, cast, false, func, or_, s
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.collation import collated
|
||||
from app.models import (
|
||||
from app.models import User
|
||||
from app.catalog.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ChannelTag,
|
||||
ExploredChannel,
|
||||
Playlist,
|
||||
PlaylistItem,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
Tag,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, Tag, VideoState
|
||||
from app.utils import iso, like_escape, to_tsquery_str
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
def _channel_url(channel_id: str, handle: str | None) -> str:
|
||||
@@ -48,7 +46,7 @@ def _saved_expr(user: User):
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(
|
||||
Playlist.user_id == user.id,
|
||||
owned_by(Playlist.user_id, user.id),
|
||||
Playlist.kind == "watch_later",
|
||||
PlaylistItem.video_id == Video.id,
|
||||
)
|
||||
@@ -103,7 +101,7 @@ def serialize(row) -> dict:
|
||||
|
||||
def _tag_visible(user: User):
|
||||
"""A ChannelTag row this user may see: a shared system tag (user_id NULL) or their own."""
|
||||
return or_(ChannelTag.user_id.is_(None), ChannelTag.user_id == user.id)
|
||||
return or_(ChannelTag.user_id.is_(None), owned_by(ChannelTag.user_id, user.id))
|
||||
|
||||
|
||||
# The pseudo-category of a selected tag id no Tag row answers (a saved view or the feed state outlives
|
||||
@@ -122,7 +120,7 @@ def tags_by_category(db: Session, user: User, tag_ids: list[int]) -> dict[str, l
|
||||
if not tag_ids:
|
||||
return {} # skip the pointless `WHERE id IN ()` round-trip on the no-tag-filter hot path
|
||||
out: dict[str, list[int]] = {}
|
||||
visible = or_(Tag.user_id.is_(None), Tag.user_id == user.id)
|
||||
visible = or_(Tag.user_id.is_(None), owned_by(Tag.user_id, user.id))
|
||||
for tid, cat in db.execute(
|
||||
select(Tag.id, Tag.category).where(Tag.id.in_(tag_ids), visible)
|
||||
).all():
|
||||
@@ -142,7 +140,7 @@ def not_blocked_cond(user: User):
|
||||
position — and the one query that copied only PART of this predicate set is exactly how that
|
||||
video came back as the landing page's suggestion after the user had said "never again"."""
|
||||
return Video.channel_id.notin_(
|
||||
select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)
|
||||
select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user.id))
|
||||
)
|
||||
|
||||
|
||||
@@ -187,7 +185,7 @@ def filtered_query(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
owned_by(Subscription.user_id, user.id),
|
||||
),
|
||||
)
|
||||
else:
|
||||
@@ -199,12 +197,12 @@ def filtered_query(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
owned_by(Subscription.user_id, user.id),
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
).outerjoin(
|
||||
SearchFind,
|
||||
and_(SearchFind.video_id == Video.id, SearchFind.user_id == user.id),
|
||||
and_(SearchFind.video_id == Video.id, owned_by(SearchFind.user_id, user.id)),
|
||||
)
|
||||
|
||||
# The state-gated views (watched/hidden/in_progress) only ever return videos that HAVE a
|
||||
@@ -214,7 +212,7 @@ def filtered_query(
|
||||
# exactly. 'unwatched'/'all' include row-less 'new' videos, so they keep the outer join.
|
||||
state_required = show in ("watched", "hidden", "in_progress")
|
||||
join_state = query.join if state_required else query.outerjoin
|
||||
query = join_state(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
query = join_state(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
|
||||
# Hide videos the maintenance job has flagged as currently unplayable (deleted/private/
|
||||
# abandoned). They're either deleted after a grace period or unhidden if they recover;
|
||||
@@ -227,8 +225,8 @@ def filtered_query(
|
||||
# catalog. Applied in BOTH scopes; the channel page reaches its ephemeral videos by
|
||||
# asking for scope=all + channel_ids (the channel is in this accessible set).
|
||||
accessible_explore = (
|
||||
select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
).union(select(ExploredChannel.channel_id).where(ExploredChannel.user_id == user.id))
|
||||
select(Subscription.channel_id).where(owned_by(Subscription.user_id, user.id))
|
||||
).union(select(ExploredChannel.channel_id).where(owned_by(ExploredChannel.user_id, user.id)))
|
||||
query = query.where(
|
||||
or_(
|
||||
Video.via_explore.is_(False),
|
||||
@@ -346,7 +344,7 @@ def filtered_query(
|
||||
if untagged:
|
||||
query = query.where(
|
||||
Video.channel_id.notin_(
|
||||
select(ChannelTag.channel_id).where(ChannelTag.user_id == user.id)
|
||||
select(ChannelTag.channel_id).where(owned_by(ChannelTag.user_id, user.id))
|
||||
)
|
||||
)
|
||||
|
||||
@@ -696,3 +694,98 @@ def decode_cursor(keys: list[dict], cursor: str) -> tuple[list, str, datetime |
|
||||
# Framework-free: the route handler translates this into an HTTP 400 so the
|
||||
# read-model stays web-layer-agnostic (see routes/feed.get_feed).
|
||||
raise ValueError("Invalid feed cursor") from exc
|
||||
|
||||
|
||||
class BadCursor(ValueError):
|
||||
"""A `cursor` this module did not write (or one from another sort): the caller's bad input."""
|
||||
|
||||
|
||||
def feed_page(
|
||||
db: Session,
|
||||
user: User,
|
||||
params: dict,
|
||||
*,
|
||||
sort: str,
|
||||
seed: int,
|
||||
limit: int,
|
||||
cursor: str | None,
|
||||
lang: str,
|
||||
) -> dict:
|
||||
"""One page of the feed, and the cursor for the next. A malformed cursor raises `BadCursor`."""
|
||||
query, rank_expr = filtered_query(db, user, **params)
|
||||
keys = sort_keys(sort, seed, rank_expr, lang)
|
||||
|
||||
vals, cursor_id, window = None, None, None
|
||||
if cursor:
|
||||
try:
|
||||
vals, cursor_id, window = decode_cursor(keys, cursor)
|
||||
except ValueError as e:
|
||||
raise BadCursor(str(e)) from e
|
||||
ranked = sort == "relevance" and rank_expr is not None
|
||||
if ranked and not cursor:
|
||||
window = rank_window_start(db, query)
|
||||
if ranked and window is not None:
|
||||
keys = sort_keys(sort, seed, windowed_rank(rank_expr, window), lang)
|
||||
|
||||
# Expose each key column on the row so we can build the next cursor from the last item.
|
||||
query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
|
||||
if vals is not None:
|
||||
for k, v in zip(keys, vals, strict=True):
|
||||
k["value"] = v
|
||||
if ranked and window is not None:
|
||||
rows = windowed_relevance_rows(db, query, keys, window, cursor_id, limit + 1)
|
||||
else:
|
||||
query = query.order_by(*order_by(keys))
|
||||
if cursor_id is not None:
|
||||
query = query.where(keyset_predicate(keys, cursor_id))
|
||||
rows = db.execute(query.limit(limit + 1)).all()
|
||||
has_more = len(rows) > limit
|
||||
page = rows[:limit]
|
||||
return {
|
||||
"items": [serialize(r) for r in page],
|
||||
"next_cursor": (
|
||||
encode_cursor(keys, page[-1], window if ranked else None)
|
||||
if (has_more and page)
|
||||
else None
|
||||
),
|
||||
"has_more": has_more,
|
||||
"limit": limit,
|
||||
}
|
||||
|
||||
|
||||
def feed_count(db: Session, user: User, **params) -> int:
|
||||
"""How many videos the filters match — the reference oracle `feed_summary`'s count is tested
|
||||
against."""
|
||||
query, _rank = filtered_query(db, user, **params)
|
||||
return db.scalar(select(func.count()).select_from(query.subquery())) or 0
|
||||
|
||||
|
||||
def facet_counts(db: Session, user: User, **params) -> dict[int, int]:
|
||||
"""Per-tag channel counts for the current filter context, one pass per category — the reference
|
||||
oracle `feed_summary`'s facets are tested against.
|
||||
|
||||
Each category is counted with every other filter applied but its own selections ignored
|
||||
(drill-down faceting: selecting one topic doesn't zero out the rest). A conjunctive (AND)
|
||||
category keeps them applied, so each remaining chip narrows to channels that ALSO have every
|
||||
selected tag, and tags that can't co-occur drop to zero."""
|
||||
visible = _tag_visible(user)
|
||||
counts: dict[int, int] = {}
|
||||
for category in ("language", "topic", "other"):
|
||||
conjunctive = params.get("tag_modes", {}).get(category) == "and"
|
||||
base, _rank = filtered_query(
|
||||
db,
|
||||
user,
|
||||
**{**params, "exclude_tag_category": None if conjunctive else category},
|
||||
)
|
||||
channels = base.with_only_columns(Video.channel_id).distinct().subquery()
|
||||
rows = db.execute(
|
||||
select(ChannelTag.tag_id, func.count(func.distinct(ChannelTag.channel_id)))
|
||||
.select_from(channels)
|
||||
.join(ChannelTag, ChannelTag.channel_id == channels.c.channel_id)
|
||||
.join(Tag, and_(Tag.id == ChannelTag.tag_id, Tag.category == category))
|
||||
.where(visible)
|
||||
.group_by(ChannelTag.tag_id)
|
||||
).all()
|
||||
for tag_id, count in rows:
|
||||
counts[tag_id] = count
|
||||
return counts
|
||||
|
||||
@@ -28,6 +28,7 @@ log = logs.get_logger(__name__)
|
||||
from app import bodylimit, collation, logsink, state, sysconfig # noqa: E402 — logging is configured first
|
||||
from app.config import settings # noqa: E402 — logging is configured first
|
||||
from app.db import SessionLocal, engine # noqa: E402 — logging is configured first
|
||||
import app.tables # noqa: E402 — every table, before the first query
|
||||
from app.downloads import service as download_service # noqa: E402 — logging is configured first
|
||||
from app.downloads import storage # noqa: E402 — logging is configured first
|
||||
from app.plex import stream as plex_stream # noqa: E402 — logging is configured first
|
||||
|
||||
+5
-856
@@ -1,13 +1,12 @@
|
||||
from datetime import date, datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
Computed,
|
||||
Date,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
@@ -19,13 +18,16 @@ from sqlalchemy import (
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship, validates
|
||||
|
||||
from app.db import Base
|
||||
from app.runstatus import RunStatus, RunTrigger, sql_in
|
||||
from app.utils import canonical_email
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.catalog.models import Subscription
|
||||
|
||||
# Every user-supplied display string in this schema shares one width, and it lives HERE so the
|
||||
# columns below are declared with it rather than with a literal: a name a person typed goes into
|
||||
# `User.display_name` (their Google profile name), `DownloadJob.display_name` and
|
||||
@@ -220,301 +222,6 @@ class Invite(Base, CanonicalEmailMixin):
|
||||
lang: Mapped[str | None] = mapped_column(String(8))
|
||||
|
||||
|
||||
class Channel(Base, TimestampMixin):
|
||||
"""A YouTube channel. Shared across all users (one channel's videos are the same
|
||||
for everyone), so its expensive metadata is fetched and stored only once."""
|
||||
|
||||
__tablename__ = "channels"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(32), primary_key=True) # "UC..." channel id
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
handle: Mapped[str | None] = mapped_column(String(255))
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
uploads_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
subscriber_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
video_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Channel "About" metadata (fetched with the same channels.list call, no extra quota):
|
||||
# lifetime view count, the channel's join date, banner image, and external links.
|
||||
total_view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
banner_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
external_links: Mapped[list | None] = mapped_column(JSON)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
country: Mapped[str | None] = mapped_column(String(8))
|
||||
# Creator keyword tags (brandingSettings.channel.keywords) — a single space-separated string,
|
||||
# multi-word tags quoted; the client parses it into chips for the About tab.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
# Sync bookkeeping.
|
||||
details_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
recent_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_rss_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# When the newest uploads were last read through the Data API because the RSS feed would not
|
||||
# answer (`sync/runner.py` `run_feed_fallback`): the once-per-`rss_fallback_hours` clock.
|
||||
feed_fallback_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
backfill_cursor: Mapped[str | None] = mapped_column(String(128))
|
||||
backfill_done: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this channel only entered the catalog because a live YouTube search
|
||||
# surfaced one of its videos (nobody subscribes to it / it has no organic upload sync).
|
||||
# Lets the Library hide search-only "noise" by default.
|
||||
from_search: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
# True when this channel exists only because a user EXPLORED it (opened its page without
|
||||
# subscribing) — an ephemeral, per-explorer addition. The cleanup job hard-deletes such a
|
||||
# channel (and its via_explore videos) once nobody keeps it (no subscription, no live
|
||||
# ExploredChannel row, no per-user state) after a grace period. Cleared when someone
|
||||
# subscribes (the "keep" signal promotes it to organic).
|
||||
from_explore: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
|
||||
videos: Mapped[list["Video"]] = relationship(back_populates="channel")
|
||||
subscriptions: Mapped[list["Subscription"]] = relationship(back_populates="channel")
|
||||
|
||||
|
||||
class Subscription(Base, TimestampMixin):
|
||||
"""Per-user link to a channel, with the user's own priority/visibility overrides."""
|
||||
|
||||
__tablename__ = "subscriptions"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# YouTube's subscription resource id, needed to unsubscribe via the API.
|
||||
yt_subscription_id: Mapped[str | None] = mapped_column(String(128))
|
||||
priority: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
hidden: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Per-user opt-in to full-history (deep) backfill of this channel. Default off so a
|
||||
# new user's unique channels don't trigger a big shared-quota burst; the deep
|
||||
# scheduler only picks up channels at least one user has requested.
|
||||
deep_requested: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
subscribed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
user: Mapped["User"] = relationship(back_populates="subscriptions")
|
||||
channel: Mapped["Channel"] = relationship(back_populates="subscriptions")
|
||||
|
||||
|
||||
# Video.live_status values that mean "currently a live/upcoming broadcast" — transient states
|
||||
# hidden from feeds/search and revisited by the refresh job until they settle into a VOD.
|
||||
LIVE_OR_UPCOMING = ("live", "upcoming")
|
||||
|
||||
|
||||
class Video(Base, TimestampMixin):
|
||||
"""A YouTube video. Shared across users; per-user state lives elsewhere."""
|
||||
|
||||
__tablename__ = "videos"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(16), primary_key=True) # video id
|
||||
# No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
|
||||
# (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
|
||||
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
|
||||
# sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
|
||||
# Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
|
||||
# Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
|
||||
channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
|
||||
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
# NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
|
||||
# forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
|
||||
# that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
|
||||
# PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
|
||||
# the real date); the ix_videos_published_at index stays for the `oldest` sort.
|
||||
published_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
|
||||
)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
# The feed's views / duration / title sorts each ride a migration-defined index in their exact
|
||||
# ORDER BY, one per direction (and per ICU collation for title): migration 0096, held to
|
||||
# `feedquery.order_by` by test_feed_sort_indexes.
|
||||
duration_seconds: Mapped[int | None] = mapped_column(Integer)
|
||||
view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
like_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
category_id: Mapped[int | None] = mapped_column(Integer)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
detected_language: Mapped[str | None] = mapped_column(String(16))
|
||||
# Creator-supplied keyword tags (snippet.tags joined by spaces) — fetched for free with the
|
||||
# snippet we already request. Indexed into search_vector so the feed search matches the
|
||||
# uploader's own keywords, not just words in the title.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
# Accumulated distinct live-search queries that surfaced this video (across all users) —
|
||||
# folds YouTube's own relevance judgement into the local search: a video YouTube returned
|
||||
# for "ti amo magyarul" becomes findable by that query locally even if its title lacks it.
|
||||
search_terms: Mapped[str | None] = mapped_column(Text)
|
||||
# No btree here or on the flag columns below: migration 0069 (PERF-3) dropped the never-picked
|
||||
# boolean btrees and replaced the useful ones with partial indexes over the target subset
|
||||
# (via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — migration-defined, like the
|
||||
# search_vector GIN.
|
||||
is_short: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Whether the youtube.com/shorts/<id> probe has run for this video.
|
||||
shorts_probed: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video first entered the catalog via a live YouTube search (not an
|
||||
# organic channel/playlist sync). Used to hide search-discovered "noise" from the
|
||||
# Library by default; left False by the organic ingest paths.
|
||||
via_search: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video entered the catalog because a user EXPLORED its (un-subscribed)
|
||||
# channel. Like via_search it's hidden from the shared Library by default, but it's
|
||||
# additionally PER-USER private: a via_explore video is only visible to users who
|
||||
# explored or subscribe to its channel (a gate in the feed query), so one user's
|
||||
# curiosity never floods everyone else's catalog. Left False by the organic ingest paths.
|
||||
via_explore: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# none | live | upcoming | premiere | was_live
|
||||
live_status: Mapped[str] = mapped_column(
|
||||
String(16), default="none", server_default="none", index=True
|
||||
)
|
||||
enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# videos.list?part=status fields (populated during enrichment); used by the
|
||||
# maintenance/validation job to decide whether a video is playable anywhere.
|
||||
embeddable: Mapped[bool | None] = mapped_column(Boolean)
|
||||
privacy_status: Mapped[str | None] = mapped_column(String(16)) # public|unlisted|private
|
||||
upload_status: Mapped[str | None] = mapped_column(
|
||||
String(16)
|
||||
) # processed|failed|rejected|deleted
|
||||
# Maintenance/validation lifecycle. last_checked_at drives the rolling re-validation
|
||||
# (least-recently-checked first); unavailable_since marks a video as currently
|
||||
# unplayable (hidden from the feed immediately, hard-deleted after a grace period);
|
||||
# unavailable_reason records why (deleted|private|paywalled|abandoned|stuck_live).
|
||||
# last_checked_at: no plain btree — migration 0069 (PERF-18) replaced it with the partial
|
||||
# ix_videos_recheck_order (last_checked_at NULLS FIRST WHERE unavailable_since IS NULL) that the
|
||||
# revalidate job's exact ORDER BY rides; unavailable_since is a partial WHERE IS NOT NULL (PERF-3).
|
||||
last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_reason: Mapped[str | None] = mapped_column(String(24))
|
||||
|
||||
# Weighted full-text search document (DB-generated, never written by the app): title (A) >
|
||||
# creator keywords + search queries (B) > truncated description (C). ts_rank honours the
|
||||
# weights so title matches outrank description matches. Backed by a GIN index (migration
|
||||
# 0032). The accent-insensitive `unaccent_simple` config comes from migration 0031.
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(keywords, '') || ' ' || "
|
||||
"coalesce(search_terms, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(description, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
channel: Mapped["Channel"] = relationship(back_populates="videos")
|
||||
|
||||
|
||||
class Tag(Base, TimestampMixin):
|
||||
"""A label. System tags (user_id NULL) are computed automatically and shared by all
|
||||
users; user tags (user_id set) are personal. category groups them in the UI."""
|
||||
|
||||
__tablename__ = "tags"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(64))
|
||||
color: Mapped[str | None] = mapped_column(String(16))
|
||||
# language | topic | other
|
||||
category: Mapped[str] = mapped_column(String(16), default="other", server_default="other")
|
||||
|
||||
|
||||
class ChannelTag(Base):
|
||||
"""Links a tag to a channel. source='auto' rows are managed by the auto-tagger and
|
||||
regenerated freely; source='user' rows are never touched by the system."""
|
||||
|
||||
__tablename__ = "channel_tags"
|
||||
__table_args__ = (UniqueConstraint("channel_id", "tag_id", "user_id", name="uq_channel_tag"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
tag_id: Mapped[int] = mapped_column(ForeignKey("tags.id", ondelete="CASCADE"), index=True)
|
||||
# NULL = system/auto link visible to everyone; set = a specific user's link.
|
||||
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
source: Mapped[str] = mapped_column(String(8), default="auto", server_default="auto")
|
||||
confidence: Mapped[float | None] = mapped_column(Float)
|
||||
|
||||
|
||||
class VideoState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a video. Rows exist only for non-default states; the
|
||||
absence of a row means 'new/unseen'."""
|
||||
|
||||
__tablename__ = "video_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
# new | watched | hidden ("saved" is now Watch-later playlist membership, not a status)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Resume position (seconds into the video) for the in-app player. Orthogonal to
|
||||
# status: a partially-watched video keeps status "new" but a non-zero position, so
|
||||
# it can render a progress bar and match the "in progress" feed filter. 0 = not
|
||||
# started / finished (trivially-early and near-finished positions are not stored).
|
||||
# int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class SearchFind(Base, TimestampMixin):
|
||||
"""Per-user record of a video the user surfaced via their own live YouTube search.
|
||||
|
||||
Lets "videos I searched for" appear in that user's own (Mine) feed and the Mine Source
|
||||
filter — distinct from the GLOBAL `Video.via_search` provenance, which marks a video as
|
||||
search-discovered by *anyone* and drives the shared Library's Source filter."""
|
||||
|
||||
__tablename__ = "search_finds"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_search_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
|
||||
class ExploredChannel(Base, TimestampMixin):
|
||||
"""Per-user record of a channel the user opened the page of WITHOUT subscribing.
|
||||
|
||||
Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
|
||||
user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
|
||||
explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
|
||||
purge: a from_explore channel with no live ExploredChannel row (and no subscription / no
|
||||
per-user state on its videos) is hard-deleted by the cleanup job. Subscribing is the "keep"
|
||||
signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
|
||||
(from TimestampMixin) is the grace clock; a re-visit refreshes it."""
|
||||
|
||||
__tablename__ = "explored_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_explored_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class BlockedChannel(Base, TimestampMixin):
|
||||
"""Per-user channel blocklist. Videos from a blocked channel are excluded from this user's
|
||||
live YouTube search (and not ingested on their behalf) and hidden from their feed / explore
|
||||
views — a personal "never show me this channel again" that doesn't affect other users."""
|
||||
|
||||
__tablename__ = "blocked_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_blocked_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class ApiQuotaUsage(Base):
|
||||
"""Tracks YouTube Data API units spent per Pacific-time day (the quota resets at
|
||||
midnight Pacific). The whole app shares one daily budget."""
|
||||
@@ -739,79 +446,6 @@ class SystemConfig(Base):
|
||||
)
|
||||
|
||||
|
||||
class Playlist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user named, ordered collection of videos from the shared catalog.
|
||||
|
||||
`kind`: 'user' (normal) or 'watch_later' (built-in, undeletable, one per user).
|
||||
`source`: 'local' (created here) or 'youtube' (mirrored from the user's YouTube
|
||||
playlists). `yt_playlist_id` links a mirrored/exported playlist to its YouTube id.
|
||||
`dirty` marks a YouTube-linked playlist with local edits not yet pushed, so the read
|
||||
sync won't clobber them. (source/yt_playlist_id/dirty are forward-looking for the YT
|
||||
sync phases; the foundation phase only uses local playlists.)"""
|
||||
|
||||
__tablename__ = "playlists"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
name: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16), default="user", server_default="user")
|
||||
source: Mapped[str] = mapped_column(String(16), default="local", server_default="local")
|
||||
yt_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
# True when the current items/name differ from `synced_fingerprint` (unpushed edits).
|
||||
dirty: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# SHA-256 of name + ordered video ids as last synced from / pushed to YouTube; the
|
||||
# baseline `dirty` is derived against. NULL = no baseline yet (treated as dirty).
|
||||
synced_fingerprint: Mapped[str | None] = mapped_column(String)
|
||||
# JSON list of the ordered video ids as of the last sync — the 3-way-merge baseline the
|
||||
# two-way YouTube sync classifies each item against (local-add vs YT-add vs real delete).
|
||||
# NULL = no baseline yet; the first merge then does a lossless union (never deletes).
|
||||
synced_video_ids: Mapped[str | None] = mapped_column(Text)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
|
||||
items: Mapped[list["PlaylistItem"]] = relationship(
|
||||
back_populates="playlist",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="PlaylistItem.position",
|
||||
)
|
||||
|
||||
|
||||
class PlaylistItem(Base):
|
||||
"""A video's membership in a playlist, with its order within that playlist."""
|
||||
|
||||
__tablename__ = "playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "video_id", name="uq_playlist_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
added_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
|
||||
|
||||
playlist: Mapped["Playlist"] = relationship(back_populates="items")
|
||||
|
||||
|
||||
class SavedView(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user named snapshot of a module's filter/sort state (a "smart view").
|
||||
|
||||
`source` names the module the view belongs to ("feed" | "plex"); `filters` is that module's
|
||||
serialized filter JSON the frontend applies with one click. `position` orders them in the sidebar
|
||||
(drag-reorder, per source); `is_default` marks the one view applied automatically on load — at most
|
||||
one per (user, source), enforced by the API on set. The name is unique per (user, source)."""
|
||||
|
||||
__tablename__ = "saved_views"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("user_id", "source", "name", name="uq_user_source_saved_view"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
source: Mapped[str] = mapped_column(String(16), default="feed", server_default="feed")
|
||||
name: Mapped[str] = mapped_column(String(80))
|
||||
filters: Mapped[dict] = mapped_column(JSON)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
is_default: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class Notification(Base):
|
||||
"""A durable, server-backed per-user notification (the inbox center, phase 1).
|
||||
|
||||
@@ -878,488 +512,3 @@ class ApiTokenGrant(Base):
|
||||
can_read: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
can_write: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
can_delete: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
# --- Plex integration (optional) -------------------------------------------------------------
|
||||
# A dedicated, self-contained catalog mirrored from a locally-reachable Plex server. Playback is
|
||||
# from the LOCAL physical file (Plex is metadata only); per-user watch state lives in plex_states
|
||||
# so it works for users without a Plex account. Mirrors the app's "shared catalog, private
|
||||
# per-user state" model. The weighted FTS search_vector uses the same unaccent_simple config as
|
||||
# `videos` (migrations 0031/0032).
|
||||
|
||||
|
||||
class PlexLibrary(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex library section. Only `enabled` sections are exposed in the feed.
|
||||
`plex_key` is the section id on the Plex server."""
|
||||
|
||||
__tablename__ = "plex_libraries"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
plex_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
title: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16)) # "movie" | "show"
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# HU-overlay support. `metadata_lang` tags the section's metadata language. An OVERLAY
|
||||
# library (overlay_of set) is the Hungarian-metadata twin of another section: it is never browsable
|
||||
# on its own — its titles' HU fields are written onto the matched EN rows (paired by `guid`).
|
||||
metadata_lang: Mapped[str] = mapped_column(String(8), default="en", server_default="en")
|
||||
overlay_of: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="SET NULL"), index=True
|
||||
)
|
||||
|
||||
|
||||
class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A TV show (grandparent of episodes) — holds poster + summary for the drill-down page, plus
|
||||
the same filterable metadata as movies (genres/rating/content_rating/studio/people) so the TV
|
||||
grid can be filtered/sorted like the movie grid. All mirrored cheaply from the show listing."""
|
||||
|
||||
__tablename__ = "plex_shows"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_shows_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
# ix_plex_shows_collections_gin already created with collections (migration 0047).
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# Canonical external id (e.g. "tmdb:12345") — the key that pairs this EN row with its HU-overlay
|
||||
# twin during sync. Indexed for the guid->row overlay match.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(
|
||||
String(512)
|
||||
) # Plex thumb path (proxied, not stored)
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer) # seasons
|
||||
collection_keys: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # Plex collections this show is in
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the show section listing (parallels PlexItem).
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True) # Plex audienceRating (~IMDb)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True) # network/studio
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True) # first air date
|
||||
# none_as_null: an empty tag list must persist as SQL NULL, not JSON `null`. Without it a Python
|
||||
# None round-trips as a JSON scalar that jsonb_array_elements_text() can't unnest ("cannot extract
|
||||
# elements from a scalar") — the trap a metadata-poor library sprang on /facets. Same on plex_items.
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
people_text: Mapped[str | None] = mapped_column(
|
||||
Text
|
||||
) # cast+directors, folded into search_vector
|
||||
# Set once the show's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: see the identical block on plex_items for the full rationale. Additive;
|
||||
# served from the row; NULL until enriched. (`budget`/`revenue` stay NULL for shows — TV has none.) ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched,
|
||||
# so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexSeason(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A season under a show — used to order episodes for prev/next stepping."""
|
||||
|
||||
__tablename__ = "plex_seasons"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
show_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
season_number: Mapped[int | None] = mapped_column(Integer, index=True)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
|
||||
|
||||
class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A playable LEAF — a movie or a single episode. This is what the feed lists and the player
|
||||
plays. Movies carry `library_id` and no parents; episodes carry show/season refs +
|
||||
season/episode numbers. `file_path` is the Plex-side absolute path of the physical media,
|
||||
mapped through `plex_path_map` to a local mount at playback time. `playable` is the
|
||||
ffprobe-derived browser-compatibility class (direct | remux | transcode)."""
|
||||
|
||||
__tablename__ = "plex_items"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_items_show_order", "show_id", "season_number", "episode_number"),
|
||||
Index("ix_plex_items_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_items_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_items_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_items_collections_gin", "collection_keys", postgresql_using="gin"),
|
||||
Index("ix_plex_items_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
kind: Mapped[str] = mapped_column(String(16), index=True) # "movie" | "episode"
|
||||
# Canonical external id (e.g. "tmdb:12345") — pairs a MOVIE row with its HU-overlay twin during
|
||||
# sync; indexed for the guid->row match. Episodes carry it too but it's unused for them.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_seasons.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_number: Mapped[int | None] = mapped_column(Integer)
|
||||
episode_number: Mapped[int | None] = mapped_column(Integer)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
duration_s: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
cast: Mapped[list | None] = mapped_column(JSON) # [role names]
|
||||
# Physical media (from the Plex Part).
|
||||
file_path: Mapped[str | None] = mapped_column(Text) # Plex-side absolute path
|
||||
part_key: Mapped[str | None] = mapped_column(String(255)) # /library/parts/... (fallback fetch)
|
||||
container: Mapped[str | None] = mapped_column(String(16))
|
||||
codec_video: Mapped[str | None] = mapped_column(String(32))
|
||||
codec_audio: Mapped[str | None] = mapped_column(String(32))
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Browser-playability class from ffprobe: "direct" | "remux" | "transcode" (None = unprobed).
|
||||
playable: Mapped[str | None] = mapped_column(String(12), index=True)
|
||||
probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Transcode HW-capability facet (the playback-class filter). NULL for non-transcode items; for a
|
||||
# `playable="transcode"` item: True = the prod iGPU can hardware-transcode it (codec is HW-decodable
|
||||
# AND the source is 8-bit SDR), False = software libx264 only (AV1/XviD, or a 10-bit/HDR source that
|
||||
# breaks the VAAPI nv12 pipeline — see plex/stream.py). Derived by `plex.sync.transcode_hw_capable`:
|
||||
# set codec-only at sync (bit depth unknown from the cheap listing), then refined by the bit-depth
|
||||
# backfill (`_probe_media_batch`) which downgrades a 10-bit/HDR HEVC to False. The one filterable
|
||||
# home of the invariant `_want_hw`'s source gate computes at play time — kept in sync via the shared
|
||||
# classifier so the catalog facet and the runtime encoder choice never disagree by codec.
|
||||
transcode_hw: Mapped[bool | None] = mapped_column(Boolean, index=True)
|
||||
# Video bit depth (8/10/12…) from the Plex media Stream, filled by the bit-depth backfill. NULL =
|
||||
# not yet probed. Surfaced on the info page (e.g. "HEVC · 10-bit") and feeds `transcode_hw`.
|
||||
video_bit_depth: Mapped[int | None] = mapped_column(Integer)
|
||||
# Watermark for the bit-depth backfill: when this item's media Stream was last probed for
|
||||
# bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
|
||||
# probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
|
||||
media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
|
||||
# `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
|
||||
# filters (GIN-indexed) and double as content signals for a future watch-habit recommender.
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True)
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# rating_keys of the Plex collections this movie belongs to (GIN — filter + card-strip siblings).
|
||||
collection_keys: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Cast + director names, space-joined — folded into search_vector (weight B) so the search box
|
||||
# finds titles by a person's name (and ranks them above summary-only mentions).
|
||||
people_text: Mapped[str | None] = mapped_column(Text)
|
||||
# Set once the movie's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: matched by the canonical `guid` ("tmdb:12345"), filled by the TMDB
|
||||
# backfill job, SERVED FROM THIS ROW (never a live TMDB call). Additive — imdb_rating/rating
|
||||
# above stay Plex-sourced; `tmdb_rating` is the distinct TMDb community score. NULL until enriched;
|
||||
# `tmdb_enriched_at` is the "tried" watermark (stamped on a definitive no-match too, so an
|
||||
# unresolvable title isn't re-fetched every run; left NULL only on a transient fetch error). ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger) # USD (movies; TV has none)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
# [{tmdb_id, title, poster, media_type}] — resolved to library rows (by guid) at serve time.
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB backdrop file_paths [str] (crossfade pool)
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched
|
||||
# (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A small cast/crew headshot cache, keyed by name, so person filter-chips (the cards for the
|
||||
actors currently in the filter) can show a photo + count without a live per-card metadata fetch.
|
||||
Populated opportunistically when an item's RICH metadata is fetched — the info-page view and the
|
||||
periodic rich re-sync — since the cheap section listing that drives the main sync only carries the
|
||||
top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
|
||||
removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
|
||||
metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
|
||||
row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
|
||||
(the headshot FILE lives in the image cache, filled by the people_enrich job)."""
|
||||
|
||||
__tablename__ = "plex_people"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String(255), unique=True, index=True)
|
||||
photo_thumb: Mapped[str | None] = mapped_column(Text) # proxied /person-image URL, or None
|
||||
# TMDB person id, linked by name from a title's TMDB credits (the /person
|
||||
# biography fetch builds on it). NULL until a TMDB-enriched title names this person.
|
||||
tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
|
||||
# --- TMDB person enrichment: fetched from /person/{tmdb_person_id} by the people_enrich job,
|
||||
# or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
|
||||
# no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
|
||||
# media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
|
||||
biography: Mapped[str | None] = mapped_column(Text)
|
||||
# TMDB's Hungarian biography, from the same request's `translations`; a Hungarian reader is
|
||||
# served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
|
||||
# keywords have no Hungarian on TMDB, so they stay in the instance language.
|
||||
biography_hu: Mapped[str | None] = mapped_column(Text)
|
||||
birthday: Mapped[str | None] = mapped_column(String(10)) # "YYYY-MM-DD", served as-is
|
||||
deathday: Mapped[str | None] = mapped_column(String(10))
|
||||
place_of_birth: Mapped[str | None] = mapped_column(Text)
|
||||
known_for_department: Mapped[str | None] = mapped_column(String(64))
|
||||
also_known_as: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
known_for: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{tmdb_id, media_type, title}]
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexCollection(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex collection (a grouping of movies or shows — franchises like Avatar, curated
|
||||
sets like MCU, or auto/community lists like IMDb Top 250). Mirrored in full by the background
|
||||
sync so reads never hit Plex's slow collection queries. Membership is stored as `collection_keys`
|
||||
on the member movies (plex_items) / shows (plex_shows). `smart` = Plex's query-based flag;
|
||||
`editable` = we may write changes back to Plex (default False = read-only; set for curated ones)."""
|
||||
|
||||
__tablename__ = "plex_collections"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer)
|
||||
smart: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
editable: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a Plex item — mirrors VideoState. Lives in Siftlode's own DB so
|
||||
it works for users WITHOUT a Plex account (the whole point of the access-layer design).
|
||||
`synced_to_plex` marks whether this state was pushed back to a linked Plex account."""
|
||||
|
||||
__tablename__ = "plex_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
synced_to_plex: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class PlexLink(Base, UpdatedAtMixin):
|
||||
"""Links a Siftlode user to a Plex account for two-way watch-state sync (one row per user).
|
||||
|
||||
For the owner (the MVP scope) the row uses the server's admin token — `uses_admin=True`,
|
||||
`plex_token_enc=None` — so no separate Plex login is needed; the admin token already reads/writes
|
||||
the owner account's watch state. `plex_account_id` is that Plex account's numeric id, used to
|
||||
filter the shared watch-history feed to this user (Phase C). Reserved for later: a friend can link
|
||||
their OWN Plex account via PIN OAuth, storing a personal `plex_token_enc` (uses_admin=False) — the
|
||||
sync loop then iterates link rows uniformly.
|
||||
|
||||
`initial_import_done` guards the one-time "Plex is master" import; `last_watch_sync_at` is the
|
||||
high-water mark for the incremental Plex→Siftlode pass."""
|
||||
|
||||
__tablename__ = "plex_link"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), unique=True, index=True
|
||||
)
|
||||
uses_admin: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
plex_token_enc: Mapped[str | None] = mapped_column(Text) # P5b friend tokens (encrypted)
|
||||
plex_account_id: Mapped[int | None] = mapped_column(Integer)
|
||||
plex_username: Mapped[str | None] = mapped_column(String(255))
|
||||
sync_enabled: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
initial_import_done: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false"
|
||||
)
|
||||
last_watch_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
linked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexPlaylist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user ORDERED watch-list of Plex items — Siftlode-native (lives in our own DB, works for
|
||||
users WITHOUT a Plex account, like plex_states). Unlike collections (shared, library-level), a
|
||||
playlist is owned by one user and its items are ordered. Optional Plex-direction sync is a later
|
||||
phase (`plex_rating_key` set once pushed back to the owner's Plex account)."""
|
||||
|
||||
__tablename__ = "plex_playlists"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
plex_rating_key: Mapped[str | None] = mapped_column(String(32))
|
||||
|
||||
|
||||
class PlexPlaylistItem(Base):
|
||||
"""One ordered entry in a playlist. `position` (0-based) orders playback."""
|
||||
|
||||
__tablename__ = "plex_playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("plex_playlists.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
|
||||
|
||||
class ComfortPin(Base, TimestampMixin):
|
||||
"""One entry on a user's comfort shelf — the anti-paralysis fallback: "just put on X".
|
||||
|
||||
Deliberately NOT a boolean on `VideoState`/`PlexState`, and NOT a polymorphic `(kind, ref)` string
|
||||
pair. A flag cannot ORDER a shelf and cannot hold a channel or a show — and comfort media is
|
||||
usually "put on X", a series or a channel, not one particular video. A string ref would order and
|
||||
hold anything, but it would also lose the foreign key: a Plex item dropped from the library, or an
|
||||
unsubscribed channel, would leave a dead tile on the one surface whose whole job is to be
|
||||
reliable. So instead: one nullable CASCADE-ing FK per pinnable kind, and a check constraint that
|
||||
exactly one of them is set.
|
||||
|
||||
That constraint is the house rule applied to a five-way union — the invariant lives in the schema,
|
||||
not in a comment every insert site has to remember. `kind`/`ref` below derive FROM the columns, so
|
||||
there is no second vocabulary to keep in sync.
|
||||
|
||||
NULLs are DISTINCT in a Postgres unique index, so the five per-kind unique constraints each stop a
|
||||
double-pin of the same thing without colliding across kinds."""
|
||||
|
||||
__tablename__ = "comfort_pins"
|
||||
__table_args__ = (
|
||||
CheckConstraint(
|
||||
"num_nonnulls(video_id, channel_id, plex_item_id, plex_show_id, playlist_id) = 1",
|
||||
name="ck_comfort_pin_one_ref",
|
||||
),
|
||||
UniqueConstraint("user_id", "video_id", name="uq_comfort_pin_video"),
|
||||
UniqueConstraint("user_id", "channel_id", name="uq_comfort_pin_channel"),
|
||||
UniqueConstraint("user_id", "plex_item_id", name="uq_comfort_pin_plex_item"),
|
||||
UniqueConstraint("user_id", "plex_show_id", name="uq_comfort_pin_plex_show"),
|
||||
UniqueConstraint("user_id", "playlist_id", name="uq_comfort_pin_playlist"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
video_id: Mapped[str | None] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str | None] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
plex_item_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE")
|
||||
)
|
||||
plex_show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE")
|
||||
)
|
||||
playlist_id: Mapped[int | None] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
|
||||
|
||||
# 0-based shelf order. The shelf is short and hand-curated, so a dense integer the reorder
|
||||
# endpoint rewrites wholesale beats a fractional index nobody would ever exercise here.
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
|
||||
@@ -9,6 +9,7 @@ table can't grow without bound for a heavy user.
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.models import Notification
|
||||
|
||||
# Keep at most this many READ notifications per user; older read ones are pruned. Unread
|
||||
@@ -48,11 +49,13 @@ def create_notification(
|
||||
|
||||
def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
"""Delete a user's oldest READ notifications beyond `cap`. Returns the number removed.
|
||||
Idempotent and cheap; safe to call after marking things read."""
|
||||
Idempotent and cheap; safe to call after marking things read. Flushes first, so a notice the
|
||||
caller has just marked read counts; the caller commits."""
|
||||
db.flush()
|
||||
ids = (
|
||||
db.execute(
|
||||
select(Notification.id)
|
||||
.where(Notification.user_id == user_id, Notification.read.is_(True))
|
||||
.where(owned_by(Notification.user_id, user_id), Notification.read.is_(True))
|
||||
.order_by(Notification.created_at.desc())
|
||||
.offset(cap)
|
||||
)
|
||||
@@ -63,5 +66,4 @@ def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
return 0
|
||||
# One bulk DELETE rather than a get+delete per row (the ids are already in hand).
|
||||
db.execute(delete(Notification).where(Notification.id.in_(ids)))
|
||||
db.commit()
|
||||
return len(ids)
|
||||
|
||||
@@ -5,7 +5,7 @@ router. Pure functions — no DB, no request state."""
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
from app.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
|
||||
|
||||
class ShowState(NamedTuple):
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
"""What a collection is to the app: where it comes from, whether it may be edited, its card, its
|
||||
effective content rating, and how its members order — shared by the collection pages, the info
|
||||
page's collection strips and the admin editor."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime
|
||||
from typing import Any, NamedTuple
|
||||
|
||||
from app.plex.models import PlexCollection
|
||||
|
||||
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
|
||||
# members (Plex shows one cert on the collection; we don't store it, so derive it).
|
||||
_CERT_RANK = {
|
||||
"G": 1,
|
||||
"TV-Y": 1,
|
||||
"TV-Y7": 1,
|
||||
"TV-G": 1,
|
||||
"PG": 2,
|
||||
"TV-PG": 2,
|
||||
"PG-13": 3,
|
||||
"TV-14": 3,
|
||||
"R": 4,
|
||||
"TV-MA": 4,
|
||||
"NC-17": 5,
|
||||
}
|
||||
|
||||
|
||||
def collection_cert(certs: list[str | None]) -> str | None:
|
||||
ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
|
||||
return max(ranked, key=lambda x: x[1])[0] if ranked else None
|
||||
|
||||
|
||||
_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
|
||||
|
||||
|
||||
def collection_source(col: PlexCollection) -> str:
|
||||
first = (col.title or "").strip().split(" ", 1)[0].lower()
|
||||
if first in _STRIP_PROVIDERS:
|
||||
return first
|
||||
if col.smart:
|
||||
return "smart"
|
||||
return "collection"
|
||||
|
||||
|
||||
def collection_editable(c: PlexCollection) -> bool:
|
||||
"""Effective editability: the admin marked it editable AND it's a plain manual collection — never
|
||||
smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
|
||||
return bool(c.editable) and not c.smart and collection_source(c) == "collection"
|
||||
|
||||
|
||||
def collection_card(c: PlexCollection) -> dict:
|
||||
return {
|
||||
"id": c.rating_key,
|
||||
"title": c.title,
|
||||
"summary": c.summary,
|
||||
"thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
|
||||
"child_count": c.child_count,
|
||||
"smart": c.smart,
|
||||
"source": collection_source(c),
|
||||
"editable": c.editable,
|
||||
"can_edit": collection_editable(
|
||||
c
|
||||
), # whether the edit controls should be offered (admin UI)
|
||||
}
|
||||
|
||||
|
||||
class Member(NamedTuple):
|
||||
"""One collection member flattened to everything the Sort control can order by. The page's two
|
||||
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
|
||||
ordering is a pure list sort over these, which is also what makes it unit-testable."""
|
||||
|
||||
kind: str # "movie" | "show"
|
||||
obj: Any # PlexItem | PlexShow, carried through to the card serializer
|
||||
title: str # lower-cased; never NULL (the column is not nullable)
|
||||
year: int | None
|
||||
rating: float | None
|
||||
added: datetime | None
|
||||
release: date | None
|
||||
duration: int | None # movies only — a show has no single runtime
|
||||
last_watched: datetime | None
|
||||
|
||||
|
||||
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
|
||||
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
|
||||
# that silently does nothing here.
|
||||
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
|
||||
"added": lambda m: m.added,
|
||||
"lastwatched": lambda m: m.last_watched,
|
||||
"release": lambda m: m.release,
|
||||
"year": lambda m: m.year,
|
||||
"rating": lambda m: m.rating,
|
||||
"duration": lambda m: m.duration,
|
||||
"title": lambda m: m.title,
|
||||
}
|
||||
|
||||
# What the page ordered by before it had a control, and still its default: a film series wants to be
|
||||
# read in release order, not by whenever Plex happened to ingest it.
|
||||
DEFAULT_MEMBER_SORT = "year"
|
||||
|
||||
|
||||
def sort_members(
|
||||
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
|
||||
) -> list[Member]:
|
||||
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
|
||||
|
||||
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
|
||||
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
|
||||
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
|
||||
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
|
||||
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
|
||||
|
||||
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
|
||||
compare as Python strings, which puts every accented initial after "z"."""
|
||||
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
|
||||
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
|
||||
if key is MEMBER_SORT_KEYS["title"]:
|
||||
key = by_title
|
||||
ordered = sorted(members, key=by_title) # stable secondary, applied first
|
||||
have = [m for m in ordered if key(m) is not None]
|
||||
missing = [m for m in ordered if key(m) is None]
|
||||
have.sort(key=key, reverse=sort_dir != "asc")
|
||||
return have + missing
|
||||
@@ -1,10 +1,11 @@
|
||||
"""Parse the "info page" rich metadata out of a full Plex metadata dict, and persist the subset that
|
||||
the detail endpoints serve from the DB row.
|
||||
|
||||
Shared by the live info-page path (routes.plex.browse) and the background rich re-sync (plex.sync) so
|
||||
the two never diverge — the same one-parser rule that keeps `people.py`'s cast handling consistent.
|
||||
`rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY fields onto a movie/show row
|
||||
(genres/directors/studio/content_rating already land from the cheap listing at `_apply_facet_fields`)."""
|
||||
Shared by the live info-page path (`plex.service.enrich_movie` / `enrich_show`) and the background
|
||||
rich re-sync (plex.sync) so the two never diverge — the same one-parser rule that keeps `people.py`'s
|
||||
cast handling consistent. `rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY
|
||||
fields onto a movie/show row (genres/directors/studio/content_rating already land from the cheap
|
||||
listing at `_apply_facet_fields`)."""
|
||||
|
||||
from app.plex import people
|
||||
from app.plex.plaintext import plain_text
|
||||
|
||||
@@ -0,0 +1,536 @@
|
||||
"""Plex Discover's reads: the rail-surface read-model.
|
||||
|
||||
A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
|
||||
Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
|
||||
separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
|
||||
recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
|
||||
every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
|
||||
their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
|
||||
card paths); the HU overlay coalesces only in the main grid.
|
||||
|
||||
The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
|
||||
Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
|
||||
grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
|
||||
the client hides). Genre-language matches the grid; duration is movie-only.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import sorted_words
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import SHOW_NEW, display_title, leaf_card, movie_card, show_card
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.query import (
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
top_art_keys,
|
||||
)
|
||||
from app.plex.tmdb import resolve_recs
|
||||
from app.userscope import owned_by
|
||||
|
||||
GENRE_TILES = 12
|
||||
SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows (6 total)
|
||||
|
||||
|
||||
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
|
||||
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
return (
|
||||
movie_ids if scope in ("movie", "both") else [],
|
||||
show_ids if scope in ("show", "both") else [],
|
||||
)
|
||||
|
||||
|
||||
def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
|
||||
"""Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
|
||||
q = apply_meta_filters(q, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
q = q.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
q = q.where(PlexItem.duration_s <= f.duration_max)
|
||||
return q
|
||||
|
||||
|
||||
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
|
||||
sts = rollup.item_state_map(db, uid, [it.id for it in items])
|
||||
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
|
||||
|
||||
|
||||
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
|
||||
statuses = rollup.state_map(db, uid, [s.id for s in shows])
|
||||
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
|
||||
|
||||
|
||||
def _recently_added(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Newest movies + shows, interleaved by `added_at` desc into one rail."""
|
||||
movies = (
|
||||
db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
.order_by(PlexItem.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
|
||||
)
|
||||
.order_by(PlexShow.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
|
||||
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
|
||||
# review found this one still on the older `(is not None, value)` form, i.e. one rule in two
|
||||
# spellings, with the two comments explaining it differently.
|
||||
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
|
||||
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
|
||||
top = tagged[:RAIL_LIMIT]
|
||||
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
|
||||
# share one), so the merge doesn't depend on object identity staying alive.
|
||||
top_movies = [o for _, kind, o in top if kind == "movie"]
|
||||
top_shows = [o for _, kind, o in top if kind == "show"]
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
|
||||
}
|
||||
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
|
||||
|
||||
|
||||
def _recently_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
|
||||
card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
|
||||
sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
|
||||
rows: list[tuple[datetime | None, dict]] = [] # (watched_at, card)
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
)
|
||||
for it, st in db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all():
|
||||
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
|
||||
if show_ids:
|
||||
eq = (
|
||||
select(PlexItem, PlexState, PlexShow)
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.join(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
for ep, st, sh in db.execute(eq).all():
|
||||
card = leaf_card(
|
||||
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
|
||||
)
|
||||
rows.append((st.watched_at, card))
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _continue_watching(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
|
||||
A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
|
||||
rows: list[tuple[datetime | None, dict]] = [] # (resumed_at, card)
|
||||
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.position_seconds > 0,
|
||||
# A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
|
||||
# `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
|
||||
or_(
|
||||
PlexState.status.is_(None),
|
||||
PlexState.status.notin_(["watched", "hidden"]),
|
||||
),
|
||||
)
|
||||
)
|
||||
mrows = db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.progress_updated_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for it, st in mrows:
|
||||
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
|
||||
|
||||
if show_ids:
|
||||
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
|
||||
# its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
|
||||
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
|
||||
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
|
||||
sq = apply_show_state_filter(
|
||||
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
|
||||
prog, prog.c.show_id == PlexShow.id
|
||||
),
|
||||
prog.c.total,
|
||||
prog.c.watched,
|
||||
prog.c.inprog,
|
||||
"in_progress",
|
||||
).where(PlexShow.library_id.in_(show_ids))
|
||||
srows = db.execute(
|
||||
apply_meta_filters(sq, PlexShow, p, lang)
|
||||
.order_by(prog.c.resumed_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for sh, resumed_at, total, watched, inprog in srows:
|
||||
state = rollup.show_state(total, watched, inprog)
|
||||
rows.append((resumed_at, show_card(sh, state, lang=lang)))
|
||||
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _genre_tiles(
|
||||
db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
|
||||
) -> list[dict]:
|
||||
"""The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
|
||||
The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
|
||||
The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
|
||||
counts: dict[str, int] = {}
|
||||
for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
|
||||
if not ids:
|
||||
continue
|
||||
gcol = genres_col(model, lang)
|
||||
base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
|
||||
model.library_id.in_(ids),
|
||||
func.jsonb_typeof(gcol) == "array",
|
||||
*meta_filter_conds(model, p, lang),
|
||||
)
|
||||
if model is PlexItem:
|
||||
base = base.where(model.kind == "movie")
|
||||
expanded = base.subquery()
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
counts[g] = counts.get(g, 0) + c
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
|
||||
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
|
||||
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
|
||||
art = top_art_keys(
|
||||
db,
|
||||
PlexItem,
|
||||
movie_ids,
|
||||
genres_col(PlexItem, lang),
|
||||
[g for g, _ in top],
|
||||
extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
|
||||
)
|
||||
return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
|
||||
|
||||
|
||||
def _because_you_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> dict | None:
|
||||
"""Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
|
||||
recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
|
||||
None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
|
||||
The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
|
||||
if not movie_ids:
|
||||
return None
|
||||
anchor = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
PlexItem.recommendations.isnot(None),
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(1)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if anchor is None:
|
||||
return None
|
||||
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
|
||||
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
|
||||
rks = [
|
||||
r["rating_key"]
|
||||
for r in resolved
|
||||
if r.get("in_library") and r.get("rating_key") != anchor.rating_key
|
||||
]
|
||||
if not rks:
|
||||
return None
|
||||
movies = {
|
||||
m.rating_key: m
|
||||
for m in db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
ordered_movies = [movies[rk] for rk in rks if rk in movies]
|
||||
ordered_shows = [shows[rk] for rk in rks if rk in shows]
|
||||
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
|
||||
}
|
||||
items: list[dict] = []
|
||||
for rk in rks:
|
||||
if rk in mcards:
|
||||
items.append(mcards[rk])
|
||||
elif rk in scards:
|
||||
items.append(scards[rk])
|
||||
if not items:
|
||||
return None
|
||||
return {
|
||||
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
def discover_page(db: Session, user: User, scope: str, lang: str, f: LibraryFilters) -> dict:
|
||||
"""The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
|
||||
sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
|
||||
is null when there's no anchor. `lang` is already normalised."""
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
return {
|
||||
"scope": scope,
|
||||
"continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
|
||||
"because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
}
|
||||
|
||||
|
||||
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
|
||||
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
|
||||
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
|
||||
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
|
||||
fresh = (
|
||||
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
|
||||
.scalars()
|
||||
.all()
|
||||
if exclude
|
||||
else []
|
||||
)
|
||||
if len(fresh) == per:
|
||||
return list(fresh)
|
||||
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
|
||||
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
|
||||
return [*fresh, *rest.all()]
|
||||
|
||||
|
||||
def _kept(
|
||||
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
|
||||
) -> list[dict]:
|
||||
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
|
||||
stays in it, marked watched, and one gone from the scope's libraries drops out."""
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexItem.rating_key.in_(keep),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
cards = {
|
||||
c["id"]: c
|
||||
for c in [
|
||||
*_movie_cards(db, uid, list(movies), lang),
|
||||
*_show_cards(db, uid, list(shows), lang),
|
||||
]
|
||||
}
|
||||
return [cards[k] for k in keep if k in cards]
|
||||
|
||||
|
||||
def surprise(
|
||||
db: Session,
|
||||
user: User,
|
||||
scope: str,
|
||||
lang: str,
|
||||
limit: int,
|
||||
exclude: list[str],
|
||||
keep: list[str],
|
||||
f: LibraryFilters,
|
||||
) -> dict:
|
||||
"""A fresh random sample for the reshuffling Surprise-me rail (a random draw each call), narrowed by
|
||||
the active filters. Only titles the user has NOT seen — a watched movie / a fully-watched show is
|
||||
excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
|
||||
SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
|
||||
one type."""
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
if keep:
|
||||
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
|
||||
per = min(limit, SURPRISE_PER_KIND)
|
||||
movies = []
|
||||
if movie_ids:
|
||||
# Exclude movies this user has watched.
|
||||
watched_movie = (
|
||||
select(PlexState.id)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
owned_by(PlexState.user_id, user),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
movies = _draw(
|
||||
db,
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
~watched_movie,
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
),
|
||||
PlexItem.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
shows = []
|
||||
if show_ids:
|
||||
# Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
|
||||
# as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
|
||||
prog = rollup.agg_subquery(db, user.id).subquery()
|
||||
not_fully_watched = or_(
|
||||
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
|
||||
)
|
||||
shows = _draw(
|
||||
db,
|
||||
apply_meta_filters(
|
||||
select(PlexShow)
|
||||
.outerjoin(prog, prog.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
|
||||
PlexShow,
|
||||
p,
|
||||
lang,
|
||||
),
|
||||
PlexShow.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
mcards = _movie_cards(db, user.id, movies, lang)
|
||||
scards = _show_cards(db, user.id, shows, lang)
|
||||
# Interleave movie/show cards, then trim to `limit`.
|
||||
items: list[dict] = []
|
||||
for i in range(max(len(mcards), len(scards))):
|
||||
if i < len(mcards):
|
||||
items.append(mcards[i])
|
||||
if i < len(scards):
|
||||
items.append(scards[i])
|
||||
return {"scope": scope, "items": items[:limit]}
|
||||
@@ -15,7 +15,7 @@ from fastapi import Response
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import PlexCollection, PlexItem, PlexSeason, PlexShow
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexSeason, PlexShow
|
||||
from app.plex.cache import CACHE_DIR
|
||||
from app.plex.lookups import by_rating_key
|
||||
|
||||
|
||||
@@ -0,0 +1,473 @@
|
||||
"""The Plex browse grid's reads: the unified movies + shows feed, its search, and its facets.
|
||||
|
||||
A user's watch state is joined through `userscope`; the feed and its facets share every filter
|
||||
(`plex.query`), so a facet count is the size of what the grid would show.
|
||||
"""
|
||||
|
||||
from sqlalchemy import Boolean, Integer, String, and_, cast, func, literal, null, or_, select
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import norm_lang
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.query import (
|
||||
FIELD_COLUMNS,
|
||||
PLAYBACK_CLASSES,
|
||||
TS_CONFIG,
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_movie_state_filter,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
field_recheck,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
movie_status_case,
|
||||
playable_conds,
|
||||
playback_class_expr,
|
||||
show_status_case,
|
||||
split_csv,
|
||||
summary_col,
|
||||
title_col,
|
||||
vector_col,
|
||||
vector_match,
|
||||
)
|
||||
from app.userscope import owned_by
|
||||
from app.utils import to_tsquery_str
|
||||
|
||||
|
||||
def sort_column(u, sort: str, lang: str = "en"):
|
||||
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
|
||||
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
|
||||
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
|
||||
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
|
||||
in the reader's alphabet (`collation`)."""
|
||||
return {
|
||||
"added": u.c.added_at,
|
||||
"title": collated(u.c.title, lang),
|
||||
"year": u.c.year,
|
||||
"rating": u.c.rating,
|
||||
"duration": u.c.duration_s,
|
||||
"release": u.c.rel,
|
||||
"lastwatched": u.c.last_watched,
|
||||
}.get(sort, u.c.added_at)
|
||||
|
||||
|
||||
def library_page(
|
||||
db: Session,
|
||||
user: User,
|
||||
*,
|
||||
scope: str,
|
||||
q: str | None,
|
||||
field: str,
|
||||
sort: str,
|
||||
sort_dir: str,
|
||||
show: str,
|
||||
lang: str,
|
||||
offset: int,
|
||||
limit: int,
|
||||
f: LibraryFilters,
|
||||
) -> tuple[int, list, list]:
|
||||
"""One page of the unified cross-library browse: (total, rows, episodes). `rows` mixes movies
|
||||
and shows (`kind` says which), scoped to movie|show|both, with the shared filters/search/watch
|
||||
state (a show's state is the aggregate of its episodes). On a search that also matches episodes
|
||||
(and shows are in scope), `episodes` holds the matching ones as (item, state, show) — the
|
||||
grouped 'Episodes' section. `lang` is already normalised."""
|
||||
p = f.as_dict()
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
|
||||
ts = to_tsquery_str(q) if q else None
|
||||
tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
|
||||
if field not in FIELD_COLUMNS and field != "all":
|
||||
field = "all"
|
||||
# `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
|
||||
# the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
|
||||
grid_field = field != "episode"
|
||||
recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
|
||||
|
||||
selects = []
|
||||
if want_movies and grid_field:
|
||||
st = aliased(PlexState)
|
||||
m_status = movie_status_case(st)
|
||||
mq = (
|
||||
select(
|
||||
PlexItem.rating_key.label("rk"),
|
||||
literal("movie").label("kind"),
|
||||
title_col(PlexItem, lang).label("title"),
|
||||
PlexItem.year.label("year"),
|
||||
PlexItem.rating.label("rating"),
|
||||
# Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
|
||||
func.left(summary_col(PlexItem, lang), 300).label("summary"),
|
||||
PlexItem.added_at.label("added_at"),
|
||||
PlexItem.originally_available_at.label("rel"),
|
||||
PlexItem.duration_s.label("duration_s"),
|
||||
PlexItem.playable.label("playable"),
|
||||
PlexItem.transcode_hw.label("transcode_hw"),
|
||||
cast(null(), Integer).label("season_count"),
|
||||
func.coalesce(st.position_seconds, 0).label("position_seconds"),
|
||||
m_status.label("status"),
|
||||
literal(False).label("resumable"),
|
||||
# "last watched" sort key: when the movie was last resumed, else when it was marked
|
||||
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
|
||||
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexItem, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(st, and_(st.item_id == PlexItem.id, owned_by(st.user_id, user)))
|
||||
.where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
|
||||
)
|
||||
mq = apply_movie_state_filter(mq, st, show)
|
||||
mq = apply_meta_filters(mq, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
mq = mq.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
mq = mq.where(PlexItem.duration_s <= f.duration_max)
|
||||
# Playback-class filter (movie-only — a show is an aggregate with no single class).
|
||||
mq = mq.where(*playable_conds(split_csv(f.playable)))
|
||||
if tsq is not None:
|
||||
mq = mq.where(vector_match(PlexItem, lang, tsq))
|
||||
if recheck is not None and (narrow := recheck(PlexItem, field, tsq, lang)) is not None:
|
||||
mq = mq.where(narrow)
|
||||
selects.append(mq)
|
||||
if want_shows and grid_field:
|
||||
agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
|
||||
tot = func.coalesce(agg.c.total, 0)
|
||||
wat = func.coalesce(agg.c.watched, 0)
|
||||
inp = func.coalesce(agg.c.inprog, 0)
|
||||
s_status = show_status_case(tot, wat, inp)
|
||||
sq = (
|
||||
select(
|
||||
PlexShow.rating_key.label("rk"),
|
||||
literal("show").label("kind"),
|
||||
title_col(PlexShow, lang).label("title"),
|
||||
PlexShow.year.label("year"),
|
||||
PlexShow.rating.label("rating"),
|
||||
func.left(summary_col(PlexShow, lang), 300).label("summary"),
|
||||
PlexShow.added_at.label("added_at"),
|
||||
PlexShow.originally_available_at.label("rel"),
|
||||
cast(null(), Integer).label("duration_s"),
|
||||
cast(null(), String).label("playable"),
|
||||
cast(null(), Boolean).label("transcode_hw"),
|
||||
PlexShow.child_count.label("season_count"),
|
||||
literal(0).label("position_seconds"),
|
||||
s_status.label("status"),
|
||||
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
|
||||
(inp > 0).label("resumable"),
|
||||
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
|
||||
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
|
||||
agg.c.resumed_at.label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexShow, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_lib_ids))
|
||||
)
|
||||
sq = apply_show_state_filter(sq, tot, wat, inp, show)
|
||||
sq = apply_meta_filters(sq, PlexShow, p, lang)
|
||||
if tsq is not None:
|
||||
sq = sq.where(vector_match(PlexShow, lang, tsq))
|
||||
if recheck is not None and (narrow := recheck(PlexShow, field, tsq, lang)) is not None:
|
||||
sq = sq.where(narrow)
|
||||
selects.append(sq)
|
||||
|
||||
total = 0
|
||||
rows: list = []
|
||||
if selects:
|
||||
union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
|
||||
u = union.subquery()
|
||||
total = db.scalar(select(func.count()).select_from(u)) or 0
|
||||
|
||||
if tsq is not None:
|
||||
order = [u.c.rank.desc()]
|
||||
else:
|
||||
col = sort_column(u, sort, lang)
|
||||
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
|
||||
order.append(u.c.rk.desc())
|
||||
rows = list(db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all())
|
||||
|
||||
# Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
|
||||
# and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
|
||||
# target the movie/show grid, so they suppress episodes.
|
||||
eps: list = []
|
||||
if tsq is not None and want_shows and field in ("all", "episode"):
|
||||
ep_st = aliased(PlexState)
|
||||
# In HU an episode is found by its Hungarian title too (the overlay fills it) — and
|
||||
# still by its English one, like `vector_match` does for the grid.
|
||||
if field == "episode":
|
||||
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
|
||||
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
|
||||
else:
|
||||
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
|
||||
hit = ep_vec.op("@@")(tsq)
|
||||
if lang == "hu":
|
||||
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
|
||||
ep_vec = ep_vec_hu
|
||||
eps = list(
|
||||
db.execute(
|
||||
select(PlexItem, ep_st, PlexShow)
|
||||
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, owned_by(ep_st.user_id, user)))
|
||||
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_lib_ids),
|
||||
hit,
|
||||
)
|
||||
.order_by(func.ts_rank(ep_vec, tsq).desc())
|
||||
.limit(24)
|
||||
).all()
|
||||
)
|
||||
|
||||
return total, rows, eps
|
||||
|
||||
|
||||
def facets(db: Session, user: User, *, scope: str, show: str, lang: str, f: LibraryFilters) -> dict:
|
||||
"""Available filter values for the unified library (scope movie|show|both), ADAPTED to the active
|
||||
filters (faceted search): each group's values/bounds are computed with all the OTHER active
|
||||
filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
|
||||
a multi-select group still lets you add more. Merged across the scope's libraries; duration is
|
||||
movie-only. (Search text is not factored in — facets track the sidebar filters.) `lang` is
|
||||
normalised here, after the empty answer."""
|
||||
empty = {
|
||||
"genres": [],
|
||||
"content_ratings": [],
|
||||
"studios": [],
|
||||
"playable": [],
|
||||
"year_min": None,
|
||||
"year_max": None,
|
||||
"rating_max": None,
|
||||
"duration_min": None,
|
||||
"duration_max": None,
|
||||
}
|
||||
all_movie_ids, all_show_ids = content_libs(db)
|
||||
movie_ids = all_movie_ids if scope in ("movie", "both") else []
|
||||
show_ids = all_show_ids if scope in ("show", "both") else []
|
||||
if not movie_ids and not show_ids:
|
||||
return empty
|
||||
lang = norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
|
||||
uid = user.id
|
||||
|
||||
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
|
||||
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
|
||||
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
|
||||
# The same filter on the same rows, so the counts do not change
|
||||
# (test_plex_facets_grid_parity).
|
||||
shows_in_scope: list[int] | None = None
|
||||
if show_ids and show in ("watched", "in_progress", "unwatched"):
|
||||
agg = rollup.agg_subquery(db, uid).subquery()
|
||||
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
in_scope = apply_show_state_filter(
|
||||
in_scope,
|
||||
func.coalesce(agg.c.total, 0),
|
||||
func.coalesce(agg.c.watched, 0),
|
||||
func.coalesce(agg.c.inprog, 0),
|
||||
show,
|
||||
)
|
||||
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
|
||||
|
||||
def wsq(query, model, ids, pp, kind_movie, with_dur):
|
||||
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
|
||||
facet aggregation query, so the facets reflect exactly the visible result set."""
|
||||
query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
|
||||
if kind_movie:
|
||||
query = query.where(model.kind == "movie")
|
||||
if with_dur and pp.get("duration_min") is not None:
|
||||
query = query.where(model.duration_s >= pp["duration_min"])
|
||||
if with_dur and pp.get("duration_max") is not None:
|
||||
query = query.where(model.duration_s <= pp["duration_max"])
|
||||
# Playback-class filter (movie-only) — applied for the OTHER groups so an active class
|
||||
# narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
|
||||
query = query.where(*playable_conds(split_csv(pp.get("playable"))))
|
||||
# Join watch-state and reuse the grid's exact movie state filter (unified_library) —
|
||||
# hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
|
||||
st = aliased(PlexState)
|
||||
query = query.outerjoin(st, and_(st.item_id == model.id, owned_by(st.user_id, uid)))
|
||||
query = apply_movie_state_filter(query, st, show)
|
||||
elif shows_in_scope is not None:
|
||||
query = query.where(model.id.in_(shows_in_scope))
|
||||
return query
|
||||
|
||||
genre_counts: dict[str, int] = {}
|
||||
cr_counts: dict[str, int] = {}
|
||||
studio_counts: dict[str, int] = {}
|
||||
playable_counts: dict[str, int] = {}
|
||||
years: list[int] = []
|
||||
ratings: list[float] = []
|
||||
durs: list[int] = []
|
||||
|
||||
def per_table(model, ids, kind_movie: bool) -> None:
|
||||
if not ids:
|
||||
return
|
||||
# Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
|
||||
# OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
|
||||
# offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
|
||||
# the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
|
||||
# every matching title carries them, so they stay togglable.)
|
||||
genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
|
||||
# Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
|
||||
# `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
|
||||
# jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
|
||||
# so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
|
||||
# whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
|
||||
# In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
|
||||
gcol = genres_col(model, lang)
|
||||
expanded = (
|
||||
wsq(
|
||||
select(func.jsonb_array_elements_text(gcol).label("g")),
|
||||
model,
|
||||
ids,
|
||||
genre_pp,
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(func.jsonb_typeof(gcol) == "array")
|
||||
.subquery()
|
||||
)
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
genre_counts[g] = genre_counts.get(g, 0) + c
|
||||
# Content ratings — exclude its own.
|
||||
crq = (
|
||||
wsq(
|
||||
select(model.content_rating, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "content_ratings": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.content_rating.isnot(None))
|
||||
.group_by(model.content_rating)
|
||||
)
|
||||
for cr, c in db.execute(crq).all():
|
||||
cr_counts[cr] = cr_counts.get(cr, 0) + c
|
||||
# Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
|
||||
stq = (
|
||||
wsq(
|
||||
select(model.studio, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "studios": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.studio.isnot(None))
|
||||
.group_by(model.studio)
|
||||
)
|
||||
for s, c in db.execute(stq).all():
|
||||
studio_counts[s] = studio_counts.get(s, 0) + c
|
||||
# Playback class (movie-only) — exclude its own so each class stays an addable OR option.
|
||||
if kind_movie:
|
||||
pcls = playback_class_expr()
|
||||
pcq = (
|
||||
wsq(
|
||||
select(pcls.label("cls"), func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "playable": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(pcls.isnot(None))
|
||||
.group_by(pcls)
|
||||
)
|
||||
for cls, c in db.execute(pcq).all():
|
||||
playable_counts[cls] = playable_counts.get(cls, 0) + c
|
||||
# Year bounds — exclude the year filter.
|
||||
by = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.year), func.max(model.year)),
|
||||
model,
|
||||
ids,
|
||||
{**p, "year_min": None, "year_max": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
).one()
|
||||
if by[0] is not None:
|
||||
years.append(by[0])
|
||||
if by[1] is not None:
|
||||
years.append(by[1])
|
||||
# Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
|
||||
# rating filter and the UI badge use (IMDb preferred, audience fallback).
|
||||
br = db.scalar(
|
||||
wsq(
|
||||
select(func.max(func.coalesce(model.imdb_rating, model.rating))),
|
||||
model,
|
||||
ids,
|
||||
{**p, "rating_min": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
)
|
||||
if br is not None:
|
||||
ratings.append(float(br))
|
||||
# Duration bounds (movies) — exclude the duration filter.
|
||||
if kind_movie:
|
||||
bd = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.duration_s), func.max(model.duration_s)),
|
||||
model,
|
||||
ids,
|
||||
p,
|
||||
kind_movie,
|
||||
False,
|
||||
)
|
||||
).one()
|
||||
if bd[0] is not None:
|
||||
durs.append(bd[0])
|
||||
if bd[1] is not None:
|
||||
durs.append(bd[1])
|
||||
|
||||
per_table(PlexItem, movie_ids, True)
|
||||
per_table(PlexShow, show_ids, False)
|
||||
# Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
|
||||
# be absent): the sidebar renders genres solely from this list and hides the whole genre section —
|
||||
# chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
|
||||
# undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
|
||||
for g in split_csv(f.genres):
|
||||
genre_counts.setdefault(g, 0)
|
||||
# Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
|
||||
for s in split_csv(f.studios):
|
||||
studio_counts.setdefault(s, 0)
|
||||
# Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
|
||||
# section when its facet list is empty, which would trap a zero-result selection with no way to undo).
|
||||
for c in split_csv(f.playable):
|
||||
if c in PLAYBACK_CLASSES:
|
||||
playable_counts.setdefault(c, 0)
|
||||
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
|
||||
return {
|
||||
"genres": [
|
||||
{"value": g, "count": c}
|
||||
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
|
||||
],
|
||||
"content_ratings": [
|
||||
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
|
||||
],
|
||||
"studios": [
|
||||
{"value": s, "count": c}
|
||||
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
|
||||
],
|
||||
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
|
||||
"playable": [
|
||||
{"value": c, "count": playable_counts[c]}
|
||||
for c in PLAYBACK_CLASSES
|
||||
if c in playable_counts
|
||||
],
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"rating_max": max(ratings) if ratings else None,
|
||||
"duration_min": min(durs) if durs else None,
|
||||
"duration_max": max(durs) if durs else None,
|
||||
}
|
||||
@@ -0,0 +1,463 @@
|
||||
"""Plex's tables: a self-contained catalog mirrored from a local Plex server — libraries, shows,
|
||||
seasons, items, people, collections, the server's playlists — and each user's private part of it:
|
||||
watch state, the account link, their own playlists. Only this unit writes them; another unit reads
|
||||
them along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
Computed,
|
||||
Date,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin, UpdatedAtMixin
|
||||
|
||||
|
||||
# --- Plex integration (optional) -------------------------------------------------------------
|
||||
# A dedicated, self-contained catalog mirrored from a locally-reachable Plex server. Playback is
|
||||
# from the LOCAL physical file (Plex is metadata only); per-user watch state lives in plex_states
|
||||
# so it works for users without a Plex account. Mirrors the app's "shared catalog, private
|
||||
# per-user state" model. The weighted FTS search_vector uses the same unaccent_simple config as
|
||||
# `videos` (migrations 0031/0032).
|
||||
|
||||
|
||||
class PlexLibrary(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex library section. Only `enabled` sections are exposed in the feed.
|
||||
`plex_key` is the section id on the Plex server."""
|
||||
|
||||
__tablename__ = "plex_libraries"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
plex_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
title: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16)) # "movie" | "show"
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# HU-overlay support. `metadata_lang` tags the section's metadata language. An OVERLAY
|
||||
# library (overlay_of set) is the Hungarian-metadata twin of another section: it is never browsable
|
||||
# on its own — its titles' HU fields are written onto the matched EN rows (paired by `guid`).
|
||||
metadata_lang: Mapped[str] = mapped_column(String(8), default="en", server_default="en")
|
||||
overlay_of: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="SET NULL"), index=True
|
||||
)
|
||||
|
||||
|
||||
class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A TV show (grandparent of episodes) — holds poster + summary for the drill-down page, plus
|
||||
the same filterable metadata as movies (genres/rating/content_rating/studio/people) so the TV
|
||||
grid can be filtered/sorted like the movie grid. All mirrored cheaply from the show listing."""
|
||||
|
||||
__tablename__ = "plex_shows"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_shows_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
# ix_plex_shows_collections_gin already created with collections (migration 0047).
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# Canonical external id (e.g. "tmdb:12345") — the key that pairs this EN row with its HU-overlay
|
||||
# twin during sync. Indexed for the guid->row overlay match.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(
|
||||
String(512)
|
||||
) # Plex thumb path (proxied, not stored)
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer) # seasons
|
||||
collection_keys: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # Plex collections this show is in
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the show section listing (parallels PlexItem).
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True) # Plex audienceRating (~IMDb)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True) # network/studio
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True) # first air date
|
||||
# none_as_null: an empty tag list must persist as SQL NULL, not JSON `null`. Without it a Python
|
||||
# None round-trips as a JSON scalar that jsonb_array_elements_text() can't unnest ("cannot extract
|
||||
# elements from a scalar") — the trap a metadata-poor library sprang on /facets. Same on plex_items.
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
people_text: Mapped[str | None] = mapped_column(
|
||||
Text
|
||||
) # cast+directors, folded into search_vector
|
||||
# Set once the show's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: see the identical block on plex_items for the full rationale. Additive;
|
||||
# served from the row; NULL until enriched. (`budget`/`revenue` stay NULL for shows — TV has none.) ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched,
|
||||
# so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexSeason(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A season under a show — used to order episodes for prev/next stepping."""
|
||||
|
||||
__tablename__ = "plex_seasons"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
show_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
season_number: Mapped[int | None] = mapped_column(Integer, index=True)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
|
||||
|
||||
class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A playable LEAF — a movie or a single episode. This is what the feed lists and the player
|
||||
plays. Movies carry `library_id` and no parents; episodes carry show/season refs +
|
||||
season/episode numbers. `file_path` is the Plex-side absolute path of the physical media,
|
||||
mapped through `plex_path_map` to a local mount at playback time. `playable` is the
|
||||
ffprobe-derived browser-compatibility class (direct | remux | transcode)."""
|
||||
|
||||
__tablename__ = "plex_items"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_items_show_order", "show_id", "season_number", "episode_number"),
|
||||
Index("ix_plex_items_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_items_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_items_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_items_collections_gin", "collection_keys", postgresql_using="gin"),
|
||||
Index("ix_plex_items_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
kind: Mapped[str] = mapped_column(String(16), index=True) # "movie" | "episode"
|
||||
# Canonical external id (e.g. "tmdb:12345") — pairs a MOVIE row with its HU-overlay twin during
|
||||
# sync; indexed for the guid->row match. Episodes carry it too but it's unused for them.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_seasons.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_number: Mapped[int | None] = mapped_column(Integer)
|
||||
episode_number: Mapped[int | None] = mapped_column(Integer)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
duration_s: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
cast: Mapped[list | None] = mapped_column(JSON) # [role names]
|
||||
# Physical media (from the Plex Part).
|
||||
file_path: Mapped[str | None] = mapped_column(Text) # Plex-side absolute path
|
||||
part_key: Mapped[str | None] = mapped_column(String(255)) # /library/parts/... (fallback fetch)
|
||||
container: Mapped[str | None] = mapped_column(String(16))
|
||||
codec_video: Mapped[str | None] = mapped_column(String(32))
|
||||
codec_audio: Mapped[str | None] = mapped_column(String(32))
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Browser-playability class from ffprobe: "direct" | "remux" | "transcode" (None = unprobed).
|
||||
playable: Mapped[str | None] = mapped_column(String(12), index=True)
|
||||
probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Transcode HW-capability facet (the playback-class filter). NULL for non-transcode items; for a
|
||||
# `playable="transcode"` item: True = the prod iGPU can hardware-transcode it (codec is HW-decodable
|
||||
# AND the source is 8-bit SDR), False = software libx264 only (AV1/XviD, or a 10-bit/HDR source that
|
||||
# breaks the VAAPI nv12 pipeline — see plex/stream.py). Derived by `plex.sync.transcode_hw_capable`:
|
||||
# set codec-only at sync (bit depth unknown from the cheap listing), then refined by the bit-depth
|
||||
# backfill (`_probe_media_batch`) which downgrades a 10-bit/HDR HEVC to False. The one filterable
|
||||
# home of the invariant `_want_hw`'s source gate computes at play time — kept in sync via the shared
|
||||
# classifier so the catalog facet and the runtime encoder choice never disagree by codec.
|
||||
transcode_hw: Mapped[bool | None] = mapped_column(Boolean, index=True)
|
||||
# Video bit depth (8/10/12…) from the Plex media Stream, filled by the bit-depth backfill. NULL =
|
||||
# not yet probed. Surfaced on the info page (e.g. "HEVC · 10-bit") and feeds `transcode_hw`.
|
||||
video_bit_depth: Mapped[int | None] = mapped_column(Integer)
|
||||
# Watermark for the bit-depth backfill: when this item's media Stream was last probed for
|
||||
# bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
|
||||
# probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
|
||||
media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
|
||||
# `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
|
||||
# filters (GIN-indexed) and double as content signals for a future watch-habit recommender.
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True)
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# rating_keys of the Plex collections this movie belongs to (GIN — filter + card-strip siblings).
|
||||
collection_keys: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Cast + director names, space-joined — folded into search_vector (weight B) so the search box
|
||||
# finds titles by a person's name (and ranks them above summary-only mentions).
|
||||
people_text: Mapped[str | None] = mapped_column(Text)
|
||||
# Set once the movie's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: matched by the canonical `guid` ("tmdb:12345"), filled by the TMDB
|
||||
# backfill job, SERVED FROM THIS ROW (never a live TMDB call). Additive — imdb_rating/rating
|
||||
# above stay Plex-sourced; `tmdb_rating` is the distinct TMDb community score. NULL until enriched;
|
||||
# `tmdb_enriched_at` is the "tried" watermark (stamped on a definitive no-match too, so an
|
||||
# unresolvable title isn't re-fetched every run; left NULL only on a transient fetch error). ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger) # USD (movies; TV has none)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
# [{tmdb_id, title, poster, media_type}] — resolved to library rows (by guid) at serve time.
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB backdrop file_paths [str] (crossfade pool)
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched
|
||||
# (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A small cast/crew headshot cache, keyed by name, so person filter-chips (the cards for the
|
||||
actors currently in the filter) can show a photo + count without a live per-card metadata fetch.
|
||||
Populated opportunistically when an item's RICH metadata is fetched — the info-page view and the
|
||||
periodic rich re-sync — since the cheap section listing that drives the main sync only carries the
|
||||
top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
|
||||
removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
|
||||
metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
|
||||
row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
|
||||
(the headshot FILE lives in the image cache, filled by the people_enrich job)."""
|
||||
|
||||
__tablename__ = "plex_people"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String(255), unique=True, index=True)
|
||||
photo_thumb: Mapped[str | None] = mapped_column(Text) # proxied /person-image URL, or None
|
||||
# TMDB person id, linked by name from a title's TMDB credits (the /person
|
||||
# biography fetch builds on it). NULL until a TMDB-enriched title names this person.
|
||||
tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
|
||||
# --- TMDB person enrichment: fetched from /person/{tmdb_person_id} by the people_enrich job,
|
||||
# or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
|
||||
# no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
|
||||
# media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
|
||||
biography: Mapped[str | None] = mapped_column(Text)
|
||||
# TMDB's Hungarian biography, from the same request's `translations`; a Hungarian reader is
|
||||
# served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
|
||||
# keywords have no Hungarian on TMDB, so they stay in the instance language.
|
||||
biography_hu: Mapped[str | None] = mapped_column(Text)
|
||||
birthday: Mapped[str | None] = mapped_column(String(10)) # "YYYY-MM-DD", served as-is
|
||||
deathday: Mapped[str | None] = mapped_column(String(10))
|
||||
place_of_birth: Mapped[str | None] = mapped_column(Text)
|
||||
known_for_department: Mapped[str | None] = mapped_column(String(64))
|
||||
also_known_as: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
known_for: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{tmdb_id, media_type, title}]
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexCollection(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex collection (a grouping of movies or shows — franchises like Avatar, curated
|
||||
sets like MCU, or auto/community lists like IMDb Top 250). Mirrored in full by the background
|
||||
sync so reads never hit Plex's slow collection queries. Membership is stored as `collection_keys`
|
||||
on the member movies (plex_items) / shows (plex_shows). `smart` = Plex's query-based flag;
|
||||
`editable` = we may write changes back to Plex (default False = read-only; set for curated ones)."""
|
||||
|
||||
__tablename__ = "plex_collections"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer)
|
||||
smart: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
editable: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a Plex item — mirrors VideoState. Lives in Siftlode's own DB so
|
||||
it works for users WITHOUT a Plex account (the whole point of the access-layer design).
|
||||
`synced_to_plex` marks whether this state was pushed back to a linked Plex account."""
|
||||
|
||||
__tablename__ = "plex_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
synced_to_plex: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class PlexLink(Base, UpdatedAtMixin):
|
||||
"""Links a Siftlode user to a Plex account for two-way watch-state sync (one row per user).
|
||||
|
||||
For the owner (the MVP scope) the row uses the server's admin token — `uses_admin=True`,
|
||||
`plex_token_enc=None` — so no separate Plex login is needed; the admin token already reads/writes
|
||||
the owner account's watch state. `plex_account_id` is that Plex account's numeric id, used to
|
||||
filter the shared watch-history feed to this user (Phase C). Reserved for later: a friend can link
|
||||
their OWN Plex account via PIN OAuth, storing a personal `plex_token_enc` (uses_admin=False) — the
|
||||
sync loop then iterates link rows uniformly.
|
||||
|
||||
`initial_import_done` guards the one-time "Plex is master" import; `last_watch_sync_at` is the
|
||||
high-water mark for the incremental Plex→Siftlode pass."""
|
||||
|
||||
__tablename__ = "plex_link"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), unique=True, index=True
|
||||
)
|
||||
uses_admin: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
plex_token_enc: Mapped[str | None] = mapped_column(Text) # P5b friend tokens (encrypted)
|
||||
plex_account_id: Mapped[int | None] = mapped_column(Integer)
|
||||
plex_username: Mapped[str | None] = mapped_column(String(255))
|
||||
sync_enabled: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
initial_import_done: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false"
|
||||
)
|
||||
last_watch_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
linked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexPlaylist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user ORDERED watch-list of Plex items — Siftlode-native (lives in our own DB, works for
|
||||
users WITHOUT a Plex account, like plex_states). Unlike collections (shared, library-level), a
|
||||
playlist is owned by one user and its items are ordered. Optional Plex-direction sync is a later
|
||||
phase (`plex_rating_key` set once pushed back to the owner's Plex account)."""
|
||||
|
||||
__tablename__ = "plex_playlists"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
plex_rating_key: Mapped[str | None] = mapped_column(String(32))
|
||||
|
||||
|
||||
class PlexPlaylistItem(Base):
|
||||
"""One ordered entry in a playlist. `position` (0-based) orders playback."""
|
||||
|
||||
__tablename__ = "plex_playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("plex_playlists.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
@@ -13,7 +13,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex.client import PlexClient
|
||||
from app.plex.genres import hu_genres
|
||||
|
||||
@@ -15,7 +15,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexPerson
|
||||
from app.plex.models import PlexPerson
|
||||
from app.plex import imgcache
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
|
||||
|
||||
@@ -0,0 +1,507 @@
|
||||
"""Plex's reads for the HTTP layer: the mirrored rows a page shows, and a user's watch state on them.
|
||||
|
||||
A user's own rows (`PlexState`) are read through `userscope`, so "someone else's row" is the same
|
||||
answer as a missing one. A rating key with a NUL in it names no row (`lookups.by_rating_key`).
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexLink,
|
||||
PlexPerson,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
)
|
||||
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app.utils import like_escape
|
||||
|
||||
# --- one row ---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def item(db: Session, rating_key: str) -> PlexItem | None:
|
||||
return by_rating_key(db, PlexItem, rating_key)
|
||||
|
||||
|
||||
def movie(db: Session, rating_key: str) -> PlexItem | None:
|
||||
"""The movie with this key — an episode's key is not one."""
|
||||
it = item(db, rating_key)
|
||||
return it if it is not None and it.kind == "movie" else None
|
||||
|
||||
|
||||
def show(db: Session, rating_key: str) -> PlexShow | None:
|
||||
return by_rating_key(db, PlexShow, rating_key)
|
||||
|
||||
|
||||
def collection(db: Session, rating_key: str) -> PlexCollection | None:
|
||||
return by_rating_key(db, PlexCollection, rating_key)
|
||||
|
||||
|
||||
def season(db: Session, rating_key: str) -> PlexSeason | None:
|
||||
return by_rating_key(db, PlexSeason, rating_key)
|
||||
|
||||
|
||||
def season_episodes(db: Session, se: PlexSeason) -> list[PlexItem]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode")
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def show_by_id(db: Session, show_id: int) -> PlexShow | None:
|
||||
return db.get(PlexShow, show_id)
|
||||
|
||||
|
||||
def season_by_id(db: Session, season_id: int) -> PlexSeason | None:
|
||||
return db.get(PlexSeason, season_id)
|
||||
|
||||
|
||||
def library(db: Session, library_id: int) -> PlexLibrary | None:
|
||||
return db.get(PlexLibrary, library_id)
|
||||
|
||||
|
||||
def library_by_key(db: Session, plex_key: str) -> PlexLibrary | None:
|
||||
return (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(plex_key)))
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def library_key(db: Session, library_id: int | None) -> str | None:
|
||||
"""The Plex section key for a library id — the info/show page passes it to the admin collection
|
||||
editor (in the unified cross-library view there's no single library prop otherwise)."""
|
||||
if library_id is None:
|
||||
return None
|
||||
lib = db.get(PlexLibrary, library_id)
|
||||
return lib.plex_key if lib else None
|
||||
|
||||
|
||||
def person(db: Session, name: str) -> PlexPerson | None:
|
||||
return db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
|
||||
|
||||
|
||||
# --- a user's watch state --------------------------------------------------------------------
|
||||
|
||||
|
||||
def state(db: Session, user: User | int, item_id: int) -> PlexState | None:
|
||||
return (
|
||||
db.execute(
|
||||
select(PlexState).where(owned_by(PlexState.user_id, user), PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
# --- a show ----------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def seasons(db: Session, sh: PlexShow) -> list[PlexSeason]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexSeason)
|
||||
.where(PlexSeason.show_id == sh.id)
|
||||
.order_by(PlexSeason.season_number.asc().nullsfirst())
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def episodes(db: Session, sh: PlexShow) -> list[PlexItem]:
|
||||
"""A show's episodes in season/episode order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def episode_keys(db: Session, show_id: int) -> list[str]:
|
||||
"""The rating keys of a show's episodes, in the order `episodes` gives."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem.rating_key)
|
||||
.where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def shows_by_keys(db: Session, rating_keys: list[str], exclude: int) -> dict[str, PlexShow]:
|
||||
"""The mirrored shows among these keys, but `exclude`, by key."""
|
||||
return {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rating_keys), PlexShow.id != exclude)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
|
||||
# --- libraries -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def browsable_libraries(db: Session, lang: str) -> list[tuple[PlexLibrary, int]]:
|
||||
"""The mirrored libraries a user can browse, each with its title count (movies, or shows). An
|
||||
HU overlay library is left out: it is never browsed, it only feeds its EN twin's Hungarian
|
||||
metadata."""
|
||||
libs = db.execute(
|
||||
select(PlexLibrary)
|
||||
.where(PlexLibrary.overlay_of.is_(None))
|
||||
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang))
|
||||
).scalars()
|
||||
out = []
|
||||
for lib in libs:
|
||||
if lib.kind == "movie":
|
||||
count = db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.library_id == lib.id, PlexItem.kind == "movie"
|
||||
)
|
||||
)
|
||||
else:
|
||||
count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
|
||||
out.append((lib, count or 0))
|
||||
return out
|
||||
|
||||
|
||||
# --- people ----------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def person_photos(db: Session, names: list[str]) -> dict[str, str | None]:
|
||||
return {
|
||||
p.name: p.photo_thumb
|
||||
for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(names))).scalars()
|
||||
}
|
||||
|
||||
|
||||
def cast_count(db: Session, name: str, movie_lib_ids: list[int], show_lib_ids: list[int]) -> int:
|
||||
"""How many movies (in `movie_lib_ids`) and shows (in `show_lib_ids`) name this person in their
|
||||
cast. An empty id list leaves its side out."""
|
||||
count = 0
|
||||
if movie_lib_ids:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
PlexItem.cast_names.contains([name]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
if show_lib_ids:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexShow.id)).where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
PlexShow.cast_names.contains([name]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return count
|
||||
|
||||
|
||||
def filmography(
|
||||
db: Session,
|
||||
kind: Literal["movie", "show"],
|
||||
name: str,
|
||||
lib_ids: list[int],
|
||||
lang: str,
|
||||
limit: int,
|
||||
) -> list:
|
||||
"""The movies or shows in these libraries whose cast or directors name this person, newest
|
||||
first."""
|
||||
model: type[PlexItem] | type[PlexShow] = PlexItem if kind == "movie" else PlexShow
|
||||
query = select(model).where(
|
||||
model.library_id.in_(lib_ids),
|
||||
or_(model.cast_names.contains([name]), model.directors.contains([name])),
|
||||
)
|
||||
if model is PlexItem:
|
||||
query = query.where(PlexItem.kind == "movie")
|
||||
return list(
|
||||
db.execute(
|
||||
query.order_by(
|
||||
model.year.desc().nullslast(), collated(title_col(model, lang), lang)
|
||||
).limit(limit)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- collections -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def collections_page(
|
||||
db: Session, library: str | None, q: str | None, keys: list[str] | None, lang: str
|
||||
) -> list[PlexCollection]:
|
||||
"""Collections, most-populated first: one library's (by its plex key), else every enabled
|
||||
library's; name-filtered by `q`; narrowed to `keys` when given (an empty list asks for none)."""
|
||||
if library is not None:
|
||||
lib = library_by_key(db, library)
|
||||
if lib is None:
|
||||
return []
|
||||
query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
|
||||
else:
|
||||
enabled_ids = list(
|
||||
db.execute(
|
||||
select(PlexLibrary.id).where(
|
||||
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
if not enabled_ids:
|
||||
return []
|
||||
query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
|
||||
term = (q or "").strip()
|
||||
if term:
|
||||
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
|
||||
if keys is not None:
|
||||
# An explicit but empty `keys=` asks for nothing, which is not the same as no `keys` (every
|
||||
# collection): answered here without a query. The empty IN () would also match nothing.
|
||||
if not keys:
|
||||
return []
|
||||
query = query.where(PlexCollection.rating_key.in_(keys))
|
||||
return list(
|
||||
db.execute(
|
||||
query.order_by(
|
||||
PlexCollection.child_count.desc().nullslast(), collated(PlexCollection.title, lang)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def library_collections(db: Session, library_id: int, lang: str) -> list[PlexCollection]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexCollection)
|
||||
.where(PlexCollection.library_id == library_id)
|
||||
.order_by(collated(PlexCollection.title, lang))
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def collections_by_keys(db: Session, keys: list[str]) -> list[PlexCollection]:
|
||||
return list(
|
||||
db.execute(select(PlexCollection).where(PlexCollection.rating_key.in_(keys))).scalars()
|
||||
)
|
||||
|
||||
|
||||
def collection_named(db: Session, library_id: int, title: str) -> PlexCollection | None:
|
||||
"""A collection of this library with this name, case aside."""
|
||||
return (
|
||||
db.execute(
|
||||
select(PlexCollection).where(
|
||||
PlexCollection.library_id == library_id,
|
||||
func.lower(PlexCollection.title) == title.lower(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def collection_members(
|
||||
db: Session, rating_key: str, lang: str
|
||||
) -> tuple[list[PlexItem], list[PlexShow]]:
|
||||
"""A collection's movies and shows, each by year then title."""
|
||||
movies = db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
).scalars()
|
||||
shows = db.execute(
|
||||
select(PlexShow)
|
||||
.where(PlexShow.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
|
||||
).scalars()
|
||||
return list(movies), list(shows)
|
||||
|
||||
|
||||
def library_movies_in(
|
||||
db: Session, library_id: int, rating_key: str, lang: str, limit: int
|
||||
) -> list[PlexItem]:
|
||||
"""The movies of one library in this collection, by year then title."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.library_id == library_id,
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.collection_keys.contains([rating_key]),
|
||||
)
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
.limit(limit)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def show_rollups(db: Session, user_id: int, show_ids: list[int]) -> list:
|
||||
"""The user's episode rollup (total, watched, in progress, last resumed) of each of these shows
|
||||
that has episodes."""
|
||||
if not show_ids:
|
||||
return []
|
||||
return list(
|
||||
db.execute(
|
||||
rollup.agg_subquery(db, user_id, include_resumed=True).where(
|
||||
PlexItem.show_id.in_(show_ids)
|
||||
)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def collection_tiles(
|
||||
db: Session, keys: list[str]
|
||||
) -> tuple[dict[str, list[str]], dict[str, list[str]], dict[str, int]]:
|
||||
"""What each collection's tile shows: the top two movie members' keys, the top two show members'
|
||||
keys (a collection lives in one library, so exactly one side has any), and its movies' total
|
||||
runtime (a show has no single duration)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
|
||||
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
|
||||
runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
|
||||
return art_m, art_s, runtime
|
||||
|
||||
|
||||
# --- the admin page --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def sample_file_path(db: Session) -> str | None:
|
||||
"""Any mirrored item's Plex-side file path — a sample to test the mount and path map against."""
|
||||
return db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
|
||||
|
||||
|
||||
def watch_link(db: Session, user: User | int) -> PlexLink | None:
|
||||
"""The user's Plex watch-sync link, if they have one."""
|
||||
return db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user))).scalars().first()
|
||||
|
||||
|
||||
# --- a user's playlists ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_playlist(
|
||||
db: Session, user: User, playlist_id: int, *, lock: bool = False
|
||||
) -> PlexPlaylist | None:
|
||||
"""The user's playlist with this id; someone else's is the same answer as a missing one. `lock`
|
||||
takes it FOR UPDATE: every add, remove and reorder does, so two of them never read the same max
|
||||
position, and the list row is always locked before its items."""
|
||||
return owned_row(db, PlexPlaylist, playlist_id, user, lock=lock)
|
||||
|
||||
|
||||
def playlists_of(db: Session, user: User) -> list[PlexPlaylist]:
|
||||
"""The user's playlists, newest-touched first."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexPlaylist)
|
||||
.where(owned_by(PlexPlaylist.user_id, user))
|
||||
.order_by(PlexPlaylist.updated_at.desc())
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def playlist_count(db: Session, playlist_id: int) -> int:
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def _playlist_rows(playlist_id: int):
|
||||
return (
|
||||
select(PlexPlaylistItem, PlexItem)
|
||||
.join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
|
||||
.where(PlexPlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
)
|
||||
|
||||
|
||||
def playlist_cover(db: Session, playlist_id: int) -> str | None:
|
||||
"""The rating key of a playlist's first item — its cover."""
|
||||
first = db.execute(_playlist_rows(playlist_id)).first()
|
||||
return first[1].rating_key if first else None
|
||||
|
||||
|
||||
def playlist_items(db: Session, playlist_id: int) -> list[PlexItem]:
|
||||
"""A playlist's items in play order."""
|
||||
return [it for _row, it in db.execute(_playlist_rows(playlist_id)).all()]
|
||||
|
||||
|
||||
def playlist_holds(db: Session, playlist_id: int, item_ids: set[int]) -> int:
|
||||
"""How many of these items the playlist holds."""
|
||||
if not item_ids:
|
||||
return 0
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id,
|
||||
PlexPlaylistItem.item_id.in_(item_ids),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def item_id(db: Session, rating_key: str) -> int | None:
|
||||
"""The id of the mirrored item with this key. Unlike `item`, a NUL in the key reaches the query,
|
||||
which the app answers as a 422 (`main._nul_in_text`), as for every other compared string."""
|
||||
return db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(rating_key)))
|
||||
|
||||
|
||||
def item_ids(db: Session, rating_keys: list[str]) -> set[int]:
|
||||
"""The ids of the mirrored items with these keys."""
|
||||
if not rating_keys:
|
||||
return set()
|
||||
return set(
|
||||
db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rating_keys))).scalars()
|
||||
)
|
||||
|
||||
|
||||
def items_by_keys(db: Session, rating_keys: list) -> list[PlexItem]:
|
||||
"""The mirrored items with these keys, in the given order; an unknown key and a repeat drop out."""
|
||||
order = [str(k) for k in rating_keys if str(k)]
|
||||
if not order:
|
||||
return []
|
||||
by_rk = {
|
||||
it.rating_key: it
|
||||
for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
|
||||
}
|
||||
seen: set[str] = set()
|
||||
out: list[PlexItem] = []
|
||||
for rk in order:
|
||||
it = by_rk.get(rk)
|
||||
if it is not None and rk not in seen:
|
||||
seen.add(rk)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
|
||||
def shows_by_ids(db: Session, show_ids: set[int]) -> dict[int, PlexShow]:
|
||||
if not show_ids:
|
||||
return {}
|
||||
return {
|
||||
sh.id: sh for sh in db.execute(select(PlexShow).where(PlexShow.id.in_(show_ids))).scalars()
|
||||
}
|
||||
@@ -10,7 +10,7 @@ from sqlalchemy import and_, case, func, or_, select, type_coerce
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import PlexItem, PlexLibrary
|
||||
from app.plex.models import PlexItem, PlexLibrary
|
||||
from app.plex.genres import hu_genre
|
||||
|
||||
TS_CONFIG = "public.unaccent_simple"
|
||||
@@ -236,7 +236,7 @@ def apply_meta_filters(q, model, p: dict, lang: str = "en"):
|
||||
# The 4 classes the feed filter + facet expose, derived from the stored `playable` + `transcode_hw`.
|
||||
# ONE definition (this SQL CASE) so the filter WHERE and the facet counts can't drift: `playable_conds`
|
||||
# filters on it, `facets` groups by it. transcode_hw IS NOT True groups NULL (unprobed) with software.
|
||||
# Public — it is the shared source of truth imported by routes.plex.browse (facet order + zero-fill).
|
||||
# Public — it is the shared source of truth imported by plex.library (facet order + zero-fill).
|
||||
PLAYBACK_CLASSES = ("direct", "remux", "transcode_hw", "transcode_sw")
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,9 @@ from datetime import datetime, timezone
|
||||
from sqlalchemy import and_, case, func, or_, select
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.models import PlexItem, PlexState
|
||||
from app.models import User
|
||||
from app.userscope import owned_by
|
||||
from app.plex.models import PlexItem, PlexState
|
||||
from app.plex.cards import ShowState
|
||||
|
||||
# Sort sentinel for a rail row whose timestamp is missing, so `key=lambda r: r[0] or NO_TS` reads the
|
||||
@@ -92,7 +94,7 @@ def agg_subquery(db: Session, user_id: int, include_resumed: bool = False):
|
||||
)
|
||||
return (
|
||||
select(*cols)
|
||||
.outerjoin(ps, and_(ps.item_id == PlexItem.id, ps.user_id == user_id))
|
||||
.outerjoin(ps, and_(ps.item_id == PlexItem.id, owned_by(ps.user_id, user_id)))
|
||||
.where(PlexItem.kind == "episode", PlexItem.show_id.isnot(None))
|
||||
.group_by(PlexItem.show_id)
|
||||
)
|
||||
@@ -105,7 +107,7 @@ def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, ShowS
|
||||
return {r.show_id: show_state(r.total, r.watched, r.inprog) for r in rows}
|
||||
|
||||
|
||||
def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int, PlexState]:
|
||||
def item_state_map(db: Session, user_id: User | int, item_ids: list[int]) -> dict[int, PlexState]:
|
||||
"""A user's PlexState rows for a set of leaf items (movies or episodes), keyed by item_id — the
|
||||
leaf analogue of `state_map` (which rolls a show's episodes up to a `ShowState`). One batched
|
||||
fetch, so the caller serializes cards without a per-card query. Empty in → empty dict (no `IN ()`)."""
|
||||
@@ -114,6 +116,8 @@ def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int,
|
||||
return {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id.in_(item_ids))
|
||||
select(PlexState).where(
|
||||
owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(item_ids)
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,413 @@
|
||||
"""Plex's writes for the HTTP layer, one function per use case. None of them commits: the handler
|
||||
does, so a page that fails to save an enrichment still renders, and a Plex write and its local
|
||||
mirror land together.
|
||||
|
||||
The rest of the mirror's writes stay with the sync (`plex.sync`): a collection's targeted re-sync
|
||||
after a write-back, its local cleanup after a delete.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import CursorResult, delete, func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.db import SessionLocal
|
||||
from app.plex import detail as plex_detail
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup
|
||||
from app.plex.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexLink,
|
||||
PlexPerson,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
)
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def enrich_show(db: Session, sh: PlexShow, meta: dict) -> None:
|
||||
"""Fill a not-yet-enriched show from its live metadata: the full cast (filterable), IMDb and
|
||||
tagline, stamped so later views serve from the row."""
|
||||
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
|
||||
plex_detail.apply_rich_detail(sh, meta)
|
||||
sh.people_enriched_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def enrich_movie(db: Session, it: PlexItem, meta: dict) -> None:
|
||||
"""The same for a movie. The listing sync stored only its top ~3 cast; the live metadata
|
||||
carries all of them."""
|
||||
cast = plex_people.cast_from_meta(meta)
|
||||
if cast:
|
||||
plex_people.enrich_row_people(db, it, cast)
|
||||
plex_detail.apply_rich_detail(it, meta)
|
||||
it.people_enriched_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def set_collection_editable(col: PlexCollection, editable: bool) -> None:
|
||||
col.editable = editable
|
||||
|
||||
|
||||
def keep_created_collection(
|
||||
db: Session, lib: PlexLibrary, rating_key: str, title: str, seed: str
|
||||
) -> None:
|
||||
"""A collection Plex made, whose follow-up read failed: its row is kept, editable. The full sync
|
||||
preserves `editable` but creates a missing row read-only, so the collection would drop out of
|
||||
every "add to" list; and the duplicate-name check reads these rows, so a retry of the same name
|
||||
would make a second one on Plex. Plex made it with exactly the seed in it, so the seed's row says
|
||||
so too — otherwise the title showed unticked in the collection it was just added to, and the
|
||||
collection empty, until that sync."""
|
||||
db.add(
|
||||
PlexCollection(
|
||||
rating_key=rating_key, library_id=lib.id, title=title, editable=True, child_count=1
|
||||
)
|
||||
)
|
||||
Model = PlexItem if lib.kind == "movie" else PlexShow
|
||||
if row := by_rating_key(db, Model, seed):
|
||||
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), rating_key]))
|
||||
db.flush()
|
||||
|
||||
|
||||
def enrich_person_bg(person_id: int) -> None:
|
||||
"""Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
|
||||
AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
|
||||
"no live TMDB call at serve time" invariant). The request's session is already closed by then, so
|
||||
open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
|
||||
next open, and the bio lands from the row by then."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
person = db.get(PlexPerson, person_id)
|
||||
if person is not None:
|
||||
plex_tmdb.enrich_person(db, person)
|
||||
except Exception: # noqa: BLE001 — background best-effort; never surfaces to a user
|
||||
db.rollback()
|
||||
log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def set_watch_link(db: Session, user: User, enabled: bool) -> PlexLink:
|
||||
"""Turn the user's two-way watch sync on or off, creating their link on first use. The link uses
|
||||
the server admin token (`uses_admin`), so it needs no Plex login of its own."""
|
||||
link = queries.watch_link(db, user)
|
||||
if link is None:
|
||||
link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
|
||||
db.add(link)
|
||||
link.sync_enabled = enabled
|
||||
db.flush()
|
||||
return link
|
||||
|
||||
|
||||
# --- a user's playlists ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_playlist(db: Session, user: User, title: str, seed: PlexItem | None) -> PlexPlaylist:
|
||||
"""A new playlist of the user's, holding `seed` when one is given."""
|
||||
pl = PlexPlaylist(user_id=user.id, title=title)
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
if seed is not None:
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=seed.id, position=0))
|
||||
db.flush()
|
||||
return pl
|
||||
|
||||
|
||||
def rename_playlist(pl: PlexPlaylist, title: str) -> None:
|
||||
pl.title = title
|
||||
|
||||
|
||||
def delete_playlist(db: Session, pl: PlexPlaylist) -> None:
|
||||
db.delete(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def _touch(pl: PlexPlaylist) -> None:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _last_position(db: Session, playlist_id: int) -> int:
|
||||
return db.execute(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
).scalar_one()
|
||||
|
||||
|
||||
def add_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
|
||||
"""Append `it` to the (locked) playlist; an item already in it stays where it is."""
|
||||
exists = db.execute(
|
||||
select(PlexPlaylistItem.id).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
).first()
|
||||
if exists is None:
|
||||
pos = _last_position(db, pl.id) + 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def remove_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
|
||||
db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def add_items(db: Session, pl: PlexPlaylist, items: list[PlexItem]) -> int:
|
||||
"""Append many items in their given order (a whole season or show); those already in the list
|
||||
are skipped. Says how many were added."""
|
||||
existing = set(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
)
|
||||
pos = _last_position(db, pl.id)
|
||||
added = 0
|
||||
for it in items:
|
||||
if it.id in existing:
|
||||
continue
|
||||
pos += 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
|
||||
existing.add(it.id)
|
||||
added += 1
|
||||
if added:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return added
|
||||
|
||||
|
||||
def insert_items_at(db: Session, pl: PlexPlaylist, placed: list[tuple[int, PlexItem]]) -> int:
|
||||
"""Insert new rows at their indices (ascending, so each index is where the row stood when the
|
||||
rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
|
||||
already in the list does not move: the add stays idempotent."""
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem)
|
||||
.where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
present = {r.item_id for r in rows}
|
||||
added = 0
|
||||
for at, it in sorted(placed, key=lambda p: p[0]):
|
||||
if it.id in present:
|
||||
continue
|
||||
row = PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0)
|
||||
db.add(row)
|
||||
rows.insert(min(at, len(rows)), row)
|
||||
present.add(it.id)
|
||||
added += 1
|
||||
for i, r in enumerate(rows):
|
||||
r.position = i
|
||||
if added:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return added
|
||||
|
||||
|
||||
def remove_items(db: Session, pl: PlexPlaylist, item_ids: list[int]) -> int:
|
||||
"""Remove these items from the playlist; says how many were there."""
|
||||
if not item_ids:
|
||||
return 0
|
||||
result = db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id.in_(item_ids)
|
||||
),
|
||||
execution_options={"synchronize_session": False},
|
||||
)
|
||||
removed = result.rowcount if isinstance(result, CursorResult) else 0
|
||||
if removed:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return removed
|
||||
|
||||
|
||||
def reorder_playlist(db: Session, pl: PlexPlaylist, rating_keys: list[str]) -> None:
|
||||
"""Set the play order from an explicit list of item keys (0-based positions)."""
|
||||
id_by_rk = {
|
||||
it.rating_key: it.id
|
||||
for it in db.execute(
|
||||
select(PlexItem).where(PlexItem.rating_key.in_(rating_keys or [""]))
|
||||
).scalars()
|
||||
}
|
||||
rows = {
|
||||
r.item_id: r
|
||||
for r in db.execute(
|
||||
select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[int] = set()
|
||||
for rk in rating_keys:
|
||||
row = rows.get(id_by_rk.get(rk, -1))
|
||||
if row is not None and row.item_id not in seen:
|
||||
row.position = pos
|
||||
pos += 1
|
||||
seen.add(row.item_id)
|
||||
# Rows the payload omitted (added in another tab meanwhile) keep their relative order after
|
||||
# the given ones, so no two rows share a position (the YouTube reorder does the same).
|
||||
for row in sorted(
|
||||
(r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
|
||||
):
|
||||
row.position = pos
|
||||
pos += 1
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- a user's watch state --------------------------------------------------------------------
|
||||
|
||||
|
||||
def mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
|
||||
"""Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
|
||||
checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
|
||||
read-then-write let both see "not watched" and both scrobble. The transition is decided by the
|
||||
write itself: an insert that did not conflict, or an update conditional on the old status —
|
||||
under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
|
||||
values = {
|
||||
"status": "watched",
|
||||
"watched_at": now,
|
||||
"position_seconds": 0,
|
||||
"progress_updated_at": now,
|
||||
"synced_to_plex": False,
|
||||
}
|
||||
inserted = db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(user_id=user_id, item_id=item_id, **values)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
.returning(PlexState.id)
|
||||
).first()
|
||||
if inserted is not None:
|
||||
return True
|
||||
mine = owned_by(PlexState.user_id, user_id) & (PlexState.item_id == item_id)
|
||||
result = db.execute(update(PlexState).where(mine, PlexState.status != "watched").values(values))
|
||||
if isinstance(result, CursorResult) and result.rowcount:
|
||||
return True
|
||||
# Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
|
||||
db.execute(update(PlexState).where(mine).values(values))
|
||||
return False
|
||||
|
||||
|
||||
def state_row(
|
||||
db: Session,
|
||||
user: User | int,
|
||||
item_id: int,
|
||||
st: PlexState | None = None,
|
||||
*,
|
||||
synced: bool = False,
|
||||
) -> PlexState:
|
||||
"""The user's state row on this item, made if the read before found none. Made by the insert
|
||||
itself (ON CONFLICT DO NOTHING) and then read: the first checkpoint of an item and the `final`
|
||||
flush beside it routinely both read "none", and a plain add let the second one fail on
|
||||
`uq_plex_user_item` — a 500, and the push it was carrying lost. Every first row goes through
|
||||
here: a checkpoint, a status, a season marked at once, a pull from Plex. `synced` is what a row
|
||||
made now says about Plex; a row another writer made first keeps its own."""
|
||||
if st is not None:
|
||||
return st
|
||||
db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(
|
||||
user_id=user if isinstance(user, int) else user.id,
|
||||
item_id=item_id,
|
||||
synced_to_plex=synced,
|
||||
)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
)
|
||||
made = queries.state(db, user, item_id)
|
||||
assert made is not None # inserted now, or committed by the request that won
|
||||
return made
|
||||
|
||||
|
||||
def clear_progress(db: Session, st: PlexState | None, now: datetime) -> None:
|
||||
"""A checkpoint too early to keep: a state that held nothing else goes, any other forgets its
|
||||
position."""
|
||||
if st is not None and st.status == "new":
|
||||
db.delete(st)
|
||||
elif st is not None:
|
||||
st.position_seconds = 0
|
||||
st.progress_updated_at = now
|
||||
db.flush()
|
||||
|
||||
|
||||
def checkpoint(
|
||||
db: Session,
|
||||
user: User,
|
||||
item_id: int,
|
||||
st: PlexState | None,
|
||||
position: int,
|
||||
now: datetime,
|
||||
*,
|
||||
dirty: bool,
|
||||
) -> PlexState:
|
||||
"""Record a resume position. `dirty` marks it not yet mirrored to Plex — set before a push, so
|
||||
the reconcile re-pushes it if the push fails and the push can settle it once Plex confirms."""
|
||||
st = state_row(db, user, item_id, st)
|
||||
st.position_seconds = position
|
||||
st.progress_updated_at = now
|
||||
if dirty:
|
||||
st.synced_to_plex = False
|
||||
db.flush()
|
||||
return st
|
||||
|
||||
|
||||
def forget_state(db: Session, st: PlexState) -> None:
|
||||
db.delete(st)
|
||||
db.flush()
|
||||
|
||||
|
||||
def set_status(
|
||||
db: Session, user: User, item_id: int, st: PlexState | None, status: str
|
||||
) -> PlexState:
|
||||
"""Set a watched or hidden status; watched also clears the resume point."""
|
||||
st = state_row(db, user, item_id, st)
|
||||
st.status = status
|
||||
if status == "watched":
|
||||
st.watched_at = datetime.now(timezone.utc)
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
db.flush()
|
||||
return st
|
||||
|
||||
|
||||
def bulk_state(
|
||||
db: Session, user_id: int, eps: list[PlexItem], watched: bool
|
||||
) -> list[tuple[int, str, str]]:
|
||||
"""Mark every episode in `eps` watched (or unwatched) for a user and say what changed, as the
|
||||
(item id, rating key, action) pushes a linked Plex account needs. Unwatching keeps the user's
|
||||
private "hidden" marks."""
|
||||
now = datetime.now(timezone.utc)
|
||||
existing = rollup.item_state_map(db, user_id, [e.id for e in eps])
|
||||
to_push: list[tuple[int, str, str]] = []
|
||||
for e in eps:
|
||||
st = existing.get(e.id)
|
||||
prev = st.status if st is not None else "new"
|
||||
if watched:
|
||||
st = state_row(db, user_id, e.id, st)
|
||||
prev = st.status # a first checkpoint beside this may have made it meanwhile
|
||||
st.status = "watched"
|
||||
st.watched_at = now
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
if prev != "watched":
|
||||
to_push.append((e.id, e.rating_key, "watched"))
|
||||
else: # → new: drop any state (but keep the user's private "hidden" marks)
|
||||
if st is not None and prev != "hidden":
|
||||
db.delete(st)
|
||||
if prev != "new":
|
||||
to_push.append((e.id, e.rating_key, "unwatched"))
|
||||
db.flush()
|
||||
return to_push
|
||||
@@ -33,7 +33,7 @@ from sqlalchemy.orm import Session as DbSession
|
||||
from app import logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.downloads.edit import encode_thread_cap
|
||||
from app.models import PlexItem
|
||||
from app.plex.models import PlexItem
|
||||
from app.plex import paths
|
||||
from app.plex.sync import HDR_TRANSFERS, HW_DECODABLE_CODECS
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.db import SessionLocal
|
||||
from app.models import PlexItem
|
||||
from app.plex.models import PlexItem
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex.cache import CACHE_DIR
|
||||
from app.plex.client import PlexClient, PlexError
|
||||
|
||||
@@ -21,7 +21,7 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
|
||||
from app.plex import detail, overlay, people
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.genres import hu_genres
|
||||
|
||||
@@ -22,7 +22,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.models import PlexItem, PlexPerson, PlexShow
|
||||
from app.plex.models import PlexItem, PlexPerson, PlexShow
|
||||
from app.plex.cards import display_title
|
||||
from app.plex.people import people_by_credits, warm_portraits
|
||||
|
||||
@@ -619,9 +619,9 @@ def enrich_person(db: Session, person: PlexPerson) -> bool:
|
||||
if not person.tmdb_person_id or not sysconfig.get_str(db, "tmdb_api_key"):
|
||||
return False
|
||||
try:
|
||||
# Called from a background task (off the request path — see people._enrich_person_bg), so it no
|
||||
# longer blocks a serve; still cap the outbound TMDB call with a short timeout so a hanging TMDB
|
||||
# doesn't tie up a threadpool worker for the default 30s.
|
||||
# Called from a background task (off the request path — see `plex.service.enrich_person_bg`),
|
||||
# so it no longer blocks a serve; still cap the outbound TMDB call with a short timeout so a
|
||||
# hanging TMDB doesn't tie up a threadpool worker for the default 30s.
|
||||
with TmdbClient(db, timeout=8.0) as tmdb:
|
||||
_fetch_person_into(tmdb, person)
|
||||
except TmdbError as exc:
|
||||
|
||||
@@ -29,8 +29,11 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import PlexItem, PlexLibrary, PlexLink, PlexState
|
||||
from app.userscope import owned_by
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexLink, PlexState
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex import queries
|
||||
from app.plex.service import state_row
|
||||
from app.plex.sync import (
|
||||
_TYPE_EPISODE,
|
||||
_TYPE_MOVIE,
|
||||
@@ -196,7 +199,7 @@ def link_for_push(db: Session, user_id: int) -> PlexLink | None:
|
||||
"""The active owner watch-push link for this user, or None when push is off / not the owner.
|
||||
Push is gated on `initial_import_done` so we never push a half-known local state over Plex's
|
||||
authoritative one before the one-time "Plex is master" import has reconciled them."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user_id)).scalars().first()
|
||||
link = db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user_id))).scalars().first()
|
||||
if link and link.uses_admin and link.sync_enabled and link.initial_import_done:
|
||||
return link
|
||||
return None
|
||||
@@ -251,13 +254,7 @@ def push_state_to_plex(
|
||||
# Flag the row synced — but an "unwatched" via status=new deletes the row, so there may be
|
||||
# nothing to flag; that's fine (unscrobble still happened). Skip the flag if the row was
|
||||
# re-edited since we captured `expect_ts` (leave it dirty for its own push / the reconcile).
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user_id, item_id)
|
||||
if st is not None and _row_matches(st, expect_ts):
|
||||
st.synced_to_plex = True
|
||||
db.commit()
|
||||
@@ -318,7 +315,7 @@ def push_bulk_state_to_plex(user_id: int, changes: list[tuple[int, str, str]]) -
|
||||
if synced_ids:
|
||||
db.execute(
|
||||
update(PlexState)
|
||||
.where(PlexState.user_id == user_id, PlexState.item_id.in_(synced_ids))
|
||||
.where(owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(synced_ids))
|
||||
.values(synced_to_plex=True)
|
||||
)
|
||||
db.commit()
|
||||
@@ -391,13 +388,7 @@ def _pull_apply(
|
||||
"""Bring one Plex-side positive signal into Siftlode under last-write-wins. Returns
|
||||
"watched"/"in_progress" if it changed the row, else None. Never clears a row (un-watch is the
|
||||
full reconcile's job) — this only pulls in watched/resume."""
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user_id, item_id)
|
||||
if _same_state(st, status, pos):
|
||||
if st is not None and not st.synced_to_plex:
|
||||
st.synced_to_plex = True # already matches Plex → just settle the flag, no churn
|
||||
@@ -418,8 +409,9 @@ def _pull_apply(
|
||||
# "almost always already pushed" assumption above excludes.
|
||||
return None
|
||||
if st is None:
|
||||
st = PlexState(user_id=user_id, item_id=item_id)
|
||||
db.add(st)
|
||||
st = state_row(db, user_id, item_id, synced=True)
|
||||
if not st.synced_to_plex: # a local change made it first, beside this pull: the guard above
|
||||
return None
|
||||
if status == "watched":
|
||||
st.status, st.position_seconds, st.watched_at, st.progress_updated_at = (
|
||||
"watched",
|
||||
@@ -449,7 +441,7 @@ def _repush_dirty(db: Session, plex: PlexClient, link: PlexLink) -> int:
|
||||
dirty = (
|
||||
db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == link.user_id,
|
||||
owned_by(PlexState.user_id, link.user_id),
|
||||
PlexState.synced_to_plex.is_(False),
|
||||
PlexState.status != "hidden",
|
||||
)
|
||||
@@ -505,7 +497,7 @@ def _reconcile_inprogress_movies(
|
||||
and aren't affected). Note: like the rest of this job, `metadata()` reads the TOKEN account (the
|
||||
owner) — a HTPC playing under a managed Plex user (accountID>1) is out of scope here as elsewhere."""
|
||||
conds = [
|
||||
PlexState.user_id == link.user_id,
|
||||
owned_by(PlexState.user_id, link.user_id),
|
||||
PlexState.status.notin_(("watched", "hidden")),
|
||||
PlexState.position_seconds > _PROGRESS_MIN_S,
|
||||
PlexItem.kind == "movie",
|
||||
@@ -627,7 +619,7 @@ def _clear_unwatched(db: Session, user_id: int, plex_states: dict, settled: set[
|
||||
select(PlexState)
|
||||
.join(PlexItem, PlexItem.id == PlexState.item_id)
|
||||
.join(PlexLibrary, PlexLibrary.id == PlexItem.library_id)
|
||||
.where(PlexState.user_id == user_id, PlexLibrary.plex_key.in_(settled))
|
||||
.where(owned_by(PlexState.user_id, user_id), PlexLibrary.plex_key.in_(settled))
|
||||
)
|
||||
for st in rows.scalars().all():
|
||||
if st.item_id in plex_states or st.status == "hidden" or not st.synced_to_plex:
|
||||
|
||||
@@ -16,6 +16,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import ApiQuotaUsage, QuotaEvent
|
||||
@@ -173,7 +174,7 @@ def actions_today(db: Session, user_id: int, action: str) -> int:
|
||||
select(func.count())
|
||||
.select_from(QuotaEvent)
|
||||
.where(
|
||||
QuotaEvent.user_id == user_id,
|
||||
owned_by(QuotaEvent.user_id, user_id),
|
||||
QuotaEvent.action == action,
|
||||
QuotaEvent.created_at >= pacific_day_start_utc(),
|
||||
)
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
"""Reader's tables: what a user does with the shared catalog — where they are in a video
|
||||
(`VideoState`), their playlists, their saved views, their labels on channels (`Tag`, `ChannelTag`)
|
||||
and the comfort shelf on Home. Only this unit writes them; another unit reads them along a declared
|
||||
edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin, UpdatedAtMixin
|
||||
|
||||
|
||||
class Tag(Base, TimestampMixin):
|
||||
"""A label. System tags (user_id NULL) are computed automatically and shared by all
|
||||
users; user tags (user_id set) are personal. category groups them in the UI."""
|
||||
|
||||
__tablename__ = "tags"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(64))
|
||||
color: Mapped[str | None] = mapped_column(String(16))
|
||||
# language | topic | other
|
||||
category: Mapped[str] = mapped_column(String(16), default="other", server_default="other")
|
||||
|
||||
|
||||
class ChannelTag(Base):
|
||||
"""Links a tag to a channel. source='auto' rows are managed by the auto-tagger and
|
||||
regenerated freely; source='user' rows are never touched by the system."""
|
||||
|
||||
__tablename__ = "channel_tags"
|
||||
__table_args__ = (UniqueConstraint("channel_id", "tag_id", "user_id", name="uq_channel_tag"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
tag_id: Mapped[int] = mapped_column(ForeignKey("tags.id", ondelete="CASCADE"), index=True)
|
||||
# NULL = system/auto link visible to everyone; set = a specific user's link.
|
||||
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
source: Mapped[str] = mapped_column(String(8), default="auto", server_default="auto")
|
||||
confidence: Mapped[float | None] = mapped_column(Float)
|
||||
|
||||
|
||||
class VideoState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a video. Rows exist only for non-default states; the
|
||||
absence of a row means 'new/unseen'."""
|
||||
|
||||
__tablename__ = "video_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
# new | watched | hidden ("saved" is now Watch-later playlist membership, not a status)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Resume position (seconds into the video) for the in-app player. Orthogonal to
|
||||
# status: a partially-watched video keeps status "new" but a non-zero position, so
|
||||
# it can render a progress bar and match the "in progress" feed filter. 0 = not
|
||||
# started / finished (trivially-early and near-finished positions are not stored).
|
||||
# int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class Playlist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user named, ordered collection of videos from the shared catalog.
|
||||
|
||||
`kind`: 'user' (normal) or 'watch_later' (built-in, undeletable, one per user).
|
||||
`source`: 'local' (created here) or 'youtube' (mirrored from the user's YouTube
|
||||
playlists). `yt_playlist_id` links a mirrored/exported playlist to its YouTube id.
|
||||
`dirty` marks a YouTube-linked playlist with local edits not yet pushed, so the read
|
||||
sync won't clobber them. (source/yt_playlist_id/dirty are forward-looking for the YT
|
||||
sync phases; the foundation phase only uses local playlists.)"""
|
||||
|
||||
__tablename__ = "playlists"
|
||||
__table_args__ = (
|
||||
# One Watch later per user, held by the database: two first saves at once each made one.
|
||||
Index(
|
||||
"uq_playlist_watch_later",
|
||||
"user_id",
|
||||
unique=True,
|
||||
postgresql_where=text("kind = 'watch_later'"),
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
name: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16), default="user", server_default="user")
|
||||
source: Mapped[str] = mapped_column(String(16), default="local", server_default="local")
|
||||
yt_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
# True when the current items/name differ from `synced_fingerprint` (unpushed edits).
|
||||
dirty: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# SHA-256 of name + ordered video ids as last synced from / pushed to YouTube; the
|
||||
# baseline `dirty` is derived against. NULL = no baseline yet (treated as dirty).
|
||||
synced_fingerprint: Mapped[str | None] = mapped_column(String)
|
||||
# JSON list of the ordered video ids as of the last sync — the 3-way-merge baseline the
|
||||
# two-way YouTube sync classifies each item against (local-add vs YT-add vs real delete).
|
||||
# NULL = no baseline yet; the first merge then does a lossless union (never deletes).
|
||||
synced_video_ids: Mapped[str | None] = mapped_column(Text)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
|
||||
items: Mapped[list["PlaylistItem"]] = relationship(
|
||||
back_populates="playlist",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="PlaylistItem.position",
|
||||
)
|
||||
|
||||
|
||||
class PlaylistItem(Base):
|
||||
"""A video's membership in a playlist, with its order within that playlist."""
|
||||
|
||||
__tablename__ = "playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "video_id", name="uq_playlist_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
added_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
|
||||
|
||||
playlist: Mapped["Playlist"] = relationship(back_populates="items")
|
||||
|
||||
|
||||
class SavedView(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user named snapshot of a module's filter/sort state (a "smart view").
|
||||
|
||||
`source` names the module the view belongs to ("feed" | "plex"); `filters` is that module's
|
||||
serialized filter JSON the frontend applies with one click. `position` orders them in the sidebar
|
||||
(drag-reorder, per source); `is_default` marks the one view applied automatically on load — at most
|
||||
one per (user, source), enforced by the API on set. The name is unique per (user, source)."""
|
||||
|
||||
__tablename__ = "saved_views"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("user_id", "source", "name", name="uq_user_source_saved_view"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
source: Mapped[str] = mapped_column(String(16), default="feed", server_default="feed")
|
||||
name: Mapped[str] = mapped_column(String(80))
|
||||
filters: Mapped[dict] = mapped_column(JSON)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
is_default: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class ComfortPin(Base, TimestampMixin):
|
||||
"""One entry on a user's comfort shelf — the anti-paralysis fallback: "just put on X".
|
||||
|
||||
Deliberately NOT a boolean on `VideoState`/`PlexState`, and NOT a polymorphic `(kind, ref)` string
|
||||
pair. A flag cannot ORDER a shelf and cannot hold a channel or a show — and comfort media is
|
||||
usually "put on X", a series or a channel, not one particular video. A string ref would order and
|
||||
hold anything, but it would also lose the foreign key: a Plex item dropped from the library, or an
|
||||
unsubscribed channel, would leave a dead tile on the one surface whose whole job is to be
|
||||
reliable. So instead: one nullable CASCADE-ing FK per pinnable kind, and a check constraint that
|
||||
exactly one of them is set.
|
||||
|
||||
That constraint is the house rule applied to a five-way union — the invariant lives in the schema,
|
||||
not in a comment every insert site has to remember. `kind`/`ref` below derive FROM the columns, so
|
||||
there is no second vocabulary to keep in sync.
|
||||
|
||||
NULLs are DISTINCT in a Postgres unique index, so the five per-kind unique constraints each stop a
|
||||
double-pin of the same thing without colliding across kinds."""
|
||||
|
||||
__tablename__ = "comfort_pins"
|
||||
__table_args__ = (
|
||||
CheckConstraint(
|
||||
"num_nonnulls(video_id, channel_id, plex_item_id, plex_show_id, playlist_id) = 1",
|
||||
name="ck_comfort_pin_one_ref",
|
||||
),
|
||||
UniqueConstraint("user_id", "video_id", name="uq_comfort_pin_video"),
|
||||
UniqueConstraint("user_id", "channel_id", name="uq_comfort_pin_channel"),
|
||||
UniqueConstraint("user_id", "plex_item_id", name="uq_comfort_pin_plex_item"),
|
||||
UniqueConstraint("user_id", "plex_show_id", name="uq_comfort_pin_plex_show"),
|
||||
UniqueConstraint("user_id", "playlist_id", name="uq_comfort_pin_playlist"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
video_id: Mapped[str | None] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str | None] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
plex_item_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE")
|
||||
)
|
||||
plex_show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE")
|
||||
)
|
||||
playlist_id: Mapped[int | None] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
|
||||
|
||||
# 0-based shelf order. The shelf is short and hand-curated, so a dense integer the reorder
|
||||
# endpoint rewrites wholesale beats a fractional index nobody would ever exercise here.
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
@@ -0,0 +1,361 @@
|
||||
"""Reader's reads: a user's labels, saved views, watch state and playlists, for the HTTP layer and
|
||||
for the other units that ask about them.
|
||||
|
||||
Every read of a user's row goes through `userscope`, so "someone else's row" is the same answer as
|
||||
a missing one. Reader reads Catalog's videos along its declared edge (a playlist's cover and
|
||||
length, whether a video exists); another unit asks the questions at the end of this module rather
|
||||
than joining Reader's tables itself.
|
||||
"""
|
||||
|
||||
from sqlalchemy import ColumnElement, and_, func, or_, select
|
||||
from sqlalchemy.orm import InstrumentedAttribute, Session, aliased
|
||||
|
||||
from app.collation import collated
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
|
||||
from app.userscope import owned_by, owned_row
|
||||
|
||||
# --- catalog, read along the edge ------------------------------------------------------------
|
||||
|
||||
|
||||
def video(db: Session, video_id: str) -> Video | None:
|
||||
return db.get(Video, video_id)
|
||||
|
||||
|
||||
# --- tags ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def tags_with_counts(db: Session, user: User) -> list[tuple[Tag, int]]:
|
||||
"""The tags this user sees (the shared system ones and their own), each with how many channel
|
||||
links it has, by category then name."""
|
||||
return [
|
||||
(tag, count)
|
||||
for tag, count in db.execute(
|
||||
select(Tag, func.count(ChannelTag.id))
|
||||
.outerjoin(ChannelTag, ChannelTag.tag_id == Tag.id)
|
||||
.where(or_(Tag.user_id.is_(None), owned_by(Tag.user_id, user)))
|
||||
.group_by(Tag.id)
|
||||
.order_by(Tag.category, Tag.name)
|
||||
).tuples()
|
||||
]
|
||||
|
||||
|
||||
def own_tag(db: Session, user: User, tag_id: int) -> Tag | None:
|
||||
"""One of the user's own tags. A system tag (no owner) is nobody's to edit, so it reads as
|
||||
missing too."""
|
||||
return owned_row(db, Tag, tag_id, user)
|
||||
|
||||
|
||||
# --- saved views -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def views(db: Session, user: User, source: str) -> list[SavedView]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(SavedView)
|
||||
.where(owned_by(SavedView.user_id, user), SavedView.source == source)
|
||||
.order_by(SavedView.position, SavedView.id)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def all_views(db: Session, user: User) -> list[SavedView]:
|
||||
return list(db.execute(select(SavedView).where(owned_by(SavedView.user_id, user))).scalars())
|
||||
|
||||
|
||||
def own_view(db: Session, user: User, view_id: int) -> SavedView | None:
|
||||
return owned_row(db, SavedView, view_id, user)
|
||||
|
||||
|
||||
# --- watch state -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def video_state(db: Session, user: User, video_id: str) -> VideoState | None:
|
||||
return db.execute(
|
||||
select(VideoState).where(
|
||||
owned_by(VideoState.user_id, user), VideoState.video_id == video_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
# --- playlists -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_playlist(
|
||||
db: Session, user: User, playlist_id: int, *, lock: bool = False
|
||||
) -> Playlist | None:
|
||||
return owned_row(db, Playlist, playlist_id, user, lock=lock)
|
||||
|
||||
|
||||
def playlists_for(db: Session, user: User) -> list[Playlist]:
|
||||
"""The user's playlists in their rail order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Playlist)
|
||||
.where(owned_by(Playlist.user_id, user))
|
||||
.order_by(Playlist.position, Playlist.created_at)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def watch_later(db: Session, user: User) -> Playlist | None:
|
||||
return db.scalar(
|
||||
select(Playlist).where(owned_by(Playlist.user_id, user), Playlist.kind == "watch_later")
|
||||
)
|
||||
|
||||
|
||||
def item_counts(db: Session, ids: list[int]) -> dict[int, int]:
|
||||
return dict(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id, func.count())
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
)
|
||||
.tuples()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def durations(db: Session, ids: list[int]) -> dict[int, int]:
|
||||
"""Each playlist's running time: the sum of its videos' lengths."""
|
||||
return {
|
||||
pid: int(total or 0)
|
||||
for pid, total in db.execute(
|
||||
select(PlaylistItem.playlist_id, func.coalesce(func.sum(Video.duration_seconds), 0))
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
).tuples()
|
||||
}
|
||||
|
||||
|
||||
def containing(db: Session, ids: list[int], video_id: str) -> set[int]:
|
||||
"""Which of these playlists already hold the video (the add-to-playlist popover's ticks)."""
|
||||
return set(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id).where(
|
||||
PlaylistItem.playlist_id.in_(ids), PlaylistItem.video_id == video_id
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def covers(db: Session, ids: list[int]) -> dict[int, str | None]:
|
||||
"""Each playlist's cover: its first item's thumbnail, for every list in ONE window query rather
|
||||
than a lookup per playlist."""
|
||||
rn = (
|
||||
func.row_number()
|
||||
.over(partition_by=PlaylistItem.playlist_id, order_by=PlaylistItem.position)
|
||||
.label("rn")
|
||||
)
|
||||
first_items = (
|
||||
select(PlaylistItem.playlist_id.label("pid"), Video.thumbnail_url.label("thumb"), rn)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.subquery()
|
||||
)
|
||||
return dict(
|
||||
db.execute(select(first_items.c.pid, first_items.c.thumb).where(first_items.c.rn == 1))
|
||||
.tuples()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def playlist_totals(db: Session, playlist_id: int) -> tuple[int, int, str | None]:
|
||||
"""One playlist's item count, running time and cover. An outer join, so an empty list still
|
||||
answers (0, 0, None)."""
|
||||
count, total = db.execute(
|
||||
select(func.count(PlaylistItem.id), func.coalesce(func.sum(Video.duration_seconds), 0))
|
||||
.select_from(PlaylistItem)
|
||||
.outerjoin(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
).one()
|
||||
cover = db.scalar(
|
||||
select(Video.thumbnail_url)
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlaylistItem.position)
|
||||
.limit(1)
|
||||
)
|
||||
return count or 0, total or 0, cover
|
||||
|
||||
|
||||
def playlist_rows(db: Session, user: User, playlist_id: int) -> list:
|
||||
"""A playlist's videos as feed rows (the same columns the feed serializes), in list order."""
|
||||
state = aliased(VideoState)
|
||||
return list(
|
||||
db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
# `id` breaks a position tie the same way on every read (the renumbering add does too).
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def items(db: Session, playlist_id: int) -> list[PlaylistItem]:
|
||||
"""A playlist's item rows, in list order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlaylistItem)
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def item(db: Session, playlist_id: int, video_id: str) -> PlaylistItem | None:
|
||||
return db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == playlist_id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# --- for the other units ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def kept_by_a_reader(video_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
|
||||
"""A video someone holds: watched, in progress or hidden (a `VideoState`), or in one of their
|
||||
playlists. The catalog's cleanup jobs keep such a video; as a clause, so their DELETE stays
|
||||
one statement."""
|
||||
return or_(
|
||||
select(VideoState.id).where(VideoState.video_id == video_id).exists(),
|
||||
select(PlaylistItem.id).where(PlaylistItem.video_id == video_id).exists(),
|
||||
)
|
||||
|
||||
|
||||
def playlisted_by(db: Session, video_ids: list[str]) -> list[tuple[int, str]]:
|
||||
"""(user id, video id) for every user who has one of these videos in a playlist."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Playlist.user_id, PlaylistItem.video_id)
|
||||
.join(PlaylistItem, PlaylistItem.playlist_id == Playlist.id)
|
||||
.where(PlaylistItem.video_id.in_(video_ids))
|
||||
).tuples()
|
||||
)
|
||||
|
||||
|
||||
def has_auto_tags(channel_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
|
||||
"""A channel the auto-tagger has already labelled, as a clause for its "only the missing" scan."""
|
||||
return (
|
||||
select(ChannelTag.id)
|
||||
.where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.source == "auto",
|
||||
ChannelTag.user_id.is_(None),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
|
||||
|
||||
def view_tag_ids(filters: object) -> list[int]:
|
||||
"""The tag ids a stored view's filters name; anything else in that slot names none."""
|
||||
tags = filters.get("tags") if isinstance(filters, dict) else None
|
||||
if not isinstance(tags, list):
|
||||
return []
|
||||
return [t for t in tags if type(t) is int] # `type`, not isinstance: True is not a tag id
|
||||
|
||||
|
||||
def tags_in_feed_views(db: Session) -> set[int]:
|
||||
"""Every tag id any user's saved feed view filters on. Read defensively (`view_tag_ids`): a view
|
||||
stored before its route checked the shape must not raise for everyone's job."""
|
||||
return {
|
||||
t
|
||||
for (filters,) in db.execute(select(SavedView.filters).where(SavedView.source == "feed"))
|
||||
for t in view_tag_ids(filters)
|
||||
}
|
||||
|
||||
|
||||
def held_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
|
||||
"""Which of these videos the user already holds: watched, in progress or hidden, or in one of
|
||||
their playlists (a live search sinks them below the new finds)."""
|
||||
if not video_ids:
|
||||
return set()
|
||||
has_state = (
|
||||
select(VideoState.id)
|
||||
.where(VideoState.video_id == Video.id, owned_by(VideoState.user_id, user))
|
||||
.exists()
|
||||
)
|
||||
in_playlist = (
|
||||
select(PlaylistItem.id)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(PlaylistItem.video_id == Video.id, owned_by(Playlist.user_id, user))
|
||||
.exists()
|
||||
)
|
||||
return set(
|
||||
db.scalars(select(Video.id).where(Video.id.in_(video_ids), or_(has_state, in_playlist)))
|
||||
)
|
||||
|
||||
|
||||
def feed_rows_by_ids(db: Session, user: User, video_ids: list[str]) -> list[dict]:
|
||||
"""These videos serialized with the feed's projection (their channel, this user's watch state,
|
||||
saved), in the order given."""
|
||||
if not video_ids:
|
||||
return []
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
.where(Video.id.in_(video_ids))
|
||||
).all()
|
||||
by_id = {r.id: serialize(r) for r in rows}
|
||||
return [by_id[i] for i in video_ids if i in by_id]
|
||||
|
||||
|
||||
def playlist_channels(db: Session, user: User, exclude: str | None, lang: str) -> list:
|
||||
"""The channels of the videos in the user's playlists that they don't subscribe to (`exclude`:
|
||||
their own channel), each with how many of those videos it has and across how many playlists;
|
||||
most present first, then by title in the reader's alphabet."""
|
||||
subscribed = select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
|
||||
q = (
|
||||
select(
|
||||
Channel,
|
||||
func.count(func.distinct(PlaylistItem.video_id)),
|
||||
func.count(func.distinct(Playlist.id)),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.where(owned_by(Playlist.user_id, user), Channel.id.not_in(subscribed))
|
||||
)
|
||||
if exclude:
|
||||
q = q.where(Channel.id != exclude)
|
||||
return list(
|
||||
db.execute(
|
||||
q.group_by(Channel.id).order_by(
|
||||
func.count(func.distinct(PlaylistItem.video_id)).desc(),
|
||||
collated(Channel.title, lang),
|
||||
)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def user_tag_links(db: Session, user: User, channel_id: str | None = None) -> dict[str, list[int]]:
|
||||
"""The user's own tag links by channel (one channel's, when given)."""
|
||||
stmt = select(ChannelTag.channel_id, ChannelTag.tag_id).where(
|
||||
owned_by(ChannelTag.user_id, user)
|
||||
)
|
||||
if channel_id is not None:
|
||||
stmt = stmt.where(ChannelTag.channel_id == channel_id)
|
||||
out: dict[str, list[int]] = {}
|
||||
for cid, tag_id in db.execute(stmt).tuples():
|
||||
out.setdefault(cid, []).append(tag_id)
|
||||
return out
|
||||
|
||||
|
||||
def user_tag_link(db: Session, user: User, channel_id: str, tag_id: int) -> ChannelTag | None:
|
||||
return db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == tag_id,
|
||||
owned_by(ChannelTag.user_id, user),
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
@@ -0,0 +1,497 @@
|
||||
"""Reader's writes: a user's labels and saved views, where they are in a video, their playlists'
|
||||
items and order — and the system tags the catalog's auto-tagger hands over.
|
||||
|
||||
Every function here flushes and leaves the commit to its caller (the handler, or the job that owns
|
||||
its session). A write that must not double up keeps its guard where it is written: an upsert on the
|
||||
unique key, a row lock taken before a presence check. The comfort shelf's writes are Home's
|
||||
(`discover.pins`); a playlist's YouTube sync is `sync.playlists`.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import exists, func, select, text, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import User
|
||||
from app.reader import queries
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
|
||||
from app.sync.playlists import lock_playlists, recompute_dirty
|
||||
from app.userscope import owned_by
|
||||
|
||||
TAG_CATEGORIES = ("language", "topic", "other")
|
||||
VIEW_NAME_MAX = 80
|
||||
|
||||
# --- tags ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_tag(db: Session, user: User, name: str, category: str, color: str | None) -> Tag:
|
||||
"""A tag of the user's own. An unknown category falls back to "other" rather than refusing. A
|
||||
name the user already has raises `IntegrityError` at the flush."""
|
||||
tag = Tag(
|
||||
user_id=user.id,
|
||||
name=name,
|
||||
color=color,
|
||||
category=category if category in TAG_CATEGORIES else "other",
|
||||
)
|
||||
db.add(tag)
|
||||
db.flush()
|
||||
return tag
|
||||
|
||||
|
||||
def update_tag(tag: Tag, *, sent: set[str], name: str | None, color: str | None) -> None:
|
||||
"""Change the fields the caller sent: `color` sent as None clears it."""
|
||||
if "name" in sent and name is not None:
|
||||
tag.name = name
|
||||
if "color" in sent:
|
||||
tag.color = color
|
||||
|
||||
|
||||
def delete_tag(db: Session, tag: Tag) -> None:
|
||||
db.delete(tag) # its channel links cascade
|
||||
db.flush()
|
||||
|
||||
|
||||
def ensure_system_tag(db: Session, name: str, category: str, color: str | None) -> Tag:
|
||||
"""The shared system tag of this name, made the first time a channel earns it."""
|
||||
tag = db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), Tag.name == name)
|
||||
).scalar_one_or_none()
|
||||
if tag is None:
|
||||
tag = Tag(user_id=None, name=name, category=category, color=color)
|
||||
db.add(tag)
|
||||
db.flush()
|
||||
return tag
|
||||
|
||||
|
||||
def set_auto_tags(db: Session, channel_id: str, desired: dict[int, float | None]) -> None:
|
||||
"""Make a channel's automatic tag links exactly `desired` (tag id → confidence). Only the
|
||||
auto-tagger's own links move; a user's links on the channel are never touched."""
|
||||
existing = {
|
||||
ct.tag_id: ct
|
||||
for ct in db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.user_id.is_(None),
|
||||
ChannelTag.source == "auto",
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
for tag_id, ct in existing.items():
|
||||
if tag_id not in desired:
|
||||
db.delete(ct)
|
||||
for tag_id, conf in desired.items():
|
||||
ct = existing.get(tag_id)
|
||||
if ct is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel_id,
|
||||
tag_id=tag_id,
|
||||
user_id=None,
|
||||
source="auto",
|
||||
confidence=conf,
|
||||
)
|
||||
)
|
||||
else:
|
||||
ct.confidence = conf
|
||||
db.flush()
|
||||
|
||||
|
||||
def prune_orphan_system_tags(db: Session) -> int:
|
||||
"""Delete the system tags no channel links to any more (after a mapping change) — but not one a
|
||||
saved feed view filters on. Views hold tag IDS, and a deleted tag comes back under a new id when
|
||||
a channel earns it again, so the view's filter would point at nothing; the feed drops an unknown
|
||||
id, and the view showed the whole feed under its old name. The kept ids are filtered here rather
|
||||
than in SQL: a `NOT IN` over every view's ids could pass the driver's bind-parameter limit."""
|
||||
in_views = queries.tags_in_feed_views(db)
|
||||
orphans = [
|
||||
tag
|
||||
for tag in db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), ~exists().where(ChannelTag.tag_id == Tag.id))
|
||||
).scalars()
|
||||
if tag.id not in in_views
|
||||
]
|
||||
for tag in orphans:
|
||||
db.delete(tag)
|
||||
db.flush()
|
||||
return len(orphans)
|
||||
|
||||
|
||||
# --- saved views -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_view(db: Session, user: User, source: str, name: str, filters: dict) -> SavedView:
|
||||
"""A view appended after the user's others of the same module (each module numbers its own
|
||||
0..N). A name the user already has there raises `IntegrityError` at the flush."""
|
||||
maxpos = db.execute(
|
||||
select(func.coalesce(func.max(SavedView.position), -1)).where(
|
||||
owned_by(SavedView.user_id, user), SavedView.source == source
|
||||
)
|
||||
).scalar_one()
|
||||
view = SavedView(
|
||||
user_id=user.id,
|
||||
source=source,
|
||||
name=name[:VIEW_NAME_MAX],
|
||||
filters=filters,
|
||||
position=maxpos + 1,
|
||||
)
|
||||
db.add(view)
|
||||
db.flush()
|
||||
return view
|
||||
|
||||
|
||||
def update_view(
|
||||
db: Session,
|
||||
view: SavedView,
|
||||
*,
|
||||
sent: set[str],
|
||||
name: str | None = None,
|
||||
filters: dict | None = None,
|
||||
is_default: bool | None = None,
|
||||
) -> None:
|
||||
"""Change the fields the caller sent. Making a view the default clears the flag on the user's
|
||||
other views of the same module first: at most one default per (user, source)."""
|
||||
if "name" in sent and name is not None:
|
||||
view.name = name[:VIEW_NAME_MAX]
|
||||
if "filters" in sent and filters is not None:
|
||||
view.filters = filters
|
||||
if "is_default" in sent:
|
||||
make_default = bool(is_default)
|
||||
if make_default:
|
||||
db.execute(
|
||||
update(SavedView)
|
||||
.where(
|
||||
owned_by(SavedView.user_id, view.user_id),
|
||||
SavedView.source == view.source,
|
||||
SavedView.id != view.id,
|
||||
)
|
||||
.values(is_default=False)
|
||||
)
|
||||
view.is_default = make_default
|
||||
db.flush()
|
||||
|
||||
|
||||
def delete_view(db: Session, view: SavedView) -> None:
|
||||
db.delete(view)
|
||||
db.flush()
|
||||
|
||||
|
||||
def reorder_views(db: Session, user: User, ids: list[int]) -> None:
|
||||
"""Number the listed views 0..n in the listed order; ids that are not the user's are skipped."""
|
||||
by_id = {v.id: v for v in queries.all_views(db, user)}
|
||||
pos = 0
|
||||
for vid in ids:
|
||||
v = by_id.get(vid)
|
||||
if v is not None:
|
||||
v.position = pos
|
||||
pos += 1
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- watch state -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def unmark(db: Session, user: User, video_id: str) -> None:
|
||||
"""Back to "new". A row that still holds a resume point keeps it (un-marking "watched" restores
|
||||
the in-progress state, not where the user left off); otherwise the row goes. The player fires
|
||||
this beside a checkpoint that may delete the same row: the caller's commit then raises
|
||||
`StaleDataError`, which is that lost race and nothing worse."""
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is None:
|
||||
return
|
||||
if row.position_seconds:
|
||||
row.status = "new"
|
||||
row.watched_at = None
|
||||
else:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
|
||||
|
||||
def mark(db: Session, user: User, video_id: str, status: str) -> None:
|
||||
""" "watched" or "hidden", as one atomic upsert on (user, video). The player marks "watched" at
|
||||
the end of playback at the instant a checkpoint may delete the in-progress row; a SELECT then
|
||||
UPDATE raised `StaleDataError` on that race, and an INSERT … ON CONFLICT is immune to it.
|
||||
"hidden" keeps any earlier `watched_at`."""
|
||||
now = datetime.now(timezone.utc)
|
||||
set_: dict = {"status": status, "updated_at": now}
|
||||
if status == "watched":
|
||||
set_["watched_at"] = now
|
||||
db.execute(
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
status=status,
|
||||
watched_at=now if status == "watched" else None,
|
||||
)
|
||||
.on_conflict_do_update(constraint="uq_user_video", set_=set_)
|
||||
)
|
||||
|
||||
|
||||
def clear_state(db: Session, user: User, video_id: str) -> None:
|
||||
"""The video back to pristine for this user: status, resume point and `watched_at` all go.
|
||||
Playlist membership lives elsewhere and stays."""
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
|
||||
|
||||
def set_progress(db: Session, user: User, video_id: str, position: int) -> None:
|
||||
"""The player's resume point. 0 clears it: a row holding nothing else goes, one with a status
|
||||
(watched, hidden) keeps it with the position zeroed. The status itself is never changed here.
|
||||
A position is one upsert on (user, video): the first checkpoint and the end-of-playback mark
|
||||
beside it both read "no row", and a plain add let the second one fail on `uq_user_video`."""
|
||||
now = datetime.now(timezone.utc)
|
||||
if position:
|
||||
db.execute(
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
position_seconds=position,
|
||||
progress_updated_at=now,
|
||||
)
|
||||
.on_conflict_do_update(
|
||||
constraint="uq_user_video",
|
||||
set_={"position_seconds": position, "progress_updated_at": now, "updated_at": now},
|
||||
)
|
||||
)
|
||||
return
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is None:
|
||||
return
|
||||
if row.status == "new":
|
||||
db.delete(row)
|
||||
else:
|
||||
row.position_seconds = 0
|
||||
row.progress_updated_at = now
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- playlists -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MoveRefused(Exception):
|
||||
"""A move between playlists that cannot happen: `reason` is the code the UI translates."""
|
||||
|
||||
def __init__(self, reason: str):
|
||||
super().__init__(reason)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def touch(pl: Playlist) -> None:
|
||||
"""Bump the playlist's `updated_at` for an edit that only writes `playlist_items`.
|
||||
|
||||
`UpdatedAtMixin`'s `onupdate` fires on an UPDATE of the PLAYLIST row, and add / remove /
|
||||
reorder / move write the item rows instead — so without this the rail's "recently changed"
|
||||
sort would only ever see renames. The assignment is what makes the column mean "last time
|
||||
this list changed", which is what the rail promises the reader.
|
||||
|
||||
The DATABASE's clock, like the mixin's own `onupdate`: one sort key written from two clocks
|
||||
(this process's and Postgres's) could order a rename and an item edit the wrong way round."""
|
||||
pl.updated_at = func.now()
|
||||
|
||||
|
||||
def _next_playlist_position(db: Session, user_id: int) -> int:
|
||||
"""Append position for a new playlist in this user's rail."""
|
||||
return (
|
||||
db.execute(
|
||||
select(func.coalesce(func.max(Playlist.position), -1)).where(
|
||||
owned_by(Playlist.user_id, user_id)
|
||||
)
|
||||
).scalar_one()
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def next_item_position(db: Session, playlist_id: int) -> int:
|
||||
"""Append position for a new item at the end of a playlist. Locks the playlist row first: two
|
||||
adds (two tabs, the rail and the page) otherwise both read the same max and land on one
|
||||
position, and the list's order between them is then arbitrary."""
|
||||
lock_playlists(db, playlist_id)
|
||||
return (
|
||||
db.execute(
|
||||
select(func.coalesce(func.max(PlaylistItem.position), -1)).where(
|
||||
PlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
).scalar_one()
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def create_playlist(db: Session, user: User, name: str) -> Playlist:
|
||||
pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id))
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
return pl
|
||||
|
||||
|
||||
def watch_later(db: Session, user: User) -> Playlist:
|
||||
"""The user's built-in Watch later playlist, created on first use. Created by the insert itself
|
||||
(ON CONFLICT DO NOTHING on the one-per-user index) and then read: the first two saves at once
|
||||
both read "none", and each used to make a list of its own."""
|
||||
pl = queries.watch_later(db, user)
|
||||
if pl is None:
|
||||
db.execute(
|
||||
pg_insert(Playlist)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
name="Watch later",
|
||||
kind="watch_later",
|
||||
position=_next_playlist_position(db, user.id),
|
||||
)
|
||||
.on_conflict_do_nothing(
|
||||
index_elements=["user_id"], index_where=text("kind = 'watch_later'")
|
||||
)
|
||||
)
|
||||
pl = queries.watch_later(db, user)
|
||||
assert pl is not None # inserted now, or committed by the request that won
|
||||
return pl
|
||||
|
||||
|
||||
def add_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Append a video to a playlist if not already present (idempotent). Returns whether it
|
||||
was added. `mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later).
|
||||
Locked BEFORE the presence check: two adds of one video both saw "absent" and the second one's
|
||||
insert hit uq_playlist_video as a 500."""
|
||||
lock_playlists(db, pl.id)
|
||||
if queries.item(db, pl.id, video_id) is not None:
|
||||
return False
|
||||
db.add(
|
||||
PlaylistItem(playlist_id=pl.id, video_id=video_id, position=next_item_position(db, pl.id))
|
||||
)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
|
||||
def remove_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Remove a video from a playlist if present (idempotent). Returns whether it was removed.
|
||||
`mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later)."""
|
||||
lock_playlists(db, pl.id)
|
||||
found = queries.item(db, pl.id, video_id)
|
||||
if found is None:
|
||||
return False
|
||||
db.delete(found)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
|
||||
def place_item(db: Session, pl: Playlist, video_id: str, position: int) -> None:
|
||||
"""Put a video already in the list at `position` (clamped to the end), renumbering the whole
|
||||
list: positions can have gaps after removals, so "index N" is only meaningful over the
|
||||
ordered rows."""
|
||||
rows = queries.items(db, pl.id)
|
||||
new = next(r for r in rows if r.video_id == video_id)
|
||||
rows.remove(new)
|
||||
rows.insert(min(position, len(rows)), new)
|
||||
for pos, row in enumerate(rows):
|
||||
row.position = pos
|
||||
db.flush()
|
||||
|
||||
|
||||
def add_to_playlist(db: Session, pl: Playlist, video_id: str, position: int | None) -> None:
|
||||
"""Add a video to one of the user's lists, at `position` when given (the list page's "Undo" of
|
||||
a removal puts the row back where it was in ONE request), else at the end. A video already in
|
||||
the list stays where it is."""
|
||||
added = add_item(db, pl, video_id, mark_dirty=False)
|
||||
if added and position is not None:
|
||||
place_item(db, pl, video_id, position)
|
||||
recompute_dirty(db, pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def rename_playlist(db: Session, pl: Playlist, name: str) -> None:
|
||||
"""A new name moves `dirty`; the caller holds the playlist's lock (the scheduled mirror refresh
|
||||
reads `dirty` under it)."""
|
||||
if name != pl.name:
|
||||
pl.name = name
|
||||
recompute_dirty(db, pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def delete_playlist(db: Session, pl: Playlist) -> None:
|
||||
"""The playlist first, as every item write: the cascade deletes the items after the list row
|
||||
is locked."""
|
||||
lock_playlists(db, pl.id)
|
||||
db.delete(pl) # items cascade
|
||||
db.flush()
|
||||
|
||||
|
||||
def move_item(db: Session, src: Playlist, dst: Playlist, video_id: str) -> None:
|
||||
"""Move one video from `src` into `dst`, in the caller's one transaction.
|
||||
|
||||
The client-side shape — add to the target, then remove from the source — leaves the video in
|
||||
BOTH lists whenever the second call fails. Idempotent: a video already in the target is simply
|
||||
dropped from the source (moving into a list that has it is a removal, not a duplicate). Raises
|
||||
`MoveRefused` for the same list twice, or a video a concurrent edit already took out."""
|
||||
if src.id == dst.id:
|
||||
raise MoveRefused("same_playlist")
|
||||
# Both lists, in id order: a move A→B beside a move B→A would otherwise lock crosswise.
|
||||
lock_playlists(db, src.id, dst.id)
|
||||
row = queries.item(db, src.id, video_id)
|
||||
if row is None:
|
||||
raise MoveRefused("not_in_playlist")
|
||||
dst_changed = False
|
||||
if queries.item(db, dst.id, video_id) is None:
|
||||
# Re-parent the existing row rather than delete+insert: `added_at` ("when I saved this")
|
||||
# survives the move, and there is no window where the video belongs to neither list. The
|
||||
# check above holds: both lists are locked, and an insert into the target (another tab's
|
||||
# add, a YouTube merge) waits for the lock — its foreign key needs the target row.
|
||||
row.playlist_id = dst.id
|
||||
row.position = next_item_position(db, dst.id)
|
||||
dst_changed = True
|
||||
if not dst_changed:
|
||||
db.delete(row)
|
||||
# The source always changed; the target only when the row landed in it. Touching a target
|
||||
# that already had the video promoted an untouched list in the rail's "recently changed" sort.
|
||||
recompute_dirty(db, src)
|
||||
touch(src)
|
||||
if dst_changed:
|
||||
recompute_dirty(db, dst)
|
||||
touch(dst)
|
||||
db.flush()
|
||||
|
||||
|
||||
def reorder_items(db: Session, pl: Playlist, order: list[str]) -> int:
|
||||
"""Number the listed videos 0..n in the listed order. Items the list left out (added in another
|
||||
tab meanwhile) keep their relative order after them, so no two items share a position. The
|
||||
caller holds the playlist's lock. Returns how many items the list has."""
|
||||
by_video = {it.video_id: it for it in queries.items(db, pl.id)}
|
||||
pos = 0
|
||||
seen: set[str] = set()
|
||||
for vid in order:
|
||||
it = by_video.get(vid)
|
||||
if it is not None:
|
||||
it.position = pos
|
||||
pos += 1
|
||||
seen.add(vid)
|
||||
for it in sorted(
|
||||
(it for vid, it in by_video.items() if vid not in seen), key=lambda i: i.position
|
||||
):
|
||||
it.position = pos
|
||||
pos += 1
|
||||
recompute_dirty(db, pl)
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return pos
|
||||
|
||||
|
||||
def attach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
|
||||
"""The user's own label on a channel (idempotent). The caller has checked the tag is theirs."""
|
||||
if queries.user_tag_link(db, user, channel_id, tag_id) is None:
|
||||
db.add(ChannelTag(channel_id=channel_id, tag_id=tag_id, user_id=user.id, source="user"))
|
||||
db.flush()
|
||||
|
||||
|
||||
def detach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
|
||||
row = queries.user_tag_link(db, user, channel_id, tag_id)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
@@ -89,8 +89,9 @@ def owned_or_404(
|
||||
) -> M:
|
||||
"""Fetch a row by primary key that `user` owns, or raise 404 (`lock`: see `get_or_404`). A row owned by someone
|
||||
else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger.
|
||||
It replaced the near-identical per-route ownership helpers, and is called directly — a thin
|
||||
`_own_x` wrapper around it is the ad-hoc shape CLAUDE.md's patterns table rules out."""
|
||||
A core route calls it directly. A unit's route cannot (it may not name a unit's model as a value):
|
||||
it asks its unit's `queries.own_x`, which goes through the same `owned_row`, and answers
|
||||
`not_found` itself."""
|
||||
obj = owned_row(db, model, id_, user, owner_attr=owner_attr, lock=lock)
|
||||
if obj is None:
|
||||
raise not_found(what)
|
||||
|
||||
+107
-108
@@ -5,9 +5,10 @@ helpers) stays in app/auth.py, imported below."""
|
||||
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import NamedTuple
|
||||
from typing import Literal, NamedTuple
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from authlib.common.security import generate_token
|
||||
from authlib.integrations.starlette_client import OAuth, OAuthError
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
|
||||
from fastapi.responses import JSONResponse, RedirectResponse
|
||||
@@ -16,6 +17,7 @@ from sqlalchemy import select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -24,10 +26,11 @@ from app import sysconfig
|
||||
from app.auth import (
|
||||
_YOUTUBE_SCOPES,
|
||||
BASE_SCOPES,
|
||||
MAX_SESSION_ACCOUNTS,
|
||||
EXPECTED_ACCOUNT_HEADER,
|
||||
PASSWORD_MIN_LEN,
|
||||
READ_SCOPES,
|
||||
WRITE_SCOPES,
|
||||
built_for_another_account,
|
||||
current_user,
|
||||
env_admin_canonicals,
|
||||
env_allowed_canonicals,
|
||||
@@ -36,7 +39,9 @@ from app.auth import (
|
||||
is_allowed,
|
||||
log,
|
||||
operator_contact,
|
||||
resolved_user_id,
|
||||
optional_current_user,
|
||||
session_user_id,
|
||||
stamp_session,
|
||||
)
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal, get_db
|
||||
@@ -319,6 +324,25 @@ class SetPasswordIn(BaseModel):
|
||||
current_password: str | None = None
|
||||
|
||||
|
||||
# A connect in flight, in the session: `{state, uid, explicit}`. It names the OAuth state of the flow
|
||||
# that set it, because the session is the browser's, not the flow's: a callback coming back with
|
||||
# another state (an older flow, which authlib then refuses) must not spend the marker of the connect
|
||||
# still open, or that connect would return as a plain sign-in — signing someone else in, or storing
|
||||
# a YouTube grant on whichever account the Google identity already belongs to.
|
||||
_LINK_MARKER = "oauth_link"
|
||||
|
||||
# Where a navigation from a tab showing another account than the one signed in lands: the app, which
|
||||
# boots as the account signed in now and says why (the frontend's `ACCOUNT_CHANGED_FLAG`).
|
||||
ACCOUNT_CHANGED_LANDING = "/?account=changed"
|
||||
|
||||
|
||||
def _take_link_marker(request: Request) -> dict | None:
|
||||
marker = request.session.get(_LINK_MARKER)
|
||||
if not isinstance(marker, dict) or marker.get("state") != request.query_params.get("state"):
|
||||
return None
|
||||
return request.session.pop(_LINK_MARKER)
|
||||
|
||||
|
||||
@router.get("/login")
|
||||
async def login(request: Request, db: Session = Depends(get_db)):
|
||||
# access_type=offline ensures a refresh_token on first authorization. We avoid
|
||||
@@ -329,6 +353,10 @@ async def login(request: Request, db: Session = Depends(get_db)):
|
||||
# Google sign-in isn't configured on this instance — land back on the login page (the UI
|
||||
# hides the button, so this is just a safety net for a stale/direct hit).
|
||||
return RedirectResponse(url="/")
|
||||
# A sign-in, never a link. Starting it retires every earlier flow's state (authlib keeps only the
|
||||
# newest), so a connect's marker left from before — abandoned at Google, or still open in
|
||||
# another tab — names a flow that can no longer finish.
|
||||
request.session.pop(_LINK_MARKER, None)
|
||||
return await client.authorize_redirect(
|
||||
request,
|
||||
_oauth_redirect_uri(request),
|
||||
@@ -349,24 +377,23 @@ async def callback(
|
||||
if client is None:
|
||||
return RedirectResponse(url="/")
|
||||
# Linking mode: an already-authenticated account is attaching this Google identity (a
|
||||
# password account enabling YouTube or connecting SSO). The markers are set by /auth/link and
|
||||
# /auth/upgrade; when present we attach to the current user instead of identifying by sub.
|
||||
# Pop them BEFORE the token exchange: if the user aborts Google's consent screen the exchange
|
||||
# throws below, and a marker left behind would silently turn the NEXT clean sign-in into a link
|
||||
# attempt. Popping up front means an aborted link flow leaves no residue in the session.
|
||||
link_uid = request.session.pop("oauth_link_uid", None)
|
||||
link_explicit = request.session.pop("oauth_link_explicit", False)
|
||||
# password account enabling YouTube or connecting SSO). The marker is set by
|
||||
# /auth/google/connect; when it is this flow's we attach to the current user instead of
|
||||
# identifying by sub. Taken BEFORE the token exchange: if the user aborts Google's consent screen
|
||||
# the exchange throws below, and a marker left behind would silently turn the NEXT clean sign-in
|
||||
# into a link attempt. A connect that never comes back here at all is cleared by /auth/login.
|
||||
link = _take_link_marker(request)
|
||||
try:
|
||||
token = await client.authorize_access_token(request)
|
||||
except OAuthError as exc:
|
||||
# The user cancelled or denied consent at Google — land back with a friendly notice instead
|
||||
# of a raw 400 JSON error page. An already-signed-in user (adding another account via
|
||||
# /auth/login, or linking/upgrading via /auth/link|upgrade) never sees the pre-auth login
|
||||
# banner, so surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on
|
||||
# the login page with the oauth_cancelled banner instead. Keyed on the live session, not on
|
||||
# link_uid, so the add-account path (no marker) is covered too.
|
||||
# of a raw 400 JSON error page. An already-signed-in user (signing in again via /auth/login,
|
||||
# or connecting via /auth/google/connect) never sees the pre-auth login banner, so
|
||||
# surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on the login page
|
||||
# with the oauth_cancelled banner instead. Keyed on the live session, not on link_uid, so a
|
||||
# sign-in started while signed in (no marker) is covered too.
|
||||
log.info("Google OAuth cancelled/failed: %s", exc.error)
|
||||
if request.session.get("user_id") is not None:
|
||||
if session_user_id(request) is not None:
|
||||
return RedirectResponse(url="/?oauth=cancelled")
|
||||
return RedirectResponse(url="/?login=oauth_cancelled")
|
||||
|
||||
@@ -377,8 +404,8 @@ async def callback(
|
||||
sub = userinfo["sub"]
|
||||
email = (userinfo.get("email") or "").lower()
|
||||
|
||||
if link_uid is not None:
|
||||
return _complete_link(request, db, link_uid, link_explicit, sub, userinfo, token)
|
||||
if link is not None:
|
||||
return _complete_link(request, db, link["uid"], link["explicit"], sub, userinfo, token)
|
||||
|
||||
if not is_allowed(db, email):
|
||||
# No uid to name — having no account here is the branch. And the address is left out on
|
||||
@@ -579,19 +606,16 @@ def _complete_link(
|
||||
token: dict,
|
||||
):
|
||||
"""Attach the just-authorized Google identity to the already-signed-in account that started
|
||||
the link/upgrade. No is_allowed gate — they're already an active user. Refuses to hijack a
|
||||
the connect. No is_allowed gate — they're already an active user. Refuses to hijack a
|
||||
Google identity owned by another account, or to silently swap a different linked identity.
|
||||
`explicit` (the popped oauth_link_explicit marker) distinguishes a "Connect Google" click from
|
||||
a YouTube upgrade — the caller pops it up front so an aborted flow leaves nothing behind."""
|
||||
# SEC-11: the marker must name an account signed into THIS browser — the session default OR any
|
||||
# account in the wallet — not only the default. A tab acting as a secondary account links Google to
|
||||
# ITSELF (the marker was set from current_user, which resolves the ?account= it navigated with), so
|
||||
# accepting wallet membership stops the identity/token landing on the default account A instead of B.
|
||||
accounts = request.session.get("account_ids") or []
|
||||
if link_uid != request.session.get("user_id") and link_uid not in accounts:
|
||||
return RedirectResponse(url="/?link=error")
|
||||
user = db.get(User, link_uid)
|
||||
if user is None:
|
||||
`explicit` (from the link marker) distinguishes a "Connect Google" click from a YouTube grant —
|
||||
the caller takes the marker up front so an aborted flow leaves nothing behind."""
|
||||
# The marker must name the account still signed into THIS browser, and that session must still
|
||||
# hold: a sign-in as someone else between the start and the callback replaced it (the identity
|
||||
# must not land on the newcomer), and a session revoked meanwhile ("log out everywhere", a
|
||||
# password change, a suspension) must not finish what it started.
|
||||
user = optional_current_user(request, db)
|
||||
if user is None or user.id != link_uid:
|
||||
return RedirectResponse(url="/?link=error")
|
||||
other = db.execute(
|
||||
select(User).where(User.google_sub == sub, User.id != user.id)
|
||||
@@ -611,45 +635,31 @@ def _complete_link(
|
||||
_store_token(db, user, token)
|
||||
db.commit()
|
||||
log.info("Google linked: uid=%s scopes=%s", user.id, token.get("scope"))
|
||||
# An explicit "Connect Google" lands on Settings with a confirmation; a YouTube upgrade
|
||||
# An explicit "Connect Google" lands on Settings with a confirmation; a YouTube grant
|
||||
# (wizard / Settings access rows) returns to the app so the onboarding flow resumes as before.
|
||||
return RedirectResponse(url="/?link=ok" if explicit else "/")
|
||||
|
||||
|
||||
@router.post("/logout")
|
||||
async def logout(request: Request):
|
||||
"""Sign the requesting tab's active account out of this browser (removing it from the wallet).
|
||||
Per-tab aware: the account is taken from the X-Siftlode-Account header when present, so logging
|
||||
out in one tab doesn't disturb another tab running a different account. If the account being
|
||||
removed was the session default, promote the most recent remaining account (or clear the
|
||||
session when none remain)."""
|
||||
# A tab still pinned to an account already signed out in another tab resolves to nobody, so it
|
||||
# signs nobody out — it used to fall back to the default and sign THAT account out in its place.
|
||||
active, is_default = resolved_user_id(request)
|
||||
remaining = [a for a in (request.session.get("account_ids") or []) if a != active]
|
||||
request.session["account_ids"] = remaining
|
||||
if is_default:
|
||||
if remaining:
|
||||
request.session["user_id"] = remaining[-1]
|
||||
return JSONResponse({"ok": True, "switched": True})
|
||||
request.session.clear()
|
||||
return JSONResponse({"ok": True, "switched": False})
|
||||
# A non-default (per-tab) account signed out: it's gone from the wallet and the session default
|
||||
# is untouched. The tab drops its own override and falls back to the default on reload.
|
||||
return JSONResponse({"ok": True, "switched": False})
|
||||
"""Sign this browser out: the session holds one account, and it ends — unless the tab asking was
|
||||
built for another account than the one signed in now (a sign-in as someone else in another tab):
|
||||
its Sign out would end THAT person's session (see `EXPECTED_ACCOUNT_HEADER`)."""
|
||||
signed_in = session_user_id(request)
|
||||
if signed_in is not None and built_for_another_account(
|
||||
signed_in, request.headers.get(EXPECTED_ACCOUNT_HEADER)
|
||||
):
|
||||
raise refused(401, "account_changed")
|
||||
request.session.clear()
|
||||
return JSONResponse({"ok": True})
|
||||
|
||||
|
||||
def _establish_session(request: Request, user: User) -> None:
|
||||
"""Sign `user` in for this browser session — the ONE sign-in path, shared by the Google callback
|
||||
and password login, so both join the same account switcher and both stamp `last_login_at` (the
|
||||
caller commits). Multi-session: every account that authenticated in this browser is remembered so
|
||||
the user can switch between them; only accounts that actually proved themselves HERE land in that
|
||||
wallet (it is the proof they may be switched to)."""
|
||||
accounts = [a for a in (request.session.get("account_ids") or []) if a != user.id]
|
||||
accounts.append(user.id)
|
||||
request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
|
||||
and password login, so both stamp `last_login_at` (the caller commits). One account per session:
|
||||
the sign-in replaces whoever was signed in here before."""
|
||||
request.session["user_id"] = user.id
|
||||
_remember_epoch(request, user)
|
||||
stamp_session(request, user)
|
||||
user.last_login_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
@@ -1124,7 +1134,7 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
|
||||
# Burn any other outstanding reset tokens for this user.
|
||||
for row in db.execute(
|
||||
select(AuthToken).where(
|
||||
AuthToken.user_id == user.id,
|
||||
owned_by(AuthToken.user_id, user),
|
||||
AuthToken.kind.in_(_PASSWORD_TOKEN_KINDS),
|
||||
AuthToken.used_at.is_(None),
|
||||
)
|
||||
@@ -1134,15 +1144,6 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
def _remember_epoch(request: Request, user: User) -> None:
|
||||
"""Record this account's current `session_epoch` in the signed cookie (SA4), so `current_user`
|
||||
can later reject the cookie if the account's epoch was bumped (password reset/change, or
|
||||
"log out everywhere"). Per-account so one account's revocation doesn't evict the others."""
|
||||
epochs = dict(request.session.get("epochs") or {})
|
||||
epochs[str(user.id)] = user.session_epoch
|
||||
request.session["epochs"] = epochs
|
||||
|
||||
|
||||
def bump_session_epoch(db: Session, user: User) -> None:
|
||||
"""Invalidate every EXISTING signed cookie for this account by advancing its epoch (SA4). Cookies
|
||||
minted before the bump carry a now-stale epoch and are rejected by `current_user`. The account's
|
||||
@@ -1219,35 +1220,11 @@ def logout_others(
|
||||
# session at a newer epoch than the DB (see set_password).
|
||||
bump_session_epoch(db, user)
|
||||
db.commit()
|
||||
_remember_epoch(request, user)
|
||||
stamp_session(request, user)
|
||||
log.info("Logout-others: uid=%s (epoch=%s)", user.id, user.session_epoch)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.get("/link")
|
||||
async def link_google(
|
||||
request: Request,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Start linking a Google account to the signed-in (e.g. password) account. Requests only
|
||||
the identity scopes — YouTube access is granted separately via /auth/upgrade. The callback
|
||||
sees `oauth_link_uid` and attaches the identity instead of creating a new account."""
|
||||
client = google_oauth(db)
|
||||
if client is None:
|
||||
return RedirectResponse(url="/?link=error")
|
||||
request.session["oauth_link_uid"] = user.id
|
||||
request.session["oauth_link_explicit"] = True
|
||||
return await client.authorize_redirect(
|
||||
request,
|
||||
_oauth_redirect_uri(request),
|
||||
access_type="offline",
|
||||
prompt="select_account",
|
||||
include_granted_scopes="true",
|
||||
scope=BASE_SCOPES,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/set-password")
|
||||
def set_password(
|
||||
body: SetPasswordIn,
|
||||
@@ -1271,35 +1248,57 @@ def set_password(
|
||||
# can't leave the cookie at a newer epoch than the DB (which would wrongly 401 THIS session).
|
||||
bump_session_epoch(db, user)
|
||||
db.commit()
|
||||
_remember_epoch(request, user)
|
||||
stamp_session(request, user)
|
||||
log.info("Password set/changed: uid=%s", user.id)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.get("/upgrade")
|
||||
async def upgrade(
|
||||
# What each Google connect asks Google for. `identity` attaches a Google sign-in to the account
|
||||
# (Settings → Connect Google) and lets the person pick which Google account; `read` and `write` grant
|
||||
# YouTube access (the onboarding wizard, the Settings access rows) and always show the consent screen,
|
||||
# so a scope granted earlier and since revoked at Google is asked for again.
|
||||
_CONNECT: dict[str, tuple[str, str]] = {
|
||||
"identity": (BASE_SCOPES, "select_account"),
|
||||
"read": (READ_SCOPES, "consent"),
|
||||
"write": (WRITE_SCOPES, "consent"),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/google/connect")
|
||||
async def google_connect(
|
||||
request: Request,
|
||||
access: str = "read",
|
||||
access: Literal["identity", "read", "write"],
|
||||
expect: str | None = None,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Incremental consent for the onboarding wizard. `access=read` grants YouTube
|
||||
read-only (enough to build the feed); `access=write` additionally grants management
|
||||
(unsubscribe). The shared callback stores whatever Google returns, so `can_read` /
|
||||
`can_write` flip on once granted. Anything other than an explicit `write` defaults to
|
||||
the least-privileged read scope."""
|
||||
"""Attach Google to the signed-in account: its identity, or YouTube read/write access on top.
|
||||
The shared callback sees the link marker and attaches what Google returns to THIS account —
|
||||
without it a password account (no google_sub) would get a brand-new, separate Google user
|
||||
created instead. It stores whatever scopes Google grants, so `can_read` / `can_write` flip on
|
||||
once granted.
|
||||
|
||||
`expect` is the account the tab was built for. A navigation carries no headers, so the check
|
||||
every request gets (`EXPECTED_ACCOUNT_HEADER`) rides the query here: a tab still showing the
|
||||
account signed in before would otherwise attach a Google identity to the one signed in now."""
|
||||
if built_for_another_account(user.id, expect):
|
||||
return RedirectResponse(url=ACCOUNT_CHANGED_LANDING)
|
||||
client = google_oauth(db)
|
||||
if client is None:
|
||||
return RedirectResponse(url="/")
|
||||
# Attach the grant to THIS account in the callback. Without it a password account (no
|
||||
# google_sub) would get a brand-new, separate Google user created instead of being linked.
|
||||
request.session["oauth_link_uid"] = user.id
|
||||
scope = WRITE_SCOPES if access == "write" else READ_SCOPES
|
||||
return RedirectResponse(url="/?link=error")
|
||||
scope, prompt = _CONNECT[access]
|
||||
state = generate_token(48)
|
||||
request.session[_LINK_MARKER] = {
|
||||
"state": state,
|
||||
"uid": user.id,
|
||||
"explicit": access == "identity",
|
||||
}
|
||||
return await client.authorize_redirect(
|
||||
request,
|
||||
_oauth_redirect_uri(request),
|
||||
state=state,
|
||||
access_type="offline",
|
||||
prompt="consent",
|
||||
prompt=prompt,
|
||||
include_granted_scopes="true",
|
||||
scope=scope,
|
||||
)
|
||||
|
||||
+70
-261
@@ -4,31 +4,22 @@ hidden, tags) and per-channel sync state. Local reads/writes plus YouTube-side w
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from functools import partial
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import Select, func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app.mailtext import lang_for_user
|
||||
from app import logs, quota, scheduler
|
||||
from app.auth import admin_user, current_user, require_write_scope
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ChannelTag,
|
||||
ExploredChannel,
|
||||
Playlist,
|
||||
PlaylistItem,
|
||||
Subscription,
|
||||
Tag,
|
||||
User,
|
||||
Video,
|
||||
)
|
||||
from app.routes._common import get_or_404, not_found, owned_or_404, youtube_refusal
|
||||
from app.models import User
|
||||
from app.catalog import queries, service
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
from app.routes._common import not_found, youtube_refusal
|
||||
from app.sync.explore import explore_ingest_page
|
||||
from app.sync.runner import backfill_channel_now, run_recent_backfill
|
||||
from app.sync.subscriptions import apply_channel_details, details_stale
|
||||
@@ -36,35 +27,17 @@ from app.utils import iso
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.catalog.models import Channel, Subscription
|
||||
|
||||
router = APIRouter(prefix="/api/channels", tags=["channels"])
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def channel_aggregates(channel_ids: list[str]) -> Select:
|
||||
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
|
||||
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
|
||||
alone; a test holds that (test_channel_list_index_only)."""
|
||||
return (
|
||||
select(
|
||||
Video.channel_id,
|
||||
func.count(Video.id),
|
||||
func.max(Video.published_at),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.where(Video.channel_id.in_(channel_ids))
|
||||
.group_by(Video.channel_id)
|
||||
)
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_channels(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
rows = db.execute(
|
||||
select(Subscription, Channel)
|
||||
.join(Channel, Channel.id == Subscription.channel_id)
|
||||
.where(Subscription.user_id == user.id)
|
||||
.order_by(Subscription.priority.desc(), collated(Channel.title, lang_for_user(user)))
|
||||
).all()
|
||||
rows = queries.subscriptions(db, user, lang_for_user(user))
|
||||
|
||||
# Per-channel aggregates over the shared catalog (channel_aggregates), computed on read rather than
|
||||
# denormalised: an Index Only Scan over a ~20 MB index, where it used to read every video's heap page
|
||||
@@ -73,36 +46,21 @@ def list_channels(user: User = Depends(current_user), db: Session = Depends(get_
|
||||
# computed here — the manager table dropped its N/S/L column.)
|
||||
channel_ids = [c.id for _, c in rows]
|
||||
agg: dict[str, dict] = {}
|
||||
if channel_ids:
|
||||
for cid, n, last_up, total_dur in db.execute(channel_aggregates(channel_ids)).all():
|
||||
agg[cid] = {
|
||||
"stored": n,
|
||||
"last_video_at": iso(last_up),
|
||||
"total_duration_seconds": int(total_dur or 0),
|
||||
}
|
||||
for cid, n, last_up, total_dur in queries.aggregates(db, channel_ids):
|
||||
agg[cid] = {
|
||||
"stored": n,
|
||||
"last_video_at": iso(last_up),
|
||||
"total_duration_seconds": int(total_dur or 0),
|
||||
}
|
||||
|
||||
# Channels already in the *global* deep-backfill queue — i.e. at least one user (any
|
||||
# user) has requested full history and it isn't done yet. Their whole back-catalogue is
|
||||
# coming for everyone (video data is shared), so the UI shouldn't offer "get full
|
||||
# history" to a user who simply hasn't personally opted in.
|
||||
deep_queued: set[str] = set()
|
||||
if channel_ids:
|
||||
for (cid,) in db.execute(
|
||||
select(Subscription.channel_id)
|
||||
.where(
|
||||
Subscription.channel_id.in_(channel_ids),
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.distinct()
|
||||
).all():
|
||||
deep_queued.add(cid)
|
||||
deep_queued = queries.deep_queued(db, channel_ids)
|
||||
|
||||
# The user's personal tag links, grouped by channel.
|
||||
tags_by_channel: dict[str, list[int]] = {}
|
||||
for cid, tag_id in db.execute(
|
||||
select(ChannelTag.channel_id, ChannelTag.tag_id).where(ChannelTag.user_id == user.id)
|
||||
).all():
|
||||
tags_by_channel.setdefault(cid, []).append(tag_id)
|
||||
tags_by_channel = reader_queries.user_tag_links(db, user)
|
||||
|
||||
return [
|
||||
{
|
||||
@@ -131,27 +89,7 @@ DISCOVERY_ENRICH_CAP = 200
|
||||
def _discovery_rows(db: Session, user: User) -> list:
|
||||
"""The user's playlist channels they don't subscribe to (and that aren't their own),
|
||||
each with how many playlist videos come from it and across how many playlists."""
|
||||
subscribed = select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
q = (
|
||||
select(
|
||||
Channel,
|
||||
func.count(func.distinct(PlaylistItem.video_id)),
|
||||
func.count(func.distinct(Playlist.id)),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.where(Playlist.user_id == user.id, Channel.id.not_in(subscribed))
|
||||
)
|
||||
if user.yt_channel_id:
|
||||
q = q.where(Channel.id != user.yt_channel_id)
|
||||
return db.execute(
|
||||
q.group_by(Channel.id).order_by(
|
||||
func.count(func.distinct(PlaylistItem.video_id)).desc(),
|
||||
collated(Channel.title, lang_for_user(user)),
|
||||
)
|
||||
).all()
|
||||
return reader_queries.playlist_channels(db, user, user.yt_channel_id, lang_for_user(user))
|
||||
|
||||
|
||||
@router.get("/discovery")
|
||||
@@ -173,7 +111,7 @@ def discover_channels(
|
||||
):
|
||||
own = yt.get_my_channel_id()
|
||||
if own:
|
||||
user.yt_channel_id = own
|
||||
service.remember_own_channel(user, own)
|
||||
db.commit()
|
||||
except YouTubeError as exc:
|
||||
# No traceback anywhere in this file's YouTube handlers: for an upstream answer
|
||||
@@ -229,7 +167,7 @@ def discover_channels(
|
||||
]
|
||||
|
||||
|
||||
def _channel_summary(ch: Channel) -> dict:
|
||||
def _channel_summary(ch: "Channel") -> dict:
|
||||
"""The channel fields common to the list and discovery projections."""
|
||||
return {
|
||||
"id": ch.id,
|
||||
@@ -243,7 +181,7 @@ def _channel_summary(ch: Channel) -> dict:
|
||||
|
||||
|
||||
def _channel_detail_dict(
|
||||
channel: Channel,
|
||||
channel: "Channel",
|
||||
*,
|
||||
subscribed: bool,
|
||||
explored: bool,
|
||||
@@ -312,7 +250,7 @@ def channel_detail(
|
||||
needed (a stub) and refreshes it once it is stale (`DETAILS_MAX_AGE`), which is how a channel nobody subscribes to (an explored one, outside the
|
||||
subscription import) gets a banner that still resolves. The enrich uses the API key (1 unit,
|
||||
cached after)."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
# No `published_at is None` sentinel: a terminated channel never gets one, so it re-fetched on
|
||||
# every view; rows enriched before that field existed are long past DETAILS_MAX_AGE anyway.
|
||||
stale = details_stale(channel, datetime.now(timezone.utc))
|
||||
@@ -324,7 +262,7 @@ def channel_detail(
|
||||
):
|
||||
apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
|
||||
db.commit()
|
||||
channel = db.get(Channel, channel_id)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
except YouTubeError as exc:
|
||||
log.warning(
|
||||
"channel detail enrich failed: %s",
|
||||
@@ -334,41 +272,15 @@ def channel_detail(
|
||||
),
|
||||
)
|
||||
db.rollback()
|
||||
sub = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
sub = queries.subscription(db, user, channel_id)
|
||||
subscribed = sub is not None
|
||||
explored = (
|
||||
db.execute(
|
||||
select(ExploredChannel.id).where(
|
||||
ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
blocked = _is_blocked(db, user, channel_id)
|
||||
explored = queries.exploration(db, user, channel_id) is not None
|
||||
blocked = queries.is_blocked(db, user, channel_id)
|
||||
# Stored total + the Normal/Shorts/Live split in one grouped pass (same filters as the manager
|
||||
# list), so the channel page can show the per-type breakdown that moved off the table.
|
||||
n_all, n_short, n_live = db.execute(
|
||||
select(
|
||||
func.count(Video.id),
|
||||
func.count(Video.id).filter(Video.is_short.is_(True)),
|
||||
func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
|
||||
).where(Video.channel_id == channel_id)
|
||||
).one()
|
||||
stored = int(n_all or 0)
|
||||
count_short = int(n_short or 0)
|
||||
count_live = int(n_live or 0)
|
||||
stored, count_short, count_live = queries.video_counts(db, channel_id)
|
||||
count_normal = max(0, stored - count_short - count_live)
|
||||
tag_ids = list(
|
||||
db.scalars(
|
||||
select(ChannelTag.tag_id).where(
|
||||
ChannelTag.user_id == user.id, ChannelTag.channel_id == channel_id
|
||||
)
|
||||
).all()
|
||||
)
|
||||
tag_ids = reader_queries.user_tag_links(db, user, channel_id).get(channel_id, [])
|
||||
return _channel_detail_dict(
|
||||
channel,
|
||||
subscribed=subscribed,
|
||||
@@ -398,28 +310,15 @@ def explore_channel(
|
||||
it, then ingests one page of its uploads (newest-first) marked via_explore + enriched.
|
||||
Returns the next page token so the page can "load more". Guarded against the shared
|
||||
backfill quota reserve."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
if _is_blocked(db, user, channel_id):
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
if queries.is_blocked(db, user, channel_id):
|
||||
raise refused(403, "channel_blocked")
|
||||
if quota.backfill_reserve_reached():
|
||||
raise refused(429, "quota_reserve")
|
||||
|
||||
# Record/refresh the exploration (DB only) before ingesting, so the explorer can see the
|
||||
# videos as soon as they land. Flag the channel ephemeral unless it's organically followed.
|
||||
existing = db.execute(
|
||||
select(ExploredChannel).where(
|
||||
ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is None:
|
||||
db.add(ExploredChannel(user_id=user.id, channel_id=channel_id))
|
||||
else:
|
||||
existing.created_at = datetime.now(timezone.utc) # refresh the grace clock
|
||||
followed = db.execute(
|
||||
select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
|
||||
).first()
|
||||
if followed is None and not channel.from_explore:
|
||||
channel.from_explore = True
|
||||
service.explore(db, user, channel)
|
||||
db.commit()
|
||||
|
||||
try:
|
||||
@@ -433,7 +332,7 @@ def explore_channel(
|
||||
if details_stale(channel, datetime.now(timezone.utc)):
|
||||
apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
|
||||
db.commit()
|
||||
channel = db.get(Channel, channel_id)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
result = explore_ingest_page(db, yt, channel, page_token)
|
||||
except YouTubeError as exc:
|
||||
raise youtube_refusal(exc) from exc
|
||||
@@ -445,30 +344,20 @@ def explore_channel(
|
||||
}
|
||||
|
||||
|
||||
def _user_subscription(db: Session, user: User, channel_id: str) -> Subscription:
|
||||
sub = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
def _channel_or_404(db: Session, channel_id: str) -> "Channel":
|
||||
channel = queries.channel(db, channel_id)
|
||||
if channel is None:
|
||||
raise not_found("channel")
|
||||
return channel
|
||||
|
||||
|
||||
def _user_subscription(db: Session, user: User, channel_id: str) -> "Subscription":
|
||||
sub = queries.subscription(db, user, channel_id)
|
||||
if sub is None:
|
||||
raise not_found("subscription")
|
||||
return sub
|
||||
|
||||
|
||||
def _is_blocked(db: Session, user: User, channel_id: str) -> bool:
|
||||
"""Whether this user has blocked this channel."""
|
||||
return (
|
||||
db.execute(
|
||||
select(BlockedChannel.id).where(
|
||||
BlockedChannel.user_id == user.id,
|
||||
BlockedChannel.channel_id == channel_id,
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
class ChannelUpdateIn(BaseModel):
|
||||
# PATCH semantics: a field left unset (None) is untouched. Every field is optional so the
|
||||
# UI can send just the one control it changed. int/bool coercion + rejection of garbage
|
||||
@@ -486,19 +375,14 @@ def update_channel(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
sub = _user_subscription(db, user, channel_id)
|
||||
channel = db.get(Channel, channel_id)
|
||||
deep_turned_on = False
|
||||
if body.priority is not None:
|
||||
sub.priority = body.priority
|
||||
if body.hidden is not None:
|
||||
sub.hidden = body.hidden
|
||||
if body.deep_requested is not None:
|
||||
# Opt this channel into (or out of) full-history backfill. The deep scheduler
|
||||
# picks the flag up on its next run; turning it off won't undo already-fetched
|
||||
# videos, it just stops further deep paging if nobody else still wants it.
|
||||
want = body.deep_requested
|
||||
deep_turned_on = want and not sub.deep_requested
|
||||
sub.deep_requested = want
|
||||
channel = queries.channel(db, channel_id)
|
||||
deep_turned_on = service.update_subscription(
|
||||
db,
|
||||
sub,
|
||||
priority=body.priority,
|
||||
hidden=body.hidden,
|
||||
deep_requested=body.deep_requested,
|
||||
)
|
||||
db.commit()
|
||||
|
||||
# Opting a channel into full history should give an immediate feed: if we haven't even
|
||||
@@ -542,19 +426,8 @@ def reset_backfill(
|
||||
state (a "reset" trigger). Clears the channel's backfill markers, opts it back into deep
|
||||
backfill, and re-runs its recent pull immediately; the deep scheduler re-pages the full
|
||||
back-catalog on its next run. Idempotent — videos upsert by id, so nothing duplicates."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
channel.backfill_done = False
|
||||
channel.backfill_cursor = None
|
||||
channel.recent_synced_at = None
|
||||
# Re-arm the demand-driven deep scheduler for this channel. run_deep_backfill gates on an
|
||||
# EXISTS over subscriptions with deep_requested=True, so flip it on EVERY subscriber rather
|
||||
# than requiring the admin to be personally subscribed (they reset from the manager, not
|
||||
# their own feed). No-op row count if the channel has no subscribers.
|
||||
db.execute(
|
||||
update(Subscription)
|
||||
.where(Subscription.channel_id == channel_id)
|
||||
.values(deep_requested=True)
|
||||
)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
service.reset_backfill(db, channel)
|
||||
db.commit()
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_BACKFILL_RECENT):
|
||||
run_recent_backfill(db, [channel], max_channels=1)
|
||||
@@ -572,13 +445,8 @@ def subscribe(
|
||||
local subscription with YouTube's returned resource id and enriches the channel's metadata
|
||||
if it's still a stub, so the new row renders properly in the manager. It does NOT pull the
|
||||
channel's videos — the scheduler picks those up on its next run, like any subscribed channel."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
existing = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
if queries.subscription(db, user, channel_id) is not None:
|
||||
raise refused(409, "already_subscribed")
|
||||
with (
|
||||
quota.attribute(user.id, quota.QuotaAction.CHANNELS_SUBSCRIBE),
|
||||
@@ -618,24 +486,7 @@ def subscribe(
|
||||
reason=exc.reason,
|
||||
),
|
||||
)
|
||||
db.add(
|
||||
Subscription(
|
||||
user_id=user.id,
|
||||
channel_id=channel_id,
|
||||
yt_subscription_id=yt_sub_id or None,
|
||||
subscribed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
# Subscribing is the "keep" signal: a previously-explored channel becomes a normal organic
|
||||
# one. Drop its ephemeral flags so the cleanup job leaves it alone and its videos stop being
|
||||
# per-user private (they join the shared Library like any subscribed channel's uploads).
|
||||
if channel.from_explore:
|
||||
channel.from_explore = False
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.channel_id == channel_id, Video.via_explore.is_(True))
|
||||
.values(via_explore=False)
|
||||
)
|
||||
service.record_subscription(db, user, channel, yt_sub_id)
|
||||
db.commit()
|
||||
return {"ok": True, "subscribed": channel_id}
|
||||
|
||||
@@ -662,7 +513,7 @@ def unsubscribe(
|
||||
# 422 (a real, explainable error → speaking modal), not 502 which the client treats as
|
||||
# a transient "connection lost" gateway blip.
|
||||
raise youtube_refusal(exc) from exc
|
||||
db.delete(sub)
|
||||
service.drop_subscription(db, sub)
|
||||
db.commit()
|
||||
return {"ok": True, "unsubscribed": channel_id}
|
||||
|
||||
@@ -670,16 +521,7 @@ def unsubscribe(
|
||||
@router.get("/blocked/list")
|
||||
def list_blocked(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
"""The user's blocked channels (videos from these are excluded from their search/feed/explore)."""
|
||||
rows = (
|
||||
db.execute(
|
||||
select(Channel)
|
||||
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
|
||||
.where(BlockedChannel.user_id == user.id)
|
||||
.order_by(collated(Channel.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = queries.blocked_channels(db, user, lang_for_user(user))
|
||||
return [
|
||||
{"id": c.id, "title": c.title, "handle": c.handle, "thumbnail_url": c.thumbnail_url}
|
||||
for c in rows
|
||||
@@ -695,16 +537,9 @@ def block_channel(
|
||||
"""Block a channel for this user: its videos won't appear in their live search (and won't be
|
||||
ingested for them) and are hidden from their feed / explore. Idempotent. The now-hidden
|
||||
un-kept search/explore videos are reclaimed by the discovery-cleanup job in due course."""
|
||||
get_or_404(db, Channel, channel_id, what="channel")
|
||||
if not _is_blocked(db, user, channel_id):
|
||||
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
|
||||
# Stop any active exploration of it so it can be cleaned up.
|
||||
db.execute(
|
||||
ExploredChannel.__table__.delete().where(
|
||||
(ExploredChannel.user_id == user.id) & (ExploredChannel.channel_id == channel_id)
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
_channel_or_404(db, channel_id)
|
||||
service.block(db, user, channel_id)
|
||||
db.commit()
|
||||
return {"ok": True, "blocked": channel_id}
|
||||
|
||||
|
||||
@@ -714,11 +549,7 @@ def unblock_channel(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
db.execute(
|
||||
BlockedChannel.__table__.delete().where(
|
||||
(BlockedChannel.user_id == user.id) & (BlockedChannel.channel_id == channel_id)
|
||||
)
|
||||
)
|
||||
service.unblock(db, user, channel_id)
|
||||
db.commit()
|
||||
return {"ok": True, "unblocked": channel_id}
|
||||
|
||||
@@ -735,26 +566,12 @@ def attach_tag(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
_user_subscription(db, user, channel_id)
|
||||
# Only the user's own tags may be attached as personal links (owned_or_404 → 404 if the tag
|
||||
# is missing OR belongs to someone else, without leaking which).
|
||||
owned_or_404(db, Tag, body.tag_id, user, what="tag")
|
||||
exists = db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == body.tag_id,
|
||||
ChannelTag.user_id == user.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if exists is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel_id,
|
||||
tag_id=body.tag_id,
|
||||
user_id=user.id,
|
||||
source="user",
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
# Only the user's own tags may be attached as personal links (a 404 if the tag is missing OR
|
||||
# belongs to someone else, without leaking which).
|
||||
if reader_queries.own_tag(db, user, body.tag_id) is None:
|
||||
raise not_found("tag")
|
||||
reader_service.attach_user_tag(db, user, channel_id, body.tag_id)
|
||||
db.commit()
|
||||
return {"ok": True, "channel_id": channel_id, "tag_id": body.tag_id}
|
||||
|
||||
|
||||
@@ -765,14 +582,6 @@ def detach_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
row = db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == tag_id,
|
||||
ChannelTag.user_id == user.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.commit()
|
||||
reader_service.detach_user_tag(db, user, channel_id, tag_id)
|
||||
db.commit()
|
||||
return {"ok": True, "channel_id": channel_id, "tag_id": tag_id}
|
||||
|
||||
@@ -18,6 +18,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs, logsink
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -95,7 +96,7 @@ def stop_session(
|
||||
def list_sessions(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sessions = db.scalars(
|
||||
select(DebugSession)
|
||||
.where(DebugSession.user_id == user.id)
|
||||
.where(owned_by(DebugSession.user_id, user))
|
||||
.order_by(DebugSession.started_at.desc())
|
||||
.limit(LIST_MAX)
|
||||
).all()
|
||||
@@ -219,6 +220,7 @@ def send_issue(sid: str, user: User = Depends(current_user), db: Session = Depen
|
||||
raise refused(409, "issue_sending")
|
||||
new = session.issue_url is None
|
||||
if not new and not debugsession.unfinished(session):
|
||||
db.commit() # nothing to send: the lock ends here, not when the request's session closes
|
||||
return {"ok": True, "session": _view(db, session)}
|
||||
key = str(user.id)
|
||||
if new:
|
||||
|
||||
@@ -13,7 +13,6 @@ from typing import Literal
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -22,9 +21,9 @@ from app.db import get_db
|
||||
from app.discover import hero as hero_mod
|
||||
from app.discover import pins, rails
|
||||
from app.discover.budget import Budget, DEFAULT_BUDGET
|
||||
from app.models import ComfortPin, User
|
||||
from app.models import User
|
||||
from app.plex.cards import norm_lang
|
||||
from app.routes._common import owned_or_404, not_found
|
||||
from app.routes._common import not_found
|
||||
|
||||
router = APIRouter(prefix="/api/discover", tags=["discover"])
|
||||
|
||||
@@ -140,30 +139,19 @@ def add_pin(
|
||||
raise not_found("item")
|
||||
column, value = resolved
|
||||
|
||||
existing = db.execute(
|
||||
select(ComfortPin).where(
|
||||
ComfortPin.user_id == user.id,
|
||||
getattr(ComfortPin, column) == value,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
existing = pins.pin_of(db, user, column, value)
|
||||
if existing is not None:
|
||||
return {"ok": True, "id": existing.id, "created": False}
|
||||
|
||||
pin = ComfortPin(user_id=user.id, position=pins.next_position(db, user), **{column: value})
|
||||
db.add(pin)
|
||||
try:
|
||||
pin = pins.add(db, user, column, value)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
# A concurrent pin of the same thing from a second tab. The unique constraint did its job;
|
||||
# the user's intent ("be on my shelf") is satisfied either way, so report the winner rather
|
||||
# than a 500 the user can do nothing with.
|
||||
db.rollback()
|
||||
winner = db.execute(
|
||||
select(ComfortPin).where(
|
||||
ComfortPin.user_id == user.id,
|
||||
getattr(ComfortPin, column) == value,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
winner = pins.pin_of(db, user, column, value)
|
||||
if winner is None:
|
||||
# Not the unique constraint, then: the only other way in is the target row being pruned
|
||||
# between `resolve_ref` confirming it and this INSERT (a Plex sync dropping a removed
|
||||
@@ -180,8 +168,10 @@ def remove_pin(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pin = owned_or_404(db, ComfortPin, pin_id, user, what="pin")
|
||||
db.delete(pin)
|
||||
pin = pins.own_pin(db, user, pin_id)
|
||||
if pin is None:
|
||||
raise not_found("pin")
|
||||
pins.remove(db, pin)
|
||||
db.commit()
|
||||
return {"deleted": pin_id}
|
||||
|
||||
|
||||
+51
-183
@@ -1,9 +1,7 @@
|
||||
from datetime import date, datetime, timezone
|
||||
from datetime import date
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm.exc import StaleDataError
|
||||
|
||||
@@ -11,34 +9,10 @@ from app import quota
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.mailtext import lang_for_user
|
||||
from app.feedquery import (
|
||||
decode_cursor,
|
||||
encode_cursor,
|
||||
feed_summary,
|
||||
feed_years,
|
||||
filtered_query,
|
||||
keyset_predicate,
|
||||
order_by,
|
||||
rank_window_start,
|
||||
serialize,
|
||||
sort_keys,
|
||||
windowed_rank,
|
||||
windowed_relevance_rows,
|
||||
)
|
||||
from app.models import (
|
||||
ChannelTag,
|
||||
Tag,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.routes._common import (
|
||||
ProgressIn,
|
||||
get_or_404,
|
||||
kept_position,
|
||||
not_found,
|
||||
youtube_refusal,
|
||||
)
|
||||
from app.feedquery import BadCursor, facet_counts, feed_count, feed_page, feed_summary, feed_years
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.routes._common import ProgressIn, kept_position, not_found, youtube_refusal
|
||||
from app.sync.videos import parse_iso8601_duration
|
||||
from app.utils import iso
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
@@ -108,49 +82,20 @@ def get_feed(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
query, rank_expr = filtered_query(db, user, **params)
|
||||
lang = lang_for_user(user)
|
||||
keys = sort_keys(sort, seed, rank_expr, lang)
|
||||
|
||||
vals, cursor_id, window = None, None, None
|
||||
if cursor:
|
||||
# The read-model raises a plain ValueError on a malformed cursor; translate it to the
|
||||
# HTTP 400 here so feedquery stays web-framework-free.
|
||||
try:
|
||||
vals, cursor_id, window = decode_cursor(keys, cursor)
|
||||
except ValueError:
|
||||
# `from None`: the ValueError is an expected bad-input signal, not an error to chain.
|
||||
raise refused(400, "invalid_cursor") from None
|
||||
ranked = sort == "relevance" and rank_expr is not None
|
||||
if ranked and not cursor:
|
||||
window = rank_window_start(db, query)
|
||||
if ranked and window is not None:
|
||||
keys = sort_keys(sort, seed, windowed_rank(rank_expr, window), lang)
|
||||
|
||||
# Expose each key column on the row so we can build the next cursor from the last item.
|
||||
query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
|
||||
if vals is not None:
|
||||
for k, v in zip(keys, vals, strict=True):
|
||||
k["value"] = v
|
||||
if ranked and window is not None:
|
||||
rows = windowed_relevance_rows(db, query, keys, window, cursor_id, limit + 1)
|
||||
else:
|
||||
query = query.order_by(*order_by(keys))
|
||||
if cursor_id is not None:
|
||||
query = query.where(keyset_predicate(keys, cursor_id))
|
||||
rows = db.execute(query.limit(limit + 1)).all()
|
||||
has_more = len(rows) > limit
|
||||
page = rows[:limit]
|
||||
return {
|
||||
"items": [serialize(r) for r in page],
|
||||
"next_cursor": (
|
||||
encode_cursor(keys, page[-1], window if ranked else None)
|
||||
if (has_more and page)
|
||||
else None
|
||||
),
|
||||
"has_more": has_more,
|
||||
"limit": limit,
|
||||
}
|
||||
try:
|
||||
return feed_page(
|
||||
db,
|
||||
user,
|
||||
params,
|
||||
sort=sort,
|
||||
seed=seed,
|
||||
limit=limit,
|
||||
cursor=cursor,
|
||||
lang=lang_for_user(user),
|
||||
)
|
||||
except BadCursor:
|
||||
# `from None`: a malformed cursor is an expected bad-input signal, not an error to chain.
|
||||
raise refused(400, "invalid_cursor") from None
|
||||
|
||||
|
||||
@router.get("/feed/count")
|
||||
@@ -162,9 +107,7 @@ def get_feed_count(
|
||||
# SUPERSEDED by /feed/summary (PERF-13) — the app no longer calls this. Kept as the reference
|
||||
# oracle that test_feed_summary_matches_count_plus_facets asserts /feed/summary's `count` against;
|
||||
# remove both together with that test if this endpoint is ever retired.
|
||||
query, _rank = filtered_query(db, user, **params)
|
||||
total = db.scalar(select(func.count()).select_from(query.subquery()))
|
||||
return {"count": total or 0}
|
||||
return {"count": feed_count(db, user, **params)}
|
||||
|
||||
|
||||
@router.get("/facets")
|
||||
@@ -186,31 +129,7 @@ def get_facets(
|
||||
SUPERSEDED by /feed/summary (PERF-13) — the app no longer calls this. Kept as the reference
|
||||
oracle for test_feed_summary_matches_count_plus_facets (it asserts /feed/summary's `facets`
|
||||
against these `counts`); retire the two together."""
|
||||
visible = or_(ChannelTag.user_id.is_(None), ChannelTag.user_id == user.id)
|
||||
counts: dict[int, int] = {}
|
||||
for category in ("language", "topic", "other"):
|
||||
# Disjunctive (OR) facets drop the category's own selections so its chips keep
|
||||
# independent counts (you can OR more of them in). Conjunctive (AND, topics only)
|
||||
# keeps them applied, so each remaining chip narrows to channels that ALSO have all
|
||||
# already-selected topics — and tags that can't co-occur drop to zero (hidden).
|
||||
conjunctive = params.get("tag_modes", {}).get(category) == "and"
|
||||
base, _rank = filtered_query(
|
||||
db,
|
||||
user,
|
||||
**{**params, "exclude_tag_category": None if conjunctive else category},
|
||||
)
|
||||
channels = base.with_only_columns(Video.channel_id).distinct().subquery()
|
||||
rows = db.execute(
|
||||
select(ChannelTag.tag_id, func.count(func.distinct(ChannelTag.channel_id)))
|
||||
.select_from(channels)
|
||||
.join(ChannelTag, ChannelTag.channel_id == channels.c.channel_id)
|
||||
.join(Tag, and_(Tag.id == ChannelTag.tag_id, Tag.category == category))
|
||||
.where(visible)
|
||||
.group_by(ChannelTag.tag_id)
|
||||
).all()
|
||||
for tag_id, count in rows:
|
||||
counts[tag_id] = count
|
||||
return {"counts": counts}
|
||||
return {"counts": facet_counts(db, user, **params)}
|
||||
|
||||
|
||||
@router.get("/feed/summary")
|
||||
@@ -239,6 +158,11 @@ def get_feed_years(
|
||||
return {"years": feed_years(db, user, **params)}
|
||||
|
||||
|
||||
def _video_or_404(db: Session, video_id: str) -> None:
|
||||
if queries.video(db, video_id) is None:
|
||||
raise not_found("video")
|
||||
|
||||
|
||||
class VideoStateIn(BaseModel):
|
||||
status: str
|
||||
|
||||
@@ -253,50 +177,17 @@ def set_video_state(
|
||||
status = body.status
|
||||
if status not in VALID_STATES:
|
||||
raise refused(400, "invalid_status")
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
_video_or_404(db, video_id)
|
||||
if status == "new":
|
||||
# Un-marking: keep the row if it still holds a resume position (un-marking
|
||||
# "watched" should restore the in-progress state, not wipe where the user left
|
||||
# off); otherwise drop it. The in-app player fires this alongside a progress
|
||||
# checkpoint that may DELETE the same row, so tolerate it vanishing under us.
|
||||
row = db.execute(
|
||||
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
if row.position_seconds:
|
||||
row.status = "new"
|
||||
row.watched_at = None
|
||||
else:
|
||||
db.delete(row)
|
||||
try:
|
||||
db.commit()
|
||||
# silent-ok: the documented lost race — a concurrent player checkpoint already removed
|
||||
# or moved this row. The state the caller asked for is the state that holds either way.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
try:
|
||||
service.unmark(db, user, video_id)
|
||||
db.commit()
|
||||
# silent-ok: the documented lost race — a concurrent player checkpoint already removed or
|
||||
# moved this row (`service.unmark`). The state the caller asked for holds either way.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
return {"ok": True, "video_id": video_id, "status": "new"}
|
||||
|
||||
# watched / hidden: atomic upsert. The player marks "watched" at end-of-playback at
|
||||
# the same instant a progress checkpoint may DELETE the in-progress row; a plain
|
||||
# SELECT-then-UPDATE then raises StaleDataError ("0 rows matched") on that race. An
|
||||
# INSERT … ON CONFLICT DO UPDATE keyed on uq_user_video is immune to it.
|
||||
set_: dict = {"status": status, "updated_at": now}
|
||||
if status == "watched":
|
||||
set_["watched_at"] = now # hidden keeps any existing watched_at
|
||||
stmt = (
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
status=status,
|
||||
watched_at=now if status == "watched" else None,
|
||||
)
|
||||
.on_conflict_do_update(constraint="uq_user_video", set_=set_)
|
||||
)
|
||||
db.execute(stmt)
|
||||
service.mark(db, user, video_id, status)
|
||||
db.commit()
|
||||
return {"ok": True, "video_id": video_id, "status": status}
|
||||
|
||||
@@ -310,16 +201,12 @@ def clear_video_state(
|
||||
"""Reset a video to pristine for this user — drop the whole VideoState row (status,
|
||||
watch position and watched_at), as if it had never been opened. Saved/playlist membership
|
||||
lives elsewhere and is untouched."""
|
||||
row = db.execute(
|
||||
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
try:
|
||||
db.commit()
|
||||
# silent-ok: the row went under the delete, which is the outcome the delete wanted.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
try:
|
||||
service.clear_state(db, user, video_id)
|
||||
db.commit()
|
||||
# silent-ok: the row went under the delete, which is the outcome the delete wanted.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
return {"ok": True, "video_id": video_id, "status": "new", "position_seconds": 0}
|
||||
|
||||
|
||||
@@ -335,36 +222,17 @@ def set_video_progress(
|
||||
partially-watched video can render a progress bar and match the "in progress" filter.
|
||||
Trivially-early and near-finished positions clear the position rather than store it."""
|
||||
position = kept_position(body)
|
||||
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
|
||||
row = db.execute(
|
||||
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
_video_or_404(db, video_id)
|
||||
if not position:
|
||||
# Nothing meaningful to store: drop a status-less row entirely, otherwise just
|
||||
# zero the position (a saved/hidden video keeps its status).
|
||||
if row is not None:
|
||||
if row.status == "new":
|
||||
db.delete(row)
|
||||
else:
|
||||
row.position_seconds = 0
|
||||
row.progress_updated_at = datetime.now(timezone.utc)
|
||||
# A concurrent state change (e.g. the end-of-playback "watched" mark) may
|
||||
# have removed/updated this row already — tolerate the lost race.
|
||||
try:
|
||||
db.commit()
|
||||
# silent-ok: exactly the race the comment above describes; the response stays true.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
try:
|
||||
service.set_progress(db, user, video_id, 0)
|
||||
db.commit()
|
||||
# silent-ok: a concurrent state change (the end-of-playback "watched" mark) removed or
|
||||
# updated this row already — the lost race; the response stays true.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
return {"ok": True, "video_id": video_id, "position_seconds": 0}
|
||||
|
||||
if row is None:
|
||||
row = VideoState(user_id=user.id, video_id=video_id)
|
||||
db.add(row)
|
||||
row.position_seconds = position
|
||||
row.progress_updated_at = datetime.now(timezone.utc)
|
||||
service.set_progress(db, user, video_id, position)
|
||||
db.commit()
|
||||
return {"ok": True, "video_id": video_id, "position_seconds": position}
|
||||
|
||||
@@ -382,7 +250,7 @@ def get_video_detail(
|
||||
our DB (e.g. a YouTube link inside another video's description that the in-app
|
||||
player navigated to) is resolved via the YouTube API (videos.list, 1 unit,
|
||||
attributed to the requesting user)."""
|
||||
v = db.get(Video, video_id)
|
||||
v = queries.video(db, video_id)
|
||||
if v is not None:
|
||||
return {
|
||||
"id": v.id,
|
||||
|
||||
@@ -375,6 +375,9 @@ def update_folder(
|
||||
folder = own_folder(db, user, folder_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
# Which folder may be renamed is answered before what the name is, as it always was.
|
||||
if not folders.renamable(folder):
|
||||
raise refused(409, "file_folder_not_own")
|
||||
try:
|
||||
folders.rename(db, folder, _folder_name(body.name or ""))
|
||||
except folders.FolderError as e:
|
||||
|
||||
+10
-81
@@ -9,11 +9,9 @@ from app.auth import (
|
||||
count_admins,
|
||||
count_pending_activations,
|
||||
current_user,
|
||||
drop_from_wallet,
|
||||
google_enabled,
|
||||
has_read_scope,
|
||||
has_write_scope,
|
||||
is_allowed,
|
||||
optional_current_user,
|
||||
purge_user,
|
||||
)
|
||||
@@ -26,79 +24,10 @@ from app.db import get_db
|
||||
from app.models import Invite, User
|
||||
from app.utils import iso
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
router = APIRouter(prefix="/api/me", tags=["me"])
|
||||
|
||||
|
||||
@router.get("/accounts")
|
||||
def my_accounts(
|
||||
request: Request,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> list[dict]:
|
||||
"""The accounts that have authenticated in this browser session (switchable without a
|
||||
new Google sign-in). The active one is flagged."""
|
||||
ids = request.session.get("account_ids") or [user.id]
|
||||
rows = {u.id: u for u in db.execute(select(User).where(User.id.in_(ids))).scalars()}
|
||||
out = []
|
||||
for uid in ids: # preserve recency order from the session
|
||||
u = rows.get(uid)
|
||||
if u is not None:
|
||||
out.append(
|
||||
{
|
||||
"id": u.id,
|
||||
"email": u.email,
|
||||
"display_name": u.display_name,
|
||||
"avatar_url": u.avatar_url,
|
||||
"active": u.id == user.id,
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
class SwitchAccountIn(BaseModel):
|
||||
user_id: int
|
||||
|
||||
|
||||
@router.post("/switch")
|
||||
def switch_account(
|
||||
body: SwitchAccountIn,
|
||||
request: Request,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Switch the active account to another one already authenticated in this browser. No
|
||||
Google round-trip — but only accounts present in the session list (proof they signed in
|
||||
here) are allowed, and access is re-checked in case the invite was revoked since."""
|
||||
target = body.user_id
|
||||
if target not in (request.session.get("account_ids") or []):
|
||||
raise refused(403, "not_your_account")
|
||||
# An account `_session_user` would refuse — gone, deactivated, suspended, or its session revoked — is
|
||||
# refused here too, and dropped from the switch list the way `_session_user` drops it. Made the session
|
||||
# default instead, the next request's current_user would refuse it as the DEFAULT and clear the whole
|
||||
# wallet session, logging out every account here. (B-core-9)
|
||||
u = db.get(User, target)
|
||||
if u is None:
|
||||
drop_from_wallet(request.session, target)
|
||||
raise not_found("account")
|
||||
if not is_allowed(db, u.email):
|
||||
raise refused(403, "account_no_access")
|
||||
if not u.is_active:
|
||||
drop_from_wallet(request.session, target)
|
||||
raise refused(403, "account_no_access")
|
||||
if u.is_suspended:
|
||||
drop_from_wallet(request.session, target)
|
||||
raise refused(403, "account_suspended")
|
||||
# Revoked: a password reset/change or "log out everywhere" bumped the account's epoch.
|
||||
epochs = request.session.get("epochs") or {}
|
||||
if int(epochs.get(str(target), 0)) != (u.session_epoch or 0):
|
||||
drop_from_wallet(request.session, target)
|
||||
raise refused(403, "account_signed_out")
|
||||
request.session["user_id"] = target
|
||||
return {"ok": True, "user_id": target}
|
||||
|
||||
|
||||
@router.get("")
|
||||
def get_me(
|
||||
user: User | None = Depends(optional_current_user), db: Session = Depends(get_db)
|
||||
@@ -187,17 +116,11 @@ def delete_my_account(
|
||||
surfaces)."""
|
||||
if user.role == "admin" and count_admins(db) <= 1:
|
||||
raise refused(400, "only_admin_delete")
|
||||
user_id = user.id
|
||||
# Full erasure (cascades) + access-request cleanup + Google-grant revocation; shared with the
|
||||
# admin delete path. Capture the id first — `user` is detached once purge_user deletes the row.
|
||||
# admin delete path.
|
||||
purge_user(db, user, background)
|
||||
# Drop this account from the browser session; switch to another signed-in account if any.
|
||||
remaining = [a for a in (request.session.get("account_ids") or []) if a != user_id]
|
||||
if remaining:
|
||||
request.session["account_ids"] = remaining
|
||||
request.session["user_id"] = remaining[-1]
|
||||
else:
|
||||
request.session.clear()
|
||||
# The session's one account is gone: so is the session.
|
||||
request.session.clear()
|
||||
return {"deleted": True}
|
||||
|
||||
|
||||
@@ -213,7 +136,13 @@ def update_preferences(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Merge so partial updates don't wipe other keys.
|
||||
# Merge so partial updates don't wipe other keys — over the stored row read again under a row
|
||||
# lock: a client saves keys one at a time (several at once on a boot that moves this browser's
|
||||
# copies to the account), and two merges over the same earlier read would keep only the last.
|
||||
# FOR NO KEY UPDATE, as bump_session_epoch: it still orders two merges, and a transaction that
|
||||
# only added a row referring to the account (a playlist pull, then minutes of YouTube) does not
|
||||
# hold the save up.
|
||||
db.refresh(user, with_for_update={"key_share": True})
|
||||
merged = dict(user.preferences or {})
|
||||
merged.update(preferences.root)
|
||||
user.preferences = merged
|
||||
|
||||
@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import Notification, User
|
||||
@@ -36,7 +37,7 @@ def list_notifications(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
where = [Notification.user_id == user.id]
|
||||
where = [owned_by(Notification.user_id, user)]
|
||||
if not include_dismissed:
|
||||
where.append(Notification.dismissed.is_(False))
|
||||
total = db.scalar(select(func.count()).select_from(Notification).where(*where)) or 0
|
||||
@@ -61,7 +62,7 @@ def unread_count(user: User = Depends(current_user), db: Session = Depends(get_d
|
||||
select(func.count())
|
||||
.select_from(Notification)
|
||||
.where(
|
||||
Notification.user_id == user.id,
|
||||
owned_by(Notification.user_id, user),
|
||||
Notification.read.is_(False),
|
||||
Notification.dismissed.is_(False),
|
||||
)
|
||||
@@ -77,8 +78,8 @@ def mark_read(
|
||||
) -> dict:
|
||||
n = owned_or_404(db, Notification, notification_id, user, what="notification")
|
||||
n.read = True
|
||||
db.commit()
|
||||
trim_read(db, user.id)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -86,11 +87,11 @@ def mark_read(
|
||||
def mark_all_read(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.read.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.read.is_(False))
|
||||
.values(read=True)
|
||||
)
|
||||
db.commit()
|
||||
trim_read(db, user.id)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -113,7 +114,7 @@ def clear_all(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"""Dismiss every (still-visible) notification for this user."""
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.dismissed.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.dismissed.is_(False))
|
||||
.values(dismissed=True, read=True)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
+61
-304
@@ -3,31 +3,34 @@ CRUD (create / rename / delete, add / remove / reorder items) plus YouTube two-w
|
||||
mirror the user's own YouTube playlists in (`/sync-youtube`) and push/revert local edits back
|
||||
out (`/{id}/push-plan`, `/{id}/revert-youtube`), behind the optional write scope."""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, select, and_
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import quota
|
||||
from app.auth import current_user, has_read_scope, has_write_scope, require_write_scope
|
||||
from app.db import get_db
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.models import Channel, Playlist, PlaylistItem, User, Video, VideoState
|
||||
from app.feedquery import serialize
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.sync.playlists import (
|
||||
PlaylistGone,
|
||||
lock_playlists,
|
||||
merge_playlist,
|
||||
plan_merge,
|
||||
recompute_dirty,
|
||||
pull_user_playlists,
|
||||
repull_playlist,
|
||||
sync_claim,
|
||||
user_sync_claim,
|
||||
)
|
||||
from app.ratelimit import RateLimiter
|
||||
from app.routes._common import get_or_404, not_found, owned_or_404, youtube_refusal
|
||||
from app.routes._common import not_found, youtube_refusal
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.reader.models import Playlist
|
||||
|
||||
router = APIRouter(prefix="/api/playlists", tags=["playlists"])
|
||||
|
||||
# SEC-14, same reasoning as the sync module's limiter: a playlist pull lists every playlist the
|
||||
@@ -56,90 +59,20 @@ def _refused(status: int, reason: str) -> HTTPException:
|
||||
return HTTPException(status_code=status, detail={"code": "playlist", "reason": reason})
|
||||
|
||||
|
||||
def _touch(pl: Playlist) -> None:
|
||||
"""Bump the playlist's `updated_at` for an edit that only writes `playlist_items`.
|
||||
|
||||
`UpdatedAtMixin`'s `onupdate` fires on an UPDATE of the PLAYLIST row, and add / remove /
|
||||
reorder / move write the item rows instead — so without this the rail's "recently changed"
|
||||
sort would only ever see renames. The assignment is what makes the column mean "last time
|
||||
this list changed", which is what the rail promises the reader.
|
||||
|
||||
The DATABASE's clock, like the mixin's own `onupdate`: one sort key written from two clocks
|
||||
(this process's and Postgres's) could order a rename and an item edit the wrong way round."""
|
||||
pl.updated_at = func.now()
|
||||
def _own_playlist(db: Session, user: User, playlist_id: int, *, lock: bool = False) -> "Playlist":
|
||||
pl = queries.own_playlist(db, user, playlist_id, lock=lock)
|
||||
if pl is None:
|
||||
raise not_found("playlist")
|
||||
return pl
|
||||
|
||||
|
||||
def _next_playlist_position(db: Session, user_id: int) -> int:
|
||||
"""Append position for a new playlist in this user's rail."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(Playlist.position), -1)).where(
|
||||
Playlist.user_id == user_id
|
||||
)
|
||||
)
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def _next_item_position(db: Session, playlist_id: int) -> int:
|
||||
"""Append position for a new item at the end of a playlist. Locks the playlist row first: two
|
||||
adds (two tabs, the rail and the page) otherwise both read the same max and land on one
|
||||
position, and the list's order between them is then arbitrary."""
|
||||
lock_playlists(db, playlist_id)
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(PlaylistItem.position), -1)).where(
|
||||
PlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
)
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def _add_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Append a video to a playlist if not already present (idempotent). Returns whether it
|
||||
was added. `mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later).
|
||||
Locked BEFORE the presence check: two adds of one video both saw "absent" and the second one's
|
||||
insert hit uq_playlist_video as a 500."""
|
||||
lock_playlists(db, pl.id)
|
||||
exists = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == pl.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
if exists is not None:
|
||||
return False
|
||||
db.add(
|
||||
PlaylistItem(playlist_id=pl.id, video_id=video_id, position=_next_item_position(db, pl.id))
|
||||
)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
_touch(pl)
|
||||
db.flush() # the handler commits (CLAUDE.md: helpers flush)
|
||||
return True
|
||||
|
||||
|
||||
def _remove_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Remove a video from a playlist if present (idempotent). Returns whether it was removed.
|
||||
`mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later)."""
|
||||
lock_playlists(db, pl.id)
|
||||
item = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == pl.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
if item is None:
|
||||
return False
|
||||
db.delete(item)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
_touch(pl)
|
||||
db.flush() # the handler commits (CLAUDE.md: helpers flush)
|
||||
return True
|
||||
def _video_or_404(db: Session, video_id: str) -> None:
|
||||
if queries.video(db, video_id) is None:
|
||||
raise not_found("video")
|
||||
|
||||
|
||||
def _summary(
|
||||
pl: Playlist,
|
||||
pl: "Playlist",
|
||||
count: int,
|
||||
total_duration: int = 0,
|
||||
has_video: bool | None = None,
|
||||
@@ -156,7 +89,7 @@ def _summary(
|
||||
"total_duration_seconds": total_duration,
|
||||
"cover_thumbnail": cover,
|
||||
# When this list last CHANGED (rename, add/remove/move/reorder) — the rail's
|
||||
# "recently changed first" sort reads it. See _touch for why it isn't automatic.
|
||||
# "recently changed first" sort reads it. See `service.touch` for why it isn't automatic.
|
||||
"updated_at": pl.updated_at,
|
||||
}
|
||||
if has_video is not None:
|
||||
@@ -172,73 +105,14 @@ def list_playlists(
|
||||
) -> list[dict]:
|
||||
"""The user's playlists (newest custom order). With `contains=<video_id>`, each entry
|
||||
also reports whether that video is already in it — used by the add-to-playlist popover."""
|
||||
pls = (
|
||||
db.execute(
|
||||
select(Playlist)
|
||||
.where(Playlist.user_id == user.id)
|
||||
.order_by(Playlist.position, Playlist.created_at)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
pls = queries.playlists_for(db, user)
|
||||
if not pls:
|
||||
return []
|
||||
ids = [p.id for p in pls]
|
||||
counts = dict(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id, func.count())
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
).all()
|
||||
)
|
||||
durations = dict(
|
||||
db.execute(
|
||||
select(
|
||||
PlaylistItem.playlist_id,
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
).all()
|
||||
)
|
||||
member: set[int] = set()
|
||||
if contains:
|
||||
member = set(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id).where(
|
||||
PlaylistItem.playlist_id.in_(ids),
|
||||
PlaylistItem.video_id == contains,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
# Batch the cover thumbnails (first item per playlist by position) in ONE window query
|
||||
# instead of a per-playlist SELECT inside _summary (the old N+1).
|
||||
rn = (
|
||||
func.row_number()
|
||||
.over(
|
||||
partition_by=PlaylistItem.playlist_id,
|
||||
order_by=PlaylistItem.position,
|
||||
)
|
||||
.label("rn")
|
||||
)
|
||||
first_items = (
|
||||
select(
|
||||
PlaylistItem.playlist_id.label("pid"),
|
||||
Video.thumbnail_url.label("thumb"),
|
||||
rn,
|
||||
)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.subquery()
|
||||
)
|
||||
covers = dict(
|
||||
db.execute(
|
||||
select(first_items.c.pid, first_items.c.thumb).where(first_items.c.rn == 1)
|
||||
).all()
|
||||
)
|
||||
counts = queries.item_counts(db, ids)
|
||||
durations = queries.durations(db, ids)
|
||||
member = queries.containing(db, ids, contains) if contains else set()
|
||||
covers = queries.covers(db, ids)
|
||||
return [
|
||||
_summary(
|
||||
p,
|
||||
@@ -269,29 +143,11 @@ def create_playlist(
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise _refused(400, "name_required")
|
||||
pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id))
|
||||
db.add(pl)
|
||||
pl = service.create_playlist(db, user, name)
|
||||
db.commit()
|
||||
return _summary(pl, 0)
|
||||
|
||||
|
||||
def _watch_later(db: Session, user: User) -> Playlist:
|
||||
"""The user's built-in Watch later playlist, created on first use."""
|
||||
pl = db.scalar(
|
||||
select(Playlist).where(Playlist.user_id == user.id, Playlist.kind == "watch_later")
|
||||
)
|
||||
if pl is None:
|
||||
pl = Playlist(
|
||||
user_id=user.id,
|
||||
name="Watch later",
|
||||
kind="watch_later",
|
||||
position=_next_playlist_position(db, user.id),
|
||||
)
|
||||
db.add(pl)
|
||||
db.flush() # the id, for the item the caller adds next; the handler commits
|
||||
return pl
|
||||
|
||||
|
||||
@router.post("/watch-later")
|
||||
def add_watch_later(
|
||||
body: VideoRefIn,
|
||||
@@ -300,9 +156,9 @@ def add_watch_later(
|
||||
) -> dict:
|
||||
"""Bookmark shortcut: add a video to the built-in Watch later playlist."""
|
||||
video_id = body.video_id
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
pl = _watch_later(db, user)
|
||||
_add_item(db, pl, video_id, mark_dirty=False)
|
||||
_video_or_404(db, video_id)
|
||||
pl = service.watch_later(db, user)
|
||||
service.add_item(db, pl, video_id, mark_dirty=False)
|
||||
db.commit()
|
||||
return {"ok": True, "saved": True, "playlist_id": pl.id}
|
||||
|
||||
@@ -313,11 +169,9 @@ def remove_watch_later(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = db.scalar(
|
||||
select(Playlist).where(Playlist.user_id == user.id, Playlist.kind == "watch_later")
|
||||
)
|
||||
pl = queries.watch_later(db, user)
|
||||
if pl is not None:
|
||||
_remove_item(db, pl, video_id, mark_dirty=False)
|
||||
service.remove_item(db, pl, video_id, mark_dirty=False)
|
||||
db.commit()
|
||||
return {"ok": True, "saved": False}
|
||||
|
||||
@@ -354,7 +208,7 @@ def revert_youtube(
|
||||
"""Reset one YouTube-linked playlist to its current YouTube state, discarding local
|
||||
edits (and clearing the dirty flag). The user-facing 'undo all my changes' for a list
|
||||
once the in-session undo history is gone."""
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
if not pl.yt_playlist_id:
|
||||
raise _refused(400, "not_on_youtube")
|
||||
if not has_read_scope(user):
|
||||
@@ -375,10 +229,10 @@ def revert_youtube(
|
||||
raise youtube_refusal(exc, playlist=True) from exc
|
||||
|
||||
|
||||
def _pushable(db: Session, user: User, playlist_id: int) -> Playlist:
|
||||
def _pushable(db: Session, user: User, playlist_id: int) -> "Playlist":
|
||||
"""A local playlist the user may push to YouTube (create or update). The built-in
|
||||
Watch later and read-only YouTube mirrors are never pushable."""
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
if pl.kind == "watch_later":
|
||||
raise _refused(400, "watch_later_not_pushable")
|
||||
return pl
|
||||
@@ -448,17 +302,8 @@ def get_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
# `id` breaks a position tie the same way on every read (the renumbering add does too).
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).all()
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
rows = queries.playlist_rows(db, user, pl.id)
|
||||
return {
|
||||
"id": pl.id,
|
||||
"name": pl.name,
|
||||
@@ -482,34 +327,15 @@ def rename_playlist(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked: the new name moves `dirty`, which the scheduled mirror refresh reads under this lock.
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist", lock=True)
|
||||
pl = _own_playlist(db, user, playlist_id, lock=True)
|
||||
if "name" in body.model_fields_set:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise _refused(400, "name_required")
|
||||
if name != pl.name:
|
||||
pl.name = name
|
||||
recompute_dirty(db, pl)
|
||||
service.rename_playlist(db, pl, name)
|
||||
db.commit()
|
||||
# One aggregate for count + duration (outer join so an empty playlist still returns 0),
|
||||
# plus a single cover lookup — was 3 separate round-trips.
|
||||
count, total = db.execute(
|
||||
select(
|
||||
func.count(PlaylistItem.id),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.outerjoin(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
).one()
|
||||
cover = db.scalar(
|
||||
select(Video.thumbnail_url)
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlaylistItem.position)
|
||||
.limit(1)
|
||||
)
|
||||
return _summary(pl, count or 0, total or 0, cover=cover)
|
||||
count, total, cover = queries.playlist_totals(db, pl.id)
|
||||
return _summary(pl, count, total, cover=cover)
|
||||
|
||||
|
||||
@router.delete("/{playlist_id}")
|
||||
@@ -519,7 +345,7 @@ def delete_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
if pl.kind == "watch_later":
|
||||
raise _refused(400, "watch_later_undeletable")
|
||||
# Not in the middle of a sync of it (`sync_claim`): a push that read the list before the delete
|
||||
@@ -540,9 +366,7 @@ def delete_playlist(
|
||||
yt.delete_playlist(pl.yt_playlist_id)
|
||||
except YouTubeError as exc:
|
||||
raise youtube_refusal(exc, playlist=True) from exc
|
||||
# The playlist first, as every item write: the cascade deletes the items before the list row.
|
||||
lock_playlists(db, pl.id)
|
||||
db.delete(pl) # items cascade
|
||||
service.delete_playlist(db, pl)
|
||||
db.commit()
|
||||
return {"deleted": playlist_id}
|
||||
|
||||
@@ -561,26 +385,10 @@ def add_item(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
video_id = body.video_id
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
added = _add_item(db, pl, video_id, mark_dirty=False)
|
||||
if added and body.position is not None:
|
||||
# Renumber the whole list with the new row at `position` (clamped to the end): positions
|
||||
# can have gaps after removals, so "index N" is only meaningful over the ordered rows.
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlaylistItem)
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
new = next(r for r in rows if r.video_id == video_id)
|
||||
rows.remove(new)
|
||||
rows.insert(min(body.position, len(rows)), new)
|
||||
for pos, row in enumerate(rows):
|
||||
row.position = pos
|
||||
recompute_dirty(db, pl)
|
||||
_video_or_404(db, video_id)
|
||||
service.add_to_playlist(db, pl, video_id, body.position)
|
||||
db.commit()
|
||||
return {"ok": True, "playlist_id": pl.id, "video_id": video_id}
|
||||
|
||||
@@ -592,8 +400,8 @@ def remove_item(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
_remove_item(db, pl, video_id, mark_dirty=True)
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
service.remove_item(db, pl, video_id, mark_dirty=True)
|
||||
db.commit()
|
||||
return {"ok": True, "playlist_id": pl.id, "video_id": video_id}
|
||||
|
||||
@@ -617,49 +425,21 @@ def move_item(
|
||||
exactly the kind of half-applied state this codebase treats as a bug rather than an edge
|
||||
case. Idempotent: if the video is already in the target, the source row is simply dropped
|
||||
(moving into a list that already has it is a removal, not a duplicate)."""
|
||||
src = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
src = _own_playlist(db, user, playlist_id)
|
||||
# Its own reason: "this playlist doesn't exist any more" about a TARGET that is gone was read as
|
||||
# the list on screen — which is still there.
|
||||
dst = owned_or_404(db, Playlist, body.target_playlist_id, user, what="move_target")
|
||||
if src.id == dst.id:
|
||||
dst = queries.own_playlist(db, user, body.target_playlist_id)
|
||||
if dst is None:
|
||||
raise not_found("move_target")
|
||||
try:
|
||||
service.move_item(db, src, dst, video_id)
|
||||
except service.MoveRefused as e:
|
||||
# A same-list move is a bad request; a video a concurrent edit (another tab, a YouTube
|
||||
# re-pull) already took out is gone.
|
||||
status = 400 if e.reason == "same_playlist" else 404
|
||||
raise HTTPException(
|
||||
status_code=400, detail={"code": "playlist_move", "reason": "same_playlist"}
|
||||
)
|
||||
# Both lists, in id order: a move A→B beside a move B→A would otherwise lock crosswise.
|
||||
lock_playlists(db, src.id, dst.id)
|
||||
item = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == src.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
if item is None:
|
||||
# A concurrent edit (another tab, or a YouTube re-pull) already took it out.
|
||||
raise HTTPException(
|
||||
status_code=404, detail={"code": "playlist_move", "reason": "not_in_playlist"}
|
||||
)
|
||||
already = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == dst.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
dst_changed = False
|
||||
if already is None:
|
||||
# Re-parent the existing row rather than delete+insert: `added_at` ("when I saved this")
|
||||
# survives the move, and there is no window where the video belongs to neither list. The
|
||||
# check above holds: both lists are locked, and an insert into the target (another tab's
|
||||
# add, a YouTube merge) waits for the lock — its foreign key needs the target row.
|
||||
item.playlist_id = dst.id
|
||||
item.position = _next_item_position(db, dst.id)
|
||||
dst_changed = True
|
||||
if not dst_changed:
|
||||
db.delete(item)
|
||||
# The source always changed; the target only when the row landed in it. `_touch` on a target
|
||||
# that already had the video promoted an untouched list in the rail's "recently changed" sort.
|
||||
recompute_dirty(db, src)
|
||||
_touch(src)
|
||||
if dst_changed:
|
||||
recompute_dirty(db, dst)
|
||||
_touch(dst)
|
||||
status_code=status, detail={"code": "playlist_move", "reason": e.reason}
|
||||
) from None
|
||||
db.commit()
|
||||
return {"ok": True, "from_playlist_id": src.id, "to_playlist_id": dst.id, "video_id": video_id}
|
||||
|
||||
@@ -676,30 +456,7 @@ def reorder_items(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked: a concurrent add between reading the items and writing the order would collide.
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist", lock=True)
|
||||
order = body.video_ids
|
||||
items = {
|
||||
it.video_id: it
|
||||
for it in db.execute(
|
||||
select(PlaylistItem).where(PlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[str] = set()
|
||||
for vid in order:
|
||||
it = items.get(vid)
|
||||
if it is not None:
|
||||
it.position = pos
|
||||
pos += 1
|
||||
seen.add(vid)
|
||||
# Any items the payload omitted (e.g. added concurrently in another tab) keep their relative
|
||||
# order but get fresh trailing positions, so no two items collide on the same position.
|
||||
for it in sorted(
|
||||
(it for vid, it in items.items() if vid not in seen), key=lambda i: i.position
|
||||
):
|
||||
it.position = pos
|
||||
pos += 1
|
||||
recompute_dirty(db, pl)
|
||||
_touch(pl)
|
||||
pl = _own_playlist(db, user, playlist_id, lock=True)
|
||||
pos = service.reorder_items(db, pl, body.video_ids)
|
||||
db.commit()
|
||||
return {"ok": True, "playlist_id": pl.id, "count": pos}
|
||||
|
||||
@@ -1,18 +1,22 @@
|
||||
"""Shared helpers, serializers and constants for the plex route package."""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import PlexItem
|
||||
from app.plex import queries
|
||||
from app.plex.cards import norm_lang as _norm_lang # noqa: F401 — the package's historic name
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def item_or_404(db: Session, rating_key: str) -> PlexItem:
|
||||
it = by_rating_key(db, PlexItem, rating_key)
|
||||
def item_or_404(db: Session, rating_key: str) -> "PlexItem":
|
||||
it = queries.item(db, rating_key)
|
||||
if it is None:
|
||||
raise not_found("plex_item")
|
||||
return it
|
||||
|
||||
@@ -1,27 +1,26 @@
|
||||
"""Plex admin: connection test, catalog sync, and owner watch-link config."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexLink,
|
||||
User,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex import queries, service
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex import watch_sync as plex_watch
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.utils import iso
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexLink
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
@@ -69,7 +68,7 @@ def _sample_plex_path(db: Session, plex: PlexClient, sections: list[dict]) -> st
|
||||
"""A real Plex-side media file path to validate the mount + path-map against. Prefers the
|
||||
already-mirrored catalog (cheap); before the first sync, pulls one movie leaf from an enabled
|
||||
section straight from Plex."""
|
||||
fp = db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
|
||||
fp = queries.sample_file_path(db)
|
||||
if fp:
|
||||
return fp
|
||||
wanted = plex_sync._enabled_section_keys(db)
|
||||
@@ -106,7 +105,7 @@ def trigger_sync(_: User = Depends(admin_user), db: Session = Depends(get_db)) -
|
||||
return {"ok": True, **plex_sync.sync(db)}
|
||||
|
||||
|
||||
def _link_dict(link: PlexLink | None) -> dict:
|
||||
def _link_dict(link: "PlexLink | None") -> dict:
|
||||
return {
|
||||
"linked": link is not None,
|
||||
"sync_enabled": bool(link and link.sync_enabled),
|
||||
@@ -119,8 +118,7 @@ def _link_dict(link: PlexLink | None) -> dict:
|
||||
@router.get("/watch/link")
|
||||
def watch_link_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""This user's Plex watch-sync link status (drives the Settings toggle)."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
return _link_dict(link)
|
||||
return _link_dict(queries.watch_link(db, user))
|
||||
|
||||
|
||||
class WatchLinkIn(BaseModel):
|
||||
@@ -139,11 +137,7 @@ def watch_link_set(
|
||||
if not sysconfig.get_bool(db, "plex_enabled"):
|
||||
raise refused(409, "plex_disabled")
|
||||
enabled = body.enabled
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
if link is None:
|
||||
link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
|
||||
db.add(link)
|
||||
link.sync_enabled = enabled
|
||||
link = service.set_watch_link(db, user, enabled)
|
||||
db.commit()
|
||||
|
||||
result: dict = {}
|
||||
@@ -156,7 +150,7 @@ def watch_link_set(
|
||||
def watch_import_now(user: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""Admin: (re-)run the "Plex is master" import for this account — Plex re-wins on the
|
||||
intersection; Siftlode-only states are preserved. Idempotent."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
link = queries.watch_link(db, user)
|
||||
if link is None or not link.sync_enabled:
|
||||
raise refused(409, "watch_sync_off")
|
||||
result = plex_watch.import_owner_watch_state(db, link)
|
||||
|
||||
@@ -1,110 +1,30 @@
|
||||
"""Plex browse: the library list, the unified cross-library browse feed, and its faceted filters."""
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
Integer,
|
||||
String,
|
||||
and_,
|
||||
cast,
|
||||
func,
|
||||
literal,
|
||||
null,
|
||||
or_,
|
||||
select,
|
||||
)
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated, sorted_words
|
||||
from app import sysconfig
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.mailtext import lang_for_user
|
||||
from app.models import User
|
||||
from app.plex import library as plex_library
|
||||
from app.plex import queries
|
||||
from app.plex.cards import display_title, episode_card
|
||||
from app.plex.query import (
|
||||
FIELD_COLUMNS,
|
||||
PLAYBACK_CLASSES,
|
||||
TS_CONFIG,
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_movie_state_filter,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
field_recheck,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
movie_status_case,
|
||||
playable_conds,
|
||||
playback_class_expr,
|
||||
show_status_case,
|
||||
split_csv,
|
||||
summary_col,
|
||||
title_col,
|
||||
vector_col,
|
||||
vector_match,
|
||||
)
|
||||
from app.plex.query import LibraryFilters
|
||||
from app.routes.plex._shared import _norm_lang
|
||||
from app.utils import to_tsquery_str
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def sort_column(u, sort: str, lang: str = "en"):
|
||||
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
|
||||
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
|
||||
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
|
||||
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
|
||||
in the reader's alphabet (`collation`)."""
|
||||
return {
|
||||
"added": u.c.added_at,
|
||||
"title": collated(u.c.title, lang),
|
||||
"year": u.c.year,
|
||||
"rating": u.c.rating,
|
||||
"duration": u.c.duration_s,
|
||||
"release": u.c.rel,
|
||||
"lastwatched": u.c.last_watched,
|
||||
}.get(sort, u.c.added_at)
|
||||
|
||||
|
||||
@router.get("/libraries")
|
||||
def list_libraries(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""The mirrored libraries the user can browse (drives the Plex scope selector). HU overlay
|
||||
libraries are excluded — they're not browsable, they only feed the EN twin's Hungarian metadata."""
|
||||
libs = (
|
||||
db.execute(
|
||||
select(PlexLibrary)
|
||||
.where(PlexLibrary.overlay_of.is_(None))
|
||||
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
out = []
|
||||
for lib in libs:
|
||||
if lib.kind == "movie":
|
||||
count = db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.library_id == lib.id, PlexItem.kind == "movie"
|
||||
)
|
||||
)
|
||||
else:
|
||||
count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
|
||||
out.append(
|
||||
{
|
||||
"key": lib.plex_key,
|
||||
"title": lib.title,
|
||||
"kind": lib.kind,
|
||||
"count": count or 0,
|
||||
}
|
||||
)
|
||||
out = [
|
||||
{"key": lib.plex_key, "title": lib.title, "kind": lib.kind, "count": count}
|
||||
for lib, count in queries.browsable_libraries(db, lang_for_user(user))
|
||||
]
|
||||
return {"enabled": sysconfig.get_bool(db, "plex_enabled"), "libraries": out}
|
||||
|
||||
|
||||
@@ -129,193 +49,60 @@ def unified_library(
|
||||
episodes come back in a separate `episodes` list (grouped result — the 'Episodes' section)."""
|
||||
offset = max(0, offset)
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
|
||||
ts = to_tsquery_str(q) if q else None
|
||||
tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
|
||||
if field not in FIELD_COLUMNS and field != "all":
|
||||
field = "all"
|
||||
# `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
|
||||
# the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
|
||||
grid_field = field != "episode"
|
||||
recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
|
||||
|
||||
selects = []
|
||||
if want_movies and grid_field:
|
||||
st = aliased(PlexState)
|
||||
m_status = movie_status_case(st)
|
||||
mq = (
|
||||
select(
|
||||
PlexItem.rating_key.label("rk"),
|
||||
literal("movie").label("kind"),
|
||||
title_col(PlexItem, lang).label("title"),
|
||||
PlexItem.year.label("year"),
|
||||
PlexItem.rating.label("rating"),
|
||||
# Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
|
||||
func.left(summary_col(PlexItem, lang), 300).label("summary"),
|
||||
PlexItem.added_at.label("added_at"),
|
||||
PlexItem.originally_available_at.label("rel"),
|
||||
PlexItem.duration_s.label("duration_s"),
|
||||
PlexItem.playable.label("playable"),
|
||||
PlexItem.transcode_hw.label("transcode_hw"),
|
||||
cast(null(), Integer).label("season_count"),
|
||||
func.coalesce(st.position_seconds, 0).label("position_seconds"),
|
||||
m_status.label("status"),
|
||||
literal(False).label("resumable"),
|
||||
# "last watched" sort key: when the movie was last resumed, else when it was marked
|
||||
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
|
||||
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexItem, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(st, and_(st.item_id == PlexItem.id, st.user_id == user.id))
|
||||
.where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
|
||||
)
|
||||
mq = apply_movie_state_filter(mq, st, show)
|
||||
mq = apply_meta_filters(mq, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
mq = mq.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
mq = mq.where(PlexItem.duration_s <= f.duration_max)
|
||||
# Playback-class filter (movie-only — a show is an aggregate with no single class).
|
||||
mq = mq.where(*playable_conds(split_csv(f.playable)))
|
||||
if tsq is not None:
|
||||
mq = mq.where(vector_match(PlexItem, lang, tsq))
|
||||
if recheck is not None:
|
||||
mq = mq.where(recheck(PlexItem, field, tsq, lang))
|
||||
selects.append(mq)
|
||||
if want_shows and grid_field:
|
||||
agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
|
||||
tot = func.coalesce(agg.c.total, 0)
|
||||
wat = func.coalesce(agg.c.watched, 0)
|
||||
inp = func.coalesce(agg.c.inprog, 0)
|
||||
s_status = show_status_case(tot, wat, inp)
|
||||
sq = (
|
||||
select(
|
||||
PlexShow.rating_key.label("rk"),
|
||||
literal("show").label("kind"),
|
||||
title_col(PlexShow, lang).label("title"),
|
||||
PlexShow.year.label("year"),
|
||||
PlexShow.rating.label("rating"),
|
||||
func.left(summary_col(PlexShow, lang), 300).label("summary"),
|
||||
PlexShow.added_at.label("added_at"),
|
||||
PlexShow.originally_available_at.label("rel"),
|
||||
cast(null(), Integer).label("duration_s"),
|
||||
cast(null(), String).label("playable"),
|
||||
cast(null(), Boolean).label("transcode_hw"),
|
||||
PlexShow.child_count.label("season_count"),
|
||||
literal(0).label("position_seconds"),
|
||||
s_status.label("status"),
|
||||
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
|
||||
(inp > 0).label("resumable"),
|
||||
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
|
||||
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
|
||||
agg.c.resumed_at.label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexShow, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_lib_ids))
|
||||
)
|
||||
sq = apply_show_state_filter(sq, tot, wat, inp, show)
|
||||
sq = apply_meta_filters(sq, PlexShow, p, lang)
|
||||
if tsq is not None:
|
||||
sq = sq.where(vector_match(PlexShow, lang, tsq))
|
||||
if recheck is not None:
|
||||
sq = sq.where(recheck(PlexShow, field, tsq, lang))
|
||||
selects.append(sq)
|
||||
|
||||
total, rows, eps = plex_library.library_page(
|
||||
db,
|
||||
user,
|
||||
scope=scope,
|
||||
q=q,
|
||||
field=field,
|
||||
sort=sort,
|
||||
sort_dir=sort_dir,
|
||||
show=show,
|
||||
lang=lang,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
f=f,
|
||||
)
|
||||
items = []
|
||||
total = 0
|
||||
if selects:
|
||||
union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
|
||||
u = union.subquery()
|
||||
total = db.scalar(select(func.count()).select_from(u)) or 0
|
||||
|
||||
if tsq is not None:
|
||||
order = [u.c.rank.desc()]
|
||||
else:
|
||||
col = sort_column(u, sort, lang)
|
||||
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
|
||||
order.append(u.c.rk.desc())
|
||||
rows = db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all()
|
||||
|
||||
for r in rows:
|
||||
if r.kind == "movie":
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "movie",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"duration_seconds": r.duration_s,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"playable": r.playable,
|
||||
"transcode_hw": r.transcode_hw,
|
||||
"status": r.status,
|
||||
"position_seconds": r.position_seconds,
|
||||
}
|
||||
)
|
||||
else:
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "show",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"season_count": r.season_count,
|
||||
"status": r.status,
|
||||
"resumable": r.resumable,
|
||||
}
|
||||
)
|
||||
|
||||
# Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
|
||||
# and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
|
||||
# target the movie/show grid, so they suppress episodes.
|
||||
episodes = []
|
||||
if tsq is not None and want_shows and field in ("all", "episode"):
|
||||
ep_st = aliased(PlexState)
|
||||
# In HU an episode is found by its Hungarian title too (the overlay fills it) — and
|
||||
# still by its English one, like `vector_match` does for the grid.
|
||||
if field == "episode":
|
||||
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
|
||||
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
|
||||
else:
|
||||
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
|
||||
hit = ep_vec.op("@@")(tsq)
|
||||
if lang == "hu":
|
||||
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
|
||||
ep_vec = ep_vec_hu
|
||||
eps = db.execute(
|
||||
select(PlexItem, ep_st, PlexShow)
|
||||
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, ep_st.user_id == user.id))
|
||||
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_lib_ids),
|
||||
hit,
|
||||
for r in rows:
|
||||
if r.kind == "movie":
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "movie",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"duration_seconds": r.duration_s,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"playable": r.playable,
|
||||
"transcode_hw": r.transcode_hw,
|
||||
"status": r.status,
|
||||
"position_seconds": r.position_seconds,
|
||||
}
|
||||
)
|
||||
.order_by(func.ts_rank(ep_vec, tsq).desc())
|
||||
.limit(24)
|
||||
).all()
|
||||
for e, s, sh in eps:
|
||||
card = episode_card(e, s, lang=lang)
|
||||
card["show_title"] = display_title(sh, lang) if sh is not None else None
|
||||
episodes.append(card)
|
||||
else:
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "show",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"season_count": r.season_count,
|
||||
"status": r.status,
|
||||
"resumable": r.resumable,
|
||||
}
|
||||
)
|
||||
|
||||
episodes = []
|
||||
for e, s, sh in eps:
|
||||
card = episode_card(e, s, lang=lang)
|
||||
card["show_title"] = display_title(sh, lang) if sh is not None else None
|
||||
episodes.append(card)
|
||||
|
||||
return {
|
||||
"scope": scope,
|
||||
@@ -341,237 +128,4 @@ def facets(
|
||||
filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
|
||||
a multi-select group still lets you add more. Merged across the scope's libraries; duration is
|
||||
movie-only. (Search text is not factored in — facets track the sidebar filters.)"""
|
||||
empty = {
|
||||
"genres": [],
|
||||
"content_ratings": [],
|
||||
"studios": [],
|
||||
"playable": [],
|
||||
"year_min": None,
|
||||
"year_max": None,
|
||||
"rating_max": None,
|
||||
"duration_min": None,
|
||||
"duration_max": None,
|
||||
}
|
||||
all_movie_ids, all_show_ids = content_libs(db)
|
||||
movie_ids = all_movie_ids if scope in ("movie", "both") else []
|
||||
show_ids = all_show_ids if scope in ("show", "both") else []
|
||||
if not movie_ids and not show_ids:
|
||||
return empty
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
|
||||
uid = user.id
|
||||
|
||||
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
|
||||
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
|
||||
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
|
||||
# The same filter on the same rows, so the counts do not change
|
||||
# (test_plex_facets_grid_parity).
|
||||
shows_in_scope: list[int] | None = None
|
||||
if show_ids and show in ("watched", "in_progress", "unwatched"):
|
||||
agg = rollup.agg_subquery(db, uid).subquery()
|
||||
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
in_scope = apply_show_state_filter(
|
||||
in_scope,
|
||||
func.coalesce(agg.c.total, 0),
|
||||
func.coalesce(agg.c.watched, 0),
|
||||
func.coalesce(agg.c.inprog, 0),
|
||||
show,
|
||||
)
|
||||
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
|
||||
|
||||
def wsq(query, model, ids, pp, kind_movie, with_dur):
|
||||
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
|
||||
facet aggregation query, so the facets reflect exactly the visible result set."""
|
||||
query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
|
||||
if kind_movie:
|
||||
query = query.where(model.kind == "movie")
|
||||
if with_dur and pp.get("duration_min") is not None:
|
||||
query = query.where(model.duration_s >= pp["duration_min"])
|
||||
if with_dur and pp.get("duration_max") is not None:
|
||||
query = query.where(model.duration_s <= pp["duration_max"])
|
||||
# Playback-class filter (movie-only) — applied for the OTHER groups so an active class
|
||||
# narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
|
||||
query = query.where(*playable_conds(split_csv(pp.get("playable"))))
|
||||
# Join watch-state and reuse the grid's exact movie state filter (unified_library) —
|
||||
# hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
|
||||
st = aliased(PlexState)
|
||||
query = query.outerjoin(st, and_(st.item_id == model.id, st.user_id == uid))
|
||||
query = apply_movie_state_filter(query, st, show)
|
||||
elif shows_in_scope is not None:
|
||||
query = query.where(model.id.in_(shows_in_scope))
|
||||
return query
|
||||
|
||||
genre_counts: dict[str, int] = {}
|
||||
cr_counts: dict[str, int] = {}
|
||||
studio_counts: dict[str, int] = {}
|
||||
playable_counts: dict[str, int] = {}
|
||||
years: list[int] = []
|
||||
ratings: list[float] = []
|
||||
durs: list[int] = []
|
||||
|
||||
def per_table(model, ids, kind_movie: bool) -> None:
|
||||
if not ids:
|
||||
return
|
||||
# Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
|
||||
# OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
|
||||
# offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
|
||||
# the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
|
||||
# every matching title carries them, so they stay togglable.)
|
||||
genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
|
||||
# Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
|
||||
# `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
|
||||
# jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
|
||||
# so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
|
||||
# whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
|
||||
# In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
|
||||
gcol = genres_col(model, lang)
|
||||
expanded = (
|
||||
wsq(
|
||||
select(func.jsonb_array_elements_text(gcol).label("g")),
|
||||
model,
|
||||
ids,
|
||||
genre_pp,
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(func.jsonb_typeof(gcol) == "array")
|
||||
.subquery()
|
||||
)
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
genre_counts[g] = genre_counts.get(g, 0) + c
|
||||
# Content ratings — exclude its own.
|
||||
crq = (
|
||||
wsq(
|
||||
select(model.content_rating, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "content_ratings": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.content_rating.isnot(None))
|
||||
.group_by(model.content_rating)
|
||||
)
|
||||
for cr, c in db.execute(crq).all():
|
||||
cr_counts[cr] = cr_counts.get(cr, 0) + c
|
||||
# Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
|
||||
stq = (
|
||||
wsq(
|
||||
select(model.studio, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "studios": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.studio.isnot(None))
|
||||
.group_by(model.studio)
|
||||
)
|
||||
for s, c in db.execute(stq).all():
|
||||
studio_counts[s] = studio_counts.get(s, 0) + c
|
||||
# Playback class (movie-only) — exclude its own so each class stays an addable OR option.
|
||||
if kind_movie:
|
||||
pcls = playback_class_expr()
|
||||
pcq = (
|
||||
wsq(
|
||||
select(pcls.label("cls"), func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "playable": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(pcls.isnot(None))
|
||||
.group_by(pcls)
|
||||
)
|
||||
for cls, c in db.execute(pcq).all():
|
||||
playable_counts[cls] = playable_counts.get(cls, 0) + c
|
||||
# Year bounds — exclude the year filter.
|
||||
by = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.year), func.max(model.year)),
|
||||
model,
|
||||
ids,
|
||||
{**p, "year_min": None, "year_max": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
).first()
|
||||
if by[0] is not None:
|
||||
years.append(by[0])
|
||||
if by[1] is not None:
|
||||
years.append(by[1])
|
||||
# Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
|
||||
# rating filter and the UI badge use (IMDb preferred, audience fallback).
|
||||
br = db.scalar(
|
||||
wsq(
|
||||
select(func.max(func.coalesce(model.imdb_rating, model.rating))),
|
||||
model,
|
||||
ids,
|
||||
{**p, "rating_min": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
)
|
||||
if br is not None:
|
||||
ratings.append(float(br))
|
||||
# Duration bounds (movies) — exclude the duration filter.
|
||||
if kind_movie:
|
||||
bd = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.duration_s), func.max(model.duration_s)),
|
||||
model,
|
||||
ids,
|
||||
p,
|
||||
kind_movie,
|
||||
False,
|
||||
)
|
||||
).first()
|
||||
if bd[0] is not None:
|
||||
durs.append(bd[0])
|
||||
if bd[1] is not None:
|
||||
durs.append(bd[1])
|
||||
|
||||
per_table(PlexItem, movie_ids, True)
|
||||
per_table(PlexShow, show_ids, False)
|
||||
# Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
|
||||
# be absent): the sidebar renders genres solely from this list and hides the whole genre section —
|
||||
# chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
|
||||
# undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
|
||||
for g in split_csv(f.genres):
|
||||
genre_counts.setdefault(g, 0)
|
||||
# Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
|
||||
for s in split_csv(f.studios):
|
||||
studio_counts.setdefault(s, 0)
|
||||
# Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
|
||||
# section when its facet list is empty, which would trap a zero-result selection with no way to undo).
|
||||
for c in split_csv(f.playable):
|
||||
if c in PLAYBACK_CLASSES:
|
||||
playable_counts.setdefault(c, 0)
|
||||
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
|
||||
return {
|
||||
"genres": [
|
||||
{"value": g, "count": c}
|
||||
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
|
||||
],
|
||||
"content_ratings": [
|
||||
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
|
||||
],
|
||||
"studios": [
|
||||
{"value": s, "count": c}
|
||||
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
|
||||
],
|
||||
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
|
||||
"playable": [
|
||||
{"value": c, "count": playable_counts[c]}
|
||||
for c in PLAYBACK_CLASSES
|
||||
if c in playable_counts
|
||||
],
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"rating_max": max(ratings) if ratings else None,
|
||||
"duration_min": min(durs) if durs else None,
|
||||
"duration_max": max(durs) if durs else None,
|
||||
}
|
||||
return plex_library.facets(db, user, scope=scope, show=show, lang=lang, f=f)
|
||||
|
||||
@@ -1,96 +1,41 @@
|
||||
"""Plex collections: browse mirrored collections + the admin manual-collection editor."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime
|
||||
from typing import Any, NamedTuple
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app import logs
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex.cards import SHOW_NEW, display_title, movie_card, norm_lang, show_card
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
|
||||
from app.utils import like_escape
|
||||
from app.plex.collections import (
|
||||
DEFAULT_MEMBER_SORT,
|
||||
Member,
|
||||
collection_card,
|
||||
collection_cert,
|
||||
collection_editable,
|
||||
collection_source,
|
||||
sort_members,
|
||||
)
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexCollection, PlexState
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
|
||||
# members (Plex shows one cert on the collection; we don't store it, so derive it).
|
||||
_CERT_RANK = {
|
||||
"G": 1,
|
||||
"TV-Y": 1,
|
||||
"TV-Y7": 1,
|
||||
"TV-G": 1,
|
||||
"PG": 2,
|
||||
"TV-PG": 2,
|
||||
"PG-13": 3,
|
||||
"TV-14": 3,
|
||||
"R": 4,
|
||||
"TV-MA": 4,
|
||||
"NC-17": 5,
|
||||
}
|
||||
|
||||
|
||||
def _collection_cert(certs: list[str | None]) -> str | None:
|
||||
ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
|
||||
return max(ranked, key=lambda x: x[1])[0] if ranked else None
|
||||
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
|
||||
|
||||
|
||||
def _collection_source(col: PlexCollection) -> str:
|
||||
first = (col.title or "").strip().split(" ", 1)[0].lower()
|
||||
if first in _STRIP_PROVIDERS:
|
||||
return first
|
||||
if col.smart:
|
||||
return "smart"
|
||||
return "collection"
|
||||
|
||||
|
||||
def _collection_editable(c: PlexCollection) -> bool:
|
||||
"""Effective editability: the admin marked it editable AND it's a plain manual collection — never
|
||||
smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
|
||||
return bool(c.editable) and not c.smart and _collection_source(c) == "collection"
|
||||
|
||||
|
||||
def _collection_card(c: PlexCollection) -> dict:
|
||||
return {
|
||||
"id": c.rating_key,
|
||||
"title": c.title,
|
||||
"summary": c.summary,
|
||||
"thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
|
||||
"child_count": c.child_count,
|
||||
"smart": c.smart,
|
||||
"source": _collection_source(c),
|
||||
"editable": c.editable,
|
||||
"can_edit": _collection_editable(
|
||||
c
|
||||
), # whether the edit controls should be offered (admin UI)
|
||||
}
|
||||
|
||||
|
||||
@router.get("/collections")
|
||||
def collections(
|
||||
@@ -110,64 +55,22 @@ def collections(
|
||||
sidebar token and the applied-filter chip had nothing to render but the raw id. Resolving them
|
||||
through this endpoint costs one small row each; pulling the whole list to find two titles is
|
||||
~200 KB."""
|
||||
if library is not None:
|
||||
lib = (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(library)))
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if lib is None:
|
||||
return {"collections": []}
|
||||
query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
|
||||
else:
|
||||
enabled_ids = list(
|
||||
db.execute(
|
||||
select(PlexLibrary.id).where(
|
||||
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
if not enabled_ids:
|
||||
return {"collections": []}
|
||||
query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
|
||||
term = (q or "").strip()
|
||||
if term:
|
||||
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
|
||||
if keys is not None:
|
||||
wanted = [k for k in (keys or "").split(",") if k]
|
||||
# An explicit but empty `keys=` asks for nothing — answer nothing, rather than falling through
|
||||
# to "every collection" (which is what an unguarded empty IN () would have to become).
|
||||
if not wanted:
|
||||
return {"collections": []}
|
||||
query = query.where(PlexCollection.rating_key.in_(wanted))
|
||||
rows = (
|
||||
db.execute(
|
||||
query.order_by(
|
||||
PlexCollection.child_count.desc().nullslast(),
|
||||
collated(PlexCollection.title, lang_for_user(user)),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
wanted = [k for k in keys.split(",") if k] if keys is not None else None
|
||||
rows = queries.collections_page(db, library, q, wanted, lang_for_user(user))
|
||||
# A montage of the top titles per tile + the collection's total runtime. Art unions MOVIE and SHOW
|
||||
# members (a collection lives in one library, so exactly one side has art) — a show collection would
|
||||
# otherwise render blank; the collection's own poster is the next fallback, then the colour hash.
|
||||
# Runtime is movie-only (shows have no single duration).
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
keys = [c.rating_key for c in rows]
|
||||
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
|
||||
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
|
||||
runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
|
||||
art_m, art_s, runtime = queries.collection_tiles(db, [c.rating_key for c in rows])
|
||||
|
||||
def _art(c: PlexCollection) -> list[str]:
|
||||
def _art(c: "PlexCollection") -> list[str]:
|
||||
member = art_m.get(c.rating_key) or art_s.get(c.rating_key)
|
||||
return member or ([c.rating_key] if c.thumb_key else [])
|
||||
|
||||
return {
|
||||
"collections": [
|
||||
{
|
||||
**_collection_card(c),
|
||||
**collection_card(c),
|
||||
"art": _art(c),
|
||||
"total_runtime_s": runtime.get(c.rating_key, 0),
|
||||
}
|
||||
@@ -185,89 +88,21 @@ def collections_for_item(
|
||||
"""What a movie or show card's "Add to collection" lists: the editable collections of the title's
|
||||
library (a collection lives in one library, and only a manual one can take a title — the same
|
||||
`can_edit` the editor offers), each saying whether the title is in it already, by name."""
|
||||
movie = by_rating_key(db, PlexItem, item_rating_key)
|
||||
title: PlexItem | PlexShow | None = (
|
||||
movie if movie is not None and movie.kind == "movie" else None
|
||||
) or by_rating_key(db, PlexShow, item_rating_key)
|
||||
movie = queries.movie(db, item_rating_key)
|
||||
title = movie or queries.show(db, item_rating_key)
|
||||
if title is None:
|
||||
raise not_found("item")
|
||||
member_of = set(title.collection_keys or [])
|
||||
rows = (
|
||||
db.execute(
|
||||
select(PlexCollection)
|
||||
.where(PlexCollection.library_id == title.library_id)
|
||||
.order_by(collated(PlexCollection.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = queries.library_collections(db, title.library_id, lang_for_user(user))
|
||||
return {
|
||||
"collections": [
|
||||
{**_collection_card(c), "contains": c.rating_key in member_of}
|
||||
{**collection_card(c), "contains": c.rating_key in member_of}
|
||||
for c in rows
|
||||
if _collection_editable(c)
|
||||
if collection_editable(c)
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
class Member(NamedTuple):
|
||||
"""One collection member flattened to everything the Sort control can order by. The page's two
|
||||
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
|
||||
ordering is a pure list sort over these, which is also what makes it unit-testable."""
|
||||
|
||||
kind: str # "movie" | "show"
|
||||
obj: Any # PlexItem | PlexShow, carried through to the card serializer
|
||||
title: str # lower-cased; never NULL (the column is not nullable)
|
||||
year: int | None
|
||||
rating: float | None
|
||||
added: datetime | None
|
||||
release: date | None
|
||||
duration: int | None # movies only — a show has no single runtime
|
||||
last_watched: datetime | None
|
||||
|
||||
|
||||
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
|
||||
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
|
||||
# that silently does nothing here.
|
||||
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
|
||||
"added": lambda m: m.added,
|
||||
"lastwatched": lambda m: m.last_watched,
|
||||
"release": lambda m: m.release,
|
||||
"year": lambda m: m.year,
|
||||
"rating": lambda m: m.rating,
|
||||
"duration": lambda m: m.duration,
|
||||
"title": lambda m: m.title,
|
||||
}
|
||||
|
||||
# What the page ordered by before it had a control, and still its default: a film series wants to be
|
||||
# read in release order, not by whenever Plex happened to ingest it.
|
||||
DEFAULT_MEMBER_SORT = "year"
|
||||
|
||||
|
||||
def sort_members(
|
||||
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
|
||||
) -> list[Member]:
|
||||
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
|
||||
|
||||
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
|
||||
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
|
||||
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
|
||||
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
|
||||
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
|
||||
|
||||
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
|
||||
compare as Python strings, which puts every accented initial after "z"."""
|
||||
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
|
||||
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
|
||||
if key is MEMBER_SORT_KEYS["title"]:
|
||||
key = by_title
|
||||
ordered = sorted(members, key=by_title) # stable secondary, applied first
|
||||
have = [m for m in ordered if key(m) is not None]
|
||||
missing = [m for m in ordered if key(m) is None]
|
||||
have.sort(key=key, reverse=sort_dir != "asc")
|
||||
return have + missing
|
||||
|
||||
|
||||
@router.get("/collections/{rating_key}")
|
||||
def collection_detail(
|
||||
rating_key: str,
|
||||
@@ -283,45 +118,20 @@ def collection_detail(
|
||||
what this page did before it had a control. Read-only here; edits go through the admin mutation
|
||||
endpoints below."""
|
||||
lang = norm_lang(lang)
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow)
|
||||
.where(PlexShow.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
movies, shows = queries.collection_members(db, rating_key, lang)
|
||||
mstate = rollup.item_state_map(db, user.id, [m.id for m in movies])
|
||||
# ONE rollup pass over the show members, for both the aggregate badge status and `resumed_at` —
|
||||
# the browse grid's "last watched" for a show (the newest resume/watched across its episodes). The
|
||||
# cheaper `rollup.state_map` gives only the status, which would leave `lastwatched` unorderable on
|
||||
# a show collection while the control still offered it.
|
||||
show_rows = (
|
||||
db.execute(
|
||||
rollup.agg_subquery(db, user.id, include_resumed=True).where(
|
||||
PlexItem.show_id.in_([s.id for s in shows])
|
||||
)
|
||||
).all()
|
||||
if shows
|
||||
else []
|
||||
)
|
||||
show_rows = queries.show_rollups(db, user.id, [s.id for s in shows])
|
||||
sstate = {r.show_id: rollup.show_state(r.total, r.watched, r.inprog) for r in show_rows}
|
||||
sresumed = {r.show_id: r.resumed_at for r in show_rows}
|
||||
|
||||
def _movie_last_watched(st: PlexState | None) -> datetime | None:
|
||||
def _movie_last_watched(st: "PlexState | None") -> datetime | None:
|
||||
# Same definition as the browse grid's: when it was last resumed, else when it was marked
|
||||
# watched (coalesce(progress_updated_at, watched_at)).
|
||||
return (st.progress_updated_at or st.watched_at) if st else None
|
||||
@@ -365,8 +175,8 @@ def collection_detail(
|
||||
years = [m.year for m in members if m.year is not None]
|
||||
certs = [m.content_rating for m in movies] + [s.content_rating for s in shows]
|
||||
return {
|
||||
**_collection_card(col),
|
||||
"content_rating": _collection_cert(certs),
|
||||
**collection_card(col),
|
||||
"content_rating": collection_cert(certs),
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"total_runtime_s": sum(m.duration_s or 0 for m in movies),
|
||||
@@ -374,11 +184,11 @@ def collection_detail(
|
||||
}
|
||||
|
||||
|
||||
def _editable_collection_or_403(db: Session, rating_key: str) -> PlexCollection:
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
def _editable_collection_or_403(db: Session, rating_key: str) -> "PlexCollection":
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
if not _collection_editable(col):
|
||||
if not collection_editable(col):
|
||||
raise refused(403, "collection_read_only")
|
||||
return col
|
||||
|
||||
@@ -389,16 +199,7 @@ def _reject_duplicate_name(
|
||||
"""No two collections in a section may share a name: Plex CONFLATES same-named collections'
|
||||
membership (both rating_keys resolve to one merged member list — a real footgun), so forbid it up
|
||||
front on create + rename rather than mirror the mess."""
|
||||
dup = (
|
||||
db.execute(
|
||||
select(PlexCollection).where(
|
||||
PlexCollection.library_id == library_id,
|
||||
func.lower(PlexCollection.title) == title.lower(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
dup = queries.collection_named(db, library_id, title)
|
||||
if dup is not None and dup.rating_key != exclude_rk:
|
||||
# Structured, so the bilingual UI says it in the reader's language (it was toasted in
|
||||
# Hungarian beside an English error dialog saying the same thing).
|
||||
@@ -441,15 +242,13 @@ def create_collection(
|
||||
seed = body.item_rating_key
|
||||
if not title or not seed:
|
||||
raise refused(400, "collection_fields_required")
|
||||
seed_show = by_rating_key(db, PlexShow, seed)
|
||||
seed_show = queries.show(db, seed)
|
||||
if library:
|
||||
lib = (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == library)).scalars().first()
|
||||
)
|
||||
lib = queries.library_by_key(db, library)
|
||||
else:
|
||||
# The "New collection" flow only knows the seed title → derive its library.
|
||||
seed_item = by_rating_key(db, PlexItem, seed) or seed_show
|
||||
lib = db.get(PlexLibrary, seed_item.library_id) if seed_item else None
|
||||
seed_item = queries.item(db, seed) or seed_show
|
||||
lib = queries.library(db, seed_item.library_id) if seed_item else None
|
||||
if lib is None:
|
||||
raise not_found("library")
|
||||
_reject_duplicate_name(db, lib.id, title)
|
||||
@@ -462,28 +261,16 @@ def create_collection(
|
||||
wrote = True
|
||||
new_rk = str(new.get("ratingKey") or "")
|
||||
col = plex_sync.resync_collection(db, plex, lib, new_rk)
|
||||
col.editable = True
|
||||
service.set_collection_editable(col, True)
|
||||
db.commit()
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
if wrote and new_rk:
|
||||
# It is on Plex now, and only the follow-up read failed. Its row is kept, editable: the
|
||||
# full sync preserves `editable` but creates a missing row read-only, so the collection
|
||||
# would drop out of every "add to" list; and the duplicate-name check reads these rows,
|
||||
# so a retry of the same name would make a second one on Plex. Plex made it with exactly
|
||||
# the seed in it, so the seed's row says so too — otherwise the title showed unticked in
|
||||
# the collection it was just added to, and the collection empty, until that sync.
|
||||
db.add(
|
||||
PlexCollection(
|
||||
rating_key=new_rk, library_id=lib.id, title=title, editable=True, child_count=1
|
||||
)
|
||||
)
|
||||
Model = PlexItem if lib.kind == "movie" else PlexShow
|
||||
if row := by_rating_key(db, Model, seed):
|
||||
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), new_rk]))
|
||||
# It is on Plex now, and only the follow-up read failed: keep its row (see the service).
|
||||
service.keep_created_collection(db, lib, new_rk, title, seed)
|
||||
db.commit()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.post("/collections/{rating_key}/items/{item_rating_key}")
|
||||
@@ -494,7 +281,7 @@ def collection_add_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -505,7 +292,7 @@ def collection_add_item(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.delete("/collections/{rating_key}/items/{item_rating_key}")
|
||||
@@ -516,7 +303,7 @@ def collection_remove_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -527,7 +314,7 @@ def collection_remove_item(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
class CollectionRenameIn(BaseModel):
|
||||
@@ -546,7 +333,7 @@ def collection_rename(
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
_reject_duplicate_name(db, col.library_id, title, exclude_rk=rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -557,7 +344,7 @@ def collection_rename(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.delete("/collections/{rating_key}")
|
||||
@@ -565,7 +352,7 @@ def collection_delete(
|
||||
rating_key: str, _: User = Depends(admin_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
plex.delete_collection(rating_key)
|
||||
@@ -590,12 +377,12 @@ def collection_set_editable(
|
||||
) -> dict:
|
||||
"""Mark an existing Plex collection as 'mine to edit' (or unmark). Smart / auto-list collections
|
||||
(IMDb/TMDb/…) can never be made editable — they're regenerated outside Plex."""
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
want = body.editable
|
||||
if want and (col.smart or _collection_source(col) != "collection"):
|
||||
if want and (col.smart or collection_source(col) != "collection"):
|
||||
raise refused(400, "collection_smart")
|
||||
col.editable = want
|
||||
service.set_collection_editable(col, want)
|
||||
db.commit()
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
@@ -4,52 +4,34 @@ import re
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app import logs
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.discover import queries
|
||||
from app.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.discover import queries as discover_queries
|
||||
from app.models import User
|
||||
from app.plex import detail as plex_detail
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import rollup, subtitles
|
||||
from app.plex.query import title_col
|
||||
from app.plex import queries, rollup, service, subtitles
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.collections import collection_source
|
||||
from app.plex.cards import SHOW_NEW, episode_card, movie_card, show_card
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes.plex._shared import _norm_lang, item_or_404
|
||||
from app.routes.plex.collections import _collection_source
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem, PlexSeason
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def _lib_key(db: Session, library_id: int | None) -> str | None:
|
||||
"""The Plex section key for a library id — the info/show page passes it to the admin collection
|
||||
editor (in the unified cross-library view there's no single library prop otherwise)."""
|
||||
if library_id is None:
|
||||
return None
|
||||
lib = db.get(PlexLibrary, library_id)
|
||||
return lib.plex_key if lib else None
|
||||
|
||||
|
||||
_SHOW_ENRICH_TTL_S = 300
|
||||
|
||||
_show_enrich_cache: dict[str, tuple[float, dict, list]] = {}
|
||||
@@ -113,39 +95,12 @@ def show_detail(
|
||||
"""A show's seasons + episodes (the drill-down view) with per-user watch state."""
|
||||
lang = _norm_lang(lang)
|
||||
hu = lang == "hu"
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
seasons = (
|
||||
db.execute(
|
||||
select(PlexSeason)
|
||||
.where(PlexSeason.show_id == sh.id)
|
||||
.order_by(PlexSeason.season_number.asc().nullsfirst())
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.item_id.in_([e.id for e in eps] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
seasons = queries.seasons(db, sh)
|
||||
eps = queries.episodes(db, sh)
|
||||
states = rollup.item_state_map(db, user, [e.id for e in eps])
|
||||
by_season: dict[int | None, list] = {}
|
||||
all_cards: list[dict] = [] # every episode, in season/episode order → show-level rollup
|
||||
for e in eps:
|
||||
@@ -163,9 +118,7 @@ def show_detail(
|
||||
meta, related_raw = _show_enrichment(db, sh.rating_key)
|
||||
related = _relatedshow_cards(db, user.id, related_raw, exclude=sh.id, lang=lang)
|
||||
if sh.people_enriched_at is None and meta:
|
||||
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
|
||||
plex_detail.apply_rich_detail(sh, meta) # imdb/tagline/full_cast for future views
|
||||
sh.people_enriched_at = datetime.now(timezone.utc)
|
||||
service.enrich_show(db, sh, meta) # imdb/tagline/full_cast for future views
|
||||
try:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — a failed enrich must not fail the page
|
||||
@@ -198,7 +151,7 @@ def show_detail(
|
||||
"year": sh.year,
|
||||
"thumb": f"/api/plex/image/{sh.rating_key}",
|
||||
"art": f"/api/plex/image/{sh.rating_key}?variant=art",
|
||||
"library": _lib_key(db, sh.library_id),
|
||||
"library": queries.library_key(db, sh.library_id),
|
||||
"content_rating": sh.content_rating,
|
||||
"rating": sh.rating,
|
||||
"genres": (sh.genres_hu or sh.genres if hu else sh.genres) or [],
|
||||
@@ -234,13 +187,13 @@ def show_next(
|
||||
Asked one show at a time, when a card is hovered or played, rather than carried on every card of a
|
||||
grid: three probes a show would cost a 60-card page 180 queries for the few cards anyone plays.
|
||||
`next` is null for a show with no episodes."""
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
return {"next": queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
|
||||
return {"next": discover_queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
|
||||
|
||||
|
||||
def _season_block(se: PlexSeason, cards: list[dict]) -> dict:
|
||||
def _season_block(se: "PlexSeason", cards: list[dict]) -> dict:
|
||||
"""A season's card for the show page (with its aggregate rollup) plus its episodes (used by the
|
||||
season subpage, read from the same cached show-detail payload)."""
|
||||
roll = rollup.rollup(cards)
|
||||
@@ -265,12 +218,7 @@ def _relatedshow_cards(
|
||||
rks = [str(m.get("ratingKey")) for m in related if m.get("ratingKey")]
|
||||
if not rks:
|
||||
return []
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks[:60]), PlexShow.id != exclude)
|
||||
).scalars()
|
||||
}
|
||||
shows = queries.shows_by_keys(db, rks[:60], exclude)
|
||||
statuses = rollup.state_map(db, user_id, [s.id for s in shows.values()])
|
||||
out: list[dict] = []
|
||||
seen: set[str] = set()
|
||||
@@ -285,73 +233,36 @@ def _relatedshow_cards(
|
||||
return out
|
||||
|
||||
|
||||
def _collection_strips(db: Session, it: PlexItem, user_id: int, lang: str) -> list[dict]:
|
||||
def _collection_strips(db: Session, it: "PlexItem", user_id: int, lang: str) -> list[dict]:
|
||||
"""The collections this movie belongs to, each with its member titles (playable cards) — the
|
||||
'Avatar Collection' strip on the info page. Pure local lookup (no Plex call)."""
|
||||
keys = it.collection_keys or []
|
||||
if not keys or it.kind != "movie":
|
||||
return []
|
||||
cols = {
|
||||
c.rating_key: c
|
||||
for c in db.execute(
|
||||
select(PlexCollection).where(PlexCollection.rating_key.in_(keys))
|
||||
).scalars()
|
||||
}
|
||||
cols = {c.rating_key: c for c in queries.collections_by_keys(db, keys)}
|
||||
# Smallest (most specific — e.g. a franchise) first; the big auto-lists (IMDb/TMDb Popular) last.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [c.title for c in cols.values()], lang))}
|
||||
ordered = sorted(cols.values(), key=lambda c: (c.child_count or 0, alpha[c.title]))
|
||||
out = []
|
||||
for col in ordered:
|
||||
rk = col.rating_key
|
||||
members = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.library_id == it.library_id,
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.collection_keys.contains([rk]),
|
||||
)
|
||||
.order_by(
|
||||
PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang)
|
||||
)
|
||||
.limit(60)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
ids = [m.id for m in members]
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user_id, PlexState.item_id.in_(ids or [0])
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
members = queries.library_movies_in(db, it.library_id, rk, lang, limit=60)
|
||||
states = rollup.item_state_map(db, user_id, [m.id for m in members])
|
||||
out.append(
|
||||
{
|
||||
"id": rk,
|
||||
"title": col.title,
|
||||
"source": _collection_source(col),
|
||||
"source": collection_source(col),
|
||||
"items": [movie_card(m, states.get(m.id), lang=lang) for m in members],
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _episode_nav(db: Session, it: PlexItem) -> tuple[str | None, str | None]:
|
||||
def _episode_nav(db: Session, it: "PlexItem") -> tuple[str | None, str | None]:
|
||||
if it.kind != "episode" or it.show_id is None:
|
||||
return None, None
|
||||
keys = list(
|
||||
db.execute(
|
||||
select(PlexItem.rating_key)
|
||||
.where(PlexItem.show_id == it.show_id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
keys = queries.episode_keys(db, it.show_id)
|
||||
try:
|
||||
i = keys.index(it.rating_key)
|
||||
except ValueError:
|
||||
@@ -398,13 +309,7 @@ def item_detail(
|
||||
prev/next, and the per-user resume position + watch status."""
|
||||
lang = _norm_lang(lang)
|
||||
it = item_or_404(db, rating_key)
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
|
||||
# Markers + audio/subtitle streams are file-specific, so they stay LIVE; the rich detail (cast/IMDb/
|
||||
# tagline/genres) is served from the DB row for movies (persisted below / by the backfill) and only
|
||||
@@ -485,11 +390,7 @@ def item_detail(
|
||||
# only stored the listing's top ~3), so fill a not-yet-enriched movie — its cast becomes filterable
|
||||
# and future views serve from the row. Episodes aren't enriched; a show's cast is filled from /show.
|
||||
if it.kind == "movie" and it.people_enriched_at is None and meta:
|
||||
cast = plex_people.cast_from_meta(meta)
|
||||
if cast:
|
||||
plex_people.enrich_row_people(db, it, cast)
|
||||
plex_detail.apply_rich_detail(it, meta)
|
||||
it.people_enriched_at = datetime.now(timezone.utc)
|
||||
service.enrich_movie(db, it, meta)
|
||||
try:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — a failed enrich must not fail the page
|
||||
@@ -522,8 +423,8 @@ def item_detail(
|
||||
rich = plex_detail.rich_detail(meta)
|
||||
|
||||
prev_id, next_id = _episode_nav(db, it)
|
||||
show = db.get(PlexShow, it.show_id) if it.kind == "episode" and it.show_id else None
|
||||
season = db.get(PlexSeason, it.season_id) if show and it.season_id else None
|
||||
show = queries.show_by_id(db, it.show_id) if it.kind == "episode" and it.show_id else None
|
||||
season = queries.season_by_id(db, it.season_id) if show and it.season_id else None
|
||||
return {
|
||||
"id": it.rating_key,
|
||||
"kind": it.kind,
|
||||
@@ -538,7 +439,7 @@ def item_detail(
|
||||
"video_bit_depth": it.video_bit_depth,
|
||||
"thumb": f"/api/plex/image/{it.rating_key}",
|
||||
"art": f"/api/plex/image/{it.rating_key}?variant=art",
|
||||
"library": _lib_key(db, it.library_id),
|
||||
"library": queries.library_key(db, it.library_id),
|
||||
"cast": rich.get("cast", []),
|
||||
"imdb_rating": rich.get("imdb_rating"),
|
||||
"imdb_id": rich.get("imdb_id"),
|
||||
|
||||
@@ -1,391 +1,18 @@
|
||||
"""Plex Discover: the rail-surface read-model.
|
||||
|
||||
A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
|
||||
Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
|
||||
separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
|
||||
recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
|
||||
every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
|
||||
their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
|
||||
card paths); the HU overlay coalesces only in the main grid.
|
||||
|
||||
The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
|
||||
Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
|
||||
grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
|
||||
the client hides). Genre-language matches the grid; duration is movie-only.
|
||||
"""
|
||||
"""Plex Discover: the rail-surface landing and its Surprise-me draw (`plex.discover`)."""
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import sorted_words
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import PlexItem, PlexShow, PlexState, User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import (
|
||||
SHOW_NEW,
|
||||
display_title,
|
||||
leaf_card,
|
||||
movie_card,
|
||||
show_card,
|
||||
)
|
||||
from app.plex.query import (
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
top_art_keys,
|
||||
)
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.plex.tmdb import resolve_recs
|
||||
from app.models import User
|
||||
from app.plex import discover as plex_discover
|
||||
from app.plex.discover import SURPRISE_PER_KIND
|
||||
from app.plex.query import LibraryFilters
|
||||
from app.routes.plex._shared import _norm_lang
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
GENRE_TILES = 12
|
||||
SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows (6 total)
|
||||
|
||||
|
||||
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
|
||||
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
return (
|
||||
movie_ids if scope in ("movie", "both") else [],
|
||||
show_ids if scope in ("show", "both") else [],
|
||||
)
|
||||
|
||||
|
||||
def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
|
||||
"""Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
|
||||
q = apply_meta_filters(q, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
q = q.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
q = q.where(PlexItem.duration_s <= f.duration_max)
|
||||
return q
|
||||
|
||||
|
||||
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
|
||||
sts = rollup.item_state_map(db, uid, [it.id for it in items])
|
||||
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
|
||||
|
||||
|
||||
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
|
||||
statuses = rollup.state_map(db, uid, [s.id for s in shows])
|
||||
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
|
||||
|
||||
|
||||
def _recently_added(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Newest movies + shows, interleaved by `added_at` desc into one rail."""
|
||||
movies = (
|
||||
db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
.order_by(PlexItem.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
|
||||
)
|
||||
.order_by(PlexShow.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
|
||||
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
|
||||
# review found this one still on the older `(is not None, value)` form, i.e. one rule in two
|
||||
# spellings, with the two comments explaining it differently.
|
||||
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
|
||||
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
|
||||
top = tagged[:RAIL_LIMIT]
|
||||
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
|
||||
# share one), so the merge doesn't depend on object identity staying alive.
|
||||
top_movies = [o for _, kind, o in top if kind == "movie"]
|
||||
top_shows = [o for _, kind, o in top if kind == "show"]
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
|
||||
}
|
||||
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
|
||||
|
||||
|
||||
def _recently_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
|
||||
card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
|
||||
sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
|
||||
rows: list[tuple[object, dict]] = [] # (watched_at, card)
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
)
|
||||
for it, st in db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all():
|
||||
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
|
||||
if show_ids:
|
||||
eq = (
|
||||
select(PlexItem, PlexState, PlexShow)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.join(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
for ep, st, sh in db.execute(eq).all():
|
||||
card = leaf_card(
|
||||
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
|
||||
)
|
||||
rows.append((st.watched_at, card))
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _continue_watching(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
|
||||
A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
|
||||
rows: list[tuple[object, dict]] = [] # (resumed_at, card)
|
||||
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.position_seconds > 0,
|
||||
# A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
|
||||
# `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
|
||||
or_(
|
||||
PlexState.status.is_(None),
|
||||
PlexState.status.notin_(["watched", "hidden"]),
|
||||
),
|
||||
)
|
||||
)
|
||||
mrows = db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.progress_updated_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for it, st in mrows:
|
||||
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
|
||||
|
||||
if show_ids:
|
||||
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
|
||||
# its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
|
||||
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
|
||||
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
|
||||
sq = apply_show_state_filter(
|
||||
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
|
||||
prog, prog.c.show_id == PlexShow.id
|
||||
),
|
||||
prog.c.total,
|
||||
prog.c.watched,
|
||||
prog.c.inprog,
|
||||
"in_progress",
|
||||
).where(PlexShow.library_id.in_(show_ids))
|
||||
srows = db.execute(
|
||||
apply_meta_filters(sq, PlexShow, p, lang)
|
||||
.order_by(prog.c.resumed_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for sh, resumed_at, total, watched, inprog in srows:
|
||||
state = rollup.show_state(total, watched, inprog)
|
||||
rows.append((resumed_at, show_card(sh, state, lang=lang)))
|
||||
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _genre_tiles(
|
||||
db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
|
||||
) -> list[dict]:
|
||||
"""The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
|
||||
The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
|
||||
The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
|
||||
counts: dict[str, int] = {}
|
||||
for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
|
||||
if not ids:
|
||||
continue
|
||||
gcol = genres_col(model, lang)
|
||||
base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
|
||||
model.library_id.in_(ids),
|
||||
func.jsonb_typeof(gcol) == "array",
|
||||
*meta_filter_conds(model, p, lang),
|
||||
)
|
||||
if model is PlexItem:
|
||||
base = base.where(model.kind == "movie")
|
||||
expanded = base.subquery()
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
counts[g] = counts.get(g, 0) + c
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
|
||||
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
|
||||
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
|
||||
art = top_art_keys(
|
||||
db,
|
||||
PlexItem,
|
||||
movie_ids,
|
||||
genres_col(PlexItem, lang),
|
||||
[g for g, _ in top],
|
||||
extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
|
||||
)
|
||||
return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
|
||||
|
||||
|
||||
def _because_you_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> dict | None:
|
||||
"""Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
|
||||
recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
|
||||
None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
|
||||
The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
|
||||
if not movie_ids:
|
||||
return None
|
||||
anchor = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
PlexItem.recommendations.isnot(None),
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(1)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if anchor is None:
|
||||
return None
|
||||
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
|
||||
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
|
||||
rks = [
|
||||
r["rating_key"]
|
||||
for r in resolved
|
||||
if r.get("in_library") and r.get("rating_key") != anchor.rating_key
|
||||
]
|
||||
if not rks:
|
||||
return None
|
||||
movies = {
|
||||
m.rating_key: m
|
||||
for m in db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
ordered_movies = [movies[rk] for rk in rks if rk in movies]
|
||||
ordered_shows = [shows[rk] for rk in rks if rk in shows]
|
||||
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
|
||||
}
|
||||
items: list[dict] = []
|
||||
for rk in rks:
|
||||
if rk in mcards:
|
||||
items.append(mcards[rk])
|
||||
elif rk in scards:
|
||||
items.append(scards[rk])
|
||||
if not items:
|
||||
return None
|
||||
return {
|
||||
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/discover")
|
||||
def discover(
|
||||
@@ -398,72 +25,7 @@ def discover(
|
||||
"""The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
|
||||
sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
|
||||
is null when there's no anchor."""
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
return {
|
||||
"scope": scope,
|
||||
"continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
|
||||
"because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
}
|
||||
|
||||
|
||||
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
|
||||
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
|
||||
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
|
||||
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
|
||||
fresh = (
|
||||
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
|
||||
.scalars()
|
||||
.all()
|
||||
if exclude
|
||||
else []
|
||||
)
|
||||
if len(fresh) == per:
|
||||
return list(fresh)
|
||||
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
|
||||
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
|
||||
return [*fresh, *rest.all()]
|
||||
|
||||
|
||||
def _kept(
|
||||
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
|
||||
) -> list[dict]:
|
||||
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
|
||||
stays in it, marked watched, and one gone from the scope's libraries drops out."""
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexItem.rating_key.in_(keep),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
cards = {
|
||||
c["id"]: c
|
||||
for c in [
|
||||
*_movie_cards(db, uid, list(movies), lang),
|
||||
*_show_cards(db, uid, list(shows), lang),
|
||||
]
|
||||
}
|
||||
return [cards[k] for k in keep if k in cards]
|
||||
return plex_discover.discover_page(db, user, scope, _norm_lang(lang), f)
|
||||
|
||||
|
||||
@router.get("/surprise")
|
||||
@@ -485,69 +47,4 @@ def surprise(
|
||||
excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
|
||||
SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
|
||||
one type."""
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
if keep:
|
||||
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
|
||||
per = min(limit, SURPRISE_PER_KIND)
|
||||
movies = []
|
||||
if movie_ids:
|
||||
# Exclude movies this user has watched.
|
||||
watched_movie = (
|
||||
select(PlexState.id)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
movies = _draw(
|
||||
db,
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
~watched_movie,
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
),
|
||||
PlexItem.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
shows = []
|
||||
if show_ids:
|
||||
# Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
|
||||
# as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
|
||||
prog = rollup.agg_subquery(db, user.id).subquery()
|
||||
not_fully_watched = or_(
|
||||
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
|
||||
)
|
||||
shows = _draw(
|
||||
db,
|
||||
apply_meta_filters(
|
||||
select(PlexShow)
|
||||
.outerjoin(prog, prog.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
|
||||
PlexShow,
|
||||
p,
|
||||
lang,
|
||||
),
|
||||
PlexShow.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
mcards = _movie_cards(db, user.id, movies, lang)
|
||||
scards = _show_cards(db, user.id, shows, lang)
|
||||
# Interleave movie/show cards, then trim to `limit`.
|
||||
items: list[dict] = []
|
||||
for i in range(max(len(mcards), len(scards))):
|
||||
if i < len(mcards):
|
||||
items.append(mcards[i])
|
||||
if i < len(scards):
|
||||
items.append(scards[i])
|
||||
return {"scope": scope, "items": items[:limit]}
|
||||
return plex_discover.surprise(db, user, scope, _norm_lang(lang), limit, exclude, keep, f)
|
||||
|
||||
@@ -4,26 +4,18 @@ import threading
|
||||
import time
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated
|
||||
from app import logs, sysconfig
|
||||
from app import sysconfig
|
||||
from app.auth import current_user
|
||||
from app.db import SessionLocal, get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexPerson,
|
||||
PlexShow,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.db import get_db
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.cards import SHOW_NEW, movie_card, norm_lang, show_card
|
||||
from app.plex.query import content_libs, split_csv, title_col
|
||||
from app.plex.query import content_libs, split_csv
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
# How many "Known for" titles the person page shows (the stored pool is deeper; see plex.tmdb).
|
||||
@@ -80,35 +72,13 @@ def people_cards(
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
photos = {
|
||||
p.name: p.photo_thumb
|
||||
for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(wanted))).scalars()
|
||||
}
|
||||
out = []
|
||||
for nm in wanted:
|
||||
count = 0
|
||||
if want_movies:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
PlexItem.cast_names.contains([nm]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
if want_shows:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexShow.id)).where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
PlexShow.cast_names.contains([nm]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
out.append({"name": nm, "photo": photos.get(nm), "count": count})
|
||||
photos = queries.person_photos(db, wanted)
|
||||
movies = movie_lib_ids if want_movies else []
|
||||
shows = show_lib_ids if want_shows else []
|
||||
out = [
|
||||
{"name": nm, "photo": photos.get(nm), "count": queries.cast_count(db, nm, movies, shows)}
|
||||
for nm in wanted
|
||||
]
|
||||
return {"people": out}
|
||||
|
||||
|
||||
@@ -144,24 +114,6 @@ def _should_schedule_enrich(person_id: int) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _enrich_person_bg(person_id: int) -> None:
|
||||
"""Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
|
||||
AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
|
||||
"no live TMDB call at serve time" invariant). The request's session is already closed by then, so
|
||||
open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
|
||||
next open, and the bio lands from the row by then."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
person = db.get(PlexPerson, person_id)
|
||||
if person is not None:
|
||||
plex_tmdb.enrich_person(db, person)
|
||||
except Exception: # noqa: BLE001 — background best-effort; never surfaces to a user
|
||||
db.rollback()
|
||||
log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@router.get("/person/{name}")
|
||||
def person_detail(
|
||||
name: str,
|
||||
@@ -183,7 +135,7 @@ def person_detail(
|
||||
fetches everyone ahead of time, so this lazy path is only for a person the job hasn't reached yet.
|
||||
"""
|
||||
lang = norm_lang(lang)
|
||||
person = db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
|
||||
person = queries.person(db, name)
|
||||
# Lazy one-time bio enrich, scheduled AFTER the response so the TMDB call is never on the serve path.
|
||||
# Gated by a per-person cooldown so concurrent/rapid opens (or repeated opens while TMDB is degraded)
|
||||
# can't pile background tasks onto the shared threadpool.
|
||||
@@ -197,53 +149,18 @@ def person_detail(
|
||||
and bool(sysconfig.get_str(db, "tmdb_api_key"))
|
||||
)
|
||||
if enriching and person is not None and _should_schedule_enrich(person.id):
|
||||
background_tasks.add_task(_enrich_person_bg, person.id)
|
||||
background_tasks.add_task(service.enrich_person_bg, person.id)
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
|
||||
movies: list[dict] = []
|
||||
if movie_lib_ids:
|
||||
items = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
or_(
|
||||
PlexItem.cast_names.contains([name]),
|
||||
PlexItem.directors.contains([name]),
|
||||
),
|
||||
)
|
||||
.order_by(
|
||||
PlexItem.year.desc().nullslast(), collated(title_col(PlexItem, lang), lang)
|
||||
)
|
||||
.limit(_PERSON_SECTION_MAX)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
items = queries.filmography(db, "movie", name, movie_lib_ids, lang, _PERSON_SECTION_MAX)
|
||||
states = rollup.item_state_map(db, user.id, [it.id for it in items])
|
||||
movies = [movie_card(it, states.get(it.id), lang=lang) for it in items]
|
||||
|
||||
shows: list[dict] = []
|
||||
if show_lib_ids:
|
||||
show_rows = (
|
||||
db.execute(
|
||||
select(PlexShow)
|
||||
.where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
or_(
|
||||
PlexShow.cast_names.contains([name]),
|
||||
PlexShow.directors.contains([name]),
|
||||
),
|
||||
)
|
||||
.order_by(
|
||||
PlexShow.year.desc().nullslast(), collated(title_col(PlexShow, lang), lang)
|
||||
)
|
||||
.limit(_PERSON_SECTION_MAX)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
show_rows = queries.filmography(db, "show", name, show_lib_ids, lang, _PERSON_SECTION_MAX)
|
||||
statuses = rollup.state_map(db, user.id, [s.id for s in show_rows])
|
||||
shows = [show_card(sh, statuses.get(sh.id, SHOW_NEW), lang=lang) for sh in show_rows]
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Plex playback: image/subtitle proxies, HLS streaming, and watch-state writes."""
|
||||
|
||||
import threading
|
||||
from typing import TYPE_CHECKING
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import httpx
|
||||
@@ -16,22 +17,13 @@ from fastapi import (
|
||||
)
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user, resolved_user_id
|
||||
from app.auth import current_user, session_user_id
|
||||
from app.db import SessionLocal, get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import imgcache, subtitles
|
||||
from app.models import User
|
||||
from app.plex import imgcache, queries, service, subtitles
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import stream as plex_stream
|
||||
from app.plex import watch_sync as plex_watch
|
||||
@@ -40,6 +32,9 @@ from app.routes.plex._shared import item_or_404
|
||||
from app.refusal import refused
|
||||
from app.routes._common import ProgressIn, not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
@@ -47,7 +42,7 @@ def _require_session(request: Request) -> int:
|
||||
"""Lightweight auth for the high-fan-out image proxies: validate the signed session cookie WITHOUT
|
||||
a DB user-load, so these endpoints never hold a pool connection for the auth. Poster images are
|
||||
low-sensitivity; a valid session is enough."""
|
||||
uid, _ = resolved_user_id(request)
|
||||
uid = session_user_id(request)
|
||||
if not uid:
|
||||
raise refused(401, "not_authenticated")
|
||||
return uid
|
||||
@@ -280,14 +275,7 @@ def _push_watch(
|
||||
return
|
||||
# Capture the row's freshest timestamp NOW so the background push can tell if a newer local edit
|
||||
# lands before it settles synced_to_plex (which would otherwise strand that newer value).
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
marker = plex_watch._state_marker(st)
|
||||
marker = plex_watch._state_marker(queries.state(db, user_id, item_id))
|
||||
if action == "resume" and intent is not None:
|
||||
background.add_task(
|
||||
_push_resume_guarded, user_id, item_id, rating_key, time_ms, duration_ms, marker, intent
|
||||
@@ -305,37 +293,6 @@ def _push_watch(
|
||||
)
|
||||
|
||||
|
||||
def _mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
|
||||
"""Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
|
||||
checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
|
||||
read-then-write let both see "not watched" and both scrobble. The transition is decided by the
|
||||
write itself: an insert that did not conflict, or an update conditional on the old status —
|
||||
under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
|
||||
values = {
|
||||
"status": "watched",
|
||||
"watched_at": now,
|
||||
"position_seconds": 0,
|
||||
"progress_updated_at": now,
|
||||
"synced_to_plex": False,
|
||||
}
|
||||
inserted = db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(user_id=user_id, item_id=item_id, **values)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
.returning(PlexState.id)
|
||||
).first()
|
||||
if inserted is not None:
|
||||
return True
|
||||
mine = (PlexState.user_id == user_id) & (PlexState.item_id == item_id)
|
||||
if db.execute(
|
||||
update(PlexState).where(mine, PlexState.status != "watched").values(values)
|
||||
).rowcount:
|
||||
return True
|
||||
# Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
|
||||
db.execute(update(PlexState).where(mine).values(values))
|
||||
return False
|
||||
|
||||
|
||||
class ItemProgressIn(ProgressIn):
|
||||
"""The shared checkpoint (its bounds included), plus whether the player is leaving the item."""
|
||||
|
||||
@@ -371,7 +328,7 @@ def item_progress(
|
||||
now = datetime.now(timezone.utc)
|
||||
near_end = duration > 0 and position > duration - _FINISH_MARGIN_S
|
||||
if near_end:
|
||||
flipped = _mark_finished(db, user.id, it.id, now)
|
||||
flipped = service.mark_finished(db, user.id, it.id, now)
|
||||
# The finish supersedes every resume push still in flight for this item: stamping it as the
|
||||
# latest push makes `_resume_push_superseded` drop them. Popping the entry instead (to keep
|
||||
# the map small) removed the very guard, and an older resume push could then land at Plex
|
||||
@@ -385,33 +342,17 @@ def item_progress(
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "watched")
|
||||
return {"ok": True, "status": "watched", "position_seconds": 0}
|
||||
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
|
||||
if position < _PROGRESS_MIN_S:
|
||||
if st is not None and st.status == "new":
|
||||
db.delete(st)
|
||||
elif st is not None:
|
||||
st.position_seconds = 0
|
||||
st.progress_updated_at = now
|
||||
service.clear_progress(db, st, now)
|
||||
db.commit()
|
||||
return {"ok": True, "position_seconds": 0}
|
||||
|
||||
if st is None:
|
||||
st = PlexState(user_id=user.id, item_id=it.id)
|
||||
db.add(st)
|
||||
st.position_seconds = position
|
||||
st.progress_updated_at = now
|
||||
push_now = _should_push_resume(user.id, it.id, now, final)
|
||||
if push_now:
|
||||
# Mark dirty before the (background) push so the reconcile knows to re-push if it fails, and so
|
||||
# `push_state_to_plex` can settle synced_to_plex once Plex confirms.
|
||||
st.synced_to_plex = False
|
||||
# Marked dirty before the (background) push so the reconcile knows to re-push if it fails, and so
|
||||
# `push_state_to_plex` can settle synced_to_plex once Plex confirms.
|
||||
service.checkpoint(db, user, it.id, st, position, now, dirty=push_now)
|
||||
db.commit()
|
||||
if push_now:
|
||||
_push_watch(
|
||||
@@ -447,13 +388,7 @@ def item_state(
|
||||
status = body.status
|
||||
if status not in ("new", "watched", "hidden"):
|
||||
raise refused(400, "invalid_status")
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
prev_status = st.status if st is not None else "new"
|
||||
if status == "new":
|
||||
# Does Plex hold anything for this item that the un-mark has to clear? Two things qualify: a
|
||||
@@ -467,19 +402,12 @@ def item_state(
|
||||
st.status == "watched" or (st.position_seconds or 0) > 0
|
||||
)
|
||||
if st is not None:
|
||||
db.delete(st)
|
||||
service.forget_state(db, st)
|
||||
db.commit()
|
||||
if mirrored_on_plex:
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "unwatched")
|
||||
return {"ok": True, "status": "new"}
|
||||
if st is None:
|
||||
st = PlexState(user_id=user.id, item_id=it.id)
|
||||
db.add(st)
|
||||
st.status = status
|
||||
if status == "watched":
|
||||
st.watched_at = datetime.now(timezone.utc)
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
service.set_status(db, user, it.id, st, status)
|
||||
db.commit()
|
||||
if status == "watched" and prev_status != "watched":
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "watched")
|
||||
@@ -490,44 +418,15 @@ def _bulk_state(
|
||||
background: BackgroundTasks,
|
||||
db: Session,
|
||||
user_id: int,
|
||||
eps: list[PlexItem],
|
||||
eps: "list[PlexItem]",
|
||||
watched: bool,
|
||||
) -> int:
|
||||
"""Mark every episode in `eps` watched (or unwatched) for a user, mirroring each change to a
|
||||
linked Plex account in the background (best-effort). Returns how many rows actually changed."""
|
||||
now = datetime.now(timezone.utc)
|
||||
existing = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user_id,
|
||||
PlexState.item_id.in_([e.id for e in eps] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
linked Plex account in the background (best-effort). Returns how many rows actually changed.
|
||||
The handler commits before the response — which is when the background push runs, so the push
|
||||
always reads the committed marks."""
|
||||
pushable = plex_watch.link_for_push(db, user_id) is not None
|
||||
to_push: list[tuple[int, str, str]] = []
|
||||
for e in eps:
|
||||
st = existing.get(e.id)
|
||||
prev = st.status if st is not None else "new"
|
||||
if watched:
|
||||
if st is None:
|
||||
st = PlexState(user_id=user_id, item_id=e.id)
|
||||
db.add(st)
|
||||
st.status = "watched"
|
||||
st.watched_at = now
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
if prev != "watched":
|
||||
to_push.append((e.id, e.rating_key, "watched"))
|
||||
else: # → new: drop any state (but keep the user's private "hidden" marks)
|
||||
if st is not None and prev != "hidden":
|
||||
db.delete(st)
|
||||
if prev != "new":
|
||||
to_push.append((e.id, e.rating_key, "unwatched"))
|
||||
# Flush, not commit: the handler commits, and it does so before the response — which is when
|
||||
# the background push below runs, so the push always reads the committed marks.
|
||||
db.flush()
|
||||
to_push = service.bulk_state(db, user_id, eps, watched)
|
||||
if pushable and to_push:
|
||||
# One coalesced background task for the whole batch (reuses a single session + Plex client),
|
||||
# not one task per episode — a 200-ep show mark is 1 task, not 200.
|
||||
@@ -548,15 +447,11 @@ def show_state(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Mark a whole show watched/unwatched (all its episodes) for the current user + push to Plex."""
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
watched = body.watched
|
||||
eps = (
|
||||
db.execute(select(PlexItem).where(PlexItem.show_id == sh.id, PlexItem.kind == "episode"))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = queries.episodes(db, sh)
|
||||
changed = _bulk_state(background, db, user.id, eps, watched)
|
||||
db.commit()
|
||||
return {"ok": True, "changed": changed, "watched": watched}
|
||||
@@ -571,15 +466,11 @@ def season_state(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Mark a whole season watched/unwatched (all its episodes) for the current user + push to Plex."""
|
||||
se = by_rating_key(db, PlexSeason, rating_key)
|
||||
se = queries.season(db, rating_key)
|
||||
if se is None:
|
||||
raise not_found("season")
|
||||
watched = body.watched
|
||||
eps = (
|
||||
db.execute(select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode"))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = queries.season_episodes(db, se)
|
||||
changed = _bulk_state(background, db, user.id, eps, watched)
|
||||
db.commit()
|
||||
return {"ok": True, "changed": changed, "watched": watched}
|
||||
|
||||
@@ -1,33 +1,26 @@
|
||||
"""Plex playlists: the current user's personal, orderable playlists of playable items."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING, Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex.cards import display_title, leaf_card, norm_lang
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes._common import owned_or_404, not_found
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexPlaylist
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
|
||||
def _playlist_card(pl: "PlexPlaylist", count: int, thumb_rk: str | None) -> dict:
|
||||
return {
|
||||
"id": pl.id,
|
||||
"title": pl.title,
|
||||
@@ -36,13 +29,15 @@ def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _playlist_item_query(pid: int):
|
||||
return (
|
||||
select(PlexPlaylistItem, PlexItem)
|
||||
.join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
|
||||
.where(PlexPlaylistItem.playlist_id == pid)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
)
|
||||
def _card_of(db: Session, pl: "PlexPlaylist") -> dict:
|
||||
return _playlist_card(pl, queries.playlist_count(db, pl.id), queries.playlist_cover(db, pl.id))
|
||||
|
||||
|
||||
def _own_playlist(db: Session, user: User, pid: int, *, lock: bool = False) -> "PlexPlaylist":
|
||||
pl = queries.own_playlist(db, user, pid, lock=lock)
|
||||
if pl is None:
|
||||
raise not_found("playlist")
|
||||
return pl
|
||||
|
||||
|
||||
@router.get("/playlists")
|
||||
@@ -56,49 +51,17 @@ def playlists(
|
||||
`contains`=an item rating_key adds `has_item` per playlist (for the single 'add to playlist' dialog).
|
||||
`contains_group`=a comma-separated list of rating_keys adds `group_in` per playlist (how many of the
|
||||
group it already holds) + a top-level `group_size`, for the 'add a whole season/show' dialog."""
|
||||
contains_id = None
|
||||
if contains:
|
||||
contains_id = db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(contains)))
|
||||
contains_id = queries.item_id(db, contains) if contains else None
|
||||
group_ids: set[int] | None = None
|
||||
if contains_group is not None:
|
||||
rks = [s for s in (contains_group or "").split(",") if s]
|
||||
group_ids = set(
|
||||
db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rks or [""]))).scalars()
|
||||
)
|
||||
group_ids = queries.item_ids(db, [s for s in (contains_group or "").split(",") if s])
|
||||
out = []
|
||||
for pl in db.execute(
|
||||
select(PlexPlaylist)
|
||||
.where(PlexPlaylist.user_id == user.id)
|
||||
.order_by(PlexPlaylist.updated_at.desc())
|
||||
).scalars():
|
||||
count = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
first = db.execute(_playlist_item_query(pl.id)).first()
|
||||
card = _playlist_card(pl, count, first[1].rating_key if first else None)
|
||||
for pl in queries.playlists_of(db, user):
|
||||
card = _card_of(db, pl)
|
||||
if contains_id is not None:
|
||||
card["has_item"] = (
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.id).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id,
|
||||
PlexPlaylistItem.item_id == contains_id,
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
card["has_item"] = queries.playlist_holds(db, pl.id, {contains_id}) > 0
|
||||
if group_ids is not None:
|
||||
card["group_in"] = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id,
|
||||
PlexPlaylistItem.item_id.in_(group_ids or [0]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
card["group_in"] = queries.playlist_holds(db, pl.id, group_ids)
|
||||
out.append(card)
|
||||
resp: dict = {"playlists": out}
|
||||
if group_ids is not None:
|
||||
@@ -120,19 +83,10 @@ def create_playlist(
|
||||
title = body.title.strip()
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
pl = PlexPlaylist(user_id=user.id, title=title)
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
seed = body.item_rating_key
|
||||
count = 0
|
||||
thumb_rk = None
|
||||
if seed:
|
||||
it = by_rating_key(db, PlexItem, seed)
|
||||
if it is not None:
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0))
|
||||
count, thumb_rk = 1, it.rating_key
|
||||
seed = queries.item(db, body.item_rating_key) if body.item_rating_key else None
|
||||
pl = service.create_playlist(db, user, title, seed)
|
||||
db.commit()
|
||||
return _playlist_card(pl, count, thumb_rk)
|
||||
return _playlist_card(pl, 1 if seed else 0, seed.rating_key if seed else None)
|
||||
|
||||
|
||||
@router.get("/playlists/{pid}")
|
||||
@@ -143,26 +97,10 @@ def playlist_detail(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
lang = norm_lang(lang)
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
rows = db.execute(_playlist_item_query(pid)).all()
|
||||
items = [it for (_pi, it) in rows]
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.item_id.in_([it.id for it in items] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
shows = {
|
||||
sh.id: sh
|
||||
for sh in db.execute(
|
||||
select(PlexShow).where(
|
||||
PlexShow.id.in_([it.show_id for it in items if it.show_id] or [0])
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
pl = _own_playlist(db, user, pid)
|
||||
items = queries.playlist_items(db, pl.id)
|
||||
states = rollup.item_state_map(db, user, [it.id for it in items])
|
||||
shows = queries.shows_by_ids(db, {it.show_id for it in items if it.show_id})
|
||||
cards = [
|
||||
leaf_card(
|
||||
it,
|
||||
@@ -187,28 +125,21 @@ def rename_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
pl = _own_playlist(db, user, pid)
|
||||
title = body.title.strip()
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
pl.title = title
|
||||
service.rename_playlist(pl, title)
|
||||
db.commit()
|
||||
count = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
first = db.execute(_playlist_item_query(pl.id)).first()
|
||||
return _playlist_card(pl, count, first[1].rating_key if first else None)
|
||||
return _playlist_card(pl, queries.playlist_count(db, pl.id), queries.playlist_cover(db, pl.id))
|
||||
|
||||
|
||||
@router.delete("/playlists/{pid}")
|
||||
def delete_playlist(
|
||||
pid: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
db.delete(pl)
|
||||
pl = _own_playlist(db, user, pid)
|
||||
service.delete_playlist(db, pl)
|
||||
db.commit()
|
||||
return {"deleted": pid}
|
||||
|
||||
@@ -225,27 +156,11 @@ def playlist_add_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked (like every add / reorder here): two adds otherwise read the same max position.
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
it = by_rating_key(db, PlexItem, body.item_rating_key)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
it = queries.item(db, body.item_rating_key)
|
||||
if it is None:
|
||||
raise not_found("item")
|
||||
exists = (
|
||||
db.execute(
|
||||
select(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if exists is None:
|
||||
maxpos = db.scalar(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == pid
|
||||
)
|
||||
)
|
||||
db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=int(maxpos) + 1))
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
service.add_item(db, pl, it)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -259,15 +174,10 @@ def playlist_remove_item(
|
||||
) -> dict:
|
||||
# Locked like the adds and the reorder: the list row first, then its items — a remove that
|
||||
# locked an item row before a reorder locked the list deadlocked against it.
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
it = by_rating_key(db, PlexItem, item_rating_key)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
it = queries.item(db, item_rating_key)
|
||||
if it is not None:
|
||||
db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
service.remove_item(db, pl, it)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -276,25 +186,6 @@ class PlexItemKeysIn(BaseModel):
|
||||
item_rating_keys: list[str] = []
|
||||
|
||||
|
||||
def _items_by_rks(db: Session, rks: list) -> list[PlexItem]:
|
||||
"""Resolve rating_keys to PlexItems, preserving the input order (drops unknown keys)."""
|
||||
order = [str(k) for k in rks if str(k)]
|
||||
if not order:
|
||||
return []
|
||||
by_rk = {
|
||||
it.rating_key: it
|
||||
for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
|
||||
}
|
||||
seen: set[str] = set()
|
||||
out: list[PlexItem] = []
|
||||
for rk in order:
|
||||
it = by_rk.get(rk)
|
||||
if it is not None and rk not in seen:
|
||||
seen.add(rk)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
|
||||
class PlexItemAddBulkIn(PlexItemKeysIn):
|
||||
# Where each key goes back (the list page's "Undo" of a removal): an index over the
|
||||
# ORDERED rows, one per key. The two-call shape (append, then send the whole order) leaves the rows
|
||||
@@ -309,32 +200,6 @@ class PlexItemAddBulkIn(PlexItemKeysIn):
|
||||
return self
|
||||
|
||||
|
||||
def _insert_at(db: Session, pid: int, placed: list[tuple[int, PlexItem]]) -> int:
|
||||
"""Insert new rows at their indices (ascending, so each index is where the row stood when the
|
||||
rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
|
||||
already in the list does not move: the add stays idempotent."""
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem)
|
||||
.where(PlexPlaylistItem.playlist_id == pid)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
present = {r.item_id for r in rows}
|
||||
added = 0
|
||||
for at, it in sorted(placed, key=lambda p: p[0]):
|
||||
if it.id in present:
|
||||
continue
|
||||
row = PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=0)
|
||||
db.add(row)
|
||||
rows.insert(min(at, len(rows)), row)
|
||||
present.add(it.id)
|
||||
added += 1
|
||||
for i, r in enumerate(rows):
|
||||
r.position = i
|
||||
return added
|
||||
|
||||
|
||||
@router.post("/playlists/{pid}/items/bulk")
|
||||
def playlist_add_bulk(
|
||||
pid: int,
|
||||
@@ -345,56 +210,20 @@ def playlist_add_bulk(
|
||||
"""Append many items at once (a whole season/show). Already-present items are skipped; the input
|
||||
order is preserved for the newly-added ones. With `positions`, each key is put back at its index
|
||||
instead of appended. Returns how many were added + the new total."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
items = queries.items_by_keys(db, body.item_rating_keys)
|
||||
if body.positions is not None:
|
||||
by_rk = {it.rating_key: it for it in _items_by_rks(db, body.item_rating_keys)}
|
||||
by_rk = {it.rating_key: it for it in items}
|
||||
placed = [
|
||||
(at, by_rk[str(rk)])
|
||||
for rk, at in zip(body.item_rating_keys, body.positions, strict=True)
|
||||
if str(rk) in by_rk
|
||||
]
|
||||
added = _insert_at(db, pid, placed)
|
||||
if added:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "added": added, "item_count": total}
|
||||
items = _items_by_rks(db, body.item_rating_keys)
|
||||
existing = set(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pid)
|
||||
).scalars()
|
||||
)
|
||||
pos = int(
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == pid
|
||||
)
|
||||
)
|
||||
)
|
||||
added = 0
|
||||
for it in items:
|
||||
if it.id in existing:
|
||||
continue
|
||||
pos += 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=pos))
|
||||
existing.add(it.id)
|
||||
added += 1
|
||||
if added:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
added = service.insert_items_at(db, pl, placed)
|
||||
else:
|
||||
added = service.add_items(db, pl, items)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "added": added, "item_count": total}
|
||||
return {"ok": True, "added": added, "item_count": queries.playlist_count(db, pid)}
|
||||
|
||||
|
||||
@router.post("/playlists/{pid}/items/remove-bulk")
|
||||
@@ -406,26 +235,11 @@ def playlist_remove_bulk(
|
||||
) -> dict:
|
||||
"""Remove many items at once (a whole season/show, or a multi-select). Returns how many were
|
||||
removed + the new total."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
ids = [it.id for it in _items_by_rks(db, body.item_rating_keys)]
|
||||
removed = 0
|
||||
if ids:
|
||||
removed = db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id.in_(ids)
|
||||
),
|
||||
execution_options={"synchronize_session": False},
|
||||
).rowcount
|
||||
if removed:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
ids = [it.id for it in queries.items_by_keys(db, body.item_rating_keys)]
|
||||
removed = service.remove_items(db, pl, ids)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "removed": removed, "item_count": total}
|
||||
return {"ok": True, "removed": removed, "item_count": queries.playlist_count(db, pid)}
|
||||
|
||||
|
||||
@router.put("/playlists/{pid}/order")
|
||||
@@ -436,35 +250,7 @@ def reorder_playlist(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Set the playlist order from an explicit list of item rating_keys (0-based positions)."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
order = body.item_rating_keys
|
||||
id_by_rk = {
|
||||
it.rating_key: it.id
|
||||
for it in db.execute(
|
||||
select(PlexItem).where(PlexItem.rating_key.in_(order or [""]))
|
||||
).scalars()
|
||||
}
|
||||
rows = {
|
||||
r.item_id: r
|
||||
for r in db.execute(
|
||||
select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pid)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[int] = set()
|
||||
for rk in order:
|
||||
row = rows.get(id_by_rk.get(rk, -1))
|
||||
if row is not None and row.item_id not in seen:
|
||||
row.position = pos
|
||||
pos += 1
|
||||
seen.add(row.item_id)
|
||||
# Rows the payload omitted (added in another tab meanwhile) keep their relative order after
|
||||
# the given ones, so no two rows share a position (the YouTube reorder does the same).
|
||||
for row in sorted(
|
||||
(r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
|
||||
):
|
||||
row.position = pos
|
||||
pos += 1
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
service.reorder_playlist(db, pl, body.item_rating_keys)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -10,6 +10,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import quota
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
@@ -27,7 +28,7 @@ def _sum(db: Session, *conds) -> int:
|
||||
@router.get("/my-usage")
|
||||
def my_usage(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
now = datetime.now(timezone.utc)
|
||||
mine = QuotaEvent.user_id == user.id
|
||||
mine = owned_by(QuotaEvent.user_id, user)
|
||||
by_action = {
|
||||
action: int(units)
|
||||
for action, units in db.execute(
|
||||
|
||||
@@ -18,7 +18,8 @@ from sqlalchemy.orm import Session
|
||||
from app import scheduler
|
||||
from app.auth import admin_user
|
||||
from app.db import get_db
|
||||
from app.models import Channel, JobRun, LogEvent, User
|
||||
from app.models import JobRun, LogEvent, User
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.routes import _account as account
|
||||
from app.routes._keyset import PAGE_MAX, accounts, older_than, page, utc
|
||||
from app.runstatus import RunStatus
|
||||
@@ -116,10 +117,7 @@ def list_runs(
|
||||
lambda r: r.started_at,
|
||||
)
|
||||
channels = {r.target for r in runs if r.job in scheduler.AD_HOC_JOBS and r.target}
|
||||
titles: dict[str, str] = {}
|
||||
if channels:
|
||||
rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(channels)))
|
||||
titles = {cid: title for cid, title in rows.tuples() if title is not None}
|
||||
titles = catalog_queries.channel_titles(db, channels)
|
||||
lines: dict[int, int] = {}
|
||||
if runs:
|
||||
counted = db.execute(
|
||||
|
||||
@@ -1,15 +1,20 @@
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel, field_validator
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import SavedView, User
|
||||
from app.routes._common import owned_or_404
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.routes._common import not_found
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.reader.models import SavedView
|
||||
|
||||
# Saved "smart views": per-user named snapshots of a module's filters (feed or plex, keyed by
|
||||
# `source`).
|
||||
router = APIRouter(prefix="/api/saved-views", tags=["saved-views"])
|
||||
@@ -23,7 +28,7 @@ def _norm_source(source: str) -> str:
|
||||
return source if source in _SOURCES else "feed"
|
||||
|
||||
|
||||
def _serialize(v: SavedView) -> dict:
|
||||
def _serialize(v: "SavedView") -> dict:
|
||||
return {
|
||||
"id": v.id,
|
||||
"name": v.name,
|
||||
@@ -33,23 +38,20 @@ def _serialize(v: SavedView) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _own_view(db: Session, user: User, view_id: int) -> "SavedView":
|
||||
view = queries.own_view(db, user, view_id)
|
||||
if view is None:
|
||||
raise not_found("view")
|
||||
return view
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_views(
|
||||
source: str = "feed",
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> list[dict]:
|
||||
source = _norm_source(source)
|
||||
rows = (
|
||||
db.execute(
|
||||
select(SavedView)
|
||||
.where(SavedView.user_id == user.id, SavedView.source == source)
|
||||
.order_by(SavedView.position, SavedView.id)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return [_serialize(v) for v in rows]
|
||||
return [_serialize(v) for v in queries.views(db, user, _norm_source(source))]
|
||||
|
||||
|
||||
# A feed view's tag pick is read by more than the feed: the autotag job keeps every tag a view names
|
||||
@@ -86,22 +88,8 @@ def create_view(
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
source = _norm_source(body.source)
|
||||
# Position is per (user, source), so each module's views number 0..N independently.
|
||||
maxpos = db.execute(
|
||||
select(func.coalesce(func.max(SavedView.position), -1)).where(
|
||||
SavedView.user_id == user.id, SavedView.source == source
|
||||
)
|
||||
).scalar_one()
|
||||
view = SavedView(
|
||||
user_id=user.id,
|
||||
source=source,
|
||||
name=name[:80],
|
||||
filters=body.filters,
|
||||
position=maxpos + 1,
|
||||
)
|
||||
db.add(view)
|
||||
try:
|
||||
view = service.create_view(db, user, _norm_source(body.source), name, body.filters)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -126,32 +114,17 @@ def update_view(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
view = owned_or_404(db, SavedView, view_id, user, what="view")
|
||||
view = _own_view(db, user, view_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
view.name = name[:80]
|
||||
if "filters" in sent:
|
||||
if body.filters is None:
|
||||
raise refused(400, "filters_not_object")
|
||||
view.filters = body.filters
|
||||
if "is_default" in sent:
|
||||
make_default = bool(body.is_default)
|
||||
if make_default:
|
||||
# At most one default per (user, source): clear the flag on this module's others first.
|
||||
db.execute(
|
||||
update(SavedView)
|
||||
.where(
|
||||
SavedView.user_id == user.id,
|
||||
SavedView.source == view.source,
|
||||
SavedView.id != view.id,
|
||||
)
|
||||
.values(is_default=False)
|
||||
)
|
||||
view.is_default = make_default
|
||||
name = (body.name or "").strip()
|
||||
if "name" in sent and not name:
|
||||
raise refused(400, "name_required")
|
||||
if "filters" in sent and body.filters is None:
|
||||
raise refused(400, "filters_not_object")
|
||||
try:
|
||||
service.update_view(
|
||||
db, view, sent=sent, name=name, filters=body.filters, is_default=body.is_default
|
||||
)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -165,8 +138,7 @@ def delete_view(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
view = owned_or_404(db, SavedView, view_id, user, what="view")
|
||||
db.delete(view)
|
||||
service.delete_view(db, _own_view(db, user, view_id))
|
||||
db.commit()
|
||||
return {"deleted": view_id}
|
||||
|
||||
@@ -181,14 +153,6 @@ def reorder_views(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
ids = body.ids
|
||||
rows = db.execute(select(SavedView).where(SavedView.user_id == user.id)).scalars().all()
|
||||
by_id = {v.id: v for v in rows}
|
||||
pos = 0
|
||||
for vid in ids:
|
||||
v = by_id.get(vid)
|
||||
if v is not None:
|
||||
v.position = pos
|
||||
pos += 1
|
||||
service.reorder_views(db, user, body.ids)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -4,7 +4,6 @@ and the shared quota picture."""
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, quota, state, sysconfig
|
||||
@@ -12,7 +11,8 @@ from app.audit import AuditAction
|
||||
from app.auth import admin_user
|
||||
from app.config import settings
|
||||
from app.db import get_db
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.scheduler import (
|
||||
JOB_INTERVALS,
|
||||
MAX_INTERVAL,
|
||||
@@ -23,7 +23,6 @@ from app.scheduler import (
|
||||
trigger_all,
|
||||
trigger_job,
|
||||
)
|
||||
from app.sync.explore import followed_channel
|
||||
from app.sync.runner import estimate_deep_backfill
|
||||
from app.routes._common import not_found
|
||||
from app.refusal import refused
|
||||
@@ -122,58 +121,14 @@ def get_scheduler(_: User = Depends(admin_user), db: Session = Depends(get_db))
|
||||
snap = scheduler_snapshot(db)
|
||||
|
||||
# Work still queued (shared catalog, so these are instance-wide, not per-user).
|
||||
deep_requested = (
|
||||
select(Subscription.id)
|
||||
.where(
|
||||
Subscription.channel_id == Channel.id,
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
deep_channels = (
|
||||
db.execute(select(Channel).where(Channel.backfill_done.is_(False), deep_requested))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
deep_eta = estimate_deep_backfill(db, deep_channels)
|
||||
|
||||
queue = {
|
||||
"channels_recent_pending": db.scalar(
|
||||
# The same selection run_recent_backfill works through, or the card counts channels
|
||||
# the job will never touch and never reaches zero.
|
||||
select(func.count())
|
||||
.select_from(Channel)
|
||||
.where(Channel.recent_synced_at.is_(None), followed_channel())
|
||||
)
|
||||
or 0,
|
||||
# The whole opted-in BACKLOG, deliberately not `run_deep_backfill`'s next-pass selection
|
||||
# (which also wants `recent_synced_at`): the same figure /api/sync/my-status shows the user,
|
||||
# and a channel missing its recent pull is one the same job tick fetches first, not one the
|
||||
# job will never touch. Narrowing it here would under-report work that is genuinely queued.
|
||||
"channels_deep_pending": deep_eta["channels_pending"],
|
||||
"deep_eta_seconds": deep_eta["eta_seconds"],
|
||||
"videos_pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_shorts": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_live_refresh": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(
|
||||
or_(
|
||||
Video.live_status.in_(("live", "upcoming")),
|
||||
and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
|
||||
)
|
||||
)
|
||||
)
|
||||
or 0,
|
||||
}
|
||||
deep_eta = estimate_deep_backfill(db, catalog_queries.deep_backlog(db))
|
||||
queue = catalog_queries.work_queue(db)
|
||||
# The whole opted-in BACKLOG, deliberately not `run_deep_backfill`'s next-pass selection
|
||||
# (which also wants `recent_synced_at`): the same figure /api/sync/my-status shows the user,
|
||||
# and a channel missing its recent pull is one the same job tick fetches first, not one the
|
||||
# job will never touch. Narrowing it here would under-report work that is genuinely queued.
|
||||
queue["channels_deep_pending"] = deep_eta["channels_pending"]
|
||||
queue["deep_eta_seconds"] = deep_eta["eta_seconds"]
|
||||
|
||||
used = quota.units_used_today(db)
|
||||
quota_info = {
|
||||
|
||||
+14
-245
@@ -11,39 +11,24 @@ regardless of source.
|
||||
Ingested results are flagged (videos.via_search / channels.from_search) so the Library can hide
|
||||
this search-discovered "noise" by default. Shorts and currently-live/upcoming videos are never
|
||||
ingested this way (Shorts are confirmed by the youtube.com/shorts probe; live/upcoming are
|
||||
dropped from the search response up front)."""
|
||||
dropped from the search response up front).
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
This is the HTTP side — the paging, the caps and the refusals; what a page of results leaves in the
|
||||
catalog is `catalog/search`."""
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, delete, func, or_, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, quota, sysconfig
|
||||
from app.auth import current_user
|
||||
from app.catalog import queries
|
||||
from app.catalog import search as catalog_search
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
Playlist,
|
||||
PlaylistItem,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.utils import like_escape
|
||||
from app.sync.subscriptions import apply_channel_details
|
||||
from app.sync.videos import insert_stubs, apply_video_details, parse_dt
|
||||
from app.models import User
|
||||
from app.reader import queries as reader_queries
|
||||
from app.youtube import search_scrape
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
from app.refusal import refused
|
||||
from app.routes._common import youtube_refusal
|
||||
|
||||
@@ -54,198 +39,6 @@ router = APIRouter(prefix="/api/search", tags=["search"])
|
||||
SEARCH_COST = 100
|
||||
|
||||
|
||||
def _classify_shorts(db: Session, videos: list[Video]) -> None:
|
||||
"""Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
|
||||
|
||||
Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
|
||||
synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
|
||||
duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
|
||||
quota). Undetermined probes are left for the scheduler to retry."""
|
||||
probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
|
||||
to_probe: list[Video] = []
|
||||
for v in videos:
|
||||
if v.shorts_probed:
|
||||
continue
|
||||
if v.enriched_at is None:
|
||||
# Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
|
||||
# so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
|
||||
# (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
|
||||
continue
|
||||
if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
|
||||
v.is_short = False
|
||||
v.shorts_probed = True
|
||||
else:
|
||||
to_probe.append(v)
|
||||
if to_probe:
|
||||
with make_client() as client:
|
||||
|
||||
def work(v: Video):
|
||||
return v, probe_is_short(client, v.id)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
for v, result in pool.map(work, to_probe):
|
||||
if result is None:
|
||||
continue
|
||||
v.is_short = result
|
||||
v.shorts_probed = True
|
||||
db.commit()
|
||||
|
||||
|
||||
def _serialize_ids(db: Session, user: User, ids: list[str]) -> list[dict]:
|
||||
"""Serialize the given video ids with the shared feed projection (joined channel + this
|
||||
user's watch state + saved), preserving the input (search relevance) order."""
|
||||
if not ids:
|
||||
return []
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
.where(Video.id.in_(ids))
|
||||
).all()
|
||||
by_id = {r.id: serialize(r) for r in rows}
|
||||
return [by_id[i] for i in ids if i in by_id]
|
||||
|
||||
|
||||
def _ingest_candidates(
|
||||
db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
|
||||
) -> list[str]:
|
||||
"""Materialise one page of search candidates into the catalog (channel stubs + enrich,
|
||||
video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
|
||||
video ids — non-Short, non-live — in the input relevance order."""
|
||||
ids = [it["id"] for it in candidates]
|
||||
if not ids:
|
||||
return []
|
||||
|
||||
# 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
|
||||
# then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
|
||||
channel_ids = list({it["channel_id"] for it in candidates})
|
||||
existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
|
||||
new_channel_ids = [c for c in channel_ids if c not in existing]
|
||||
if new_channel_ids:
|
||||
titles = {it["channel_id"]: it["channel_title"] for it in candidates}
|
||||
for cid in new_channel_ids:
|
||||
db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
|
||||
db.commit()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
|
||||
details = yt.get_channels(new_channel_ids)
|
||||
apply_channel_details(db, details, new_channel_ids)
|
||||
db.commit()
|
||||
except YouTubeError:
|
||||
# The stubs remain and the details can be backfilled later — but until they are, those
|
||||
# channels sit in the results with no thumbnail and no subscriber count, looking like
|
||||
# channels that have neither.
|
||||
log.warning(
|
||||
"the channels found by a search could not be enriched",
|
||||
extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
|
||||
)
|
||||
db.rollback() # details can be backfilled later; the stubs remain
|
||||
|
||||
# 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
|
||||
# NOTHING), so an organically-synced video stays organic.
|
||||
# published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
|
||||
# (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
|
||||
# enrich in step 3 overwrites it with the video's real snippet date.
|
||||
now = datetime.now(timezone.utc)
|
||||
insert_stubs(
|
||||
db,
|
||||
[
|
||||
{
|
||||
"id": it["id"],
|
||||
"channel_id": it["channel_id"],
|
||||
"title": it["title"],
|
||||
"published_at": parse_dt(it["published_at"]) or now,
|
||||
"thumbnail_url": it["thumbnail_url"],
|
||||
"via_search": True,
|
||||
}
|
||||
for it in candidates
|
||||
],
|
||||
)
|
||||
|
||||
# 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
|
||||
videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
|
||||
detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
|
||||
except YouTubeError:
|
||||
# Every candidate then stays unenriched, and an unenriched video cannot be judged live or
|
||||
# Short — so it is dropped. A working search returns nothing and reads as "no results".
|
||||
log.warning(
|
||||
"the videos found by a search could not be enriched",
|
||||
extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
|
||||
)
|
||||
detail_by_id = {}
|
||||
enriched_at = datetime.now(
|
||||
timezone.utc
|
||||
) # when THIS enrich ran (distinct from the stub placeholder)
|
||||
for v in videos:
|
||||
item = detail_by_id.get(v.id)
|
||||
if item is None:
|
||||
continue
|
||||
apply_video_details(v, item)
|
||||
v.enriched_at = enriched_at
|
||||
db.commit()
|
||||
|
||||
# 4) Confirm/deny Shorts (no quota) so none enter via search.
|
||||
_classify_shorts(db, videos)
|
||||
|
||||
keep = {
|
||||
v.id
|
||||
for v in videos
|
||||
if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
|
||||
}
|
||||
return [vid for vid in ids if vid in keep]
|
||||
|
||||
|
||||
def _new_first(db: Session, user: User, ids: list[str]) -> list[str]:
|
||||
"""Reorder results so genuinely-new discoveries come first and things the user already has
|
||||
(a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
|
||||
to the back — keeping the relevance order within each group."""
|
||||
if not ids:
|
||||
return ids
|
||||
sub_ch = select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
has_state = (
|
||||
select(VideoState.id)
|
||||
.where(VideoState.video_id == Video.id, VideoState.user_id == user.id)
|
||||
.exists()
|
||||
)
|
||||
in_playlist = (
|
||||
select(PlaylistItem.id)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(PlaylistItem.video_id == Video.id, Playlist.user_id == user.id)
|
||||
.exists()
|
||||
)
|
||||
have = set(
|
||||
db.scalars(
|
||||
select(Video.id).where(
|
||||
Video.id.in_(ids),
|
||||
or_(Video.channel_id.in_(sub_ch), has_state, in_playlist),
|
||||
)
|
||||
).all()
|
||||
)
|
||||
return [i for i in ids if i not in have] + [i for i in ids if i in have]
|
||||
|
||||
|
||||
def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
|
||||
"""Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
|
||||
|
||||
"Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
|
||||
video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
|
||||
wildcard — either way the term was never folded in, and local search lost that relevance."""
|
||||
padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.id.in_(video_ids))
|
||||
.where(padded.notilike(f"% {like_escape(term)} %"))
|
||||
.values(
|
||||
search_terms=func.trim(
|
||||
func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@router.get("/youtube")
|
||||
def search_youtube(
|
||||
q: str | None = None,
|
||||
@@ -287,9 +80,7 @@ def search_youtube(
|
||||
raise refused(429, "quota_spent")
|
||||
|
||||
# Channels this user has blocked are dropped before ingestion — never shown, never stored.
|
||||
blocked = set(
|
||||
db.scalars(select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)).all()
|
||||
)
|
||||
blocked = queries.blocked_ids(db, user)
|
||||
|
||||
collected: list[str] = []
|
||||
next_cursor = cursor
|
||||
@@ -330,7 +121,7 @@ def search_youtube(
|
||||
# write/shape failure degrades to a 422 with a message on this JSON endpoint instead of a
|
||||
# bare 500 hidden behind the FE's generic "search failed" (same rationale as B-core-1).
|
||||
try:
|
||||
collected.extend(_ingest_candidates(db, yt, user, candidates))
|
||||
collected.extend(catalog_search.ingest_candidates(db, yt, user, candidates))
|
||||
except Exception as exc: # any ingest failure, not just YouTubeError
|
||||
db.rollback()
|
||||
# The term is NOT in this line. A search string is exactly what the privacy
|
||||
@@ -378,17 +169,12 @@ def search_youtube(
|
||||
# Record these as THIS user's search finds (Mine feed / Source filter), and fold the
|
||||
# query into each result's search_terms so local full-text search inherits YouTube's
|
||||
# relevance. Idempotent; the generated search_vector updates automatically.
|
||||
db.execute(
|
||||
pg_insert(SearchFind)
|
||||
.values([{"user_id": user.id, "video_id": vid} for vid in ordered])
|
||||
.on_conflict_do_nothing(index_elements=["user_id", "video_id"])
|
||||
)
|
||||
_fold_term(db, ordered, term)
|
||||
catalog_search.record_finds(db, user, ordered, term)
|
||||
db.commit()
|
||||
|
||||
ordered = _new_first(db, user, ordered)
|
||||
ordered = catalog_search.new_first(db, user, ordered)
|
||||
return {
|
||||
"items": _serialize_ids(db, user, ordered),
|
||||
"items": reader_queries.feed_rows_by_ids(db, user, ordered),
|
||||
"next_cursor": next_cursor,
|
||||
"has_more": bool(next_cursor),
|
||||
"source": source,
|
||||
@@ -412,23 +198,6 @@ def clear_search(
|
||||
ids = body.video_ids
|
||||
if not ids:
|
||||
return {"ok": True, "removed": 0}
|
||||
db.execute(
|
||||
delete(SearchFind).where(SearchFind.user_id == user.id, SearchFind.video_id.in_(ids))
|
||||
)
|
||||
db.commit()
|
||||
has_state = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
in_playlist = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
removed = db.execute(
|
||||
delete(Video).where(
|
||||
Video.id.in_(ids),
|
||||
Video.via_search.is_(True),
|
||||
~has_state,
|
||||
~in_playlist,
|
||||
~subscribed,
|
||||
~other_find,
|
||||
)
|
||||
).rowcount
|
||||
removed = catalog_search.clear(db, user, ids)
|
||||
db.commit()
|
||||
return {"ok": True, "removed": removed}
|
||||
|
||||
+11
-54
@@ -1,12 +1,12 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy import and_, case, func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, quota, state
|
||||
from app.audit import AuditAction
|
||||
from app.auth import admin_user, current_user, has_read_scope
|
||||
from app.db import get_db
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog import queries, service
|
||||
from app.routes._common import youtube_refusal
|
||||
from app.ratelimit import RateLimiter
|
||||
from app.sync.runner import estimate_deep_backfill
|
||||
@@ -47,19 +47,8 @@ def sync_subscriptions(user: User = Depends(current_user), db: Session = Depends
|
||||
|
||||
@router.get("/status")
|
||||
def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sub_count = db.scalar(
|
||||
select(func.count()).select_from(Subscription).where(Subscription.user_id == user.id)
|
||||
)
|
||||
return {
|
||||
"subscriptions": sub_count,
|
||||
"channels_total": db.scalar(select(func.count()).select_from(Channel)),
|
||||
"channels_backfilling": db.scalar(
|
||||
select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
|
||||
),
|
||||
"videos_total": db.scalar(select(func.count()).select_from(Video)),
|
||||
"pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
),
|
||||
**queries.catalog_totals(db, user),
|
||||
"quota_used_today": quota.units_used_today(db),
|
||||
"quota_remaining_today": quota.remaining_today(db),
|
||||
"paused": state.is_sync_paused(db),
|
||||
@@ -70,51 +59,21 @@ def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db
|
||||
@router.get("/my-status")
|
||||
def my_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""Per-user sync progress: how far the channels *this* user subscribes to have synced."""
|
||||
sub_join = and_(Subscription.channel_id == Channel.id, Subscription.user_id == user.id)
|
||||
total, details_synced, recent_synced, all_deep_done = db.execute(
|
||||
select(
|
||||
func.count(Channel.id),
|
||||
func.count(Channel.details_synced_at),
|
||||
func.count(Channel.recent_synced_at),
|
||||
func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
|
||||
)
|
||||
.select_from(Channel)
|
||||
.join(Subscription, sub_join)
|
||||
).one()
|
||||
total = total or 0
|
||||
all_deep_done = int(all_deep_done or 0)
|
||||
total, details_synced, recent_synced, all_deep_done = queries.my_progress(db, user)
|
||||
|
||||
# Per-user full-history (deep) backfill: channels this user has opted in, and an ETA
|
||||
# for the still-pending ones (quota-bound estimate).
|
||||
deep_requested_channels = (
|
||||
db.execute(
|
||||
select(Channel).join(
|
||||
Subscription,
|
||||
and_(sub_join, Subscription.deep_requested.is_(True)),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
deep_requested_channels = queries.my_deep_requested(db, user)
|
||||
deep_eta = estimate_deep_backfill(db, deep_requested_channels)
|
||||
# "done / requested" is shown as one ratio, so both halves count the SAME channels: the ones this
|
||||
# user asked full history for. Counting every subscribed channel whose back-catalog happens to be
|
||||
# complete (another user asked, or it was short) gave "342/329".
|
||||
deep_done = sum(1 for c in deep_requested_channels if c.backfill_done)
|
||||
|
||||
my_videos = db.scalar(
|
||||
select(func.count(Video.id)).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
)
|
||||
)
|
||||
my_videos = queries.my_video_count(db, user)
|
||||
# Whole-catalog total (every video any user's subscriptions have pulled in), so the
|
||||
# header can show "yours" against the shared library size.
|
||||
total_videos = db.scalar(select(func.count(Video.id)))
|
||||
total_videos = queries.video_total(db)
|
||||
return {
|
||||
"channels_total": total,
|
||||
"channels_details_synced": details_synced,
|
||||
@@ -125,8 +84,8 @@ def my_status(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"channels_deep_requested": len(deep_requested_channels),
|
||||
"deep_pending_count": deep_eta["channels_pending"],
|
||||
"deep_eta_seconds": deep_eta["eta_seconds"],
|
||||
"my_videos": my_videos or 0,
|
||||
"total_videos": total_videos or 0,
|
||||
"my_videos": my_videos,
|
||||
"total_videos": total_videos,
|
||||
"quota_used_today": quota.units_used_today(db),
|
||||
"quota_remaining_today": quota.remaining_today(db),
|
||||
"paused": state.is_sync_paused(db),
|
||||
@@ -143,11 +102,9 @@ def deep_all(
|
||||
on: bool = True,
|
||||
) -> dict:
|
||||
"""Opt every one of the user's channels into (or out of) full-history backfill."""
|
||||
result = db.execute(
|
||||
update(Subscription).where(Subscription.user_id == user.id).values(deep_requested=on)
|
||||
)
|
||||
updated = service.set_deep_all(db, user, on)
|
||||
db.commit()
|
||||
return {"ok": True, "updated": result.rowcount, "deep_requested": on}
|
||||
return {"ok": True, "updated": updated, "deep_requested": on}
|
||||
|
||||
|
||||
@router.post("/pause")
|
||||
|
||||
+24
-32
@@ -1,19 +1,32 @@
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import ChannelTag, Tag, User
|
||||
from app.routes._common import owned_or_404
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.routes._common import not_found
|
||||
from app.sync.tagsuggest import compute_tag_suggestions
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.reader.models import Tag
|
||||
|
||||
router = APIRouter(prefix="/api/tags", tags=["tags"])
|
||||
|
||||
|
||||
def _own_tag(db: Session, user: User, tag_id: int) -> "Tag":
|
||||
"""One of the user's own tags, or a 404 — a system tag (no owner) is not theirs to edit."""
|
||||
tag = queries.own_tag(db, user, tag_id)
|
||||
if tag is None:
|
||||
raise not_found("tag")
|
||||
return tag
|
||||
|
||||
|
||||
@router.get("/suggestions")
|
||||
def tag_suggestions(
|
||||
user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
@@ -27,13 +40,6 @@ def tag_suggestions(
|
||||
|
||||
@router.get("")
|
||||
def list_tags(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
rows = db.execute(
|
||||
select(Tag, func.count(ChannelTag.id))
|
||||
.outerjoin(ChannelTag, ChannelTag.tag_id == Tag.id)
|
||||
.where(or_(Tag.user_id.is_(None), Tag.user_id == user.id))
|
||||
.group_by(Tag.id)
|
||||
.order_by(Tag.category, Tag.name)
|
||||
).all()
|
||||
return [
|
||||
{
|
||||
"id": tag.id,
|
||||
@@ -43,7 +49,7 @@ def list_tags(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"system": tag.user_id is None,
|
||||
"channel_count": count,
|
||||
}
|
||||
for tag, count in rows
|
||||
for tag, count in queries.tags_with_counts(db, user)
|
||||
]
|
||||
|
||||
|
||||
@@ -62,16 +68,8 @@ def create_tag(
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
# An unknown category silently falls back to "other" (unchanged behaviour) rather than 422.
|
||||
category = body.category if body.category in ("language", "topic", "other") else "other"
|
||||
tag = Tag(
|
||||
user_id=user.id,
|
||||
name=name,
|
||||
color=body.color,
|
||||
category=category,
|
||||
)
|
||||
db.add(tag)
|
||||
try:
|
||||
tag = service.create_tag(db, user, name, body.category, body.color)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -99,17 +97,13 @@ def update_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# System tags (user_id NULL) are not user-editable: owned_or_404 404s them too.
|
||||
tag = owned_or_404(db, Tag, tag_id, user, what="tag")
|
||||
tag = _own_tag(db, user, tag_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
tag.name = name
|
||||
if "color" in sent:
|
||||
tag.color = body.color
|
||||
name = (body.name or "").strip()
|
||||
if "name" in sent and not name:
|
||||
raise refused(400, "name_required")
|
||||
try:
|
||||
service.update_tag(tag, sent=sent, name=name, color=body.color)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -129,8 +123,6 @@ def delete_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# System tags (user_id NULL) are not user-editable: owned_or_404 404s them too.
|
||||
tag = owned_or_404(db, Tag, tag_id, user, what="tag")
|
||||
db.delete(tag) # ChannelTag rows cascade on tag delete.
|
||||
service.delete_tag(db, _own_tag(db, user, tag_id))
|
||||
db.commit()
|
||||
return {"deleted": tag_id}
|
||||
|
||||
@@ -16,6 +16,7 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens, audit
|
||||
from app.apitokens import GRANT_COLUMNS, Right
|
||||
from app.audit import AuditAction
|
||||
@@ -86,7 +87,7 @@ def _serialize(db: Session, token: ApiToken) -> dict:
|
||||
@router.get("")
|
||||
def list_tokens(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
tokens = (
|
||||
db.execute(select(ApiToken).where(ApiToken.user_id == user.id).order_by(ApiToken.id))
|
||||
db.execute(select(ApiToken).where(owned_by(ApiToken.user_id, user)).order_by(ApiToken.id))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
@@ -134,7 +135,7 @@ def create_token(
|
||||
if (epoch or 0) != (user.session_epoch or 0):
|
||||
raise refused(401, "not_authenticated")
|
||||
count = db.execute(
|
||||
select(func.count(ApiToken.id)).where(ApiToken.user_id == user.id)
|
||||
select(func.count(ApiToken.id)).where(owned_by(ApiToken.user_id, user))
|
||||
).scalar_one()
|
||||
if count >= MAX_TOKENS:
|
||||
raise refused(409, "api_token_limit", limit=MAX_TOKENS)
|
||||
|
||||
@@ -9,13 +9,13 @@ from typing import Callable
|
||||
from apscheduler.executors.pool import ThreadPoolExecutor
|
||||
from apscheduler.jobstores.base import ConflictingIdError
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from sqlalchemy import select
|
||||
|
||||
from app import audit, debugsession, jobruns, logs, logsink, progress, quota, sysconfig
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal
|
||||
from app.downloads.gc import run_download_gc
|
||||
from app.models import Channel, JobRun
|
||||
from app.models import JobRun
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.plex.sync import sync as run_plex_sync
|
||||
from app.plex.tmdb import run_enrich as run_tmdb_enrich
|
||||
from app.plex.tmdb import run_people_enrich
|
||||
@@ -305,7 +305,7 @@ def scheduler_snapshot(db) -> dict:
|
||||
"running": going is not None,
|
||||
"target": shown.target if shown else None,
|
||||
"target_title": (
|
||||
db.scalar(select(Channel.title).where(Channel.id == shown.target))
|
||||
catalog_queries.channel_title(db, shown.target)
|
||||
if shown is not None and shown.target
|
||||
else None
|
||||
),
|
||||
|
||||
+14
-87
@@ -8,11 +8,13 @@ regenerated freely; user tags are never touched here.
|
||||
|
||||
import re
|
||||
|
||||
from sqlalchemy import and_, exists, func, select
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, sysconfig
|
||||
from app.models import Channel, ChannelTag, SavedView, Tag, Video
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -209,71 +211,30 @@ def compute_channel_topics(db: Session, channel: Channel) -> set[str]:
|
||||
return topics
|
||||
|
||||
|
||||
def ensure_system_tag(db: Session, name: str, category: str, color: str | None) -> Tag:
|
||||
tag = db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), Tag.name == name)
|
||||
).scalar_one_or_none()
|
||||
if tag is None:
|
||||
tag = Tag(user_id=None, name=name, category=category, color=color)
|
||||
db.add(tag)
|
||||
db.flush()
|
||||
return tag
|
||||
|
||||
|
||||
def apply_channel_autotags(db: Session, channel: Channel) -> None:
|
||||
desired: dict[int, float | None] = {}
|
||||
|
||||
lang, confidence = detect_channel_language(db, channel)
|
||||
if lang:
|
||||
tag = ensure_system_tag(db, LANG_NAMES.get(lang, lang.upper()), "language", LANG_COLOR)
|
||||
tag = reader_service.ensure_system_tag(
|
||||
db, LANG_NAMES.get(lang, lang.upper()), "language", LANG_COLOR
|
||||
)
|
||||
desired[tag.id] = confidence
|
||||
|
||||
for topic in compute_channel_topics(db, channel):
|
||||
tag = ensure_system_tag(db, topic, "topic", TOPIC_COLORS.get(topic, DEFAULT_TOPIC_COLOR))
|
||||
tag = reader_service.ensure_system_tag(
|
||||
db, topic, "topic", TOPIC_COLORS.get(topic, DEFAULT_TOPIC_COLOR)
|
||||
)
|
||||
desired.setdefault(tag.id, None)
|
||||
|
||||
existing = {
|
||||
ct.tag_id: ct
|
||||
for ct in db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel.id,
|
||||
ChannelTag.user_id.is_(None),
|
||||
ChannelTag.source == "auto",
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
for tag_id, ct in existing.items():
|
||||
if tag_id not in desired:
|
||||
db.delete(ct)
|
||||
for tag_id, conf in desired.items():
|
||||
ct = existing.get(tag_id)
|
||||
if ct is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel.id,
|
||||
tag_id=tag_id,
|
||||
user_id=None,
|
||||
source="auto",
|
||||
confidence=conf,
|
||||
)
|
||||
)
|
||||
else:
|
||||
ct.confidence = conf
|
||||
reader_service.set_auto_tags(db, channel.id, desired)
|
||||
db.commit()
|
||||
|
||||
|
||||
def run_autotag_all(db: Session, only_missing: bool = False) -> dict:
|
||||
query = select(Channel)
|
||||
if only_missing:
|
||||
query = query.where(
|
||||
~exists().where(
|
||||
and_(
|
||||
ChannelTag.channel_id == Channel.id,
|
||||
ChannelTag.source == "auto",
|
||||
ChannelTag.user_id.is_(None),
|
||||
)
|
||||
)
|
||||
)
|
||||
query = query.where(~reader_queries.has_auto_tags(Channel.id))
|
||||
channels = db.execute(query).scalars().all()
|
||||
tagged = 0
|
||||
for i, channel in enumerate(channels, 1):
|
||||
@@ -284,40 +245,6 @@ def run_autotag_all(db: Session, only_missing: bool = False) -> dict:
|
||||
db.rollback()
|
||||
log.exception("Auto-tagging failed for channel %s", channel.id)
|
||||
progress.report(i, len(channels), "autotag")
|
||||
removed = _cleanup_orphan_system_tags(db)
|
||||
return {"channels_tagged": tagged, "orphan_tags_removed": removed}
|
||||
|
||||
|
||||
def _view_tag_ids(filters: object) -> list[int]:
|
||||
"""The tag ids a stored view's filters name; anything else in that slot names none."""
|
||||
tags = filters.get("tags") if isinstance(filters, dict) else None
|
||||
if not isinstance(tags, list):
|
||||
return []
|
||||
return [t for t in tags if type(t) is int] # `type`, not isinstance: True is not a tag id
|
||||
|
||||
|
||||
def _cleanup_orphan_system_tags(db: Session) -> int:
|
||||
"""Delete system tags that no longer link to any channel (e.g. after a mapping change) — but not
|
||||
one a saved feed view filters on. Views hold tag IDS, and a deleted tag comes back under a new id
|
||||
when a channel earns it again, so the view's filter would point at nothing; the feed drops an
|
||||
unknown id, and the view showed the whole feed under its old name.
|
||||
|
||||
The views are read defensively, and the kept ids are filtered here rather than in SQL: a view
|
||||
stored before its route checked the shape (or written around it) must not raise for everyone's
|
||||
job, nor push a `NOT IN` past the driver's bind-parameter limit."""
|
||||
in_views = {
|
||||
t
|
||||
for (filters,) in db.execute(select(SavedView.filters).where(SavedView.source == "feed"))
|
||||
for t in _view_tag_ids(filters)
|
||||
}
|
||||
orphans = [
|
||||
tag
|
||||
for tag in db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), ~exists().where(ChannelTag.tag_id == Tag.id))
|
||||
).scalars()
|
||||
if tag.id not in in_views
|
||||
]
|
||||
for tag in orphans:
|
||||
db.delete(tag)
|
||||
removed = reader_service.prune_orphan_system_tags(db)
|
||||
db.commit()
|
||||
return len(orphans)
|
||||
return {"channels_tagged": tagged, "orphan_tags_removed": removed}
|
||||
|
||||
@@ -14,15 +14,8 @@ from sqlalchemy import delete, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import (
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
PlaylistItem,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.catalog.models import Channel, ExploredChannel, SearchFind, Subscription, Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.sync.videos import insert_stubs, stub_from_playlist_item, apply_video_details
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, is_playlist_gone
|
||||
|
||||
@@ -129,15 +122,12 @@ def purge_unkept_explored(db: Session) -> dict:
|
||||
if not orphan_ids:
|
||||
return {"expired": expired, "videos_deleted": 0, "channels_deleted": 0}
|
||||
|
||||
state_exists = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
pl_exists = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
find_exists = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
videos_deleted = db.execute(
|
||||
delete(Video).where(
|
||||
Video.channel_id.in_(orphan_ids),
|
||||
Video.via_explore.is_(True),
|
||||
~state_exists,
|
||||
~pl_exists,
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~find_exists,
|
||||
)
|
||||
).rowcount
|
||||
@@ -166,15 +156,12 @@ def purge_unkept_search(db: Session, grace_days: int | None = None) -> dict:
|
||||
0 quota."""
|
||||
grace = sysconfig.get_int(db, "search_grace_days") if grace_days is None else grace_days
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=grace)
|
||||
state_exists = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
pl_exists = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
sub_exists = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
deleted = db.execute(
|
||||
delete(Video).where(
|
||||
Video.via_search.is_(True),
|
||||
Video.created_at < cutoff,
|
||||
~state_exists,
|
||||
~pl_exists,
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~sub_exists,
|
||||
)
|
||||
).rowcount
|
||||
|
||||
@@ -27,7 +27,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.config import settings
|
||||
from app.models import Playlist, PlaylistItem, Video
|
||||
from app.catalog.models import Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.notifications import create_notification
|
||||
from app.sync.runner import get_service_user
|
||||
from app.sync.videos import apply_video_details, parse_dt
|
||||
@@ -88,11 +89,7 @@ def _collect_user_impact(db: Session, videos: list[Video]) -> dict[int, list[dic
|
||||
# Anyone who had it in a playlist — which now also covers "saved": Watch later became a
|
||||
# built-in playlist, so "saved" is no longer a VideoState status (the old status=="saved"
|
||||
# scan matched nothing and under-counted impacted users).
|
||||
for user_id, video_id in db.execute(
|
||||
select(Playlist.user_id, PlaylistItem.video_id)
|
||||
.join(PlaylistItem, PlaylistItem.playlist_id == Playlist.id)
|
||||
.where(PlaylistItem.video_id.in_(ids))
|
||||
):
|
||||
for user_id, video_id in reader_queries.playlisted_by(db, ids):
|
||||
impact.setdefault(user_id, set()).add(video_id)
|
||||
return {
|
||||
user_id: [
|
||||
|
||||
@@ -12,7 +12,6 @@ import json
|
||||
from collections.abc import Callable, Iterator
|
||||
from contextlib import AbstractContextManager, contextmanager
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TypeVar
|
||||
|
||||
from sqlalchemy import Engine, create_engine, delete, func, select
|
||||
@@ -21,12 +20,15 @@ from sqlalchemy.pool import NullPool
|
||||
|
||||
from app import logs, progress, quota
|
||||
from app.auth import has_read_scope
|
||||
from app.models import Channel, Playlist, PlaylistItem, User, Video
|
||||
from app.models import User
|
||||
from app.catalog.models import Video
|
||||
from app.reader.models import Playlist, PlaylistItem
|
||||
from app.sync.playlist_merge import merge_plan
|
||||
from app.sync.runner import token_users
|
||||
from app.sync.subscriptions import blank_listing, refuse_blank_listing, vanished
|
||||
from app.sync.videos import apply_video_details, parse_dt
|
||||
from app.sync.videos import ensure_videos
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, is_playlist_gone
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
# Cost of a single YouTube write op (playlists/playlistItems insert/update/delete).
|
||||
@@ -227,55 +229,11 @@ def recompute_dirty(db: Session, pl: Playlist) -> None:
|
||||
pl.dirty = pl.synced_fingerprint != current_fingerprint(db, pl)
|
||||
|
||||
|
||||
def _ensure_videos(db: Session, yt: YouTubeClient, video_ids: list[str]) -> None:
|
||||
"""Make sure every given video id exists in the catalog, fetching + ingesting the
|
||||
missing ones (and a stub channel for any unknown channel). Deleted/private videos that
|
||||
the API won't return are simply left out."""
|
||||
if not video_ids:
|
||||
return
|
||||
have = set(db.execute(select(Video.id).where(Video.id.in_(video_ids))).scalars().all())
|
||||
missing = [v for v in dict.fromkeys(video_ids) if v not in have]
|
||||
if not missing:
|
||||
return
|
||||
items = yt.get_videos(missing)
|
||||
chan_ids = {
|
||||
it.get("snippet", {}).get("channelId")
|
||||
for it in items
|
||||
if it.get("snippet", {}).get("channelId")
|
||||
}
|
||||
existing_ch = set(
|
||||
db.execute(select(Channel.id).where(Channel.id.in_(chan_ids))).scalars().all()
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
for it in items:
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if cid and cid not in existing_ch:
|
||||
db.add(Channel(id=cid, title=sn.get("channelTitle")))
|
||||
existing_ch.add(cid)
|
||||
db.flush()
|
||||
seen: set[str] = set()
|
||||
for it in items:
|
||||
vid = it.get("id")
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if not vid or not cid or vid in seen:
|
||||
continue
|
||||
seen.add(vid)
|
||||
# published_at is NOT NULL (0068); apply_video_details re-fills a missing date, but seed a now()
|
||||
# fallback here too so the constructor never carries an explicit NULL.
|
||||
v = Video(id=vid, channel_id=cid, published_at=parse_dt(sn.get("publishedAt")) or now)
|
||||
apply_video_details(v, it)
|
||||
v.enriched_at = now
|
||||
db.add(v)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _replace_items_from_youtube(db: Session, pl: Playlist, ids: list[str]) -> list[str]:
|
||||
"""Mirror a YouTube playlist's ordered video ids into `pl`'s local items (authoritative
|
||||
replace): keep only those present in the catalog in their YT order (de-duplicated), swap the
|
||||
items out, and reset the synced fingerprint + dirty flag. The caller has ingested `ids`
|
||||
(`_ensure_videos`, which commits) and THEN taken `lock_playlists`, set `pl.name`, and commits
|
||||
(`sync.videos.ensure_videos`, which commits) and THEN taken `lock_playlists`, set `pl.name`, and commits
|
||||
after: an ingest in here re-fetched every id YouTube will not return (a private or deleted
|
||||
video) and its commit let go of the lock before the replace. Returns the final ordered ids."""
|
||||
present = (
|
||||
@@ -346,7 +304,9 @@ def pull_user_playlists(db: Session, user: User) -> dict:
|
||||
existing = {
|
||||
pl.yt_playlist_id: pl
|
||||
for pl in db.execute(
|
||||
select(Playlist).where(Playlist.user_id == user.id, Playlist.source == "youtube")
|
||||
select(Playlist).where(
|
||||
owned_by(Playlist.user_id, user.id), Playlist.source == "youtube"
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
# YouTube playlists that we already own as local (exported) playlists must not be
|
||||
@@ -354,7 +314,7 @@ def pull_user_playlists(db: Session, user: User) -> dict:
|
||||
owned_yt_ids = set(
|
||||
db.execute(
|
||||
select(Playlist.yt_playlist_id).where(
|
||||
Playlist.user_id == user.id,
|
||||
owned_by(Playlist.user_id, user.id),
|
||||
Playlist.source == "local",
|
||||
Playlist.yt_playlist_id.is_not(None),
|
||||
)
|
||||
@@ -401,7 +361,7 @@ def pull_user_playlists(db: Session, user: User) -> dict:
|
||||
continue
|
||||
# Ingest first — it commits — so the row lock below is held from the dirty check
|
||||
# to the commit that replaces the items.
|
||||
_ensure_videos(db, yt, ids)
|
||||
ensure_videos(db, yt, ids)
|
||||
# `dirty` read above is a snapshot from before the YouTube calls (sessions keep
|
||||
# loaded rows past a commit): a remove or reorder made meanwhile must not be
|
||||
# replaced by YouTube's order. Item writes lock the playlist row, so this waits for
|
||||
@@ -612,7 +572,7 @@ def merge_playlist(db: Session, user: User, pl: Playlist, *, live=None) -> dict:
|
||||
prev_unavailable = parse_unavailable(pl)
|
||||
still_listed = [v for v in yt_ids if v in prev_unavailable]
|
||||
if still_listed:
|
||||
_ensure_videos(db, yt, still_listed)
|
||||
ensure_videos(db, yt, still_listed)
|
||||
healed = set(
|
||||
db.execute(select(Video.id).where(Video.id.in_(still_listed))).scalars().all()
|
||||
)
|
||||
@@ -624,7 +584,7 @@ def merge_playlist(db: Session, user: User, pl: Playlist, *, live=None) -> dict:
|
||||
# --- Siftlode side (free, direct): pull YouTube-adds in, drop YouTube-deletes, land the
|
||||
# merged order authoritatively. Do this first so local reflects the merge even if a later
|
||||
# YouTube write trips the breaker.
|
||||
_ensure_videos(db, yt, mp.add_to_local)
|
||||
ensure_videos(db, yt, mp.add_to_local)
|
||||
# `local` was read before the YouTube calls above: an add or remove committed meanwhile
|
||||
# (another tab) would be undone by the replace below. Lock the list — the ingest's commit is
|
||||
# behind us — and plan again from what is there now. Nothing new needs ingesting: a video
|
||||
@@ -749,7 +709,7 @@ def repull_playlist(db: Session, user: User, pl: Playlist) -> dict:
|
||||
)
|
||||
ids = list(yt.iter_my_playlist_video_ids(pl.yt_playlist_id))
|
||||
refuse_blank_listing(ids, snippet.get("item_count"))
|
||||
_ensure_videos(db, yt, ids)
|
||||
ensure_videos(db, yt, ids)
|
||||
_lock_or_gone(db, pl)
|
||||
pl.name = snippet.get("title") or pl.name
|
||||
ordered = _replace_items_from_youtube(db, pl, ids)
|
||||
|
||||
@@ -16,7 +16,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import Channel, OAuthToken, Subscription, User
|
||||
from app.models import OAuthToken, User
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.sync.explore import followed_channel
|
||||
from app.sync.subscriptions import import_subscriptions
|
||||
from app.sync.videos import (
|
||||
|
||||
@@ -7,9 +7,11 @@ from sqlalchemy import or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import Channel, Subscription, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.utils import normalize_description
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -172,7 +174,7 @@ def import_subscriptions(db: Session, user: User) -> dict:
|
||||
new_channels += 1
|
||||
sub_row = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == cid
|
||||
owned_by(Subscription.user_id, user.id), Subscription.channel_id == cid
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if sub_row is None:
|
||||
@@ -185,7 +187,9 @@ def import_subscriptions(db: Session, user: User) -> dict:
|
||||
# (see `should_prune`).
|
||||
removed = 0
|
||||
existing = (
|
||||
db.execute(select(Subscription).where(Subscription.user_id == user.id)).scalars().all()
|
||||
db.execute(select(Subscription).where(owned_by(Subscription.user_id, user.id)))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
prune_skipped = not should_prune(len(fetched), len(existing))
|
||||
if prune_skipped:
|
||||
|
||||
@@ -23,7 +23,10 @@ from collections import Counter
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import ChannelTag, Subscription, Tag, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Subscription
|
||||
from app.reader.models import ChannelTag, Tag
|
||||
from app.userscope import owned_by
|
||||
|
||||
# A user tag needs at least this many manually-tagged example channels before we trust its
|
||||
# profile — fewer is noise (one gaming-cooking channel shouldn't make "cooking" mean "gaming").
|
||||
@@ -57,7 +60,9 @@ def compute_tag_suggestions(db: Session, user: User) -> dict[str, list[dict]]:
|
||||
# Channel Manager shows ghost chips only on non-hidden rows, so suggesting for hidden channels would
|
||||
# inflate the header count past what's shown and let bulk-accept tag channels the user can't see).
|
||||
subs = db.execute(
|
||||
select(Subscription.channel_id, Subscription.hidden).where(Subscription.user_id == user.id)
|
||||
select(Subscription.channel_id, Subscription.hidden).where(
|
||||
owned_by(Subscription.user_id, user.id)
|
||||
)
|
||||
).all()
|
||||
subscribed: set[str] = {cid for cid, _ in subs}
|
||||
hidden: set[str] = {cid for cid, h in subs if h}
|
||||
@@ -74,7 +79,7 @@ def compute_tag_suggestions(db: Session, user: User) -> dict[str, list[dict]]:
|
||||
user_tagged: set[str] = set()
|
||||
for tag_id, cid in db.execute(
|
||||
select(ChannelTag.tag_id, ChannelTag.channel_id).where(
|
||||
ChannelTag.user_id == user.id, ChannelTag.source == "user"
|
||||
owned_by(ChannelTag.user_id, user.id), ChannelTag.source == "user"
|
||||
)
|
||||
).all():
|
||||
if cid not in subscribed:
|
||||
|
||||
@@ -12,7 +12,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.models import LIVE_OR_UPCOMING, Channel, Video
|
||||
from app.catalog.models import LIVE_OR_UPCOMING, Channel, Video
|
||||
from app.titles import normalize_title
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail, is_playlist_gone
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
@@ -418,3 +418,48 @@ def run_shorts_classification(db: Session, limit: int | None = None) -> dict:
|
||||
progress.report(probed, len(candidates), "probe")
|
||||
db.commit()
|
||||
return {"probed": probed, "shorts": shorts}
|
||||
|
||||
|
||||
def ensure_videos(db: Session, yt: YouTubeClient, video_ids: list[str]) -> None:
|
||||
"""Make sure every given video id exists in the catalog, fetching + ingesting the
|
||||
missing ones (and a stub channel for any unknown channel). Deleted/private videos that
|
||||
the API won't return are simply left out. The catalog's write for a playlist sync that names
|
||||
videos it has never seen; commits."""
|
||||
if not video_ids:
|
||||
return
|
||||
have = set(db.execute(select(Video.id).where(Video.id.in_(video_ids))).scalars().all())
|
||||
missing = [v for v in dict.fromkeys(video_ids) if v not in have]
|
||||
if not missing:
|
||||
return
|
||||
items = yt.get_videos(missing)
|
||||
chan_ids = {
|
||||
it.get("snippet", {}).get("channelId")
|
||||
for it in items
|
||||
if it.get("snippet", {}).get("channelId")
|
||||
}
|
||||
existing_ch = set(
|
||||
db.execute(select(Channel.id).where(Channel.id.in_(chan_ids))).scalars().all()
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
for it in items:
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if cid and cid not in existing_ch:
|
||||
db.add(Channel(id=cid, title=sn.get("channelTitle")))
|
||||
existing_ch.add(cid)
|
||||
db.flush()
|
||||
seen: set[str] = set()
|
||||
for it in items:
|
||||
vid = it.get("id")
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if not vid or not cid or vid in seen:
|
||||
continue
|
||||
seen.add(vid)
|
||||
# published_at is NOT NULL (0068); apply_video_details re-fills a missing date, but seed a now()
|
||||
# fallback here too so the constructor never carries an explicit NULL.
|
||||
v = Video(id=vid, channel_id=cid, published_at=parse_dt(sn.get("publishedAt")) or now)
|
||||
apply_video_details(v, it)
|
||||
v.enriched_at = now
|
||||
db.add(v)
|
||||
db.commit()
|
||||
|
||||
@@ -5,5 +5,8 @@ names the whole schema. Whatever needs all of it — the migrations, the test da
|
||||
every mapper — imports this module.
|
||||
"""
|
||||
|
||||
import app.catalog.models # Catalog's tables
|
||||
import app.downloads.models # Capture's tables
|
||||
import app.models # noqa: F401 — the core tables
|
||||
import app.models # the core tables
|
||||
import app.plex.models # Plex's tables
|
||||
import app.reader.models # noqa: F401 — Reader's tables
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
"""Whose a row is, and who an account is to the others — one place for both.
|
||||
|
||||
**Ownership.** A user-owned table says whose a row is in one column (`user_id`, or `to_user_id` for
|
||||
something handed to them). A unit's reads and writes name that column through `owned_by` and fetch a
|
||||
something handed to them). Every read and write names that column through `owned_by` and fetches a
|
||||
row by id through `owned_row`, never with an inline `Model.user_id == …`: the rule "a row owned by
|
||||
someone else is answered like a missing one" then lives here, and a contract test
|
||||
(`test_unit_fences.py`) holds a switched-on unit to it.
|
||||
someone else is answered like a missing one" then lives here. A contract test
|
||||
(`test_unit_fences.py`) holds every module, core included, to the filter's form; a rule this module
|
||||
cannot say (own or shared, built-in or own) is written out where it applies, with its reason.
|
||||
|
||||
**Recipients.** "An active, non-suspended account" — the download share-by-email and the recipient
|
||||
picker filter on it. Kept here so the two forms (SQL clauses for a query, a Python check for a loaded
|
||||
|
||||
@@ -31,6 +31,7 @@ from functools import partial
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
import app.tables # noqa: F401 — every table, before the first query
|
||||
from app import logs
|
||||
from app import logsink
|
||||
from app import sysconfig
|
||||
|
||||
@@ -5,7 +5,8 @@
|
||||
# scans it, blocking. A security fix, or a yt-dlp release YouTube's changes force, may be ONE pin
|
||||
# bumped in place instead of a full regeneration, which would move every other package too — then
|
||||
# `pip install -r requirements.lock && pip check` must pass. In place so far: urllib3 2.7.0 -> 2.8.0 (PYSEC-2026-4175/4176/4177); yt-dlp 2026.7.4 -> 2026.8.19
|
||||
# (it marks the clients that need a stream PO token, so the downloader stops fetching into a 403).
|
||||
# (it marks the clients that need a stream PO token, so the downloader stops fetching into a 403);
|
||||
# certifi 2026.7.22 -> 2026.10.19 (GHSA-6q48-6c7w-9gw2).
|
||||
APScheduler==3.11.3
|
||||
Authlib==1.7.2
|
||||
Mako==1.4.1
|
||||
@@ -21,7 +22,7 @@ argon2-cffi-bindings==25.1.0
|
||||
argon2-cffi==25.1.0
|
||||
bgutil-ytdlp-pot-provider==1.3.1
|
||||
brotli==1.2.0
|
||||
certifi==2026.7.22
|
||||
certifi==2026.10.19
|
||||
cffi==2.1.1
|
||||
charset-normalizer==3.5.1
|
||||
click==8.4.2
|
||||
|
||||
@@ -105,6 +105,11 @@ def pytest_configure(config):
|
||||
)
|
||||
|
||||
|
||||
def _write_xact(db) -> str | None:
|
||||
"""The id of the open transaction if it has written anything, else None (a read never takes one)."""
|
||||
return db.scalar(text("SELECT pg_current_xact_id_if_assigned()::text"))
|
||||
|
||||
|
||||
def _request_dep(db, unfinished: list[str]):
|
||||
"""`get_db` for a test request: the test's own session, plus the check a shared session hides.
|
||||
|
||||
@@ -114,15 +119,16 @@ def _request_dep(db, unfinished: list[str]):
|
||||
and the suite stays green over a handler that forgot to commit (deep review, debt burn). So a
|
||||
request that ends normally with a write it did not commit — flushed, or still only staged — is
|
||||
recorded, and the test fails on it at teardown. Only the ROOT transaction's end clears it: a
|
||||
savepoint released inside the handler commits nothing. The test's own staged rows are flushed before the request starts — they are the
|
||||
test's business, not the handler's. A request that raises is rolled back, as production's
|
||||
savepoint released inside the handler commits nothing. What the test itself staged or wrote is
|
||||
committed before the request starts — it is the test's business, not the handler's, and a
|
||||
request then begins with no write open, so any write left at its end is the handler's. A request that raises is rolled back, as production's
|
||||
`close()` rolls it back: without that, what a failing handler flushed but never committed stayed
|
||||
readable to the rest of the test — the one state production cannot be in (an issue created
|
||||
on the tracker before a failed upload was kept only by an explicit commit, and nothing saw it
|
||||
missing)."""
|
||||
|
||||
def dep(conn: HTTPConnection):
|
||||
db.flush()
|
||||
db.commit()
|
||||
state = {"dirty": False}
|
||||
|
||||
def flushed(session, ctx):
|
||||
@@ -138,7 +144,12 @@ def _request_dep(db, unfinished: list[str]):
|
||||
try:
|
||||
yield db
|
||||
staged = db.new or db.deleted or any(db.is_modified(o) for o in db.dirty)
|
||||
if state["dirty"] or staged:
|
||||
# Core DML (`db.execute(update(...))`) fires no flush and stages nothing; the database
|
||||
# itself says whether a write is open — the request began with none, and a transaction
|
||||
# that wrote and was not committed holds an id (see `release_read_connection`). A row
|
||||
# lock (`FOR UPDATE`) takes one too: the handler ends what it locked.
|
||||
core_dml = _write_xact(db) is not None
|
||||
if state["dirty"] or staged or core_dml:
|
||||
unfinished.append(f"{conn.scope.get('method', 'WS')} {conn.url.path}")
|
||||
except Exception:
|
||||
db.rollback()
|
||||
@@ -220,7 +231,7 @@ def _configured(monkeypatch):
|
||||
@pytest.fixture
|
||||
def client(db, override_dep, _configured):
|
||||
"""A TestClient wired to the test DB that goes through the REAL auth/session stack — no
|
||||
`current_user` override — so login, the wallet gate and epoch revoke are exercised end to end.
|
||||
`current_user` override — so login, the one-account session and epoch revoke are exercised end to end.
|
||||
Starlette's SessionMiddleware round-trips the signed cookie, so a login in one call authenticates
|
||||
the next on this same instance. No lifespan (the scheduler / HLS sweep stay off)."""
|
||||
override_dep(get_db, db.info["request"])
|
||||
|
||||
@@ -106,7 +106,7 @@ def test_the_single_playlist_repull_shares_the_pull_cooldown(db, as_user, make_u
|
||||
# SEC-14, the sweep half: `revert-youtube` spends under the SAME QuotaAction.PLAYLISTS_PULL key
|
||||
# as the bulk `sync-youtube`, so limiting only one of them left the identical drain open one
|
||||
# route over. Both must draw on the one allowance.
|
||||
from app.models import Playlist
|
||||
from app.reader.models import Playlist
|
||||
from app.routes import playlists as plroutes
|
||||
|
||||
user = make_user()
|
||||
|
||||
@@ -9,7 +9,8 @@ from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.auth import admin_user
|
||||
from app.main import app
|
||||
from app.models import AuditLog, Channel, JobRun, LogEvent, QuotaEvent
|
||||
from app.models import AuditLog, JobRun, LogEvent, QuotaEvent
|
||||
from app.catalog.models import Channel
|
||||
from tests.test_body_limit import _route_objects
|
||||
|
||||
T0 = datetime(2026, 10, 1, 12, 0, tzinfo=timezone.utc)
|
||||
|
||||
@@ -1,21 +1,25 @@
|
||||
"""Auth / session security core (TEST-1), DB-lane.
|
||||
|
||||
The wallet impersonation gate, epoch-based session revocation, single-use tokens and the
|
||||
enumeration-safe uniform 401 are the security mag, and had zero tests (only `_oauth_redirect_uri` was
|
||||
covered). Starlette's SessionMiddleware round-trips the signed cookie under TestClient, so the whole
|
||||
browser-less flow is exercisable. Uses the real auth stack via the `client` fixture (NO current_user
|
||||
override) so login, the wallet gate and epoch revoke are the code under test.
|
||||
One account per browser session, epoch-based session revocation, single-use tokens and the
|
||||
enumeration-safe uniform 401 are the security mag. Starlette's SessionMiddleware round-trips the
|
||||
signed cookie under TestClient, so the whole browser-less flow is exercisable. Uses the real auth
|
||||
stack via the `client` fixture (NO current_user override) so login, the session and epoch revoke are
|
||||
the code under test.
|
||||
|
||||
Note: `/api/me` is the bootstrap probe — it returns 200 `{authenticated: False}` when logged out, not a
|
||||
401. A revoked session is therefore checked on a PROTECTED endpoint (`/api/feed`), which does 401.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
from datetime import timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from itsdangerous import TimestampSigner
|
||||
|
||||
from app.auth import ACTIVE_ACCOUNT_HEADER
|
||||
from app.config import settings
|
||||
from app.main import app
|
||||
|
||||
_PW = "correct horse battery"
|
||||
@@ -46,74 +50,48 @@ def test_wrong_password_and_unknown_email_give_a_byte_identical_401(client, make
|
||||
assert wrong_pw.json() == unknown.json()
|
||||
|
||||
|
||||
def test_a_non_wallet_account_header_falls_back_to_the_default(client, make_user):
|
||||
# The X-Siftlode-Account header may only re-target an account already signed into THIS browser
|
||||
# (in the session wallet). A header naming someone else must be ignored, not honored — otherwise
|
||||
# it's an impersonation gate hole.
|
||||
make_user(email="alice2@example.com", password=_PW)
|
||||
mallory = make_user(email="mallory@example.com", password=_PW)
|
||||
assert _login(client, "alice2@example.com", _PW).status_code == 200
|
||||
me = client.get("/api/me", headers={ACTIVE_ACCOUNT_HEADER: str(mallory.id)})
|
||||
assert me.status_code == 200
|
||||
assert me.json()["email"] == "alice2@example.com" # still Alice, not Mallory
|
||||
def _cookie(payload: dict) -> str:
|
||||
"""A session cookie signed the way SessionMiddleware signs one — for a cookie an older build wrote."""
|
||||
data = base64.b64encode(json.dumps(payload).encode())
|
||||
return TimestampSigner(str(settings.secret_key)).sign(data).decode()
|
||||
|
||||
|
||||
def test_a_tab_pinned_to_an_account_signed_out_elsewhere_acts_as_nobody(client, make_user):
|
||||
# Two accounts in one browser: a tab pinned to Alice, Bob the default. Alice signs out in another
|
||||
# tab. The pinned tab used to go on acting as the default, Bob — its player checkpoints and pref
|
||||
# saves landed in his account, and its own Sign out signed HIM out.
|
||||
alice = make_user(email="alice3@example.com", password=_PW)
|
||||
def test_signing_in_as_another_account_replaces_the_first(client, make_user):
|
||||
# One account per browser session: a second sign-in is a fresh session, not a second account
|
||||
# beside the first — so signing out leaves nobody signed in here.
|
||||
make_user(email="alice3@example.com", password=_PW)
|
||||
make_user(email="bob3@example.com", password=_PW)
|
||||
assert _login(client, "alice3@example.com", _PW).status_code == 200
|
||||
assert _login(client, "bob3@example.com", _PW).status_code == 200 # Bob is now the default
|
||||
tab = {ACTIVE_ACCOUNT_HEADER: str(alice.id)}
|
||||
assert client.get("/api/feed", headers=tab).status_code == 200
|
||||
assert client.post("/auth/logout", headers=tab).status_code == 200 # Alice signs out
|
||||
assert _login(client, "bob3@example.com", _PW).status_code == 200
|
||||
assert client.get("/api/me").json()["email"] == "bob3@example.com"
|
||||
|
||||
assert client.get("/api/feed", headers=tab).status_code == 401
|
||||
assert client.post("/auth/logout", headers=tab).status_code == 200
|
||||
assert client.get("/api/feed").status_code == 200 # Bob is still signed in, the session intact
|
||||
# The bootstrap probe still answers as the default: how the stale tab learns who it is now.
|
||||
assert client.get("/api/me", headers=tab).json()["email"] == "bob3@example.com"
|
||||
assert client.post("/auth/logout").status_code == 200
|
||||
assert client.get("/api/feed").status_code == 401
|
||||
assert client.get("/api/me").json() == {"authenticated": False}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ended", ["revoked", "suspended"])
|
||||
def test_a_tab_pinned_to_an_account_whose_session_ended_comes_back_as_the_default(
|
||||
client, make_user, db, ended
|
||||
):
|
||||
# A tab pinned to Alice, Bob the default. Alice's session then ends while she is still in this
|
||||
# browser's wallet ("log out everywhere" on another device, or an admin suspends her). The tab's
|
||||
# bootstrap probe used to answer "signed out": the landing page with Bob still signed in, and no way
|
||||
# off it but closing the tab (signing in as anyone else there kept the tab pinned to Alice).
|
||||
def test_no_header_or_parameter_picks_another_account(client, make_user):
|
||||
# Alice signed in here before Bob: nothing a request carries makes it act as her any more.
|
||||
alice = make_user(email="alice4@example.com", password=_PW)
|
||||
make_user(email="bob4@example.com", password=_PW)
|
||||
assert _login(client, "alice4@example.com", _PW).status_code == 200
|
||||
assert _login(client, "bob4@example.com", _PW).status_code == 200
|
||||
tab = {ACTIVE_ACCOUNT_HEADER: str(alice.id)}
|
||||
assert client.get("/api/feed", headers=tab).status_code == 200
|
||||
if ended == "revoked":
|
||||
alice.session_epoch = (alice.session_epoch or 0) + 1
|
||||
else:
|
||||
alice.is_suspended = True
|
||||
db.commit()
|
||||
me = client.get(f"/api/me?account={alice.id}", headers={"X-Siftlode-Account": str(alice.id)})
|
||||
assert me.json()["email"] == "bob4@example.com"
|
||||
|
||||
assert client.get("/api/me", headers=tab).json()["email"] == "bob4@example.com"
|
||||
assert client.get("/api/feed", headers=tab).status_code == 401 # still nobody, never Bob
|
||||
assert client.get("/api/feed").status_code == 200 # Bob's session is intact
|
||||
|
||||
@pytest.mark.parametrize("path", ["/api/me/accounts", "/api/me/switch"])
|
||||
def test_there_is_no_account_list_or_switch(client, make_user, path):
|
||||
make_user(email="alice5@example.com", password=_PW)
|
||||
assert _login(client, "alice5@example.com", _PW).status_code == 200
|
||||
r = client.get(path) if path.endswith("accounts") else client.post(path, json={"user_id": 1})
|
||||
assert r.status_code in (404, 405)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ended", ["revoked", "suspended", "deactivated"])
|
||||
def test_switching_to_an_account_whose_session_ended_refuses_it_and_drops_it_from_the_list(
|
||||
client, make_user, db, monkeypatch, ended
|
||||
):
|
||||
# The account switcher lists this browser's wallet. A revoked account was dropped from it on a refused
|
||||
# switch, but a suspended one stayed, refused on every click; a deactivated one was not refused at all —
|
||||
# made the default, the next request found it inactive and signed every account here out.
|
||||
monkeypatch.setattr("app.routes.me.is_allowed", lambda _db, _email: True) # both hold an invite
|
||||
alice = make_user(email="alice5@example.com", password=_PW)
|
||||
make_user(email="bob5@example.com", password=_PW)
|
||||
assert _login(client, "alice5@example.com", _PW).status_code == 200
|
||||
assert _login(client, "bob5@example.com", _PW).status_code == 200
|
||||
def test_a_session_whose_account_ended_signs_the_browser_out(client, make_user, db, ended):
|
||||
alice = make_user(email="alice6@example.com", password=_PW)
|
||||
assert _login(client, "alice6@example.com", _PW).status_code == 200
|
||||
if ended == "revoked":
|
||||
alice.session_epoch = (alice.session_epoch or 0) + 1
|
||||
elif ended == "suspended":
|
||||
@@ -121,10 +99,111 @@ def test_switching_to_an_account_whose_session_ended_refuses_it_and_drops_it_fro
|
||||
else:
|
||||
alice.is_active = False
|
||||
db.commit()
|
||||
assert client.get("/api/feed").status_code == 401
|
||||
assert "user_id" not in _session(client) # the refusal ended the session, not just this request
|
||||
assert client.get("/api/me").json() == {"authenticated": False}
|
||||
|
||||
assert client.post("/api/me/switch", json={"user_id": alice.id}).status_code == 403
|
||||
assert [a["email"] for a in client.get("/api/me/accounts").json()] == ["bob5@example.com"]
|
||||
assert client.get("/api/feed").status_code == 200 # Bob is still signed in
|
||||
|
||||
def test_a_cookie_from_the_multi_account_build_signs_in_again(client, make_user):
|
||||
# The old session held a wallet and one epoch per account. It is not read: whoever holds one
|
||||
# signs in once more.
|
||||
alice = make_user(email="alice7@example.com", password=_PW)
|
||||
old = {"user_id": alice.id, "account_ids": [alice.id], "epochs": {str(alice.id): 0}}
|
||||
client.cookies.set("session", _cookie(old))
|
||||
assert client.get("/api/feed").status_code == 401
|
||||
assert client.get("/api/me").json() == {"authenticated": False}
|
||||
|
||||
|
||||
def test_deleting_the_account_signs_the_browser_out(client, make_user):
|
||||
make_user(email="alice8@example.com", password=_PW)
|
||||
assert _login(client, "alice8@example.com", _PW).status_code == 200
|
||||
assert client.delete("/api/me/account").status_code == 200
|
||||
# The cookie names nobody now: the probe answers signed out without looking the account up.
|
||||
assert "user_id" not in _session(client)
|
||||
|
||||
|
||||
def _session(client) -> dict:
|
||||
"""The decoded payload of the client's session cookie."""
|
||||
raw = client.cookies.get("session")
|
||||
if not raw:
|
||||
return {}
|
||||
data = TimestampSigner(str(settings.secret_key)).unsign(raw.encode())
|
||||
return json.loads(base64.b64decode(data))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("signed_in_as", ["starter", "someone_else"])
|
||||
def test_a_google_link_lands_only_on_the_account_still_signed_in(db, make_user, signed_in_as):
|
||||
# The link marker names who started it; a sign-in as someone else before the callback must not
|
||||
# get that Google identity attached to the account that started it from the other person's session.
|
||||
from starlette.requests import Request
|
||||
|
||||
from app.routes.auth import _complete_link
|
||||
|
||||
starter, other = make_user(), make_user()
|
||||
session = {"user_id": starter.id if signed_in_as == "starter" else other.id, "epoch": 0}
|
||||
request = Request({"type": "http", "session": session, "headers": []})
|
||||
resp = _complete_link(request, db, starter.id, True, "sub-123", {"email": ""}, {"scope": ""})
|
||||
db.refresh(starter)
|
||||
if signed_in_as == "starter":
|
||||
assert resp.headers["location"] == "/?link=ok"
|
||||
assert starter.google_sub == "sub-123"
|
||||
else:
|
||||
assert resp.headers["location"] == "/?link=error"
|
||||
assert starter.google_sub is None
|
||||
|
||||
|
||||
def test_no_app_code_reads_a_second_account():
|
||||
# The multi-account machinery is gone; a module reading its session keys or its header again
|
||||
# would bring back a session that names more than one account.
|
||||
app_dir = Path(__file__).parent.parent / "app"
|
||||
hits = [
|
||||
f"{path.relative_to(app_dir)}: {word}"
|
||||
for path in app_dir.rglob("*.py")
|
||||
for word in ("account_ids", '"epochs"', "x-siftlode-account", 'query_params.get("account")')
|
||||
if word in path.read_text(encoding="utf-8").lower()
|
||||
]
|
||||
assert hits == []
|
||||
|
||||
|
||||
def test_a_tab_built_for_the_account_signed_in_before_is_refused(client, make_user):
|
||||
# Two tabs of one browser: one signed in as someone else, so the cookie is Bob's under both. The
|
||||
# tab still showing Alice says so with every request; acting as Bob there would put her player
|
||||
# positions, pref saves and Sign out on his account. Refused, and the session stays Bob's.
|
||||
alice = make_user(email="alice5@example.com", password=_PW)
|
||||
bob = make_user(email="bob5@example.com", password=_PW)
|
||||
assert _login(client, "alice5@example.com", _PW).status_code == 200
|
||||
assert _login(client, "bob5@example.com", _PW).status_code == 200
|
||||
|
||||
stale = client.get("/api/feed", headers={"X-Siftlode-Expect-Account": str(alice.id)})
|
||||
assert stale.status_code == 401
|
||||
assert stale.json()["detail"]["reason"] == "account_changed"
|
||||
assert (
|
||||
client.get("/api/feed", headers={"X-Siftlode-Expect-Account": str(bob.id)}).status_code
|
||||
== 200
|
||||
)
|
||||
assert client.get("/api/feed").status_code == 200
|
||||
# Nor may its Sign out end Bob's session.
|
||||
assert (
|
||||
client.post(
|
||||
"/auth/logout", headers={"X-Siftlode-Expect-Account": str(alice.id)}
|
||||
).status_code
|
||||
== 401
|
||||
)
|
||||
# The probe answers who is signed in now, so the stale tab can reload as Bob.
|
||||
probe = client.get("/api/me", headers={"X-Siftlode-Expect-Account": str(alice.id)})
|
||||
assert probe.json()["email"] == "bob5@example.com"
|
||||
|
||||
|
||||
def test_setting_a_password_keeps_the_caller_signed_in(client, make_user):
|
||||
# The change ends every OTHER session (the epoch moves); the caller is stamped with the new one.
|
||||
make_user(email="dave@example.com", password=_PW)
|
||||
assert _login(client, "dave@example.com", _PW).status_code == 200
|
||||
r = client.post(
|
||||
"/auth/set-password",
|
||||
json={"password": "a brand new passphrase", "current_password": _PW},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
assert client.get("/api/feed").status_code == 200
|
||||
|
||||
|
||||
def test_logout_others_revokes_the_other_session_but_keeps_the_caller(client, make_user):
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""sync/autotag's orphan cleanup (the module was 32% covered).
|
||||
"""The auto-tag job's orphan cleanup (`reader.service.prune_orphan_system_tags`).
|
||||
|
||||
A system tag no channel carries any more is deleted — unless a saved feed view filters on it. The
|
||||
views hold tag ids; a deleted tag returns under a NEW id when a channel earns it again, and the feed
|
||||
@@ -7,8 +7,11 @@ drops an unknown id, so the view silently showed the whole feed under its old na
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models import Channel, ChannelTag, SavedView, Tag
|
||||
from app.sync import autotag
|
||||
from app.catalog.models import Channel
|
||||
from app.db import SessionLocal
|
||||
from app.sync.autotag import run_autotag_all
|
||||
from app.reader.models import ChannelTag, SavedView, Tag
|
||||
from app.reader import service as reader_service
|
||||
|
||||
|
||||
def test_an_orphan_system_tag_goes_but_one_a_saved_view_filters_on_stays(db, make_user):
|
||||
@@ -30,7 +33,8 @@ def test_an_orphan_system_tag_goes_but_one_a_saved_view_filters_on_stays(db, mak
|
||||
db.add(SavedView(user_id=user.id, source="plex", name="P", filters={"genres": ["Drama"]}))
|
||||
db.commit()
|
||||
|
||||
assert autotag._cleanup_orphan_system_tags(db) == 1
|
||||
assert reader_service.prune_orphan_system_tags(db) == 1
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
names = set(db.execute(select(Tag.name)).scalars())
|
||||
assert names == {"Gaming", "Music", "Mine"} # only the unreferenced system orphan went
|
||||
@@ -48,7 +52,8 @@ def test_a_malformed_saved_view_cannot_break_the_cleanup(db, make_user):
|
||||
):
|
||||
db.add(SavedView(user_id=user.id, source="feed", name=f"bad{i}", filters=filters))
|
||||
db.commit()
|
||||
assert autotag._cleanup_orphan_system_tags(db) == 1
|
||||
assert reader_service.prune_orphan_system_tags(db) == 1
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
assert db.execute(select(Tag).where(Tag.name == "Cooking")).first() is None
|
||||
|
||||
@@ -61,4 +66,17 @@ def test_a_view_kept_tag_list_larger_than_a_bind_parameter_limit_still_cleans_up
|
||||
huge = list(range(10_000_000, 10_070_000)) # past psycopg's 65535 bind parameters
|
||||
db.add(SavedView(user_id=user.id, source="feed", name="huge", filters={"tags": huge}))
|
||||
db.commit()
|
||||
assert autotag._cleanup_orphan_system_tags(db) == 1
|
||||
assert reader_service.prune_orphan_system_tags(db) == 1
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_the_job_commits_its_cleanup(db):
|
||||
"""The prune only flushes; the job ends its own transaction. Read from a session of its own,
|
||||
the orphan is gone only once the job committed — the job's session would see its own delete
|
||||
either way."""
|
||||
orphan = Tag(user_id=None, name="Cooking", category="topic")
|
||||
db.add(orphan)
|
||||
db.commit()
|
||||
assert run_autotag_all(db, only_missing=True)["orphan_tags_removed"] == 1
|
||||
with SessionLocal() as fresh:
|
||||
assert fresh.execute(select(Tag).where(Tag.name == "Cooking")).first() is None
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
"""The channel manager's local writes, through HTTP: each one lands and is committed.
|
||||
|
||||
The service tests (`test_catalog_service.py`) hold what each write does; these hold that the route
|
||||
calls it and commits — the `db` fixture fails a test whose handler left a write uncommitted — and
|
||||
what a client sees back. None of these reach YouTube. Needs the DB.
|
||||
"""
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.catalog.models import BlockedChannel, Channel, Subscription
|
||||
from app.reader.models import ChannelTag, Tag
|
||||
|
||||
|
||||
def _seed(db, user):
|
||||
db.add(Channel(id="UCr", title="R"))
|
||||
db.flush()
|
||||
db.add(Subscription(user_id=user.id, channel_id="UCr"))
|
||||
tag = Tag(user_id=user.id, name="Mine")
|
||||
db.add(tag)
|
||||
db.commit()
|
||||
return tag
|
||||
|
||||
|
||||
def test_a_subscriptions_settings_change_and_list_back(db, make_user, as_user):
|
||||
user = make_user()
|
||||
_seed(db, user)
|
||||
client = as_user(user)
|
||||
r = client.patch("/api/channels/UCr", json={"priority": 2, "hidden": True})
|
||||
assert r.status_code == 200, r.text
|
||||
assert (r.json()["priority"], r.json()["hidden"]) == (2, True)
|
||||
listed = client.get("/api/channels").json()
|
||||
assert [(c["id"], c["priority"], c["hidden"]) for c in listed] == [("UCr", 2, True)]
|
||||
r = client.post("/api/sync/deep-all?on=false")
|
||||
assert r.json() == {"ok": True, "updated": 1, "deep_requested": False}
|
||||
|
||||
|
||||
def test_a_block_and_an_unblock(db, make_user, as_user):
|
||||
user = make_user()
|
||||
_seed(db, user)
|
||||
client = as_user(user)
|
||||
assert client.post("/api/channels/UCr/block").json() == {"ok": True, "blocked": "UCr"}
|
||||
assert client.post("/api/channels/UCr/block").status_code == 200 # idempotent
|
||||
assert [c["id"] for c in client.get("/api/channels/blocked/list").json()] == ["UCr"]
|
||||
assert client.get("/api/channels/UCr").json()["blocked"] is True
|
||||
assert client.delete("/api/channels/UCr/block").json() == {"ok": True, "unblocked": "UCr"}
|
||||
db.expire_all()
|
||||
assert db.execute(select(BlockedChannel)).first() is None
|
||||
|
||||
|
||||
def test_a_tag_attached_to_a_channel_and_detached(db, make_user, as_user):
|
||||
user, stranger = make_user(), make_user()
|
||||
tag = _seed(db, user)
|
||||
client = as_user(user)
|
||||
r = client.post("/api/channels/UCr/tags", json={"tag_id": tag.id})
|
||||
assert r.status_code == 200, r.text
|
||||
assert client.post("/api/channels/UCr/tags", json={"tag_id": tag.id}).status_code == 200
|
||||
assert client.get("/api/channels/UCr").json()["tag_ids"] == [tag.id]
|
||||
db.add(Subscription(user_id=stranger.id, channel_id="UCr"))
|
||||
db.commit()
|
||||
# Another user's tag is not theirs to attach: the same 404 as a missing one.
|
||||
r = as_user(stranger).post("/api/channels/UCr/tags", json={"tag_id": tag.id})
|
||||
assert r.json()["detail"] == {"code": "not_found", "reason": "tag"}
|
||||
assert client.delete(f"/api/channels/UCr/tags/{tag.id}").status_code == 200
|
||||
db.expire_all()
|
||||
assert db.execute(select(ChannelTag)).first() is None
|
||||
|
||||
|
||||
def test_the_sync_status_counts_the_callers_subscriptions(db, make_user, as_user):
|
||||
user = make_user()
|
||||
_seed(db, user)
|
||||
body = as_user(user).get("/api/sync/status").json()
|
||||
assert (body["subscriptions"], body["channels_total"]) == (1, 1)
|
||||
mine = as_user(user).get("/api/sync/my-status").json()
|
||||
assert (mine["channels_total"], mine["channels_recent_pending"]) == (1, 1)
|
||||
@@ -0,0 +1,295 @@
|
||||
"""Catalog's service and queries, one use case each, below the HTTP layer.
|
||||
|
||||
The routes are thin callers now, and their own tests stay the contract a client sees. These hold
|
||||
what each write does and leaves on its own: that it flushes and leaves the commit to its caller,
|
||||
that a row of someone else's reads as a missing one, and the rule each write carries (an explored
|
||||
channel flagged only while nobody follows it, a subscription that keeps it, a cleared search that
|
||||
spares what anyone holds). DB-backed (real Postgres).
|
||||
"""
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.catalog import queries, search, service
|
||||
from app.catalog.models import (
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
Video,
|
||||
)
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
|
||||
|
||||
def _channel(db, cid="UCc", **kw):
|
||||
ch = Channel(id=cid, title=kw.pop("title", cid), **kw)
|
||||
db.add(ch)
|
||||
db.commit()
|
||||
return ch
|
||||
|
||||
|
||||
def _video(db, vid, cid="UCc", **kw):
|
||||
v = Video(id=vid, channel_id=cid, published_at=datetime.now(timezone.utc), **kw)
|
||||
db.add(v)
|
||||
db.commit()
|
||||
return v
|
||||
|
||||
|
||||
def _sub(db, user, cid="UCc", **kw):
|
||||
s = Subscription(user_id=user.id, channel_id=cid, **kw)
|
||||
db.add(s)
|
||||
db.commit()
|
||||
return s
|
||||
|
||||
|
||||
def _uncommitted(db):
|
||||
return bool(db.new or db.dirty or db.deleted) or db.in_transaction()
|
||||
|
||||
|
||||
# --- whose a row is ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_users_subscriptions_blocks_and_explorations_are_theirs_alone(db, make_user):
|
||||
owner, stranger = make_user(), make_user()
|
||||
ch = _channel(db)
|
||||
_sub(db, owner)
|
||||
service.explore(db, owner, ch)
|
||||
service.block(db, owner, "UCc")
|
||||
db.commit()
|
||||
assert queries.subscription(db, owner, "UCc") is not None
|
||||
assert queries.subscription(db, stranger, "UCc") is None
|
||||
assert queries.exploration(db, stranger, "UCc") is None
|
||||
assert queries.is_blocked(db, stranger, "UCc") is False
|
||||
assert queries.blocked_ids(db, stranger) == set()
|
||||
assert [s.channel_id for s, _ in queries.subscriptions(db, stranger, "en")] == []
|
||||
|
||||
|
||||
# --- subscription settings -------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_full_history_reports_only_the_switch_on(db, make_user):
|
||||
user = make_user()
|
||||
_channel(db)
|
||||
sub = _sub(db, user)
|
||||
assert service.update_subscription(db, sub, priority=3) is False
|
||||
assert service.update_subscription(db, sub, deep_requested=True) is True
|
||||
assert service.update_subscription(db, sub, deep_requested=True) is False # already on
|
||||
assert _uncommitted(db)
|
||||
db.commit()
|
||||
assert (sub.priority, sub.hidden, sub.deep_requested) == (3, False, True)
|
||||
|
||||
|
||||
def test_full_history_for_all_moves_only_the_callers_rows(db, make_user):
|
||||
user, other = make_user(), make_user()
|
||||
_channel(db)
|
||||
_sub(db, user)
|
||||
theirs = _sub(db, other)
|
||||
assert service.set_deep_all(db, user, True) == 1
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
assert theirs.deep_requested is False
|
||||
|
||||
|
||||
def test_a_reset_asks_every_subscriber_for_full_history(db, make_user):
|
||||
a, b = make_user(), make_user()
|
||||
ch = _channel(db, backfill_done=True, recent_synced_at=datetime.now(timezone.utc))
|
||||
_sub(db, a)
|
||||
_sub(db, b)
|
||||
service.reset_backfill(db, ch)
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
assert (ch.backfill_done, ch.recent_synced_at) == (False, None)
|
||||
assert set(db.scalars(select(Subscription.deep_requested))) == {True}
|
||||
|
||||
|
||||
# --- explore, subscribe, block ---------------------------------------------------------------
|
||||
|
||||
|
||||
def test_an_exploration_is_one_row_and_flags_the_channel_only_while_nobody_follows_it(
|
||||
db, make_user
|
||||
):
|
||||
explorer, follower = make_user(), make_user()
|
||||
lone = _channel(db, "UClone")
|
||||
kept = _channel(db, "UCkept")
|
||||
_sub(db, follower, "UCkept")
|
||||
service.explore(db, explorer, lone)
|
||||
service.explore(db, explorer, kept)
|
||||
db.commit()
|
||||
first = queries.exploration(db, explorer, "UClone").created_at
|
||||
service.explore(db, explorer, lone) # refreshes the grace clock
|
||||
db.commit()
|
||||
assert db.scalar(select(ExploredChannel.id).where(ExploredChannel.channel_id == "UClone"))
|
||||
assert len(db.scalars(select(ExploredChannel.id)).all()) == 2
|
||||
assert queries.exploration(db, explorer, "UClone").created_at >= first
|
||||
assert (lone.from_explore, kept.from_explore) == (True, False)
|
||||
|
||||
|
||||
def test_subscribing_keeps_an_explored_channel_and_its_videos(db, make_user):
|
||||
user = make_user()
|
||||
ch = _channel(db, from_explore=True)
|
||||
v = _video(db, "v1", via_explore=True)
|
||||
service.record_subscription(db, user, ch, "")
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
sub = queries.subscription(db, user, "UCc")
|
||||
assert (sub.yt_subscription_id, ch.from_explore, v.via_explore) == (None, False, False)
|
||||
service.drop_subscription(db, sub)
|
||||
db.commit()
|
||||
assert queries.subscription(db, user, "UCc") is None
|
||||
|
||||
|
||||
def test_a_block_is_once_and_ends_the_exploration(db, make_user):
|
||||
user = make_user()
|
||||
ch = _channel(db)
|
||||
service.explore(db, user, ch)
|
||||
db.commit()
|
||||
service.block(db, user, "UCc")
|
||||
service.block(db, user, "UCc")
|
||||
db.commit()
|
||||
assert len(db.scalars(select(BlockedChannel.id)).all()) == 1
|
||||
assert queries.exploration(db, user, "UCc") is None
|
||||
assert [c.id for c in queries.blocked_channels(db, user, "en")] == ["UCc"]
|
||||
service.unblock(db, user, "UCc")
|
||||
db.commit()
|
||||
assert queries.is_blocked(db, user, "UCc") is False
|
||||
|
||||
|
||||
# --- what the status cards count -------------------------------------------------------------
|
||||
|
||||
|
||||
def test_my_progress_counts_only_my_channels_and_my_unhidden_videos(db, make_user):
|
||||
user, other = make_user(), make_user()
|
||||
now = datetime.now(timezone.utc)
|
||||
_channel(db, "UCa", details_synced_at=now, recent_synced_at=now, backfill_done=True)
|
||||
_channel(db, "UCb")
|
||||
_channel(db, "UCtheirs")
|
||||
_video(db, "a1", "UCa")
|
||||
_video(db, "b1", "UCb")
|
||||
_video(db, "t1", "UCtheirs")
|
||||
_sub(db, user, "UCa", deep_requested=True)
|
||||
_sub(db, user, "UCb", hidden=True)
|
||||
_sub(db, other, "UCtheirs")
|
||||
assert queries.my_progress(db, user) == (2, 1, 1, 1)
|
||||
assert [c.id for c in queries.my_deep_requested(db, user)] == ["UCa"]
|
||||
assert queries.my_video_count(db, user) == 1
|
||||
assert queries.video_total(db) == 3
|
||||
assert queries.catalog_totals(db, user)["subscriptions"] == 2
|
||||
|
||||
|
||||
def test_a_channel_with_no_title_is_left_out_of_the_titles(db):
|
||||
_channel(db, "UCt", title="Titled")
|
||||
db.add(Channel(id="UCn", title=None))
|
||||
db.commit()
|
||||
assert queries.channel_titles(db, {"UCt", "UCn", "UCgone"}) == {"UCt": "Titled"}
|
||||
assert queries.channel_title(db, "UCt") == "Titled"
|
||||
|
||||
|
||||
def test_the_deep_backlog_is_the_unfinished_channels_someone_asked_for(db, make_user):
|
||||
user = make_user()
|
||||
_channel(db, "UCwant")
|
||||
_channel(db, "UCdone", backfill_done=True)
|
||||
_channel(db, "UCnobody")
|
||||
_sub(db, user, "UCwant", deep_requested=True)
|
||||
_sub(db, user, "UCdone", deep_requested=True)
|
||||
_sub(db, user, "UCnobody")
|
||||
assert [c.id for c in queries.deep_backlog(db)] == ["UCwant"]
|
||||
assert queries.deep_queued(db, ["UCwant", "UCdone", "UCnobody"]) == {"UCwant", "UCdone"}
|
||||
|
||||
|
||||
# --- live search -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_cleared_search_spares_what_anyone_holds(db, make_user):
|
||||
user, other, follower = make_user(), make_user(), make_user()
|
||||
_channel(db, "UCs")
|
||||
_channel(db, "UCfollowed")
|
||||
_sub(db, follower, "UCfollowed")
|
||||
for vid in ("loose", "watched", "found_by_other"):
|
||||
_video(db, vid, "UCs", via_search=True)
|
||||
_video(db, "followed", "UCfollowed", via_search=True)
|
||||
_video(db, "organic", "UCs")
|
||||
ids = ["loose", "watched", "found_by_other", "followed", "organic"]
|
||||
search.record_finds(db, user, ids, "cat")
|
||||
search.record_finds(db, user, ids, "cat") # idempotent
|
||||
search.record_finds(db, other, ["found_by_other"], "dog")
|
||||
reader_service.mark(db, other, "watched", "watched")
|
||||
db.commit()
|
||||
assert search.clear(db, user, ids) == 1
|
||||
db.commit()
|
||||
left = set(db.scalars(select(Video.id)))
|
||||
assert left == {"watched", "found_by_other", "followed", "organic"}
|
||||
assert db.scalars(select(SearchFind.user_id)).all() == [other.id]
|
||||
|
||||
|
||||
def test_a_search_puts_what_the_user_already_has_last(db, make_user):
|
||||
user = make_user()
|
||||
_channel(db, "UCs")
|
||||
_channel(db, "UCmine")
|
||||
_sub(db, user, "UCmine")
|
||||
for vid, cid in (("new1", "UCs"), ("mine", "UCmine"), ("seen", "UCs"), ("new2", "UCs")):
|
||||
_video(db, vid, cid)
|
||||
reader_service.mark(db, user, "seen", "hidden")
|
||||
db.commit()
|
||||
assert search.new_first(db, user, ["mine", "new1", "seen", "new2"]) == [
|
||||
"new1",
|
||||
"new2",
|
||||
"mine",
|
||||
"seen",
|
||||
]
|
||||
|
||||
|
||||
# --- what Catalog asks Reader ----------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_users_tag_links_are_made_once_and_only_theirs_are_read(db, make_user):
|
||||
user, other = make_user(), make_user()
|
||||
_channel(db)
|
||||
mine = reader_service.create_tag(db, user, "Mine", "other", None)
|
||||
theirs = reader_service.create_tag(db, other, "Theirs", "other", None)
|
||||
reader_service.attach_user_tag(db, user, "UCc", mine.id)
|
||||
reader_service.attach_user_tag(db, user, "UCc", mine.id)
|
||||
reader_service.attach_user_tag(db, other, "UCc", theirs.id)
|
||||
db.commit()
|
||||
assert reader_queries.user_tag_links(db, user) == {"UCc": [mine.id]}
|
||||
assert reader_queries.user_tag_links(db, user, "UCelse") == {}
|
||||
reader_service.detach_user_tag(db, user, "UCc", mine.id)
|
||||
db.commit()
|
||||
assert reader_queries.user_tag_links(db, user) == {}
|
||||
assert reader_queries.user_tag_links(db, other) == {"UCc": [theirs.id]}
|
||||
|
||||
|
||||
def test_discovery_offers_the_playlist_channels_not_followed_and_not_ones_own(db, make_user):
|
||||
user = make_user()
|
||||
for cid in ("UCnew", "UCfollowed", "UCown"):
|
||||
_channel(db, cid)
|
||||
for vid, cid in (("n1", "UCnew"), ("n2", "UCnew"), ("f1", "UCfollowed"), ("o1", "UCown")):
|
||||
_video(db, vid, cid)
|
||||
_sub(db, user, "UCfollowed")
|
||||
pl = reader_service.create_playlist(db, user, "L")
|
||||
for vid in ("n1", "n2", "f1", "o1"):
|
||||
reader_service.add_item(db, pl, vid, mark_dirty=False)
|
||||
db.commit()
|
||||
rows = reader_queries.playlist_channels(db, user, "UCown", "en")
|
||||
assert [(ch.id, n, lists) for ch, n, lists in rows] == [("UCnew", 2, 1)]
|
||||
|
||||
|
||||
def test_an_exploration_older_than_its_grace_is_still_one_row(db, make_user):
|
||||
"""The grace clock is refreshed in place: re-exploring after the cutoff does not add a row."""
|
||||
user = make_user()
|
||||
ch = _channel(db)
|
||||
db.add(
|
||||
ExploredChannel(
|
||||
user_id=user.id,
|
||||
channel_id="UCc",
|
||||
created_at=datetime.now(timezone.utc) - timedelta(days=30),
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
service.explore(db, user, ch)
|
||||
db.commit()
|
||||
rows = db.scalars(select(ExploredChannel)).all()
|
||||
assert len(rows) == 1
|
||||
assert rows[0].created_at > datetime.now(timezone.utc) - timedelta(minutes=1)
|
||||
@@ -12,7 +12,8 @@ import app.state as state
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.main import app
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
from app.sync import runner, videos
|
||||
|
||||
_PUB = datetime(2020, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
@@ -10,7 +10,7 @@ from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from app.models import Channel, Subscription
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.routes import channels
|
||||
from app.sync import subscriptions
|
||||
from app.sync.subscriptions import DETAILS_MAX_AGE, import_subscriptions
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user