Compare commits
19
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
909732b3c1 | ||
|
|
215f022fe3 | ||
|
|
6ea750a698 | ||
|
|
c5f121139b | ||
|
|
b04b2cbd62 | ||
|
|
3a5212cd8a | ||
|
|
b376373af5 | ||
|
|
8206384c91 | ||
|
|
e27656ee7e | ||
|
|
d73cdcee97 | ||
|
|
5ce292cd28 | ||
|
|
ad6ccbac05 | ||
|
|
531eb2eef1 | ||
|
|
2cf1b688d9 | ||
|
|
3b0e365e26 | ||
|
|
533571f572 | ||
|
|
4b6db47083 | ||
|
|
5229fd96be | ||
|
|
238e6f8a61 |
@@ -7,6 +7,17 @@ edit the release notes and re-run the generator.
|
||||
|
||||
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
|
||||
|
||||
## 0.108.1 — 2026-10-20
|
||||
|
||||
### New
|
||||
|
||||
- Development: the backend is split into four units (downloads, reader, catalog, Plex), each owning its tables. A contract test holds what each one reads and writes, and that every query names whose rows it asks for.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Watch later: saving the first two videos at once no longer makes two lists. An existing second list is merged into the first, its videos and its comfort-shelf pin with it.
|
||||
- Playback: the first resume point of a video or a Plex item no longer fails when it is saved together with the watched mark, a whole season marked at once, or a Plex sync.
|
||||
|
||||
## 0.108.0 — 2026-10-18
|
||||
|
||||
### New
|
||||
|
||||
@@ -63,9 +63,16 @@ how it gets built. Concretely:
|
||||
## Architecture shape (don't fight it)
|
||||
|
||||
`routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
|
||||
`downloads/`, `reader/`, `discover/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`downloads/`, `reader/`, `discover/`, `catalog/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `ratelimit`, `bodylimit`, `progress`, `collation`,
|
||||
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
|
||||
The domain is four **units**, each owning its tables: Capture (`downloads/`), Reader (`reader/`, `discover/`, the playlist
|
||||
sync), Catalog (`catalog/`, the `sync/` ingests) and Plex (`plex/`). A unit's route does no database work: it reads through
|
||||
the unit's `queries`, writes through its `service` (which flushes) and commits. A unit's code reads another unit's tables
|
||||
only along a declared edge and writes them only through that unit's service; core reaches a unit's tables through its
|
||||
`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`) — as far as an AST sees: what it
|
||||
cannot (an ORM write on another unit's loaded row, SQL in `text()`, a commit by `begin()`) is review's to catch, listed in
|
||||
[`backend/app/README.md`](backend/app/README.md) with the conventions.
|
||||
Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
|
||||
entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
|
||||
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
|
||||
@@ -82,36 +89,36 @@ CQRS, a repository/DI framework, a router library, a generic CRUD-router factory
|
||||
|
||||
## Canonical patterns — when you have a choice, pick THIS
|
||||
|
||||
| Area | Use | Not |
|
||||
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; helpers `flush` (test-held list) | helpers that commit (only `quota` and `jobruns` own their session) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
|
||||
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
|
||||
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
|
||||
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
|
||||
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
|
||||
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
|
||||
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect | a bare `catch {}` or `notify()` with no error attached |
|
||||
| FE floating or full-screen layer | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers") | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]` |
|
||||
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
|
||||
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
|
||||
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
|
||||
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
|
||||
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback` | a render-phase `ref.current = x` — the React Compiler skips the component |
|
||||
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
|
||||
| FE popover / scroll-fade ref | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}` | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips |
|
||||
| FE async handler with cleanup | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
|
||||
| Area | Use | Not |
|
||||
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404`. A composite rule (own or shared, built-in or own) says so in its `queries` | an inline owner filter — `Model.user_id == …`, an alias's, `.in_()`, `filter_by(user_id=…)` (fence-held, every module) — or a `db.get` and a compare where `owned_row` fits |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; a unit's `service` and every helper `flush` (fence-held, every module) | a helper that commits — only a listed one (what owns its session, or a core step that must land whatever its caller does next), each with its reason: the fence's `UNIT_COMMITS` / `CORE_COMMITS`, a route's `COMMITTING_HELPERS` (`test_route_conventions`) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
|
||||
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
|
||||
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
|
||||
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
|
||||
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
|
||||
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
|
||||
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect | a bare `catch {}` or `notify()` with no error attached |
|
||||
| FE floating or full-screen layer | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers") | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]` |
|
||||
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
|
||||
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
|
||||
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
|
||||
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
|
||||
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback` | a render-phase `ref.current = x` — the React Compiler skips the component |
|
||||
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
|
||||
| FE popover / scroll-fade ref | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}` | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips |
|
||||
| FE async handler with cleanup | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
|
||||
|
||||
Not repeated here, because a lint rule refuses the wrong one and its message names the right one:
|
||||
buttons (`ui/Button`, `IconButton`, `Pressable`, `Chip`, `DragHandle`), text fields and native
|
||||
|
||||
@@ -640,48 +640,6 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/channels.py": [
|
||||
{
|
||||
"code": "reportReturnType",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 11,
|
||||
"lineCount": 6
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 49,
|
||||
"endColumn": 56,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 38,
|
||||
"endColumn": 44,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 33,
|
||||
"endColumn": 39,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/discover.py": [
|
||||
{
|
||||
"code": "reportInvalidTypeForm",
|
||||
@@ -692,121 +650,7 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/browse.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 61,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 61,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 13,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 13,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 15,
|
||||
"endColumn": 17,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 28,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 15,
|
||||
"endColumn": 17,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 28,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/collections.py": [
|
||||
{
|
||||
"code": "reportAssignmentType",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 39,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 76,
|
||||
"endColumn": 80,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 75,
|
||||
"endColumn": 79,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 80,
|
||||
"endColumn": 84,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
@@ -864,117 +708,7 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/discover.py": [
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 4,
|
||||
"endColumn": 63,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 18,
|
||||
"endColumn": 48,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 4,
|
||||
"endColumn": 63,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 18,
|
||||
"endColumn": 48,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/playback.py": [
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 6,
|
||||
"endColumn": 14,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 51,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 51,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/playlists.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 77,
|
||||
"endColumn": 83,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 77,
|
||||
"endColumn": 83,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 10,
|
||||
"endColumn": 18,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/scheduler.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 55,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
@@ -992,42 +726,6 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/search.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 31,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 6,
|
||||
"endColumn": 14,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/sync.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 65,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 50,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/sync/explore.py": [
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""One Watch later list per user, held by the database
|
||||
|
||||
Revision ID: 0112_one_watch_later
|
||||
Revises: 0111_audit_request_id
|
||||
Create Date: 2026-10-20
|
||||
|
||||
Watch later was made on a user's first save by a read then an insert, so two first saves at once
|
||||
each made a list, and the reader then picked between them at random. A partial unique index holds
|
||||
"one per user" now. An instance that already has a second list keeps the oldest: the others' videos
|
||||
move into it (after its own, in their order; one it already holds is not added twice), a Home pin
|
||||
of one of them moves to it (unless it is pinned itself; of several, the first on the shelf), then
|
||||
they go — a pin left on them goes with them, as the kept list already holds one.
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0112_one_watch_later"
|
||||
down_revision: str | None = "0111_audit_request_id"
|
||||
branch_labels: str | None = None
|
||||
depends_on: str | None = None
|
||||
|
||||
_EXTRA = """
|
||||
SELECT p.id, k.keep_id
|
||||
FROM playlists p
|
||||
JOIN (
|
||||
SELECT user_id, min(id) AS keep_id FROM playlists WHERE kind = 'watch_later' GROUP BY user_id
|
||||
) k ON k.user_id = p.user_id
|
||||
WHERE p.kind = 'watch_later' AND p.id <> k.keep_id
|
||||
"""
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
f"""
|
||||
INSERT INTO playlist_items (playlist_id, video_id, position, added_at)
|
||||
SELECT e.keep_id, i.video_id,
|
||||
(SELECT coalesce(max(position), -1) FROM playlist_items WHERE playlist_id = e.keep_id)
|
||||
+ row_number() OVER (PARTITION BY e.keep_id ORDER BY i.playlist_id, i.position, i.id),
|
||||
i.added_at
|
||||
FROM playlist_items i
|
||||
JOIN ({_EXTRA}) e ON e.id = i.playlist_id
|
||||
ON CONFLICT ON CONSTRAINT uq_playlist_video DO NOTHING
|
||||
"""
|
||||
)
|
||||
op.execute(
|
||||
f"""
|
||||
UPDATE comfort_pins c SET playlist_id = m.keep_id
|
||||
FROM (
|
||||
SELECT DISTINCT ON (e.keep_id) p.id AS pin_id, e.keep_id
|
||||
FROM comfort_pins p
|
||||
JOIN ({_EXTRA}) e ON e.id = p.playlist_id
|
||||
WHERE NOT EXISTS (SELECT 1 FROM comfort_pins k WHERE k.playlist_id = e.keep_id)
|
||||
ORDER BY e.keep_id, p.position, p.id
|
||||
) m
|
||||
WHERE c.id = m.pin_id
|
||||
"""
|
||||
)
|
||||
op.execute(f"DELETE FROM playlists WHERE id IN (SELECT id FROM ({_EXTRA}) e)")
|
||||
op.create_index(
|
||||
"uq_playlist_watch_later",
|
||||
"playlists",
|
||||
["user_id"],
|
||||
unique=True,
|
||||
postgresql_where=sa.text("kind = 'watch_later'"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("uq_playlist_watch_later", table_name="playlists")
|
||||
+56
-16
@@ -1,9 +1,8 @@
|
||||
# Backend conventions
|
||||
|
||||
The rules below are already followed across `backend/app` — they lived only in a few module
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
|
||||
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
|
||||
I copy".
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them. When you add code,
|
||||
match these so the next reader doesn't have to guess "which pattern do I copy".
|
||||
|
||||
## Module layering
|
||||
|
||||
@@ -13,12 +12,13 @@ Three layers, by *who* runs the code — not by feature:
|
||||
response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
|
||||
Business logic that a request happens to trigger does **not** live here if anything else runs it too.
|
||||
|
||||
2. **`plex/`, `downloads/`, `sync/`, `issues/` — domain logic shared by more than one caller.** Anything the
|
||||
**scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
|
||||
route, the worker, and the scheduler share one definition. Examples from this codebase:
|
||||
`downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
|
||||
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline;
|
||||
`issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
|
||||
2. **The units and the other domain packages — logic shared by more than one caller.** A unit's
|
||||
database work (below), and anything the **scheduler or the worker** runs (not just a request), lives
|
||||
in a domain package, so the route, the worker, and the scheduler share one definition. Examples from
|
||||
this codebase: `downloads/service.py` owns enqueue + the job state transitions (routes call it);
|
||||
`plex/sync.py` is run by both the admin route and the scheduler; `downloads/pipeline.py` is the
|
||||
worker's child pipeline; `issues/bundle.py` is the one definition of a bug report's files, for the
|
||||
zip export and the tracker.
|
||||
|
||||
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's
|
||||
folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every
|
||||
@@ -34,19 +34,59 @@ Three layers, by *who* runs the code — not by feature:
|
||||
a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
|
||||
pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
|
||||
|
||||
## Units
|
||||
|
||||
The domain is four units, each owning its tables (`models.py` in its package):
|
||||
|
||||
| Unit | Package | Its routes |
|
||||
| --- | --- | --- |
|
||||
| Capture | `downloads/`, the worker | `downloads.py`, `files.py`, `public.py` |
|
||||
| Reader | `reader/`, `discover/`, `feedquery.py`, the playlist sync and tag suggestions in `sync/` | `feed.py`, `playlists.py`, `saved_views.py`, `discover.py`, `tags.py` |
|
||||
| Catalog | `catalog/`, the ingests in `sync/` | `channels.py`, `search.py`, `sync.py` |
|
||||
| Plex | `plex/` | `plex/` |
|
||||
|
||||
- **A unit's route does no database work.** It validates, reads through the unit's `queries.py`,
|
||||
writes through its `service.py`, serializes, and commits. It may name a unit's model as a type
|
||||
(under `TYPE_CHECKING`), never use one as a value, and it passes a kind (`"movie"`), not a model, to
|
||||
a query.
|
||||
- **A unit's code keeps to its edges.** It reads another unit's tables only along a declared edge
|
||||
(Capture and Reader read Catalog's and Plex's: the feed joins `Video` with `VideoState` in one
|
||||
query) and writes them only through the owner's service. Core code reaches a unit's tables through
|
||||
that unit's `queries`.
|
||||
- **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id,
|
||||
user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the
|
||||
same answer as a missing one). A unit's `queries.own_x` returns None and its route answers
|
||||
`not_found`; a core route calls `owned_or_404`. A rule `owned_row` cannot say — a job its owner
|
||||
shared, a built-in profile nobody owns, a folder read under `FOR SHARE` — is written out in the
|
||||
unit's own code, beside the reason.
|
||||
|
||||
All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's
|
||||
packages and edges (`UNIT_PACKAGES`, `READS`) and what may commit (`UNIT_COMMITS`, `CORE_COMMITS`)
|
||||
are listed there. It reads the code's AST, so it holds the forms written here: an owner column
|
||||
compared inline (a model's, an import alias's or an `aliased()` one's, by `==`, `!=`, `.in_()`, or a
|
||||
`filter_by(user_id=…)`), a model built or an `insert`/`update`/`delete` (under any import name, of a
|
||||
model or its `__table__`) across units, and a `.commit()` call off the lists all fail it. What an AST
|
||||
cannot tell is review's to catch: an ORM write on another unit's loaded row (an attribute, a
|
||||
`db.delete`), SQL in `text()`, a column or model reached another way (bound to a variable, through
|
||||
`__table__.c` or a module path), and a commit made by a `begin()` block or a `commit` handed in.
|
||||
|
||||
## Transaction ownership
|
||||
|
||||
This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
|
||||
(only `quota` and `jobruns` own their session)"** — so read it that way, not as an exception to it. The commit belongs
|
||||
to the **operation owner**; anything called *inside* another transaction only stages its change
|
||||
This follows CLAUDE.md's canonical rule — **the handler commits; a unit's `service` and every helper
|
||||
flush** — so read it that way, not as an exception to it. The commit belongs to the **operation
|
||||
owner**; anything called *inside* another transaction only stages its change
|
||||
(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
|
||||
`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
|
||||
mutation it records, so an audit entry can never exist without its change (or vice-versa).
|
||||
|
||||
- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
|
||||
whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
|
||||
a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
|
||||
`jobruns`' run rows — the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
|
||||
- The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it
|
||||
makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the
|
||||
request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, the
|
||||
scheduler's jobs, a unit's jobs (the worker's `jobdb` writes, the syncs and ingests that commit per
|
||||
page or batch so a later failure loses nothing already stored), and a few core steps that must land
|
||||
whatever their caller does next (a token's last-used stamp, a refreshed YouTube grant) — the fence's
|
||||
`UNIT_COMMITS` and `CORE_COMMITS` name each with its reason, `test_route_conventions` a route's own
|
||||
committing helpers. Everything else flushes.
|
||||
- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
|
||||
loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
|
||||
attribute — see `service.decr_ref_count`.
|
||||
|
||||
@@ -26,6 +26,7 @@ from fastapi import Depends, Request
|
||||
from sqlalchemy import delete, or_, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -223,7 +224,9 @@ def revoke_all(db: Session, user_id: int) -> int:
|
||||
change, "log out everywhere", a revoked untrusted password — ends its tokens too: one minted
|
||||
from a stolen session must not outlive the owner's answer to that theft. Returns how many."""
|
||||
gone = (
|
||||
db.execute(delete(ApiToken).where(ApiToken.user_id == user_id).returning(ApiToken.id))
|
||||
db.execute(
|
||||
delete(ApiToken).where(owned_by(ApiToken.user_id, user_id)).returning(ApiToken.id)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
@@ -110,10 +110,9 @@ def record(
|
||||
|
||||
def gc(db: Session, retention_days: int) -> int:
|
||||
"""Delete audit rows older than `retention_days`. Returns the number removed. A
|
||||
retention_days <= 0 disables pruning (keep forever)."""
|
||||
retention_days <= 0 disables pruning (keep forever). Flushes; the job commits."""
|
||||
if retention_days <= 0:
|
||||
return 0
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
|
||||
result = db.execute(delete(AuditLog).where(AuditLog.created_at < cutoff))
|
||||
db.commit()
|
||||
return result.rowcount or 0
|
||||
|
||||
+2
-1
@@ -4,6 +4,7 @@ from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.requests import HTTPConnection
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import debugsession
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -230,7 +231,7 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
|
||||
# request's access line, a queued row, a later background task would otherwise put rows back
|
||||
# under the id after this DELETE (logs.forget_user).
|
||||
logs.forget_user(user_id)
|
||||
db.execute(delete(LogEvent).where(LogEvent.user_id == user_id))
|
||||
db.execute(delete(LogEvent).where(owned_by(LogEvent.user_id, user_id)))
|
||||
# Every canonically-equal invite goes, not just the literal spelling: is_allowed matches on the
|
||||
# canonical form, so leaving a sibling row behind would keep granting access to a mailbox whose
|
||||
# account was just erased — and prod really does carry approved pairs for one mailbox.
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
"""Catalog's tables: the shared YouTube world every user reads — channels and their videos — and
|
||||
each user's footprint in it: what they subscribe to, block, explored or found by search. Only this
|
||||
unit writes them; another unit reads them along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
Computed,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.models import User
|
||||
|
||||
|
||||
class Channel(Base, TimestampMixin):
|
||||
"""A YouTube channel. Shared across all users (one channel's videos are the same
|
||||
for everyone), so its expensive metadata is fetched and stored only once."""
|
||||
|
||||
__tablename__ = "channels"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(32), primary_key=True) # "UC..." channel id
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
handle: Mapped[str | None] = mapped_column(String(255))
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
uploads_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
subscriber_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
video_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Channel "About" metadata (fetched with the same channels.list call, no extra quota):
|
||||
# lifetime view count, the channel's join date, banner image, and external links.
|
||||
total_view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
banner_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
external_links: Mapped[list | None] = mapped_column(JSON)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
country: Mapped[str | None] = mapped_column(String(8))
|
||||
# Creator keyword tags (brandingSettings.channel.keywords) — a single space-separated string,
|
||||
# multi-word tags quoted; the client parses it into chips for the About tab.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
# Sync bookkeeping.
|
||||
details_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
recent_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_rss_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# When the newest uploads were last read through the Data API because the RSS feed would not
|
||||
# answer (`sync/runner.py` `run_feed_fallback`): the once-per-`rss_fallback_hours` clock.
|
||||
feed_fallback_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
backfill_cursor: Mapped[str | None] = mapped_column(String(128))
|
||||
backfill_done: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this channel only entered the catalog because a live YouTube search
|
||||
# surfaced one of its videos (nobody subscribes to it / it has no organic upload sync).
|
||||
# Lets the Library hide search-only "noise" by default.
|
||||
from_search: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
# True when this channel exists only because a user EXPLORED it (opened its page without
|
||||
# subscribing) — an ephemeral, per-explorer addition. The cleanup job hard-deletes such a
|
||||
# channel (and its via_explore videos) once nobody keeps it (no subscription, no live
|
||||
# ExploredChannel row, no per-user state) after a grace period. Cleared when someone
|
||||
# subscribes (the "keep" signal promotes it to organic).
|
||||
from_explore: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
|
||||
videos: Mapped[list["Video"]] = relationship(back_populates="channel")
|
||||
subscriptions: Mapped[list["Subscription"]] = relationship(back_populates="channel")
|
||||
|
||||
|
||||
class Subscription(Base, TimestampMixin):
|
||||
"""Per-user link to a channel, with the user's own priority/visibility overrides."""
|
||||
|
||||
__tablename__ = "subscriptions"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# YouTube's subscription resource id, needed to unsubscribe via the API.
|
||||
yt_subscription_id: Mapped[str | None] = mapped_column(String(128))
|
||||
priority: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
hidden: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Per-user opt-in to full-history (deep) backfill of this channel. Default off so a
|
||||
# new user's unique channels don't trigger a big shared-quota burst; the deep
|
||||
# scheduler only picks up channels at least one user has requested.
|
||||
deep_requested: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
subscribed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
user: Mapped["User"] = relationship(back_populates="subscriptions")
|
||||
channel: Mapped["Channel"] = relationship(back_populates="subscriptions")
|
||||
|
||||
|
||||
# Video.live_status values that mean "currently a live/upcoming broadcast" — transient states
|
||||
# hidden from feeds/search and revisited by the refresh job until they settle into a VOD.
|
||||
LIVE_OR_UPCOMING = ("live", "upcoming")
|
||||
|
||||
|
||||
class Video(Base, TimestampMixin):
|
||||
"""A YouTube video. Shared across users; per-user state lives elsewhere."""
|
||||
|
||||
__tablename__ = "videos"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(16), primary_key=True) # video id
|
||||
# No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
|
||||
# (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
|
||||
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
|
||||
# sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
|
||||
# Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
|
||||
# Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
|
||||
channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
|
||||
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
# NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
|
||||
# forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
|
||||
# that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
|
||||
# PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
|
||||
# the real date); the ix_videos_published_at index stays for the `oldest` sort.
|
||||
published_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
|
||||
)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
# The feed's views / duration / title sorts each ride a migration-defined index in their exact
|
||||
# ORDER BY, one per direction (and per ICU collation for title): migration 0096, held to
|
||||
# `feedquery.order_by` by test_feed_sort_indexes.
|
||||
duration_seconds: Mapped[int | None] = mapped_column(Integer)
|
||||
view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
like_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
category_id: Mapped[int | None] = mapped_column(Integer)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
detected_language: Mapped[str | None] = mapped_column(String(16))
|
||||
# Creator-supplied keyword tags (snippet.tags joined by spaces) — fetched for free with the
|
||||
# snippet we already request. Indexed into search_vector so the feed search matches the
|
||||
# uploader's own keywords, not just words in the title.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
# Accumulated distinct live-search queries that surfaced this video (across all users) —
|
||||
# folds YouTube's own relevance judgement into the local search: a video YouTube returned
|
||||
# for "ti amo magyarul" becomes findable by that query locally even if its title lacks it.
|
||||
search_terms: Mapped[str | None] = mapped_column(Text)
|
||||
# No btree here or on the flag columns below: migration 0069 (PERF-3) dropped the never-picked
|
||||
# boolean btrees and replaced the useful ones with partial indexes over the target subset
|
||||
# (via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — migration-defined, like the
|
||||
# search_vector GIN.
|
||||
is_short: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Whether the youtube.com/shorts/<id> probe has run for this video.
|
||||
shorts_probed: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video first entered the catalog via a live YouTube search (not an
|
||||
# organic channel/playlist sync). Used to hide search-discovered "noise" from the
|
||||
# Library by default; left False by the organic ingest paths.
|
||||
via_search: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video entered the catalog because a user EXPLORED its (un-subscribed)
|
||||
# channel. Like via_search it's hidden from the shared Library by default, but it's
|
||||
# additionally PER-USER private: a via_explore video is only visible to users who
|
||||
# explored or subscribe to its channel (a gate in the feed query), so one user's
|
||||
# curiosity never floods everyone else's catalog. Left False by the organic ingest paths.
|
||||
via_explore: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# none | live | upcoming | premiere | was_live
|
||||
live_status: Mapped[str] = mapped_column(
|
||||
String(16), default="none", server_default="none", index=True
|
||||
)
|
||||
enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# videos.list?part=status fields (populated during enrichment); used by the
|
||||
# maintenance/validation job to decide whether a video is playable anywhere.
|
||||
embeddable: Mapped[bool | None] = mapped_column(Boolean)
|
||||
privacy_status: Mapped[str | None] = mapped_column(String(16)) # public|unlisted|private
|
||||
upload_status: Mapped[str | None] = mapped_column(
|
||||
String(16)
|
||||
) # processed|failed|rejected|deleted
|
||||
# Maintenance/validation lifecycle. last_checked_at drives the rolling re-validation
|
||||
# (least-recently-checked first); unavailable_since marks a video as currently
|
||||
# unplayable (hidden from the feed immediately, hard-deleted after a grace period);
|
||||
# unavailable_reason records why (deleted|private|paywalled|abandoned|stuck_live).
|
||||
# last_checked_at: no plain btree — migration 0069 (PERF-18) replaced it with the partial
|
||||
# ix_videos_recheck_order (last_checked_at NULLS FIRST WHERE unavailable_since IS NULL) that the
|
||||
# revalidate job's exact ORDER BY rides; unavailable_since is a partial WHERE IS NOT NULL (PERF-3).
|
||||
last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_reason: Mapped[str | None] = mapped_column(String(24))
|
||||
|
||||
# Weighted full-text search document (DB-generated, never written by the app): title (A) >
|
||||
# creator keywords + search queries (B) > truncated description (C). ts_rank honours the
|
||||
# weights so title matches outrank description matches. Backed by a GIN index (migration
|
||||
# 0032). The accent-insensitive `unaccent_simple` config comes from migration 0031.
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(keywords, '') || ' ' || "
|
||||
"coalesce(search_terms, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(description, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
channel: Mapped["Channel"] = relationship(back_populates="videos")
|
||||
|
||||
|
||||
class SearchFind(Base, TimestampMixin):
|
||||
"""Per-user record of a video the user surfaced via their own live YouTube search.
|
||||
|
||||
Lets "videos I searched for" appear in that user's own (Mine) feed and the Mine Source
|
||||
filter — distinct from the GLOBAL `Video.via_search` provenance, which marks a video as
|
||||
search-discovered by *anyone* and drives the shared Library's Source filter."""
|
||||
|
||||
__tablename__ = "search_finds"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_search_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
|
||||
class ExploredChannel(Base, TimestampMixin):
|
||||
"""Per-user record of a channel the user opened the page of WITHOUT subscribing.
|
||||
|
||||
Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
|
||||
user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
|
||||
explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
|
||||
purge: a from_explore channel with no live ExploredChannel row (and no subscription / no
|
||||
per-user state on its videos) is hard-deleted by the cleanup job. Subscribing is the "keep"
|
||||
signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
|
||||
(from TimestampMixin) is the grace clock; a re-visit refreshes it."""
|
||||
|
||||
__tablename__ = "explored_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_explored_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class BlockedChannel(Base, TimestampMixin):
|
||||
"""Per-user channel blocklist. Videos from a blocked channel are excluded from this user's
|
||||
live YouTube search (and not ingested on their behalf) and hidden from their feed / explore
|
||||
views — a personal "never show me this channel again" that doesn't affect other users."""
|
||||
|
||||
__tablename__ = "blocked_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_blocked_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
@@ -0,0 +1,296 @@
|
||||
"""Catalog's reads: channels and videos, and each user's footprint in them (subscriptions, blocks,
|
||||
explorations), for the HTTP layer and for the core code that asks about them.
|
||||
|
||||
A user's row is read through `userscope`, so someone else's is the same answer as a missing one.
|
||||
Catalog does not read another unit's tables: what it needs from Reader (a playlist's channels, a
|
||||
user's tag links) it asks `reader.queries`.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Select, and_, case, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
Subscription,
|
||||
Video,
|
||||
)
|
||||
from app.collation import collated
|
||||
from app.models import User
|
||||
from app.sync.explore import followed_channel
|
||||
from app.userscope import owned_by
|
||||
|
||||
# --- channels --------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def channel(db: Session, channel_id: str) -> Channel | None:
|
||||
return db.get(Channel, channel_id)
|
||||
|
||||
|
||||
def channel_title(db: Session, channel_id: str) -> str | None:
|
||||
return db.scalar(select(Channel.title).where(Channel.id == channel_id))
|
||||
|
||||
|
||||
def channel_titles(db: Session, ids: set[str]) -> dict[str, str]:
|
||||
"""Each channel's title by id; a channel with no title yet is left out."""
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(ids)))
|
||||
return {cid: title for cid, title in rows.tuples() if title is not None}
|
||||
|
||||
|
||||
def channel_aggregates(channel_ids: list[str]) -> Select:
|
||||
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
|
||||
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
|
||||
alone; a test holds that (test_channel_list_index_only)."""
|
||||
return (
|
||||
select(
|
||||
Video.channel_id,
|
||||
func.count(Video.id),
|
||||
func.max(Video.published_at),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.where(Video.channel_id.in_(channel_ids))
|
||||
.group_by(Video.channel_id)
|
||||
)
|
||||
|
||||
|
||||
def aggregates(db: Session, channel_ids: list[str]) -> list[tuple[str, int, datetime | None, int]]:
|
||||
"""`channel_aggregates`, run: (channel id, stored videos, last upload, total seconds)."""
|
||||
if not channel_ids:
|
||||
return []
|
||||
return list(db.execute(channel_aggregates(channel_ids)).tuples())
|
||||
|
||||
|
||||
def video_counts(db: Session, channel_id: str) -> tuple[int, int, int]:
|
||||
"""A channel's stored videos, its Shorts and its live/upcoming ones, in one grouped pass."""
|
||||
n_all, n_short, n_live = db.execute(
|
||||
select(
|
||||
func.count(Video.id),
|
||||
func.count(Video.id).filter(Video.is_short.is_(True)),
|
||||
func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
|
||||
).where(Video.channel_id == channel_id)
|
||||
).one()
|
||||
return int(n_all or 0), int(n_short or 0), int(n_live or 0)
|
||||
|
||||
|
||||
def deep_queued(db: Session, channel_ids: list[str]) -> set[str]:
|
||||
"""Which of these channels at least one user (any user) has asked full history for."""
|
||||
if not channel_ids:
|
||||
return set()
|
||||
return set(
|
||||
db.execute(
|
||||
select(Subscription.channel_id)
|
||||
.where(
|
||||
Subscription.channel_id.in_(channel_ids),
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.distinct()
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def followed(db: Session, channel_id: str) -> bool:
|
||||
"""Whether anyone subscribes to the channel."""
|
||||
return (
|
||||
db.execute(
|
||||
select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
# --- a user's subscriptions, blocks and explorations ----------------------------------------
|
||||
|
||||
|
||||
def subscriptions(db: Session, user: User, lang: str) -> list[tuple[Subscription, Channel]]:
|
||||
"""The user's subscriptions with their channels, highest priority first, then by title in the
|
||||
reader's alphabet."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Subscription, Channel)
|
||||
.join(Channel, Channel.id == Subscription.channel_id)
|
||||
.where(owned_by(Subscription.user_id, user))
|
||||
.order_by(Subscription.priority.desc(), collated(Channel.title, lang))
|
||||
).tuples()
|
||||
)
|
||||
|
||||
|
||||
def subscription(db: Session, user: User, channel_id: str) -> Subscription | None:
|
||||
return db.execute(
|
||||
select(Subscription).where(
|
||||
owned_by(Subscription.user_id, user), Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def subscribed_channels(user: User):
|
||||
"""The channel ids the user subscribes to, as a subquery."""
|
||||
return select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
|
||||
|
||||
|
||||
def exploration(db: Session, user: User, channel_id: str) -> ExploredChannel | None:
|
||||
return db.execute(
|
||||
select(ExploredChannel).where(
|
||||
owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def is_blocked(db: Session, user: User, channel_id: str) -> bool:
|
||||
return (
|
||||
db.execute(
|
||||
select(BlockedChannel.id).where(
|
||||
owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def blocked_ids(db: Session, user: User) -> set[str]:
|
||||
return set(
|
||||
db.scalars(select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user)))
|
||||
)
|
||||
|
||||
|
||||
def blocked_channels(db: Session, user: User, lang: str) -> list[Channel]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel)
|
||||
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
|
||||
.where(owned_by(BlockedChannel.user_id, user))
|
||||
.order_by(collated(Channel.title, lang))
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- sync progress ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def catalog_totals(db: Session, user: User) -> dict:
|
||||
"""The instance's catalog size and this user's subscription count (the sync status card)."""
|
||||
return {
|
||||
"subscriptions": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Subscription)
|
||||
.where(owned_by(Subscription.user_id, user))
|
||||
),
|
||||
"channels_total": db.scalar(select(func.count()).select_from(Channel)),
|
||||
"channels_backfilling": db.scalar(
|
||||
select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
|
||||
),
|
||||
"videos_total": db.scalar(select(func.count()).select_from(Video)),
|
||||
"pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def my_progress(db: Session, user: User) -> tuple[int, int, int, int]:
|
||||
"""Over the channels this user subscribes to: how many, how many have details, how many their
|
||||
recent uploads, how many their whole history."""
|
||||
total, details_synced, recent_synced, all_deep_done = db.execute(
|
||||
select(
|
||||
func.count(Channel.id),
|
||||
func.count(Channel.details_synced_at),
|
||||
func.count(Channel.recent_synced_at),
|
||||
func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
|
||||
)
|
||||
.select_from(Channel)
|
||||
.join(
|
||||
Subscription,
|
||||
and_(Subscription.channel_id == Channel.id, owned_by(Subscription.user_id, user)),
|
||||
)
|
||||
).one()
|
||||
return total or 0, details_synced or 0, recent_synced or 0, int(all_deep_done or 0)
|
||||
|
||||
|
||||
def my_deep_requested(db: Session, user: User) -> list[Channel]:
|
||||
"""The channels this user asked full history for."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Channel.id,
|
||||
owned_by(Subscription.user_id, user),
|
||||
Subscription.deep_requested.is_(True),
|
||||
),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def my_video_count(db: Session, user: User) -> int:
|
||||
"""The videos of the channels this user subscribes to and has not hidden."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(Video.id)).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
owned_by(Subscription.user_id, user),
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def video_total(db: Session) -> int:
|
||||
return db.scalar(select(func.count(Video.id))) or 0
|
||||
|
||||
|
||||
def deep_backlog(db: Session) -> list[Channel]:
|
||||
"""Every channel someone asked full history for that does not have it yet."""
|
||||
deep_requested = (
|
||||
select(Subscription.id)
|
||||
.where(Subscription.channel_id == Channel.id, Subscription.deep_requested.is_(True))
|
||||
.exists()
|
||||
)
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel).where(Channel.backfill_done.is_(False), deep_requested)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def work_queue(db: Session) -> dict:
|
||||
"""What the catalog's jobs still have queued, instance-wide (the scheduler card)."""
|
||||
return {
|
||||
# The same selection run_recent_backfill works through, or the card counts channels the
|
||||
# job will never touch and never reaches zero.
|
||||
"channels_recent_pending": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Channel)
|
||||
.where(Channel.recent_synced_at.is_(None), followed_channel())
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_shorts": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_live_refresh": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(
|
||||
or_(
|
||||
Video.live_status.in_(("live", "upcoming")),
|
||||
and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
|
||||
)
|
||||
)
|
||||
)
|
||||
or 0,
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Live YouTube search, the catalog's side: what a page of results leaves in the shared catalog.
|
||||
|
||||
A result becomes a catalog row so it renders with the normal feed cards and the in-app player and
|
||||
gains per-user state. The ingest runs in steps with a YouTube call between each, and each step's
|
||||
rows are committed before the next call: an enrich that fails rolls back to the stubs, which stay
|
||||
(the FK targets for what follows, backfilled later). Results are flagged (`videos.via_search`,
|
||||
`channels.from_search`) so the Library can hide this search-discovered noise; Shorts and
|
||||
live/upcoming videos are never kept. The HTTP layer (`routes/search.py`) runs the paging, the caps
|
||||
and the refusals.
|
||||
"""
|
||||
|
||||
from collections.abc import Sequence
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete, func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, quota, sysconfig
|
||||
from app.catalog import queries
|
||||
from app.catalog.models import LIVE_OR_UPCOMING, Channel, SearchFind, Subscription, Video
|
||||
from app.models import User
|
||||
from app.reader import queries as reader_queries
|
||||
from app.sync.subscriptions import apply_channel_details
|
||||
from app.sync.videos import apply_video_details, insert_stubs, parse_dt
|
||||
from app.userscope import owned_by
|
||||
from app.utils import like_escape
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def classify_shorts(db: Session, videos: Sequence[Video]) -> None:
|
||||
"""Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
|
||||
|
||||
Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
|
||||
synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
|
||||
duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
|
||||
quota). Undetermined probes are left for the scheduler to retry."""
|
||||
probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
|
||||
to_probe: list[Video] = []
|
||||
for v in videos:
|
||||
if v.shorts_probed:
|
||||
continue
|
||||
if v.enriched_at is None:
|
||||
# Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
|
||||
# so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
|
||||
# (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
|
||||
continue
|
||||
if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
|
||||
v.is_short = False
|
||||
v.shorts_probed = True
|
||||
else:
|
||||
to_probe.append(v)
|
||||
if to_probe:
|
||||
with make_client() as client:
|
||||
|
||||
def work(v: Video):
|
||||
return v, probe_is_short(client, v.id)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
for v, result in pool.map(work, to_probe):
|
||||
if result is None:
|
||||
continue
|
||||
v.is_short = result
|
||||
v.shorts_probed = True
|
||||
db.commit()
|
||||
|
||||
|
||||
def ingest_candidates(
|
||||
db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
|
||||
) -> list[str]:
|
||||
"""Materialise one page of search candidates into the catalog (channel stubs + enrich,
|
||||
video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
|
||||
video ids — non-Short, non-live — in the input relevance order."""
|
||||
ids = [it["id"] for it in candidates]
|
||||
if not ids:
|
||||
return []
|
||||
|
||||
# 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
|
||||
# then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
|
||||
channel_ids = list({it["channel_id"] for it in candidates})
|
||||
existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
|
||||
new_channel_ids = [c for c in channel_ids if c not in existing]
|
||||
if new_channel_ids:
|
||||
titles = {it["channel_id"]: it["channel_title"] for it in candidates}
|
||||
for cid in new_channel_ids:
|
||||
db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
|
||||
db.commit()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
|
||||
details = yt.get_channels(new_channel_ids)
|
||||
apply_channel_details(db, details, new_channel_ids)
|
||||
db.commit()
|
||||
except YouTubeError:
|
||||
# The stubs remain and the details can be backfilled later — but until they are, those
|
||||
# channels sit in the results with no thumbnail and no subscriber count, looking like
|
||||
# channels that have neither.
|
||||
log.warning(
|
||||
"the channels found by a search could not be enriched",
|
||||
extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
|
||||
)
|
||||
db.rollback() # details can be backfilled later; the stubs remain
|
||||
|
||||
# 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
|
||||
# NOTHING), so an organically-synced video stays organic.
|
||||
# published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
|
||||
# (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
|
||||
# enrich in step 3 overwrites it with the video's real snippet date.
|
||||
now = datetime.now(timezone.utc)
|
||||
insert_stubs(
|
||||
db,
|
||||
[
|
||||
{
|
||||
"id": it["id"],
|
||||
"channel_id": it["channel_id"],
|
||||
"title": it["title"],
|
||||
"published_at": parse_dt(it["published_at"]) or now,
|
||||
"thumbnail_url": it["thumbnail_url"],
|
||||
"via_search": True,
|
||||
}
|
||||
for it in candidates
|
||||
],
|
||||
)
|
||||
|
||||
# 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
|
||||
videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
|
||||
detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
|
||||
except YouTubeError:
|
||||
# Every candidate then stays unenriched, and an unenriched video cannot be judged live or
|
||||
# Short — so it is dropped. A working search returns nothing and reads as "no results".
|
||||
log.warning(
|
||||
"the videos found by a search could not be enriched",
|
||||
extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
|
||||
)
|
||||
detail_by_id = {}
|
||||
enriched_at = datetime.now(
|
||||
timezone.utc
|
||||
) # when THIS enrich ran (distinct from the stub placeholder)
|
||||
for v in videos:
|
||||
item = detail_by_id.get(v.id)
|
||||
if item is None:
|
||||
continue
|
||||
apply_video_details(v, item)
|
||||
v.enriched_at = enriched_at
|
||||
db.commit()
|
||||
|
||||
# 4) Confirm/deny Shorts (no quota) so none enter via search.
|
||||
classify_shorts(db, videos)
|
||||
|
||||
keep = {
|
||||
v.id
|
||||
for v in videos
|
||||
if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
|
||||
}
|
||||
return [vid for vid in ids if vid in keep]
|
||||
|
||||
|
||||
def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
|
||||
"""Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
|
||||
|
||||
"Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
|
||||
video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
|
||||
wildcard — either way the term was never folded in, and local search lost that relevance."""
|
||||
padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.id.in_(video_ids))
|
||||
.where(padded.notilike(f"% {like_escape(term)} %"))
|
||||
.values(
|
||||
search_terms=func.trim(
|
||||
func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def new_first(db: Session, user: User, ids: list[str]) -> list[str]:
|
||||
"""Reorder results so genuinely-new discoveries come first and things the user already has
|
||||
(a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
|
||||
to the back — keeping the relevance order within each group."""
|
||||
if not ids:
|
||||
return ids
|
||||
have = set(
|
||||
db.scalars(
|
||||
select(Video.id).where(
|
||||
Video.id.in_(ids), Video.channel_id.in_(queries.subscribed_channels(user))
|
||||
)
|
||||
)
|
||||
) | reader_queries.held_by(db, user, ids)
|
||||
return [i for i in ids if i not in have] + [i for i in ids if i in have]
|
||||
|
||||
|
||||
def record_finds(db: Session, user: User, video_ids: list[str], term: str) -> None:
|
||||
"""These results as THIS user's search finds (the Mine feed, the Source filter), and the term
|
||||
folded into each one's `search_terms` so local full-text search inherits YouTube's relevance.
|
||||
Idempotent."""
|
||||
db.execute(
|
||||
pg_insert(SearchFind)
|
||||
.values([{"user_id": user.id, "video_id": vid} for vid in video_ids])
|
||||
.on_conflict_do_nothing(index_elements=["user_id", "video_id"])
|
||||
)
|
||||
_fold_term(db, video_ids, term)
|
||||
|
||||
|
||||
def clear(db: Session, user: User, video_ids: list[str]) -> int:
|
||||
"""Discard these results "as if never added": drop this user's finds of them, then delete the
|
||||
ones nobody kept — a `via_search` video no reader holds (`reader.queries.kept_by_a_reader`),
|
||||
whose channel nobody subscribes to, and that no other user still has as a find. Returns how
|
||||
many videos went."""
|
||||
db.execute(
|
||||
delete(SearchFind).where(
|
||||
owned_by(SearchFind.user_id, user), SearchFind.video_id.in_(video_ids)
|
||||
)
|
||||
)
|
||||
subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
result = db.execute(
|
||||
delete(Video).where(
|
||||
Video.id.in_(video_ids),
|
||||
Video.via_search.is_(True),
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~subscribed,
|
||||
~other_find,
|
||||
)
|
||||
)
|
||||
return result.rowcount if isinstance(result, CursorResult) else 0
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Catalog's writes a request makes: a user's subscription settings, what they explore or block, a
|
||||
subscription recorded or dropped, a channel's sync markers reset.
|
||||
|
||||
Every function here flushes and leaves the commit to its caller (the handler). The YouTube side of a
|
||||
subscribe or an unsubscribe is the handler's (it needs the user's client and its refusals); what
|
||||
lands here is the local record of it. The ingest jobs (`sync/`, `catalog/search`) write the
|
||||
catalog's shared rows themselves.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete, update
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog import queries
|
||||
from app.catalog.models import BlockedChannel, Channel, ExploredChannel, Subscription, Video
|
||||
from app.models import User
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
def remember_own_channel(user: User, channel_id: str) -> None:
|
||||
"""The user's own YouTube channel, so discovery never offers it to them."""
|
||||
user.yt_channel_id = channel_id
|
||||
|
||||
|
||||
def update_subscription(
|
||||
db: Session,
|
||||
sub: Subscription,
|
||||
*,
|
||||
priority: int | None = None,
|
||||
hidden: bool | None = None,
|
||||
deep_requested: bool | None = None,
|
||||
) -> bool:
|
||||
"""Change the settings given (None leaves one as it is). Returns whether full history was just
|
||||
turned ON — the caller then pulls the channel now rather than at the scheduler's next tick."""
|
||||
turned_on = False
|
||||
if priority is not None:
|
||||
sub.priority = priority
|
||||
if hidden is not None:
|
||||
sub.hidden = hidden
|
||||
if deep_requested is not None:
|
||||
# Turning it off won't undo already-fetched videos; it stops further deep paging if nobody
|
||||
# else still wants it.
|
||||
turned_on = deep_requested and not sub.deep_requested
|
||||
sub.deep_requested = deep_requested
|
||||
db.flush()
|
||||
return turned_on
|
||||
|
||||
|
||||
def set_deep_all(db: Session, user: User, on: bool) -> int:
|
||||
"""Every one of the user's channels in (or out of) full-history backfill; how many rows moved."""
|
||||
result = db.execute(
|
||||
update(Subscription).where(owned_by(Subscription.user_id, user)).values(deep_requested=on)
|
||||
)
|
||||
return result.rowcount if isinstance(result, CursorResult) else 0
|
||||
|
||||
|
||||
def explore(db: Session, user: User, channel: Channel) -> None:
|
||||
"""Record (or refresh the grace clock of) the user's exploration of a channel, and flag the
|
||||
channel ephemeral unless someone follows it — the cleanup job's keep rule reads both."""
|
||||
existing = queries.exploration(db, user, channel.id)
|
||||
if existing is None:
|
||||
db.add(ExploredChannel(user_id=user.id, channel_id=channel.id))
|
||||
else:
|
||||
existing.created_at = datetime.now(timezone.utc)
|
||||
if not channel.from_explore and not queries.followed(db, channel.id):
|
||||
channel.from_explore = True
|
||||
db.flush()
|
||||
|
||||
|
||||
def reset_backfill(db: Session, channel: Channel) -> None:
|
||||
"""Fetch the channel again from scratch: its backfill markers cleared, and full history asked
|
||||
for on EVERY subscription to it — the deep scheduler gates on an EXISTS over those, so the admin
|
||||
who resets it need not subscribe themselves."""
|
||||
channel.backfill_done = False
|
||||
channel.backfill_cursor = None
|
||||
channel.recent_synced_at = None
|
||||
db.execute(
|
||||
update(Subscription)
|
||||
.where(Subscription.channel_id == channel.id)
|
||||
.values(deep_requested=True)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def record_subscription(db: Session, user: User, channel: Channel, yt_subscription_id: str) -> None:
|
||||
"""The subscription YouTube now holds, recorded here. Subscribing is the "keep" signal: an
|
||||
explored channel becomes an organic one, so the cleanup job leaves it alone and its videos join
|
||||
the shared Library like any subscribed channel's uploads."""
|
||||
db.add(
|
||||
Subscription(
|
||||
user_id=user.id,
|
||||
channel_id=channel.id,
|
||||
yt_subscription_id=yt_subscription_id or None,
|
||||
subscribed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
if channel.from_explore:
|
||||
channel.from_explore = False
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.channel_id == channel.id, Video.via_explore.is_(True))
|
||||
.values(via_explore=False)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def drop_subscription(db: Session, sub: Subscription) -> None:
|
||||
db.delete(sub)
|
||||
db.flush()
|
||||
|
||||
|
||||
def block(db: Session, user: User, channel_id: str) -> None:
|
||||
"""Block a channel for this user (idempotent), and end any exploration of it so the cleanup job
|
||||
can reclaim it."""
|
||||
if queries.is_blocked(db, user, channel_id):
|
||||
return
|
||||
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
|
||||
db.execute(
|
||||
delete(ExploredChannel).where(
|
||||
owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def unblock(db: Session, user: User, channel_id: str) -> None:
|
||||
db.execute(
|
||||
delete(BlockedChannel).where(
|
||||
owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
|
||||
)
|
||||
)
|
||||
@@ -26,6 +26,7 @@ from uuid import uuid4
|
||||
from sqlalchemy import and_, delete, func, not_, select, update
|
||||
from sqlalchemy.orm import Session, defer
|
||||
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app import diagnostics
|
||||
from app.models import DebugScreenshot, DebugSession, LogEvent, User
|
||||
|
||||
@@ -76,7 +77,9 @@ def open_session(db: Session, user: User) -> DebugSession | None:
|
||||
|
||||
def _unclosed(db: Session, user: User) -> DebugSession | None:
|
||||
return db.scalar(
|
||||
select(DebugSession).where(DebugSession.user_id == user.id, DebugSession.ended_at.is_(None))
|
||||
select(DebugSession).where(
|
||||
owned_by(DebugSession.user_id, user), DebugSession.ended_at.is_(None)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -126,12 +129,8 @@ def from_header(db: Session, user: User, value: str | None) -> DebugSession | No
|
||||
"""The session a request names, if it is the caller's own and still live; else None."""
|
||||
if not value or len(value) != _ID_LEN or not all(c in "0123456789abcdef" for c in value):
|
||||
return None
|
||||
session = db.get(DebugSession, value)
|
||||
if (
|
||||
session is None
|
||||
or session.user_id != user.id
|
||||
or not is_live(session, datetime.now(timezone.utc))
|
||||
):
|
||||
session = owned_row(db, DebugSession, value, user)
|
||||
if session is None or not is_live(session, datetime.now(timezone.utc)):
|
||||
return None
|
||||
return session
|
||||
|
||||
@@ -354,7 +353,7 @@ def stored_screenshot_bytes(db: Session, user_id: int) -> int:
|
||||
db.scalar(
|
||||
select(func.coalesce(func.sum(DebugScreenshot.size), 0))
|
||||
.join(DebugSession, DebugSession.id == DebugScreenshot.session_id)
|
||||
.where(DebugSession.user_id == user_id)
|
||||
.where(owned_by(DebugSession.user_id, user_id))
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
@@ -22,7 +22,7 @@ to the client, which has both the native payload and the translations.
|
||||
"""
|
||||
|
||||
from app.discover.budget import remaining_seconds
|
||||
from app.models import Channel
|
||||
from app.catalog.models import Channel
|
||||
from app.reader.models import Playlist
|
||||
|
||||
# Plex artwork is a poster for the things you browse and a landscape still for the things you play.
|
||||
|
||||
@@ -22,7 +22,9 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.discover import cards, queries
|
||||
from app.models import Channel, PlexItem, PlexShow, User, Video
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import ComfortPin, Playlist
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import SHOW_NEW, leaf_card, show_card
|
||||
@@ -70,11 +72,11 @@ def resolve_ref(db: Session, user: User, kind: str, ref: str) -> tuple[str, obje
|
||||
# such pin target", which is the same answer this function gives for one that does not exist.
|
||||
except ValueError:
|
||||
return None
|
||||
pl = db.get(Playlist, pid)
|
||||
# A playlist is per-user, unlike the other four (which are shared catalog rows whose PRIVACY
|
||||
# lives in the state tables). Someone else's list must read as "no such thing", not as a 403 —
|
||||
# the same reason `owned_or_404` 404s rather than 403s.
|
||||
return (col, pl.id) if pl is not None and pl.user_id == user.id else None
|
||||
pl = owned_row(db, Playlist, pid, user)
|
||||
return (col, pl.id) if pl is not None else None
|
||||
|
||||
|
||||
def reorder(db: Session, user: User, ids: list[int]) -> None:
|
||||
|
||||
@@ -6,7 +6,7 @@ Two rules shape this module:
|
||||
**Reuse the query layer, not the route.** The YouTube half calls `feedquery.filtered_query` /
|
||||
`sort_keys`, so the Home surface inherits the feed's own definitions of subscribed, blocked,
|
||||
explore-visible and in-progress for free, and cannot drift from them. The Plex half calls
|
||||
`plex.rollup` and the `plex.cards` serializers rather than `routes/plex/discover.py`: that route
|
||||
`plex.rollup` and the `plex.cards` serializers rather than `plex/discover.py`: that read-model
|
||||
narrows EVERY rail by the Plex sidebar's `LibraryFilters`, which is right for a surface that HAS a
|
||||
sidebar and wrong for a landing page that does not.
|
||||
|
||||
@@ -29,7 +29,9 @@ from app.feedquery import (
|
||||
serialize,
|
||||
sort_keys,
|
||||
)
|
||||
from app.models import Channel, PlexItem, PlexShow, PlexState, User, Video
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import PlaylistItem, VideoState
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import display_title, leaf_card, movie_card, show_card
|
||||
@@ -37,7 +39,7 @@ from app.plex.query import apply_show_state_filter, content_libs
|
||||
from app.userscope import owned_by
|
||||
|
||||
# How many rows a single rail may carry — Home's rails and the Plex module's Discover rails alike
|
||||
# (`routes/plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
|
||||
# (`plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
|
||||
# is a rail. One definition, because a comment was all that held two copies in step.
|
||||
RAIL_LIMIT = 20
|
||||
|
||||
@@ -91,7 +93,7 @@ def yt_in_progress(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[tup
|
||||
q = (
|
||||
select(*feed_columns(user, st), st.progress_updated_at.label("progressed_at"))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.join(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.join(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(
|
||||
st.position_seconds > 0,
|
||||
st.status.notin_(("watched", "hidden")),
|
||||
@@ -134,7 +136,7 @@ def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
|
||||
row = db.execute(
|
||||
select(*feed_columns(user, st))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(Video.id == video_id)
|
||||
).first()
|
||||
return serialize(row) if row is not None else None
|
||||
@@ -177,7 +179,7 @@ def playlists_summary(
|
||||
select(*feed_columns(user, st), PlaylistItem.playlist_id.label("_playlist"))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(PlaylistItem.playlist_id.in_(playlist_ids))
|
||||
.order_by(PlaylistItem.playlist_id, PlaylistItem.position.asc())
|
||||
).all()
|
||||
@@ -219,7 +221,7 @@ def plex_in_progress(
|
||||
) -> list[tuple[object, dict]]:
|
||||
"""In-progress movies + partially-watched shows as (progressed_at, card).
|
||||
|
||||
Mirrors `routes/plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
|
||||
Mirrors `plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
|
||||
NULL status still counts as started, so `notin_` alone would drop it), and the same shared
|
||||
`rollup.agg_subquery` + `apply_show_state_filter` for shows, so the two rails cannot disagree on
|
||||
what "in progress" means. What it does NOT inherit is that route's `LibraryFilters` narrowing:
|
||||
|
||||
@@ -11,7 +11,8 @@ from sqlalchemy.orm import Session
|
||||
from app import sysconfig
|
||||
from app.discover import cards, queries
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.models import PlexShow, User
|
||||
from app.models import User
|
||||
from app.plex.models import PlexShow
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.rollup import NO_TS
|
||||
|
||||
@@ -75,7 +76,7 @@ def surprise(
|
||||
) -> list[dict]:
|
||||
"""The hero's draw pool: unseen things that fit the budget, from both sources under ONE seed.
|
||||
|
||||
Interleaved rather than concatenated — the same alternation `routes/plex/discover.py` uses for its
|
||||
Interleaved rather than concatenated — the same alternation `plex/discover.py` uses for its
|
||||
movie/show split — so a short pool never comes back all-YouTube or all-Plex just because one side
|
||||
happened to be queried first.
|
||||
|
||||
|
||||
@@ -328,9 +328,14 @@ def create_own(db: Session, user_id: int, name: str) -> FileFolder:
|
||||
return folder
|
||||
|
||||
|
||||
def renamable(folder: FileFolder) -> bool:
|
||||
"""Only a folder of the user's own making takes a name: a source folder's is its origin's."""
|
||||
return folder.kind == FolderKind.OWN
|
||||
|
||||
|
||||
def rename(db: Session, folder: FileFolder, name: str) -> None:
|
||||
"""Rename one of the user's own folders (a source folder's name is its origin's)."""
|
||||
if folder.kind != FolderKind.OWN:
|
||||
if not renamable(folder):
|
||||
raise FolderError("file_folder_not_own")
|
||||
folder.name = name
|
||||
try:
|
||||
|
||||
@@ -28,7 +28,8 @@ from app.downloads.jobdb import (
|
||||
)
|
||||
from app.downloads.stated import StatedFailure
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DISPLAY_TEXT_MAX, PlexItem
|
||||
from app.models import DISPLAY_TEXT_MAX
|
||||
from app.plex.models import PlexItem
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
from app.plex import lookups as plex_lookups
|
||||
from app.plex import paths as plex_paths
|
||||
|
||||
@@ -296,13 +296,11 @@ def own_link(db: Session, user: User, link_id: int) -> DownloadLink | None:
|
||||
if link is None:
|
||||
return None
|
||||
target = (
|
||||
db.get(DownloadJob, link.job_id)
|
||||
owned_row(db, DownloadJob, link.job_id, user)
|
||||
if link.job_id is not None
|
||||
else db.get(FileFolder, link.folder_id)
|
||||
else owned_row(db, FileFolder, link.folder_id, user)
|
||||
)
|
||||
if target is None or target.user_id != user.id:
|
||||
return None
|
||||
return link
|
||||
return link if target is not None else None
|
||||
|
||||
|
||||
def links_to(
|
||||
|
||||
@@ -16,6 +16,7 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import app.tables # noqa: F401 — a fresh interpreter: every table, before the first query
|
||||
from app import logs
|
||||
from app import logsink
|
||||
from app.config import settings
|
||||
|
||||
@@ -24,7 +24,9 @@ from app.downloads import credentials
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import folders, formats, mediakind, quota, storage
|
||||
from app.downloads.status import HOLDING, JobKind, JobStatus
|
||||
from app.models import DISPLAY_TEXT_MAX, Channel, PlexItem, PlexShow, User, Video
|
||||
from app.models import DISPLAY_TEXT_MAX, User
|
||||
from app.plex.models import PlexItem, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.downloads.models import (
|
||||
DownloadJob,
|
||||
DownloadProfile,
|
||||
|
||||
@@ -17,14 +17,14 @@ from sqlalchemy import BigInteger, Select, and_, case, cast, false, func, or_, s
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.collation import collated
|
||||
from app.models import (
|
||||
from app.models import User
|
||||
from app.catalog.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
User,
|
||||
Video,
|
||||
)
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, Tag, VideoState
|
||||
@@ -212,7 +212,7 @@ def filtered_query(
|
||||
# exactly. 'unwatched'/'all' include row-less 'new' videos, so they keep the outer join.
|
||||
state_required = show in ("watched", "hidden", "in_progress")
|
||||
join_state = query.join if state_required else query.outerjoin
|
||||
query = join_state(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
query = join_state(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
|
||||
# Hide videos the maintenance job has flagged as currently unplayable (deleted/private/
|
||||
# abandoned). They're either deleted after a grace period or unhidden if they recover;
|
||||
|
||||
@@ -28,6 +28,7 @@ log = logs.get_logger(__name__)
|
||||
from app import bodylimit, collation, logsink, state, sysconfig # noqa: E402 — logging is configured first
|
||||
from app.config import settings # noqa: E402 — logging is configured first
|
||||
from app.db import SessionLocal, engine # noqa: E402 — logging is configured first
|
||||
import app.tables # noqa: E402 — every table, before the first query
|
||||
from app.downloads import service as download_service # noqa: E402 — logging is configured first
|
||||
from app.downloads import storage # noqa: E402 — logging is configured first
|
||||
from app.plex import stream as plex_stream # noqa: E402 — logging is configured first
|
||||
|
||||
+5
-676
@@ -1,13 +1,12 @@
|
||||
from datetime import date, datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
Computed,
|
||||
Date,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
@@ -19,13 +18,16 @@ from sqlalchemy import (
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship, validates
|
||||
|
||||
from app.db import Base
|
||||
from app.runstatus import RunStatus, RunTrigger, sql_in
|
||||
from app.utils import canonical_email
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.catalog.models import Subscription
|
||||
|
||||
# Every user-supplied display string in this schema shares one width, and it lives HERE so the
|
||||
# columns below are declared with it rather than with a literal: a name a person typed goes into
|
||||
# `User.display_name` (their Google profile name), `DownloadJob.display_name` and
|
||||
@@ -220,243 +222,6 @@ class Invite(Base, CanonicalEmailMixin):
|
||||
lang: Mapped[str | None] = mapped_column(String(8))
|
||||
|
||||
|
||||
class Channel(Base, TimestampMixin):
|
||||
"""A YouTube channel. Shared across all users (one channel's videos are the same
|
||||
for everyone), so its expensive metadata is fetched and stored only once."""
|
||||
|
||||
__tablename__ = "channels"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(32), primary_key=True) # "UC..." channel id
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
handle: Mapped[str | None] = mapped_column(String(255))
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
uploads_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
subscriber_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
video_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Channel "About" metadata (fetched with the same channels.list call, no extra quota):
|
||||
# lifetime view count, the channel's join date, banner image, and external links.
|
||||
total_view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
banner_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
external_links: Mapped[list | None] = mapped_column(JSON)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
country: Mapped[str | None] = mapped_column(String(8))
|
||||
# Creator keyword tags (brandingSettings.channel.keywords) — a single space-separated string,
|
||||
# multi-word tags quoted; the client parses it into chips for the About tab.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
# Sync bookkeeping.
|
||||
details_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
recent_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_rss_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# When the newest uploads were last read through the Data API because the RSS feed would not
|
||||
# answer (`sync/runner.py` `run_feed_fallback`): the once-per-`rss_fallback_hours` clock.
|
||||
feed_fallback_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
backfill_cursor: Mapped[str | None] = mapped_column(String(128))
|
||||
backfill_done: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this channel only entered the catalog because a live YouTube search
|
||||
# surfaced one of its videos (nobody subscribes to it / it has no organic upload sync).
|
||||
# Lets the Library hide search-only "noise" by default.
|
||||
from_search: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
# True when this channel exists only because a user EXPLORED it (opened its page without
|
||||
# subscribing) — an ephemeral, per-explorer addition. The cleanup job hard-deletes such a
|
||||
# channel (and its via_explore videos) once nobody keeps it (no subscription, no live
|
||||
# ExploredChannel row, no per-user state) after a grace period. Cleared when someone
|
||||
# subscribes (the "keep" signal promotes it to organic).
|
||||
from_explore: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
|
||||
videos: Mapped[list["Video"]] = relationship(back_populates="channel")
|
||||
subscriptions: Mapped[list["Subscription"]] = relationship(back_populates="channel")
|
||||
|
||||
|
||||
class Subscription(Base, TimestampMixin):
|
||||
"""Per-user link to a channel, with the user's own priority/visibility overrides."""
|
||||
|
||||
__tablename__ = "subscriptions"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# YouTube's subscription resource id, needed to unsubscribe via the API.
|
||||
yt_subscription_id: Mapped[str | None] = mapped_column(String(128))
|
||||
priority: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
hidden: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Per-user opt-in to full-history (deep) backfill of this channel. Default off so a
|
||||
# new user's unique channels don't trigger a big shared-quota burst; the deep
|
||||
# scheduler only picks up channels at least one user has requested.
|
||||
deep_requested: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
subscribed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
user: Mapped["User"] = relationship(back_populates="subscriptions")
|
||||
channel: Mapped["Channel"] = relationship(back_populates="subscriptions")
|
||||
|
||||
|
||||
# Video.live_status values that mean "currently a live/upcoming broadcast" — transient states
|
||||
# hidden from feeds/search and revisited by the refresh job until they settle into a VOD.
|
||||
LIVE_OR_UPCOMING = ("live", "upcoming")
|
||||
|
||||
|
||||
class Video(Base, TimestampMixin):
|
||||
"""A YouTube video. Shared across users; per-user state lives elsewhere."""
|
||||
|
||||
__tablename__ = "videos"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(16), primary_key=True) # video id
|
||||
# No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
|
||||
# (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
|
||||
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
|
||||
# sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
|
||||
# Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
|
||||
# Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
|
||||
channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
|
||||
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
# NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
|
||||
# forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
|
||||
# that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
|
||||
# PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
|
||||
# the real date); the ix_videos_published_at index stays for the `oldest` sort.
|
||||
published_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
|
||||
)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
# The feed's views / duration / title sorts each ride a migration-defined index in their exact
|
||||
# ORDER BY, one per direction (and per ICU collation for title): migration 0096, held to
|
||||
# `feedquery.order_by` by test_feed_sort_indexes.
|
||||
duration_seconds: Mapped[int | None] = mapped_column(Integer)
|
||||
view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
like_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
category_id: Mapped[int | None] = mapped_column(Integer)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
detected_language: Mapped[str | None] = mapped_column(String(16))
|
||||
# Creator-supplied keyword tags (snippet.tags joined by spaces) — fetched for free with the
|
||||
# snippet we already request. Indexed into search_vector so the feed search matches the
|
||||
# uploader's own keywords, not just words in the title.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
# Accumulated distinct live-search queries that surfaced this video (across all users) —
|
||||
# folds YouTube's own relevance judgement into the local search: a video YouTube returned
|
||||
# for "ti amo magyarul" becomes findable by that query locally even if its title lacks it.
|
||||
search_terms: Mapped[str | None] = mapped_column(Text)
|
||||
# No btree here or on the flag columns below: migration 0069 (PERF-3) dropped the never-picked
|
||||
# boolean btrees and replaced the useful ones with partial indexes over the target subset
|
||||
# (via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — migration-defined, like the
|
||||
# search_vector GIN.
|
||||
is_short: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Whether the youtube.com/shorts/<id> probe has run for this video.
|
||||
shorts_probed: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video first entered the catalog via a live YouTube search (not an
|
||||
# organic channel/playlist sync). Used to hide search-discovered "noise" from the
|
||||
# Library by default; left False by the organic ingest paths.
|
||||
via_search: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video entered the catalog because a user EXPLORED its (un-subscribed)
|
||||
# channel. Like via_search it's hidden from the shared Library by default, but it's
|
||||
# additionally PER-USER private: a via_explore video is only visible to users who
|
||||
# explored or subscribe to its channel (a gate in the feed query), so one user's
|
||||
# curiosity never floods everyone else's catalog. Left False by the organic ingest paths.
|
||||
via_explore: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# none | live | upcoming | premiere | was_live
|
||||
live_status: Mapped[str] = mapped_column(
|
||||
String(16), default="none", server_default="none", index=True
|
||||
)
|
||||
enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# videos.list?part=status fields (populated during enrichment); used by the
|
||||
# maintenance/validation job to decide whether a video is playable anywhere.
|
||||
embeddable: Mapped[bool | None] = mapped_column(Boolean)
|
||||
privacy_status: Mapped[str | None] = mapped_column(String(16)) # public|unlisted|private
|
||||
upload_status: Mapped[str | None] = mapped_column(
|
||||
String(16)
|
||||
) # processed|failed|rejected|deleted
|
||||
# Maintenance/validation lifecycle. last_checked_at drives the rolling re-validation
|
||||
# (least-recently-checked first); unavailable_since marks a video as currently
|
||||
# unplayable (hidden from the feed immediately, hard-deleted after a grace period);
|
||||
# unavailable_reason records why (deleted|private|paywalled|abandoned|stuck_live).
|
||||
# last_checked_at: no plain btree — migration 0069 (PERF-18) replaced it with the partial
|
||||
# ix_videos_recheck_order (last_checked_at NULLS FIRST WHERE unavailable_since IS NULL) that the
|
||||
# revalidate job's exact ORDER BY rides; unavailable_since is a partial WHERE IS NOT NULL (PERF-3).
|
||||
last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_reason: Mapped[str | None] = mapped_column(String(24))
|
||||
|
||||
# Weighted full-text search document (DB-generated, never written by the app): title (A) >
|
||||
# creator keywords + search queries (B) > truncated description (C). ts_rank honours the
|
||||
# weights so title matches outrank description matches. Backed by a GIN index (migration
|
||||
# 0032). The accent-insensitive `unaccent_simple` config comes from migration 0031.
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(keywords, '') || ' ' || "
|
||||
"coalesce(search_terms, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(description, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
channel: Mapped["Channel"] = relationship(back_populates="videos")
|
||||
|
||||
|
||||
class SearchFind(Base, TimestampMixin):
|
||||
"""Per-user record of a video the user surfaced via their own live YouTube search.
|
||||
|
||||
Lets "videos I searched for" appear in that user's own (Mine) feed and the Mine Source
|
||||
filter — distinct from the GLOBAL `Video.via_search` provenance, which marks a video as
|
||||
search-discovered by *anyone* and drives the shared Library's Source filter."""
|
||||
|
||||
__tablename__ = "search_finds"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_search_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
|
||||
class ExploredChannel(Base, TimestampMixin):
|
||||
"""Per-user record of a channel the user opened the page of WITHOUT subscribing.
|
||||
|
||||
Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
|
||||
user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
|
||||
explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
|
||||
purge: a from_explore channel with no live ExploredChannel row (and no subscription / no
|
||||
per-user state on its videos) is hard-deleted by the cleanup job. Subscribing is the "keep"
|
||||
signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
|
||||
(from TimestampMixin) is the grace clock; a re-visit refreshes it."""
|
||||
|
||||
__tablename__ = "explored_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_explored_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class BlockedChannel(Base, TimestampMixin):
|
||||
"""Per-user channel blocklist. Videos from a blocked channel are excluded from this user's
|
||||
live YouTube search (and not ingested on their behalf) and hidden from their feed / explore
|
||||
views — a personal "never show me this channel again" that doesn't affect other users."""
|
||||
|
||||
__tablename__ = "blocked_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_blocked_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class ApiQuotaUsage(Base):
|
||||
"""Tracks YouTube Data API units spent per Pacific-time day (the quota resets at
|
||||
midnight Pacific). The whole app shares one daily budget."""
|
||||
@@ -747,439 +512,3 @@ class ApiTokenGrant(Base):
|
||||
can_read: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
can_write: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
can_delete: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
# --- Plex integration (optional) -------------------------------------------------------------
|
||||
# A dedicated, self-contained catalog mirrored from a locally-reachable Plex server. Playback is
|
||||
# from the LOCAL physical file (Plex is metadata only); per-user watch state lives in plex_states
|
||||
# so it works for users without a Plex account. Mirrors the app's "shared catalog, private
|
||||
# per-user state" model. The weighted FTS search_vector uses the same unaccent_simple config as
|
||||
# `videos` (migrations 0031/0032).
|
||||
|
||||
|
||||
class PlexLibrary(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex library section. Only `enabled` sections are exposed in the feed.
|
||||
`plex_key` is the section id on the Plex server."""
|
||||
|
||||
__tablename__ = "plex_libraries"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
plex_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
title: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16)) # "movie" | "show"
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# HU-overlay support. `metadata_lang` tags the section's metadata language. An OVERLAY
|
||||
# library (overlay_of set) is the Hungarian-metadata twin of another section: it is never browsable
|
||||
# on its own — its titles' HU fields are written onto the matched EN rows (paired by `guid`).
|
||||
metadata_lang: Mapped[str] = mapped_column(String(8), default="en", server_default="en")
|
||||
overlay_of: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="SET NULL"), index=True
|
||||
)
|
||||
|
||||
|
||||
class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A TV show (grandparent of episodes) — holds poster + summary for the drill-down page, plus
|
||||
the same filterable metadata as movies (genres/rating/content_rating/studio/people) so the TV
|
||||
grid can be filtered/sorted like the movie grid. All mirrored cheaply from the show listing."""
|
||||
|
||||
__tablename__ = "plex_shows"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_shows_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
# ix_plex_shows_collections_gin already created with collections (migration 0047).
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# Canonical external id (e.g. "tmdb:12345") — the key that pairs this EN row with its HU-overlay
|
||||
# twin during sync. Indexed for the guid->row overlay match.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(
|
||||
String(512)
|
||||
) # Plex thumb path (proxied, not stored)
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer) # seasons
|
||||
collection_keys: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # Plex collections this show is in
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the show section listing (parallels PlexItem).
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True) # Plex audienceRating (~IMDb)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True) # network/studio
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True) # first air date
|
||||
# none_as_null: an empty tag list must persist as SQL NULL, not JSON `null`. Without it a Python
|
||||
# None round-trips as a JSON scalar that jsonb_array_elements_text() can't unnest ("cannot extract
|
||||
# elements from a scalar") — the trap a metadata-poor library sprang on /facets. Same on plex_items.
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
people_text: Mapped[str | None] = mapped_column(
|
||||
Text
|
||||
) # cast+directors, folded into search_vector
|
||||
# Set once the show's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: see the identical block on plex_items for the full rationale. Additive;
|
||||
# served from the row; NULL until enriched. (`budget`/`revenue` stay NULL for shows — TV has none.) ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched,
|
||||
# so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexSeason(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A season under a show — used to order episodes for prev/next stepping."""
|
||||
|
||||
__tablename__ = "plex_seasons"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
show_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
season_number: Mapped[int | None] = mapped_column(Integer, index=True)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
|
||||
|
||||
class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A playable LEAF — a movie or a single episode. This is what the feed lists and the player
|
||||
plays. Movies carry `library_id` and no parents; episodes carry show/season refs +
|
||||
season/episode numbers. `file_path` is the Plex-side absolute path of the physical media,
|
||||
mapped through `plex_path_map` to a local mount at playback time. `playable` is the
|
||||
ffprobe-derived browser-compatibility class (direct | remux | transcode)."""
|
||||
|
||||
__tablename__ = "plex_items"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_items_show_order", "show_id", "season_number", "episode_number"),
|
||||
Index("ix_plex_items_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_items_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_items_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_items_collections_gin", "collection_keys", postgresql_using="gin"),
|
||||
Index("ix_plex_items_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
kind: Mapped[str] = mapped_column(String(16), index=True) # "movie" | "episode"
|
||||
# Canonical external id (e.g. "tmdb:12345") — pairs a MOVIE row with its HU-overlay twin during
|
||||
# sync; indexed for the guid->row match. Episodes carry it too but it's unused for them.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_seasons.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_number: Mapped[int | None] = mapped_column(Integer)
|
||||
episode_number: Mapped[int | None] = mapped_column(Integer)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
duration_s: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
cast: Mapped[list | None] = mapped_column(JSON) # [role names]
|
||||
# Physical media (from the Plex Part).
|
||||
file_path: Mapped[str | None] = mapped_column(Text) # Plex-side absolute path
|
||||
part_key: Mapped[str | None] = mapped_column(String(255)) # /library/parts/... (fallback fetch)
|
||||
container: Mapped[str | None] = mapped_column(String(16))
|
||||
codec_video: Mapped[str | None] = mapped_column(String(32))
|
||||
codec_audio: Mapped[str | None] = mapped_column(String(32))
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Browser-playability class from ffprobe: "direct" | "remux" | "transcode" (None = unprobed).
|
||||
playable: Mapped[str | None] = mapped_column(String(12), index=True)
|
||||
probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Transcode HW-capability facet (the playback-class filter). NULL for non-transcode items; for a
|
||||
# `playable="transcode"` item: True = the prod iGPU can hardware-transcode it (codec is HW-decodable
|
||||
# AND the source is 8-bit SDR), False = software libx264 only (AV1/XviD, or a 10-bit/HDR source that
|
||||
# breaks the VAAPI nv12 pipeline — see plex/stream.py). Derived by `plex.sync.transcode_hw_capable`:
|
||||
# set codec-only at sync (bit depth unknown from the cheap listing), then refined by the bit-depth
|
||||
# backfill (`_probe_media_batch`) which downgrades a 10-bit/HDR HEVC to False. The one filterable
|
||||
# home of the invariant `_want_hw`'s source gate computes at play time — kept in sync via the shared
|
||||
# classifier so the catalog facet and the runtime encoder choice never disagree by codec.
|
||||
transcode_hw: Mapped[bool | None] = mapped_column(Boolean, index=True)
|
||||
# Video bit depth (8/10/12…) from the Plex media Stream, filled by the bit-depth backfill. NULL =
|
||||
# not yet probed. Surfaced on the info page (e.g. "HEVC · 10-bit") and feeds `transcode_hw`.
|
||||
video_bit_depth: Mapped[int | None] = mapped_column(Integer)
|
||||
# Watermark for the bit-depth backfill: when this item's media Stream was last probed for
|
||||
# bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
|
||||
# probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
|
||||
media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
|
||||
# `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
|
||||
# filters (GIN-indexed) and double as content signals for a future watch-habit recommender.
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True)
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# rating_keys of the Plex collections this movie belongs to (GIN — filter + card-strip siblings).
|
||||
collection_keys: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Cast + director names, space-joined — folded into search_vector (weight B) so the search box
|
||||
# finds titles by a person's name (and ranks them above summary-only mentions).
|
||||
people_text: Mapped[str | None] = mapped_column(Text)
|
||||
# Set once the movie's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: matched by the canonical `guid` ("tmdb:12345"), filled by the TMDB
|
||||
# backfill job, SERVED FROM THIS ROW (never a live TMDB call). Additive — imdb_rating/rating
|
||||
# above stay Plex-sourced; `tmdb_rating` is the distinct TMDb community score. NULL until enriched;
|
||||
# `tmdb_enriched_at` is the "tried" watermark (stamped on a definitive no-match too, so an
|
||||
# unresolvable title isn't re-fetched every run; left NULL only on a transient fetch error). ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger) # USD (movies; TV has none)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
# [{tmdb_id, title, poster, media_type}] — resolved to library rows (by guid) at serve time.
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB backdrop file_paths [str] (crossfade pool)
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched
|
||||
# (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A small cast/crew headshot cache, keyed by name, so person filter-chips (the cards for the
|
||||
actors currently in the filter) can show a photo + count without a live per-card metadata fetch.
|
||||
Populated opportunistically when an item's RICH metadata is fetched — the info-page view and the
|
||||
periodic rich re-sync — since the cheap section listing that drives the main sync only carries the
|
||||
top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
|
||||
removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
|
||||
metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
|
||||
row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
|
||||
(the headshot FILE lives in the image cache, filled by the people_enrich job)."""
|
||||
|
||||
__tablename__ = "plex_people"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String(255), unique=True, index=True)
|
||||
photo_thumb: Mapped[str | None] = mapped_column(Text) # proxied /person-image URL, or None
|
||||
# TMDB person id, linked by name from a title's TMDB credits (the /person
|
||||
# biography fetch builds on it). NULL until a TMDB-enriched title names this person.
|
||||
tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
|
||||
# --- TMDB person enrichment: fetched from /person/{tmdb_person_id} by the people_enrich job,
|
||||
# or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
|
||||
# no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
|
||||
# media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
|
||||
biography: Mapped[str | None] = mapped_column(Text)
|
||||
# TMDB's Hungarian biography, from the same request's `translations`; a Hungarian reader is
|
||||
# served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
|
||||
# keywords have no Hungarian on TMDB, so they stay in the instance language.
|
||||
biography_hu: Mapped[str | None] = mapped_column(Text)
|
||||
birthday: Mapped[str | None] = mapped_column(String(10)) # "YYYY-MM-DD", served as-is
|
||||
deathday: Mapped[str | None] = mapped_column(String(10))
|
||||
place_of_birth: Mapped[str | None] = mapped_column(Text)
|
||||
known_for_department: Mapped[str | None] = mapped_column(String(64))
|
||||
also_known_as: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
known_for: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{tmdb_id, media_type, title}]
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexCollection(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex collection (a grouping of movies or shows — franchises like Avatar, curated
|
||||
sets like MCU, or auto/community lists like IMDb Top 250). Mirrored in full by the background
|
||||
sync so reads never hit Plex's slow collection queries. Membership is stored as `collection_keys`
|
||||
on the member movies (plex_items) / shows (plex_shows). `smart` = Plex's query-based flag;
|
||||
`editable` = we may write changes back to Plex (default False = read-only; set for curated ones)."""
|
||||
|
||||
__tablename__ = "plex_collections"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer)
|
||||
smart: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
editable: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a Plex item — mirrors VideoState. Lives in Siftlode's own DB so
|
||||
it works for users WITHOUT a Plex account (the whole point of the access-layer design).
|
||||
`synced_to_plex` marks whether this state was pushed back to a linked Plex account."""
|
||||
|
||||
__tablename__ = "plex_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
synced_to_plex: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class PlexLink(Base, UpdatedAtMixin):
|
||||
"""Links a Siftlode user to a Plex account for two-way watch-state sync (one row per user).
|
||||
|
||||
For the owner (the MVP scope) the row uses the server's admin token — `uses_admin=True`,
|
||||
`plex_token_enc=None` — so no separate Plex login is needed; the admin token already reads/writes
|
||||
the owner account's watch state. `plex_account_id` is that Plex account's numeric id, used to
|
||||
filter the shared watch-history feed to this user (Phase C). Reserved for later: a friend can link
|
||||
their OWN Plex account via PIN OAuth, storing a personal `plex_token_enc` (uses_admin=False) — the
|
||||
sync loop then iterates link rows uniformly.
|
||||
|
||||
`initial_import_done` guards the one-time "Plex is master" import; `last_watch_sync_at` is the
|
||||
high-water mark for the incremental Plex→Siftlode pass."""
|
||||
|
||||
__tablename__ = "plex_link"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), unique=True, index=True
|
||||
)
|
||||
uses_admin: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
plex_token_enc: Mapped[str | None] = mapped_column(Text) # P5b friend tokens (encrypted)
|
||||
plex_account_id: Mapped[int | None] = mapped_column(Integer)
|
||||
plex_username: Mapped[str | None] = mapped_column(String(255))
|
||||
sync_enabled: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
initial_import_done: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false"
|
||||
)
|
||||
last_watch_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
linked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexPlaylist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user ORDERED watch-list of Plex items — Siftlode-native (lives in our own DB, works for
|
||||
users WITHOUT a Plex account, like plex_states). Unlike collections (shared, library-level), a
|
||||
playlist is owned by one user and its items are ordered. Optional Plex-direction sync is a later
|
||||
phase (`plex_rating_key` set once pushed back to the owner's Plex account)."""
|
||||
|
||||
__tablename__ = "plex_playlists"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
plex_rating_key: Mapped[str | None] = mapped_column(String(32))
|
||||
|
||||
|
||||
class PlexPlaylistItem(Base):
|
||||
"""One ordered entry in a playlist. `position` (0-based) orders playback."""
|
||||
|
||||
__tablename__ = "plex_playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("plex_playlists.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
|
||||
@@ -9,6 +9,7 @@ table can't grow without bound for a heavy user.
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.models import Notification
|
||||
|
||||
# Keep at most this many READ notifications per user; older read ones are pruned. Unread
|
||||
@@ -48,11 +49,13 @@ def create_notification(
|
||||
|
||||
def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
"""Delete a user's oldest READ notifications beyond `cap`. Returns the number removed.
|
||||
Idempotent and cheap; safe to call after marking things read."""
|
||||
Idempotent and cheap; safe to call after marking things read. Flushes first, so a notice the
|
||||
caller has just marked read counts; the caller commits."""
|
||||
db.flush()
|
||||
ids = (
|
||||
db.execute(
|
||||
select(Notification.id)
|
||||
.where(Notification.user_id == user_id, Notification.read.is_(True))
|
||||
.where(owned_by(Notification.user_id, user_id), Notification.read.is_(True))
|
||||
.order_by(Notification.created_at.desc())
|
||||
.offset(cap)
|
||||
)
|
||||
@@ -63,5 +66,4 @@ def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
return 0
|
||||
# One bulk DELETE rather than a get+delete per row (the ids are already in hand).
|
||||
db.execute(delete(Notification).where(Notification.id.in_(ids)))
|
||||
db.commit()
|
||||
return len(ids)
|
||||
|
||||
@@ -5,7 +5,7 @@ router. Pure functions — no DB, no request state."""
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
from app.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
|
||||
|
||||
class ShowState(NamedTuple):
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
"""What a collection is to the app: where it comes from, whether it may be edited, its card, its
|
||||
effective content rating, and how its members order — shared by the collection pages, the info
|
||||
page's collection strips and the admin editor."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime
|
||||
from typing import Any, NamedTuple
|
||||
|
||||
from app.plex.models import PlexCollection
|
||||
|
||||
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
|
||||
# members (Plex shows one cert on the collection; we don't store it, so derive it).
|
||||
_CERT_RANK = {
|
||||
"G": 1,
|
||||
"TV-Y": 1,
|
||||
"TV-Y7": 1,
|
||||
"TV-G": 1,
|
||||
"PG": 2,
|
||||
"TV-PG": 2,
|
||||
"PG-13": 3,
|
||||
"TV-14": 3,
|
||||
"R": 4,
|
||||
"TV-MA": 4,
|
||||
"NC-17": 5,
|
||||
}
|
||||
|
||||
|
||||
def collection_cert(certs: list[str | None]) -> str | None:
|
||||
ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
|
||||
return max(ranked, key=lambda x: x[1])[0] if ranked else None
|
||||
|
||||
|
||||
_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
|
||||
|
||||
|
||||
def collection_source(col: PlexCollection) -> str:
|
||||
first = (col.title or "").strip().split(" ", 1)[0].lower()
|
||||
if first in _STRIP_PROVIDERS:
|
||||
return first
|
||||
if col.smart:
|
||||
return "smart"
|
||||
return "collection"
|
||||
|
||||
|
||||
def collection_editable(c: PlexCollection) -> bool:
|
||||
"""Effective editability: the admin marked it editable AND it's a plain manual collection — never
|
||||
smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
|
||||
return bool(c.editable) and not c.smart and collection_source(c) == "collection"
|
||||
|
||||
|
||||
def collection_card(c: PlexCollection) -> dict:
|
||||
return {
|
||||
"id": c.rating_key,
|
||||
"title": c.title,
|
||||
"summary": c.summary,
|
||||
"thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
|
||||
"child_count": c.child_count,
|
||||
"smart": c.smart,
|
||||
"source": collection_source(c),
|
||||
"editable": c.editable,
|
||||
"can_edit": collection_editable(
|
||||
c
|
||||
), # whether the edit controls should be offered (admin UI)
|
||||
}
|
||||
|
||||
|
||||
class Member(NamedTuple):
|
||||
"""One collection member flattened to everything the Sort control can order by. The page's two
|
||||
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
|
||||
ordering is a pure list sort over these, which is also what makes it unit-testable."""
|
||||
|
||||
kind: str # "movie" | "show"
|
||||
obj: Any # PlexItem | PlexShow, carried through to the card serializer
|
||||
title: str # lower-cased; never NULL (the column is not nullable)
|
||||
year: int | None
|
||||
rating: float | None
|
||||
added: datetime | None
|
||||
release: date | None
|
||||
duration: int | None # movies only — a show has no single runtime
|
||||
last_watched: datetime | None
|
||||
|
||||
|
||||
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
|
||||
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
|
||||
# that silently does nothing here.
|
||||
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
|
||||
"added": lambda m: m.added,
|
||||
"lastwatched": lambda m: m.last_watched,
|
||||
"release": lambda m: m.release,
|
||||
"year": lambda m: m.year,
|
||||
"rating": lambda m: m.rating,
|
||||
"duration": lambda m: m.duration,
|
||||
"title": lambda m: m.title,
|
||||
}
|
||||
|
||||
# What the page ordered by before it had a control, and still its default: a film series wants to be
|
||||
# read in release order, not by whenever Plex happened to ingest it.
|
||||
DEFAULT_MEMBER_SORT = "year"
|
||||
|
||||
|
||||
def sort_members(
|
||||
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
|
||||
) -> list[Member]:
|
||||
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
|
||||
|
||||
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
|
||||
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
|
||||
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
|
||||
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
|
||||
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
|
||||
|
||||
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
|
||||
compare as Python strings, which puts every accented initial after "z"."""
|
||||
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
|
||||
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
|
||||
if key is MEMBER_SORT_KEYS["title"]:
|
||||
key = by_title
|
||||
ordered = sorted(members, key=by_title) # stable secondary, applied first
|
||||
have = [m for m in ordered if key(m) is not None]
|
||||
missing = [m for m in ordered if key(m) is None]
|
||||
have.sort(key=key, reverse=sort_dir != "asc")
|
||||
return have + missing
|
||||
@@ -1,10 +1,11 @@
|
||||
"""Parse the "info page" rich metadata out of a full Plex metadata dict, and persist the subset that
|
||||
the detail endpoints serve from the DB row.
|
||||
|
||||
Shared by the live info-page path (routes.plex.browse) and the background rich re-sync (plex.sync) so
|
||||
the two never diverge — the same one-parser rule that keeps `people.py`'s cast handling consistent.
|
||||
`rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY fields onto a movie/show row
|
||||
(genres/directors/studio/content_rating already land from the cheap listing at `_apply_facet_fields`)."""
|
||||
Shared by the live info-page path (`plex.service.enrich_movie` / `enrich_show`) and the background
|
||||
rich re-sync (plex.sync) so the two never diverge — the same one-parser rule that keeps `people.py`'s
|
||||
cast handling consistent. `rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY
|
||||
fields onto a movie/show row (genres/directors/studio/content_rating already land from the cheap
|
||||
listing at `_apply_facet_fields`)."""
|
||||
|
||||
from app.plex import people
|
||||
from app.plex.plaintext import plain_text
|
||||
|
||||
@@ -0,0 +1,536 @@
|
||||
"""Plex Discover's reads: the rail-surface read-model.
|
||||
|
||||
A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
|
||||
Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
|
||||
separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
|
||||
recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
|
||||
every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
|
||||
their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
|
||||
card paths); the HU overlay coalesces only in the main grid.
|
||||
|
||||
The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
|
||||
Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
|
||||
grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
|
||||
the client hides). Genre-language matches the grid; duration is movie-only.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import sorted_words
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import SHOW_NEW, display_title, leaf_card, movie_card, show_card
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.query import (
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
top_art_keys,
|
||||
)
|
||||
from app.plex.tmdb import resolve_recs
|
||||
from app.userscope import owned_by
|
||||
|
||||
GENRE_TILES = 12
|
||||
SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows (6 total)
|
||||
|
||||
|
||||
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
|
||||
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
return (
|
||||
movie_ids if scope in ("movie", "both") else [],
|
||||
show_ids if scope in ("show", "both") else [],
|
||||
)
|
||||
|
||||
|
||||
def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
|
||||
"""Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
|
||||
q = apply_meta_filters(q, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
q = q.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
q = q.where(PlexItem.duration_s <= f.duration_max)
|
||||
return q
|
||||
|
||||
|
||||
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
|
||||
sts = rollup.item_state_map(db, uid, [it.id for it in items])
|
||||
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
|
||||
|
||||
|
||||
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
|
||||
statuses = rollup.state_map(db, uid, [s.id for s in shows])
|
||||
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
|
||||
|
||||
|
||||
def _recently_added(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Newest movies + shows, interleaved by `added_at` desc into one rail."""
|
||||
movies = (
|
||||
db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
.order_by(PlexItem.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
|
||||
)
|
||||
.order_by(PlexShow.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
|
||||
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
|
||||
# review found this one still on the older `(is not None, value)` form, i.e. one rule in two
|
||||
# spellings, with the two comments explaining it differently.
|
||||
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
|
||||
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
|
||||
top = tagged[:RAIL_LIMIT]
|
||||
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
|
||||
# share one), so the merge doesn't depend on object identity staying alive.
|
||||
top_movies = [o for _, kind, o in top if kind == "movie"]
|
||||
top_shows = [o for _, kind, o in top if kind == "show"]
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
|
||||
}
|
||||
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
|
||||
|
||||
|
||||
def _recently_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
|
||||
card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
|
||||
sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
|
||||
rows: list[tuple[datetime | None, dict]] = [] # (watched_at, card)
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
)
|
||||
for it, st in db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all():
|
||||
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
|
||||
if show_ids:
|
||||
eq = (
|
||||
select(PlexItem, PlexState, PlexShow)
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.join(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
for ep, st, sh in db.execute(eq).all():
|
||||
card = leaf_card(
|
||||
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
|
||||
)
|
||||
rows.append((st.watched_at, card))
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _continue_watching(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
|
||||
A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
|
||||
rows: list[tuple[datetime | None, dict]] = [] # (resumed_at, card)
|
||||
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.position_seconds > 0,
|
||||
# A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
|
||||
# `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
|
||||
or_(
|
||||
PlexState.status.is_(None),
|
||||
PlexState.status.notin_(["watched", "hidden"]),
|
||||
),
|
||||
)
|
||||
)
|
||||
mrows = db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.progress_updated_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for it, st in mrows:
|
||||
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
|
||||
|
||||
if show_ids:
|
||||
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
|
||||
# its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
|
||||
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
|
||||
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
|
||||
sq = apply_show_state_filter(
|
||||
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
|
||||
prog, prog.c.show_id == PlexShow.id
|
||||
),
|
||||
prog.c.total,
|
||||
prog.c.watched,
|
||||
prog.c.inprog,
|
||||
"in_progress",
|
||||
).where(PlexShow.library_id.in_(show_ids))
|
||||
srows = db.execute(
|
||||
apply_meta_filters(sq, PlexShow, p, lang)
|
||||
.order_by(prog.c.resumed_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for sh, resumed_at, total, watched, inprog in srows:
|
||||
state = rollup.show_state(total, watched, inprog)
|
||||
rows.append((resumed_at, show_card(sh, state, lang=lang)))
|
||||
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _genre_tiles(
|
||||
db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
|
||||
) -> list[dict]:
|
||||
"""The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
|
||||
The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
|
||||
The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
|
||||
counts: dict[str, int] = {}
|
||||
for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
|
||||
if not ids:
|
||||
continue
|
||||
gcol = genres_col(model, lang)
|
||||
base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
|
||||
model.library_id.in_(ids),
|
||||
func.jsonb_typeof(gcol) == "array",
|
||||
*meta_filter_conds(model, p, lang),
|
||||
)
|
||||
if model is PlexItem:
|
||||
base = base.where(model.kind == "movie")
|
||||
expanded = base.subquery()
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
counts[g] = counts.get(g, 0) + c
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
|
||||
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
|
||||
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
|
||||
art = top_art_keys(
|
||||
db,
|
||||
PlexItem,
|
||||
movie_ids,
|
||||
genres_col(PlexItem, lang),
|
||||
[g for g, _ in top],
|
||||
extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
|
||||
)
|
||||
return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
|
||||
|
||||
|
||||
def _because_you_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> dict | None:
|
||||
"""Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
|
||||
recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
|
||||
None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
|
||||
The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
|
||||
if not movie_ids:
|
||||
return None
|
||||
anchor = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
PlexItem.recommendations.isnot(None),
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(1)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if anchor is None:
|
||||
return None
|
||||
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
|
||||
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
|
||||
rks = [
|
||||
r["rating_key"]
|
||||
for r in resolved
|
||||
if r.get("in_library") and r.get("rating_key") != anchor.rating_key
|
||||
]
|
||||
if not rks:
|
||||
return None
|
||||
movies = {
|
||||
m.rating_key: m
|
||||
for m in db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
ordered_movies = [movies[rk] for rk in rks if rk in movies]
|
||||
ordered_shows = [shows[rk] for rk in rks if rk in shows]
|
||||
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
|
||||
}
|
||||
items: list[dict] = []
|
||||
for rk in rks:
|
||||
if rk in mcards:
|
||||
items.append(mcards[rk])
|
||||
elif rk in scards:
|
||||
items.append(scards[rk])
|
||||
if not items:
|
||||
return None
|
||||
return {
|
||||
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
def discover_page(db: Session, user: User, scope: str, lang: str, f: LibraryFilters) -> dict:
|
||||
"""The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
|
||||
sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
|
||||
is null when there's no anchor. `lang` is already normalised."""
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
return {
|
||||
"scope": scope,
|
||||
"continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
|
||||
"because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
}
|
||||
|
||||
|
||||
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
|
||||
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
|
||||
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
|
||||
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
|
||||
fresh = (
|
||||
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
|
||||
.scalars()
|
||||
.all()
|
||||
if exclude
|
||||
else []
|
||||
)
|
||||
if len(fresh) == per:
|
||||
return list(fresh)
|
||||
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
|
||||
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
|
||||
return [*fresh, *rest.all()]
|
||||
|
||||
|
||||
def _kept(
|
||||
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
|
||||
) -> list[dict]:
|
||||
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
|
||||
stays in it, marked watched, and one gone from the scope's libraries drops out."""
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexItem.rating_key.in_(keep),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
cards = {
|
||||
c["id"]: c
|
||||
for c in [
|
||||
*_movie_cards(db, uid, list(movies), lang),
|
||||
*_show_cards(db, uid, list(shows), lang),
|
||||
]
|
||||
}
|
||||
return [cards[k] for k in keep if k in cards]
|
||||
|
||||
|
||||
def surprise(
|
||||
db: Session,
|
||||
user: User,
|
||||
scope: str,
|
||||
lang: str,
|
||||
limit: int,
|
||||
exclude: list[str],
|
||||
keep: list[str],
|
||||
f: LibraryFilters,
|
||||
) -> dict:
|
||||
"""A fresh random sample for the reshuffling Surprise-me rail (a random draw each call), narrowed by
|
||||
the active filters. Only titles the user has NOT seen — a watched movie / a fully-watched show is
|
||||
excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
|
||||
SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
|
||||
one type."""
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
if keep:
|
||||
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
|
||||
per = min(limit, SURPRISE_PER_KIND)
|
||||
movies = []
|
||||
if movie_ids:
|
||||
# Exclude movies this user has watched.
|
||||
watched_movie = (
|
||||
select(PlexState.id)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
owned_by(PlexState.user_id, user),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
movies = _draw(
|
||||
db,
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
~watched_movie,
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
),
|
||||
PlexItem.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
shows = []
|
||||
if show_ids:
|
||||
# Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
|
||||
# as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
|
||||
prog = rollup.agg_subquery(db, user.id).subquery()
|
||||
not_fully_watched = or_(
|
||||
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
|
||||
)
|
||||
shows = _draw(
|
||||
db,
|
||||
apply_meta_filters(
|
||||
select(PlexShow)
|
||||
.outerjoin(prog, prog.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
|
||||
PlexShow,
|
||||
p,
|
||||
lang,
|
||||
),
|
||||
PlexShow.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
mcards = _movie_cards(db, user.id, movies, lang)
|
||||
scards = _show_cards(db, user.id, shows, lang)
|
||||
# Interleave movie/show cards, then trim to `limit`.
|
||||
items: list[dict] = []
|
||||
for i in range(max(len(mcards), len(scards))):
|
||||
if i < len(mcards):
|
||||
items.append(mcards[i])
|
||||
if i < len(scards):
|
||||
items.append(scards[i])
|
||||
return {"scope": scope, "items": items[:limit]}
|
||||
@@ -15,7 +15,7 @@ from fastapi import Response
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import PlexCollection, PlexItem, PlexSeason, PlexShow
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexSeason, PlexShow
|
||||
from app.plex.cache import CACHE_DIR
|
||||
from app.plex.lookups import by_rating_key
|
||||
|
||||
|
||||
@@ -0,0 +1,473 @@
|
||||
"""The Plex browse grid's reads: the unified movies + shows feed, its search, and its facets.
|
||||
|
||||
A user's watch state is joined through `userscope`; the feed and its facets share every filter
|
||||
(`plex.query`), so a facet count is the size of what the grid would show.
|
||||
"""
|
||||
|
||||
from sqlalchemy import Boolean, Integer, String, and_, cast, func, literal, null, or_, select
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import norm_lang
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.query import (
|
||||
FIELD_COLUMNS,
|
||||
PLAYBACK_CLASSES,
|
||||
TS_CONFIG,
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_movie_state_filter,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
field_recheck,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
movie_status_case,
|
||||
playable_conds,
|
||||
playback_class_expr,
|
||||
show_status_case,
|
||||
split_csv,
|
||||
summary_col,
|
||||
title_col,
|
||||
vector_col,
|
||||
vector_match,
|
||||
)
|
||||
from app.userscope import owned_by
|
||||
from app.utils import to_tsquery_str
|
||||
|
||||
|
||||
def sort_column(u, sort: str, lang: str = "en"):
|
||||
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
|
||||
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
|
||||
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
|
||||
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
|
||||
in the reader's alphabet (`collation`)."""
|
||||
return {
|
||||
"added": u.c.added_at,
|
||||
"title": collated(u.c.title, lang),
|
||||
"year": u.c.year,
|
||||
"rating": u.c.rating,
|
||||
"duration": u.c.duration_s,
|
||||
"release": u.c.rel,
|
||||
"lastwatched": u.c.last_watched,
|
||||
}.get(sort, u.c.added_at)
|
||||
|
||||
|
||||
def library_page(
|
||||
db: Session,
|
||||
user: User,
|
||||
*,
|
||||
scope: str,
|
||||
q: str | None,
|
||||
field: str,
|
||||
sort: str,
|
||||
sort_dir: str,
|
||||
show: str,
|
||||
lang: str,
|
||||
offset: int,
|
||||
limit: int,
|
||||
f: LibraryFilters,
|
||||
) -> tuple[int, list, list]:
|
||||
"""One page of the unified cross-library browse: (total, rows, episodes). `rows` mixes movies
|
||||
and shows (`kind` says which), scoped to movie|show|both, with the shared filters/search/watch
|
||||
state (a show's state is the aggregate of its episodes). On a search that also matches episodes
|
||||
(and shows are in scope), `episodes` holds the matching ones as (item, state, show) — the
|
||||
grouped 'Episodes' section. `lang` is already normalised."""
|
||||
p = f.as_dict()
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
|
||||
ts = to_tsquery_str(q) if q else None
|
||||
tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
|
||||
if field not in FIELD_COLUMNS and field != "all":
|
||||
field = "all"
|
||||
# `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
|
||||
# the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
|
||||
grid_field = field != "episode"
|
||||
recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
|
||||
|
||||
selects = []
|
||||
if want_movies and grid_field:
|
||||
st = aliased(PlexState)
|
||||
m_status = movie_status_case(st)
|
||||
mq = (
|
||||
select(
|
||||
PlexItem.rating_key.label("rk"),
|
||||
literal("movie").label("kind"),
|
||||
title_col(PlexItem, lang).label("title"),
|
||||
PlexItem.year.label("year"),
|
||||
PlexItem.rating.label("rating"),
|
||||
# Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
|
||||
func.left(summary_col(PlexItem, lang), 300).label("summary"),
|
||||
PlexItem.added_at.label("added_at"),
|
||||
PlexItem.originally_available_at.label("rel"),
|
||||
PlexItem.duration_s.label("duration_s"),
|
||||
PlexItem.playable.label("playable"),
|
||||
PlexItem.transcode_hw.label("transcode_hw"),
|
||||
cast(null(), Integer).label("season_count"),
|
||||
func.coalesce(st.position_seconds, 0).label("position_seconds"),
|
||||
m_status.label("status"),
|
||||
literal(False).label("resumable"),
|
||||
# "last watched" sort key: when the movie was last resumed, else when it was marked
|
||||
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
|
||||
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexItem, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(st, and_(st.item_id == PlexItem.id, owned_by(st.user_id, user)))
|
||||
.where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
|
||||
)
|
||||
mq = apply_movie_state_filter(mq, st, show)
|
||||
mq = apply_meta_filters(mq, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
mq = mq.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
mq = mq.where(PlexItem.duration_s <= f.duration_max)
|
||||
# Playback-class filter (movie-only — a show is an aggregate with no single class).
|
||||
mq = mq.where(*playable_conds(split_csv(f.playable)))
|
||||
if tsq is not None:
|
||||
mq = mq.where(vector_match(PlexItem, lang, tsq))
|
||||
if recheck is not None and (narrow := recheck(PlexItem, field, tsq, lang)) is not None:
|
||||
mq = mq.where(narrow)
|
||||
selects.append(mq)
|
||||
if want_shows and grid_field:
|
||||
agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
|
||||
tot = func.coalesce(agg.c.total, 0)
|
||||
wat = func.coalesce(agg.c.watched, 0)
|
||||
inp = func.coalesce(agg.c.inprog, 0)
|
||||
s_status = show_status_case(tot, wat, inp)
|
||||
sq = (
|
||||
select(
|
||||
PlexShow.rating_key.label("rk"),
|
||||
literal("show").label("kind"),
|
||||
title_col(PlexShow, lang).label("title"),
|
||||
PlexShow.year.label("year"),
|
||||
PlexShow.rating.label("rating"),
|
||||
func.left(summary_col(PlexShow, lang), 300).label("summary"),
|
||||
PlexShow.added_at.label("added_at"),
|
||||
PlexShow.originally_available_at.label("rel"),
|
||||
cast(null(), Integer).label("duration_s"),
|
||||
cast(null(), String).label("playable"),
|
||||
cast(null(), Boolean).label("transcode_hw"),
|
||||
PlexShow.child_count.label("season_count"),
|
||||
literal(0).label("position_seconds"),
|
||||
s_status.label("status"),
|
||||
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
|
||||
(inp > 0).label("resumable"),
|
||||
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
|
||||
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
|
||||
agg.c.resumed_at.label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexShow, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_lib_ids))
|
||||
)
|
||||
sq = apply_show_state_filter(sq, tot, wat, inp, show)
|
||||
sq = apply_meta_filters(sq, PlexShow, p, lang)
|
||||
if tsq is not None:
|
||||
sq = sq.where(vector_match(PlexShow, lang, tsq))
|
||||
if recheck is not None and (narrow := recheck(PlexShow, field, tsq, lang)) is not None:
|
||||
sq = sq.where(narrow)
|
||||
selects.append(sq)
|
||||
|
||||
total = 0
|
||||
rows: list = []
|
||||
if selects:
|
||||
union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
|
||||
u = union.subquery()
|
||||
total = db.scalar(select(func.count()).select_from(u)) or 0
|
||||
|
||||
if tsq is not None:
|
||||
order = [u.c.rank.desc()]
|
||||
else:
|
||||
col = sort_column(u, sort, lang)
|
||||
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
|
||||
order.append(u.c.rk.desc())
|
||||
rows = list(db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all())
|
||||
|
||||
# Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
|
||||
# and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
|
||||
# target the movie/show grid, so they suppress episodes.
|
||||
eps: list = []
|
||||
if tsq is not None and want_shows and field in ("all", "episode"):
|
||||
ep_st = aliased(PlexState)
|
||||
# In HU an episode is found by its Hungarian title too (the overlay fills it) — and
|
||||
# still by its English one, like `vector_match` does for the grid.
|
||||
if field == "episode":
|
||||
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
|
||||
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
|
||||
else:
|
||||
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
|
||||
hit = ep_vec.op("@@")(tsq)
|
||||
if lang == "hu":
|
||||
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
|
||||
ep_vec = ep_vec_hu
|
||||
eps = list(
|
||||
db.execute(
|
||||
select(PlexItem, ep_st, PlexShow)
|
||||
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, owned_by(ep_st.user_id, user)))
|
||||
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_lib_ids),
|
||||
hit,
|
||||
)
|
||||
.order_by(func.ts_rank(ep_vec, tsq).desc())
|
||||
.limit(24)
|
||||
).all()
|
||||
)
|
||||
|
||||
return total, rows, eps
|
||||
|
||||
|
||||
def facets(db: Session, user: User, *, scope: str, show: str, lang: str, f: LibraryFilters) -> dict:
|
||||
"""Available filter values for the unified library (scope movie|show|both), ADAPTED to the active
|
||||
filters (faceted search): each group's values/bounds are computed with all the OTHER active
|
||||
filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
|
||||
a multi-select group still lets you add more. Merged across the scope's libraries; duration is
|
||||
movie-only. (Search text is not factored in — facets track the sidebar filters.) `lang` is
|
||||
normalised here, after the empty answer."""
|
||||
empty = {
|
||||
"genres": [],
|
||||
"content_ratings": [],
|
||||
"studios": [],
|
||||
"playable": [],
|
||||
"year_min": None,
|
||||
"year_max": None,
|
||||
"rating_max": None,
|
||||
"duration_min": None,
|
||||
"duration_max": None,
|
||||
}
|
||||
all_movie_ids, all_show_ids = content_libs(db)
|
||||
movie_ids = all_movie_ids if scope in ("movie", "both") else []
|
||||
show_ids = all_show_ids if scope in ("show", "both") else []
|
||||
if not movie_ids and not show_ids:
|
||||
return empty
|
||||
lang = norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
|
||||
uid = user.id
|
||||
|
||||
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
|
||||
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
|
||||
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
|
||||
# The same filter on the same rows, so the counts do not change
|
||||
# (test_plex_facets_grid_parity).
|
||||
shows_in_scope: list[int] | None = None
|
||||
if show_ids and show in ("watched", "in_progress", "unwatched"):
|
||||
agg = rollup.agg_subquery(db, uid).subquery()
|
||||
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
in_scope = apply_show_state_filter(
|
||||
in_scope,
|
||||
func.coalesce(agg.c.total, 0),
|
||||
func.coalesce(agg.c.watched, 0),
|
||||
func.coalesce(agg.c.inprog, 0),
|
||||
show,
|
||||
)
|
||||
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
|
||||
|
||||
def wsq(query, model, ids, pp, kind_movie, with_dur):
|
||||
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
|
||||
facet aggregation query, so the facets reflect exactly the visible result set."""
|
||||
query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
|
||||
if kind_movie:
|
||||
query = query.where(model.kind == "movie")
|
||||
if with_dur and pp.get("duration_min") is not None:
|
||||
query = query.where(model.duration_s >= pp["duration_min"])
|
||||
if with_dur and pp.get("duration_max") is not None:
|
||||
query = query.where(model.duration_s <= pp["duration_max"])
|
||||
# Playback-class filter (movie-only) — applied for the OTHER groups so an active class
|
||||
# narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
|
||||
query = query.where(*playable_conds(split_csv(pp.get("playable"))))
|
||||
# Join watch-state and reuse the grid's exact movie state filter (unified_library) —
|
||||
# hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
|
||||
st = aliased(PlexState)
|
||||
query = query.outerjoin(st, and_(st.item_id == model.id, owned_by(st.user_id, uid)))
|
||||
query = apply_movie_state_filter(query, st, show)
|
||||
elif shows_in_scope is not None:
|
||||
query = query.where(model.id.in_(shows_in_scope))
|
||||
return query
|
||||
|
||||
genre_counts: dict[str, int] = {}
|
||||
cr_counts: dict[str, int] = {}
|
||||
studio_counts: dict[str, int] = {}
|
||||
playable_counts: dict[str, int] = {}
|
||||
years: list[int] = []
|
||||
ratings: list[float] = []
|
||||
durs: list[int] = []
|
||||
|
||||
def per_table(model, ids, kind_movie: bool) -> None:
|
||||
if not ids:
|
||||
return
|
||||
# Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
|
||||
# OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
|
||||
# offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
|
||||
# the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
|
||||
# every matching title carries them, so they stay togglable.)
|
||||
genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
|
||||
# Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
|
||||
# `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
|
||||
# jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
|
||||
# so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
|
||||
# whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
|
||||
# In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
|
||||
gcol = genres_col(model, lang)
|
||||
expanded = (
|
||||
wsq(
|
||||
select(func.jsonb_array_elements_text(gcol).label("g")),
|
||||
model,
|
||||
ids,
|
||||
genre_pp,
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(func.jsonb_typeof(gcol) == "array")
|
||||
.subquery()
|
||||
)
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
genre_counts[g] = genre_counts.get(g, 0) + c
|
||||
# Content ratings — exclude its own.
|
||||
crq = (
|
||||
wsq(
|
||||
select(model.content_rating, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "content_ratings": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.content_rating.isnot(None))
|
||||
.group_by(model.content_rating)
|
||||
)
|
||||
for cr, c in db.execute(crq).all():
|
||||
cr_counts[cr] = cr_counts.get(cr, 0) + c
|
||||
# Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
|
||||
stq = (
|
||||
wsq(
|
||||
select(model.studio, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "studios": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.studio.isnot(None))
|
||||
.group_by(model.studio)
|
||||
)
|
||||
for s, c in db.execute(stq).all():
|
||||
studio_counts[s] = studio_counts.get(s, 0) + c
|
||||
# Playback class (movie-only) — exclude its own so each class stays an addable OR option.
|
||||
if kind_movie:
|
||||
pcls = playback_class_expr()
|
||||
pcq = (
|
||||
wsq(
|
||||
select(pcls.label("cls"), func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "playable": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(pcls.isnot(None))
|
||||
.group_by(pcls)
|
||||
)
|
||||
for cls, c in db.execute(pcq).all():
|
||||
playable_counts[cls] = playable_counts.get(cls, 0) + c
|
||||
# Year bounds — exclude the year filter.
|
||||
by = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.year), func.max(model.year)),
|
||||
model,
|
||||
ids,
|
||||
{**p, "year_min": None, "year_max": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
).one()
|
||||
if by[0] is not None:
|
||||
years.append(by[0])
|
||||
if by[1] is not None:
|
||||
years.append(by[1])
|
||||
# Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
|
||||
# rating filter and the UI badge use (IMDb preferred, audience fallback).
|
||||
br = db.scalar(
|
||||
wsq(
|
||||
select(func.max(func.coalesce(model.imdb_rating, model.rating))),
|
||||
model,
|
||||
ids,
|
||||
{**p, "rating_min": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
)
|
||||
if br is not None:
|
||||
ratings.append(float(br))
|
||||
# Duration bounds (movies) — exclude the duration filter.
|
||||
if kind_movie:
|
||||
bd = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.duration_s), func.max(model.duration_s)),
|
||||
model,
|
||||
ids,
|
||||
p,
|
||||
kind_movie,
|
||||
False,
|
||||
)
|
||||
).one()
|
||||
if bd[0] is not None:
|
||||
durs.append(bd[0])
|
||||
if bd[1] is not None:
|
||||
durs.append(bd[1])
|
||||
|
||||
per_table(PlexItem, movie_ids, True)
|
||||
per_table(PlexShow, show_ids, False)
|
||||
# Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
|
||||
# be absent): the sidebar renders genres solely from this list and hides the whole genre section —
|
||||
# chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
|
||||
# undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
|
||||
for g in split_csv(f.genres):
|
||||
genre_counts.setdefault(g, 0)
|
||||
# Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
|
||||
for s in split_csv(f.studios):
|
||||
studio_counts.setdefault(s, 0)
|
||||
# Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
|
||||
# section when its facet list is empty, which would trap a zero-result selection with no way to undo).
|
||||
for c in split_csv(f.playable):
|
||||
if c in PLAYBACK_CLASSES:
|
||||
playable_counts.setdefault(c, 0)
|
||||
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
|
||||
return {
|
||||
"genres": [
|
||||
{"value": g, "count": c}
|
||||
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
|
||||
],
|
||||
"content_ratings": [
|
||||
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
|
||||
],
|
||||
"studios": [
|
||||
{"value": s, "count": c}
|
||||
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
|
||||
],
|
||||
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
|
||||
"playable": [
|
||||
{"value": c, "count": playable_counts[c]}
|
||||
for c in PLAYBACK_CLASSES
|
||||
if c in playable_counts
|
||||
],
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"rating_max": max(ratings) if ratings else None,
|
||||
"duration_min": min(durs) if durs else None,
|
||||
"duration_max": max(durs) if durs else None,
|
||||
}
|
||||
@@ -0,0 +1,463 @@
|
||||
"""Plex's tables: a self-contained catalog mirrored from a local Plex server — libraries, shows,
|
||||
seasons, items, people, collections, the server's playlists — and each user's private part of it:
|
||||
watch state, the account link, their own playlists. Only this unit writes them; another unit reads
|
||||
them along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
Computed,
|
||||
Date,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin, UpdatedAtMixin
|
||||
|
||||
|
||||
# --- Plex integration (optional) -------------------------------------------------------------
|
||||
# A dedicated, self-contained catalog mirrored from a locally-reachable Plex server. Playback is
|
||||
# from the LOCAL physical file (Plex is metadata only); per-user watch state lives in plex_states
|
||||
# so it works for users without a Plex account. Mirrors the app's "shared catalog, private
|
||||
# per-user state" model. The weighted FTS search_vector uses the same unaccent_simple config as
|
||||
# `videos` (migrations 0031/0032).
|
||||
|
||||
|
||||
class PlexLibrary(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex library section. Only `enabled` sections are exposed in the feed.
|
||||
`plex_key` is the section id on the Plex server."""
|
||||
|
||||
__tablename__ = "plex_libraries"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
plex_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
title: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16)) # "movie" | "show"
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# HU-overlay support. `metadata_lang` tags the section's metadata language. An OVERLAY
|
||||
# library (overlay_of set) is the Hungarian-metadata twin of another section: it is never browsable
|
||||
# on its own — its titles' HU fields are written onto the matched EN rows (paired by `guid`).
|
||||
metadata_lang: Mapped[str] = mapped_column(String(8), default="en", server_default="en")
|
||||
overlay_of: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="SET NULL"), index=True
|
||||
)
|
||||
|
||||
|
||||
class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A TV show (grandparent of episodes) — holds poster + summary for the drill-down page, plus
|
||||
the same filterable metadata as movies (genres/rating/content_rating/studio/people) so the TV
|
||||
grid can be filtered/sorted like the movie grid. All mirrored cheaply from the show listing."""
|
||||
|
||||
__tablename__ = "plex_shows"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_shows_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
# ix_plex_shows_collections_gin already created with collections (migration 0047).
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# Canonical external id (e.g. "tmdb:12345") — the key that pairs this EN row with its HU-overlay
|
||||
# twin during sync. Indexed for the guid->row overlay match.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(
|
||||
String(512)
|
||||
) # Plex thumb path (proxied, not stored)
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer) # seasons
|
||||
collection_keys: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # Plex collections this show is in
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the show section listing (parallels PlexItem).
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True) # Plex audienceRating (~IMDb)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True) # network/studio
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True) # first air date
|
||||
# none_as_null: an empty tag list must persist as SQL NULL, not JSON `null`. Without it a Python
|
||||
# None round-trips as a JSON scalar that jsonb_array_elements_text() can't unnest ("cannot extract
|
||||
# elements from a scalar") — the trap a metadata-poor library sprang on /facets. Same on plex_items.
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
people_text: Mapped[str | None] = mapped_column(
|
||||
Text
|
||||
) # cast+directors, folded into search_vector
|
||||
# Set once the show's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: see the identical block on plex_items for the full rationale. Additive;
|
||||
# served from the row; NULL until enriched. (`budget`/`revenue` stay NULL for shows — TV has none.) ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched,
|
||||
# so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexSeason(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A season under a show — used to order episodes for prev/next stepping."""
|
||||
|
||||
__tablename__ = "plex_seasons"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
show_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
season_number: Mapped[int | None] = mapped_column(Integer, index=True)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
|
||||
|
||||
class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A playable LEAF — a movie or a single episode. This is what the feed lists and the player
|
||||
plays. Movies carry `library_id` and no parents; episodes carry show/season refs +
|
||||
season/episode numbers. `file_path` is the Plex-side absolute path of the physical media,
|
||||
mapped through `plex_path_map` to a local mount at playback time. `playable` is the
|
||||
ffprobe-derived browser-compatibility class (direct | remux | transcode)."""
|
||||
|
||||
__tablename__ = "plex_items"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_items_show_order", "show_id", "season_number", "episode_number"),
|
||||
Index("ix_plex_items_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_items_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_items_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_items_collections_gin", "collection_keys", postgresql_using="gin"),
|
||||
Index("ix_plex_items_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
kind: Mapped[str] = mapped_column(String(16), index=True) # "movie" | "episode"
|
||||
# Canonical external id (e.g. "tmdb:12345") — pairs a MOVIE row with its HU-overlay twin during
|
||||
# sync; indexed for the guid->row match. Episodes carry it too but it's unused for them.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_seasons.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_number: Mapped[int | None] = mapped_column(Integer)
|
||||
episode_number: Mapped[int | None] = mapped_column(Integer)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
duration_s: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
cast: Mapped[list | None] = mapped_column(JSON) # [role names]
|
||||
# Physical media (from the Plex Part).
|
||||
file_path: Mapped[str | None] = mapped_column(Text) # Plex-side absolute path
|
||||
part_key: Mapped[str | None] = mapped_column(String(255)) # /library/parts/... (fallback fetch)
|
||||
container: Mapped[str | None] = mapped_column(String(16))
|
||||
codec_video: Mapped[str | None] = mapped_column(String(32))
|
||||
codec_audio: Mapped[str | None] = mapped_column(String(32))
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Browser-playability class from ffprobe: "direct" | "remux" | "transcode" (None = unprobed).
|
||||
playable: Mapped[str | None] = mapped_column(String(12), index=True)
|
||||
probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Transcode HW-capability facet (the playback-class filter). NULL for non-transcode items; for a
|
||||
# `playable="transcode"` item: True = the prod iGPU can hardware-transcode it (codec is HW-decodable
|
||||
# AND the source is 8-bit SDR), False = software libx264 only (AV1/XviD, or a 10-bit/HDR source that
|
||||
# breaks the VAAPI nv12 pipeline — see plex/stream.py). Derived by `plex.sync.transcode_hw_capable`:
|
||||
# set codec-only at sync (bit depth unknown from the cheap listing), then refined by the bit-depth
|
||||
# backfill (`_probe_media_batch`) which downgrades a 10-bit/HDR HEVC to False. The one filterable
|
||||
# home of the invariant `_want_hw`'s source gate computes at play time — kept in sync via the shared
|
||||
# classifier so the catalog facet and the runtime encoder choice never disagree by codec.
|
||||
transcode_hw: Mapped[bool | None] = mapped_column(Boolean, index=True)
|
||||
# Video bit depth (8/10/12…) from the Plex media Stream, filled by the bit-depth backfill. NULL =
|
||||
# not yet probed. Surfaced on the info page (e.g. "HEVC · 10-bit") and feeds `transcode_hw`.
|
||||
video_bit_depth: Mapped[int | None] = mapped_column(Integer)
|
||||
# Watermark for the bit-depth backfill: when this item's media Stream was last probed for
|
||||
# bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
|
||||
# probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
|
||||
media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
|
||||
# `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
|
||||
# filters (GIN-indexed) and double as content signals for a future watch-habit recommender.
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True)
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# rating_keys of the Plex collections this movie belongs to (GIN — filter + card-strip siblings).
|
||||
collection_keys: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Cast + director names, space-joined — folded into search_vector (weight B) so the search box
|
||||
# finds titles by a person's name (and ranks them above summary-only mentions).
|
||||
people_text: Mapped[str | None] = mapped_column(Text)
|
||||
# Set once the movie's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: matched by the canonical `guid` ("tmdb:12345"), filled by the TMDB
|
||||
# backfill job, SERVED FROM THIS ROW (never a live TMDB call). Additive — imdb_rating/rating
|
||||
# above stay Plex-sourced; `tmdb_rating` is the distinct TMDb community score. NULL until enriched;
|
||||
# `tmdb_enriched_at` is the "tried" watermark (stamped on a definitive no-match too, so an
|
||||
# unresolvable title isn't re-fetched every run; left NULL only on a transient fetch error). ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger) # USD (movies; TV has none)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
# [{tmdb_id, title, poster, media_type}] — resolved to library rows (by guid) at serve time.
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB backdrop file_paths [str] (crossfade pool)
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched
|
||||
# (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A small cast/crew headshot cache, keyed by name, so person filter-chips (the cards for the
|
||||
actors currently in the filter) can show a photo + count without a live per-card metadata fetch.
|
||||
Populated opportunistically when an item's RICH metadata is fetched — the info-page view and the
|
||||
periodic rich re-sync — since the cheap section listing that drives the main sync only carries the
|
||||
top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
|
||||
removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
|
||||
metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
|
||||
row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
|
||||
(the headshot FILE lives in the image cache, filled by the people_enrich job)."""
|
||||
|
||||
__tablename__ = "plex_people"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String(255), unique=True, index=True)
|
||||
photo_thumb: Mapped[str | None] = mapped_column(Text) # proxied /person-image URL, or None
|
||||
# TMDB person id, linked by name from a title's TMDB credits (the /person
|
||||
# biography fetch builds on it). NULL until a TMDB-enriched title names this person.
|
||||
tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
|
||||
# --- TMDB person enrichment: fetched from /person/{tmdb_person_id} by the people_enrich job,
|
||||
# or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
|
||||
# no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
|
||||
# media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
|
||||
biography: Mapped[str | None] = mapped_column(Text)
|
||||
# TMDB's Hungarian biography, from the same request's `translations`; a Hungarian reader is
|
||||
# served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
|
||||
# keywords have no Hungarian on TMDB, so they stay in the instance language.
|
||||
biography_hu: Mapped[str | None] = mapped_column(Text)
|
||||
birthday: Mapped[str | None] = mapped_column(String(10)) # "YYYY-MM-DD", served as-is
|
||||
deathday: Mapped[str | None] = mapped_column(String(10))
|
||||
place_of_birth: Mapped[str | None] = mapped_column(Text)
|
||||
known_for_department: Mapped[str | None] = mapped_column(String(64))
|
||||
also_known_as: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
known_for: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{tmdb_id, media_type, title}]
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexCollection(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex collection (a grouping of movies or shows — franchises like Avatar, curated
|
||||
sets like MCU, or auto/community lists like IMDb Top 250). Mirrored in full by the background
|
||||
sync so reads never hit Plex's slow collection queries. Membership is stored as `collection_keys`
|
||||
on the member movies (plex_items) / shows (plex_shows). `smart` = Plex's query-based flag;
|
||||
`editable` = we may write changes back to Plex (default False = read-only; set for curated ones)."""
|
||||
|
||||
__tablename__ = "plex_collections"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer)
|
||||
smart: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
editable: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a Plex item — mirrors VideoState. Lives in Siftlode's own DB so
|
||||
it works for users WITHOUT a Plex account (the whole point of the access-layer design).
|
||||
`synced_to_plex` marks whether this state was pushed back to a linked Plex account."""
|
||||
|
||||
__tablename__ = "plex_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
synced_to_plex: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class PlexLink(Base, UpdatedAtMixin):
|
||||
"""Links a Siftlode user to a Plex account for two-way watch-state sync (one row per user).
|
||||
|
||||
For the owner (the MVP scope) the row uses the server's admin token — `uses_admin=True`,
|
||||
`plex_token_enc=None` — so no separate Plex login is needed; the admin token already reads/writes
|
||||
the owner account's watch state. `plex_account_id` is that Plex account's numeric id, used to
|
||||
filter the shared watch-history feed to this user (Phase C). Reserved for later: a friend can link
|
||||
their OWN Plex account via PIN OAuth, storing a personal `plex_token_enc` (uses_admin=False) — the
|
||||
sync loop then iterates link rows uniformly.
|
||||
|
||||
`initial_import_done` guards the one-time "Plex is master" import; `last_watch_sync_at` is the
|
||||
high-water mark for the incremental Plex→Siftlode pass."""
|
||||
|
||||
__tablename__ = "plex_link"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), unique=True, index=True
|
||||
)
|
||||
uses_admin: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
plex_token_enc: Mapped[str | None] = mapped_column(Text) # P5b friend tokens (encrypted)
|
||||
plex_account_id: Mapped[int | None] = mapped_column(Integer)
|
||||
plex_username: Mapped[str | None] = mapped_column(String(255))
|
||||
sync_enabled: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
initial_import_done: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false"
|
||||
)
|
||||
last_watch_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
linked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexPlaylist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user ORDERED watch-list of Plex items — Siftlode-native (lives in our own DB, works for
|
||||
users WITHOUT a Plex account, like plex_states). Unlike collections (shared, library-level), a
|
||||
playlist is owned by one user and its items are ordered. Optional Plex-direction sync is a later
|
||||
phase (`plex_rating_key` set once pushed back to the owner's Plex account)."""
|
||||
|
||||
__tablename__ = "plex_playlists"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
plex_rating_key: Mapped[str | None] = mapped_column(String(32))
|
||||
|
||||
|
||||
class PlexPlaylistItem(Base):
|
||||
"""One ordered entry in a playlist. `position` (0-based) orders playback."""
|
||||
|
||||
__tablename__ = "plex_playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("plex_playlists.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
@@ -13,7 +13,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex.client import PlexClient
|
||||
from app.plex.genres import hu_genres
|
||||
|
||||
@@ -15,7 +15,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexPerson
|
||||
from app.plex.models import PlexPerson
|
||||
from app.plex import imgcache
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
|
||||
|
||||
@@ -0,0 +1,507 @@
|
||||
"""Plex's reads for the HTTP layer: the mirrored rows a page shows, and a user's watch state on them.
|
||||
|
||||
A user's own rows (`PlexState`) are read through `userscope`, so "someone else's row" is the same
|
||||
answer as a missing one. A rating key with a NUL in it names no row (`lookups.by_rating_key`).
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexLink,
|
||||
PlexPerson,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
)
|
||||
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app.utils import like_escape
|
||||
|
||||
# --- one row ---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def item(db: Session, rating_key: str) -> PlexItem | None:
|
||||
return by_rating_key(db, PlexItem, rating_key)
|
||||
|
||||
|
||||
def movie(db: Session, rating_key: str) -> PlexItem | None:
|
||||
"""The movie with this key — an episode's key is not one."""
|
||||
it = item(db, rating_key)
|
||||
return it if it is not None and it.kind == "movie" else None
|
||||
|
||||
|
||||
def show(db: Session, rating_key: str) -> PlexShow | None:
|
||||
return by_rating_key(db, PlexShow, rating_key)
|
||||
|
||||
|
||||
def collection(db: Session, rating_key: str) -> PlexCollection | None:
|
||||
return by_rating_key(db, PlexCollection, rating_key)
|
||||
|
||||
|
||||
def season(db: Session, rating_key: str) -> PlexSeason | None:
|
||||
return by_rating_key(db, PlexSeason, rating_key)
|
||||
|
||||
|
||||
def season_episodes(db: Session, se: PlexSeason) -> list[PlexItem]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode")
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def show_by_id(db: Session, show_id: int) -> PlexShow | None:
|
||||
return db.get(PlexShow, show_id)
|
||||
|
||||
|
||||
def season_by_id(db: Session, season_id: int) -> PlexSeason | None:
|
||||
return db.get(PlexSeason, season_id)
|
||||
|
||||
|
||||
def library(db: Session, library_id: int) -> PlexLibrary | None:
|
||||
return db.get(PlexLibrary, library_id)
|
||||
|
||||
|
||||
def library_by_key(db: Session, plex_key: str) -> PlexLibrary | None:
|
||||
return (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(plex_key)))
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def library_key(db: Session, library_id: int | None) -> str | None:
|
||||
"""The Plex section key for a library id — the info/show page passes it to the admin collection
|
||||
editor (in the unified cross-library view there's no single library prop otherwise)."""
|
||||
if library_id is None:
|
||||
return None
|
||||
lib = db.get(PlexLibrary, library_id)
|
||||
return lib.plex_key if lib else None
|
||||
|
||||
|
||||
def person(db: Session, name: str) -> PlexPerson | None:
|
||||
return db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
|
||||
|
||||
|
||||
# --- a user's watch state --------------------------------------------------------------------
|
||||
|
||||
|
||||
def state(db: Session, user: User | int, item_id: int) -> PlexState | None:
|
||||
return (
|
||||
db.execute(
|
||||
select(PlexState).where(owned_by(PlexState.user_id, user), PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
# --- a show ----------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def seasons(db: Session, sh: PlexShow) -> list[PlexSeason]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexSeason)
|
||||
.where(PlexSeason.show_id == sh.id)
|
||||
.order_by(PlexSeason.season_number.asc().nullsfirst())
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def episodes(db: Session, sh: PlexShow) -> list[PlexItem]:
|
||||
"""A show's episodes in season/episode order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def episode_keys(db: Session, show_id: int) -> list[str]:
|
||||
"""The rating keys of a show's episodes, in the order `episodes` gives."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem.rating_key)
|
||||
.where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def shows_by_keys(db: Session, rating_keys: list[str], exclude: int) -> dict[str, PlexShow]:
|
||||
"""The mirrored shows among these keys, but `exclude`, by key."""
|
||||
return {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rating_keys), PlexShow.id != exclude)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
|
||||
# --- libraries -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def browsable_libraries(db: Session, lang: str) -> list[tuple[PlexLibrary, int]]:
|
||||
"""The mirrored libraries a user can browse, each with its title count (movies, or shows). An
|
||||
HU overlay library is left out: it is never browsed, it only feeds its EN twin's Hungarian
|
||||
metadata."""
|
||||
libs = db.execute(
|
||||
select(PlexLibrary)
|
||||
.where(PlexLibrary.overlay_of.is_(None))
|
||||
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang))
|
||||
).scalars()
|
||||
out = []
|
||||
for lib in libs:
|
||||
if lib.kind == "movie":
|
||||
count = db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.library_id == lib.id, PlexItem.kind == "movie"
|
||||
)
|
||||
)
|
||||
else:
|
||||
count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
|
||||
out.append((lib, count or 0))
|
||||
return out
|
||||
|
||||
|
||||
# --- people ----------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def person_photos(db: Session, names: list[str]) -> dict[str, str | None]:
|
||||
return {
|
||||
p.name: p.photo_thumb
|
||||
for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(names))).scalars()
|
||||
}
|
||||
|
||||
|
||||
def cast_count(db: Session, name: str, movie_lib_ids: list[int], show_lib_ids: list[int]) -> int:
|
||||
"""How many movies (in `movie_lib_ids`) and shows (in `show_lib_ids`) name this person in their
|
||||
cast. An empty id list leaves its side out."""
|
||||
count = 0
|
||||
if movie_lib_ids:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
PlexItem.cast_names.contains([name]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
if show_lib_ids:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexShow.id)).where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
PlexShow.cast_names.contains([name]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return count
|
||||
|
||||
|
||||
def filmography(
|
||||
db: Session,
|
||||
kind: Literal["movie", "show"],
|
||||
name: str,
|
||||
lib_ids: list[int],
|
||||
lang: str,
|
||||
limit: int,
|
||||
) -> list:
|
||||
"""The movies or shows in these libraries whose cast or directors name this person, newest
|
||||
first."""
|
||||
model: type[PlexItem] | type[PlexShow] = PlexItem if kind == "movie" else PlexShow
|
||||
query = select(model).where(
|
||||
model.library_id.in_(lib_ids),
|
||||
or_(model.cast_names.contains([name]), model.directors.contains([name])),
|
||||
)
|
||||
if model is PlexItem:
|
||||
query = query.where(PlexItem.kind == "movie")
|
||||
return list(
|
||||
db.execute(
|
||||
query.order_by(
|
||||
model.year.desc().nullslast(), collated(title_col(model, lang), lang)
|
||||
).limit(limit)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- collections -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def collections_page(
|
||||
db: Session, library: str | None, q: str | None, keys: list[str] | None, lang: str
|
||||
) -> list[PlexCollection]:
|
||||
"""Collections, most-populated first: one library's (by its plex key), else every enabled
|
||||
library's; name-filtered by `q`; narrowed to `keys` when given (an empty list asks for none)."""
|
||||
if library is not None:
|
||||
lib = library_by_key(db, library)
|
||||
if lib is None:
|
||||
return []
|
||||
query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
|
||||
else:
|
||||
enabled_ids = list(
|
||||
db.execute(
|
||||
select(PlexLibrary.id).where(
|
||||
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
if not enabled_ids:
|
||||
return []
|
||||
query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
|
||||
term = (q or "").strip()
|
||||
if term:
|
||||
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
|
||||
if keys is not None:
|
||||
# An explicit but empty `keys=` asks for nothing, which is not the same as no `keys` (every
|
||||
# collection): answered here without a query. The empty IN () would also match nothing.
|
||||
if not keys:
|
||||
return []
|
||||
query = query.where(PlexCollection.rating_key.in_(keys))
|
||||
return list(
|
||||
db.execute(
|
||||
query.order_by(
|
||||
PlexCollection.child_count.desc().nullslast(), collated(PlexCollection.title, lang)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def library_collections(db: Session, library_id: int, lang: str) -> list[PlexCollection]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexCollection)
|
||||
.where(PlexCollection.library_id == library_id)
|
||||
.order_by(collated(PlexCollection.title, lang))
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def collections_by_keys(db: Session, keys: list[str]) -> list[PlexCollection]:
|
||||
return list(
|
||||
db.execute(select(PlexCollection).where(PlexCollection.rating_key.in_(keys))).scalars()
|
||||
)
|
||||
|
||||
|
||||
def collection_named(db: Session, library_id: int, title: str) -> PlexCollection | None:
|
||||
"""A collection of this library with this name, case aside."""
|
||||
return (
|
||||
db.execute(
|
||||
select(PlexCollection).where(
|
||||
PlexCollection.library_id == library_id,
|
||||
func.lower(PlexCollection.title) == title.lower(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def collection_members(
|
||||
db: Session, rating_key: str, lang: str
|
||||
) -> tuple[list[PlexItem], list[PlexShow]]:
|
||||
"""A collection's movies and shows, each by year then title."""
|
||||
movies = db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
).scalars()
|
||||
shows = db.execute(
|
||||
select(PlexShow)
|
||||
.where(PlexShow.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
|
||||
).scalars()
|
||||
return list(movies), list(shows)
|
||||
|
||||
|
||||
def library_movies_in(
|
||||
db: Session, library_id: int, rating_key: str, lang: str, limit: int
|
||||
) -> list[PlexItem]:
|
||||
"""The movies of one library in this collection, by year then title."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.library_id == library_id,
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.collection_keys.contains([rating_key]),
|
||||
)
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
.limit(limit)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def show_rollups(db: Session, user_id: int, show_ids: list[int]) -> list:
|
||||
"""The user's episode rollup (total, watched, in progress, last resumed) of each of these shows
|
||||
that has episodes."""
|
||||
if not show_ids:
|
||||
return []
|
||||
return list(
|
||||
db.execute(
|
||||
rollup.agg_subquery(db, user_id, include_resumed=True).where(
|
||||
PlexItem.show_id.in_(show_ids)
|
||||
)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def collection_tiles(
|
||||
db: Session, keys: list[str]
|
||||
) -> tuple[dict[str, list[str]], dict[str, list[str]], dict[str, int]]:
|
||||
"""What each collection's tile shows: the top two movie members' keys, the top two show members'
|
||||
keys (a collection lives in one library, so exactly one side has any), and its movies' total
|
||||
runtime (a show has no single duration)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
|
||||
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
|
||||
runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
|
||||
return art_m, art_s, runtime
|
||||
|
||||
|
||||
# --- the admin page --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def sample_file_path(db: Session) -> str | None:
|
||||
"""Any mirrored item's Plex-side file path — a sample to test the mount and path map against."""
|
||||
return db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
|
||||
|
||||
|
||||
def watch_link(db: Session, user: User | int) -> PlexLink | None:
|
||||
"""The user's Plex watch-sync link, if they have one."""
|
||||
return db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user))).scalars().first()
|
||||
|
||||
|
||||
# --- a user's playlists ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_playlist(
|
||||
db: Session, user: User, playlist_id: int, *, lock: bool = False
|
||||
) -> PlexPlaylist | None:
|
||||
"""The user's playlist with this id; someone else's is the same answer as a missing one. `lock`
|
||||
takes it FOR UPDATE: every add, remove and reorder does, so two of them never read the same max
|
||||
position, and the list row is always locked before its items."""
|
||||
return owned_row(db, PlexPlaylist, playlist_id, user, lock=lock)
|
||||
|
||||
|
||||
def playlists_of(db: Session, user: User) -> list[PlexPlaylist]:
|
||||
"""The user's playlists, newest-touched first."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexPlaylist)
|
||||
.where(owned_by(PlexPlaylist.user_id, user))
|
||||
.order_by(PlexPlaylist.updated_at.desc())
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def playlist_count(db: Session, playlist_id: int) -> int:
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def _playlist_rows(playlist_id: int):
|
||||
return (
|
||||
select(PlexPlaylistItem, PlexItem)
|
||||
.join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
|
||||
.where(PlexPlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
)
|
||||
|
||||
|
||||
def playlist_cover(db: Session, playlist_id: int) -> str | None:
|
||||
"""The rating key of a playlist's first item — its cover."""
|
||||
first = db.execute(_playlist_rows(playlist_id)).first()
|
||||
return first[1].rating_key if first else None
|
||||
|
||||
|
||||
def playlist_items(db: Session, playlist_id: int) -> list[PlexItem]:
|
||||
"""A playlist's items in play order."""
|
||||
return [it for _row, it in db.execute(_playlist_rows(playlist_id)).all()]
|
||||
|
||||
|
||||
def playlist_holds(db: Session, playlist_id: int, item_ids: set[int]) -> int:
|
||||
"""How many of these items the playlist holds."""
|
||||
if not item_ids:
|
||||
return 0
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id,
|
||||
PlexPlaylistItem.item_id.in_(item_ids),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def item_id(db: Session, rating_key: str) -> int | None:
|
||||
"""The id of the mirrored item with this key. Unlike `item`, a NUL in the key reaches the query,
|
||||
which the app answers as a 422 (`main._nul_in_text`), as for every other compared string."""
|
||||
return db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(rating_key)))
|
||||
|
||||
|
||||
def item_ids(db: Session, rating_keys: list[str]) -> set[int]:
|
||||
"""The ids of the mirrored items with these keys."""
|
||||
if not rating_keys:
|
||||
return set()
|
||||
return set(
|
||||
db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rating_keys))).scalars()
|
||||
)
|
||||
|
||||
|
||||
def items_by_keys(db: Session, rating_keys: list) -> list[PlexItem]:
|
||||
"""The mirrored items with these keys, in the given order; an unknown key and a repeat drop out."""
|
||||
order = [str(k) for k in rating_keys if str(k)]
|
||||
if not order:
|
||||
return []
|
||||
by_rk = {
|
||||
it.rating_key: it
|
||||
for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
|
||||
}
|
||||
seen: set[str] = set()
|
||||
out: list[PlexItem] = []
|
||||
for rk in order:
|
||||
it = by_rk.get(rk)
|
||||
if it is not None and rk not in seen:
|
||||
seen.add(rk)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
|
||||
def shows_by_ids(db: Session, show_ids: set[int]) -> dict[int, PlexShow]:
|
||||
if not show_ids:
|
||||
return {}
|
||||
return {
|
||||
sh.id: sh for sh in db.execute(select(PlexShow).where(PlexShow.id.in_(show_ids))).scalars()
|
||||
}
|
||||
@@ -10,7 +10,7 @@ from sqlalchemy import and_, case, func, or_, select, type_coerce
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import PlexItem, PlexLibrary
|
||||
from app.plex.models import PlexItem, PlexLibrary
|
||||
from app.plex.genres import hu_genre
|
||||
|
||||
TS_CONFIG = "public.unaccent_simple"
|
||||
@@ -236,7 +236,7 @@ def apply_meta_filters(q, model, p: dict, lang: str = "en"):
|
||||
# The 4 classes the feed filter + facet expose, derived from the stored `playable` + `transcode_hw`.
|
||||
# ONE definition (this SQL CASE) so the filter WHERE and the facet counts can't drift: `playable_conds`
|
||||
# filters on it, `facets` groups by it. transcode_hw IS NOT True groups NULL (unprobed) with software.
|
||||
# Public — it is the shared source of truth imported by routes.plex.browse (facet order + zero-fill).
|
||||
# Public — it is the shared source of truth imported by plex.library (facet order + zero-fill).
|
||||
PLAYBACK_CLASSES = ("direct", "remux", "transcode_hw", "transcode_sw")
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,9 @@ from datetime import datetime, timezone
|
||||
from sqlalchemy import and_, case, func, or_, select
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.models import PlexItem, PlexState
|
||||
from app.models import User
|
||||
from app.userscope import owned_by
|
||||
from app.plex.models import PlexItem, PlexState
|
||||
from app.plex.cards import ShowState
|
||||
|
||||
# Sort sentinel for a rail row whose timestamp is missing, so `key=lambda r: r[0] or NO_TS` reads the
|
||||
@@ -92,7 +94,7 @@ def agg_subquery(db: Session, user_id: int, include_resumed: bool = False):
|
||||
)
|
||||
return (
|
||||
select(*cols)
|
||||
.outerjoin(ps, and_(ps.item_id == PlexItem.id, ps.user_id == user_id))
|
||||
.outerjoin(ps, and_(ps.item_id == PlexItem.id, owned_by(ps.user_id, user_id)))
|
||||
.where(PlexItem.kind == "episode", PlexItem.show_id.isnot(None))
|
||||
.group_by(PlexItem.show_id)
|
||||
)
|
||||
@@ -105,7 +107,7 @@ def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, ShowS
|
||||
return {r.show_id: show_state(r.total, r.watched, r.inprog) for r in rows}
|
||||
|
||||
|
||||
def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int, PlexState]:
|
||||
def item_state_map(db: Session, user_id: User | int, item_ids: list[int]) -> dict[int, PlexState]:
|
||||
"""A user's PlexState rows for a set of leaf items (movies or episodes), keyed by item_id — the
|
||||
leaf analogue of `state_map` (which rolls a show's episodes up to a `ShowState`). One batched
|
||||
fetch, so the caller serializes cards without a per-card query. Empty in → empty dict (no `IN ()`)."""
|
||||
@@ -114,6 +116,8 @@ def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int,
|
||||
return {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id.in_(item_ids))
|
||||
select(PlexState).where(
|
||||
owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(item_ids)
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,413 @@
|
||||
"""Plex's writes for the HTTP layer, one function per use case. None of them commits: the handler
|
||||
does, so a page that fails to save an enrichment still renders, and a Plex write and its local
|
||||
mirror land together.
|
||||
|
||||
The rest of the mirror's writes stay with the sync (`plex.sync`): a collection's targeted re-sync
|
||||
after a write-back, its local cleanup after a delete.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import CursorResult, delete, func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.db import SessionLocal
|
||||
from app.plex import detail as plex_detail
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup
|
||||
from app.plex.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexLink,
|
||||
PlexPerson,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
)
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def enrich_show(db: Session, sh: PlexShow, meta: dict) -> None:
|
||||
"""Fill a not-yet-enriched show from its live metadata: the full cast (filterable), IMDb and
|
||||
tagline, stamped so later views serve from the row."""
|
||||
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
|
||||
plex_detail.apply_rich_detail(sh, meta)
|
||||
sh.people_enriched_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def enrich_movie(db: Session, it: PlexItem, meta: dict) -> None:
|
||||
"""The same for a movie. The listing sync stored only its top ~3 cast; the live metadata
|
||||
carries all of them."""
|
||||
cast = plex_people.cast_from_meta(meta)
|
||||
if cast:
|
||||
plex_people.enrich_row_people(db, it, cast)
|
||||
plex_detail.apply_rich_detail(it, meta)
|
||||
it.people_enriched_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def set_collection_editable(col: PlexCollection, editable: bool) -> None:
|
||||
col.editable = editable
|
||||
|
||||
|
||||
def keep_created_collection(
|
||||
db: Session, lib: PlexLibrary, rating_key: str, title: str, seed: str
|
||||
) -> None:
|
||||
"""A collection Plex made, whose follow-up read failed: its row is kept, editable. The full sync
|
||||
preserves `editable` but creates a missing row read-only, so the collection would drop out of
|
||||
every "add to" list; and the duplicate-name check reads these rows, so a retry of the same name
|
||||
would make a second one on Plex. Plex made it with exactly the seed in it, so the seed's row says
|
||||
so too — otherwise the title showed unticked in the collection it was just added to, and the
|
||||
collection empty, until that sync."""
|
||||
db.add(
|
||||
PlexCollection(
|
||||
rating_key=rating_key, library_id=lib.id, title=title, editable=True, child_count=1
|
||||
)
|
||||
)
|
||||
Model = PlexItem if lib.kind == "movie" else PlexShow
|
||||
if row := by_rating_key(db, Model, seed):
|
||||
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), rating_key]))
|
||||
db.flush()
|
||||
|
||||
|
||||
def enrich_person_bg(person_id: int) -> None:
|
||||
"""Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
|
||||
AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
|
||||
"no live TMDB call at serve time" invariant). The request's session is already closed by then, so
|
||||
open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
|
||||
next open, and the bio lands from the row by then."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
person = db.get(PlexPerson, person_id)
|
||||
if person is not None:
|
||||
plex_tmdb.enrich_person(db, person)
|
||||
except Exception: # noqa: BLE001 — background best-effort; never surfaces to a user
|
||||
db.rollback()
|
||||
log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def set_watch_link(db: Session, user: User, enabled: bool) -> PlexLink:
|
||||
"""Turn the user's two-way watch sync on or off, creating their link on first use. The link uses
|
||||
the server admin token (`uses_admin`), so it needs no Plex login of its own."""
|
||||
link = queries.watch_link(db, user)
|
||||
if link is None:
|
||||
link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
|
||||
db.add(link)
|
||||
link.sync_enabled = enabled
|
||||
db.flush()
|
||||
return link
|
||||
|
||||
|
||||
# --- a user's playlists ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_playlist(db: Session, user: User, title: str, seed: PlexItem | None) -> PlexPlaylist:
|
||||
"""A new playlist of the user's, holding `seed` when one is given."""
|
||||
pl = PlexPlaylist(user_id=user.id, title=title)
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
if seed is not None:
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=seed.id, position=0))
|
||||
db.flush()
|
||||
return pl
|
||||
|
||||
|
||||
def rename_playlist(pl: PlexPlaylist, title: str) -> None:
|
||||
pl.title = title
|
||||
|
||||
|
||||
def delete_playlist(db: Session, pl: PlexPlaylist) -> None:
|
||||
db.delete(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def _touch(pl: PlexPlaylist) -> None:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _last_position(db: Session, playlist_id: int) -> int:
|
||||
return db.execute(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
).scalar_one()
|
||||
|
||||
|
||||
def add_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
|
||||
"""Append `it` to the (locked) playlist; an item already in it stays where it is."""
|
||||
exists = db.execute(
|
||||
select(PlexPlaylistItem.id).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
).first()
|
||||
if exists is None:
|
||||
pos = _last_position(db, pl.id) + 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def remove_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
|
||||
db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def add_items(db: Session, pl: PlexPlaylist, items: list[PlexItem]) -> int:
|
||||
"""Append many items in their given order (a whole season or show); those already in the list
|
||||
are skipped. Says how many were added."""
|
||||
existing = set(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
)
|
||||
pos = _last_position(db, pl.id)
|
||||
added = 0
|
||||
for it in items:
|
||||
if it.id in existing:
|
||||
continue
|
||||
pos += 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
|
||||
existing.add(it.id)
|
||||
added += 1
|
||||
if added:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return added
|
||||
|
||||
|
||||
def insert_items_at(db: Session, pl: PlexPlaylist, placed: list[tuple[int, PlexItem]]) -> int:
|
||||
"""Insert new rows at their indices (ascending, so each index is where the row stood when the
|
||||
rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
|
||||
already in the list does not move: the add stays idempotent."""
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem)
|
||||
.where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
present = {r.item_id for r in rows}
|
||||
added = 0
|
||||
for at, it in sorted(placed, key=lambda p: p[0]):
|
||||
if it.id in present:
|
||||
continue
|
||||
row = PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0)
|
||||
db.add(row)
|
||||
rows.insert(min(at, len(rows)), row)
|
||||
present.add(it.id)
|
||||
added += 1
|
||||
for i, r in enumerate(rows):
|
||||
r.position = i
|
||||
if added:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return added
|
||||
|
||||
|
||||
def remove_items(db: Session, pl: PlexPlaylist, item_ids: list[int]) -> int:
|
||||
"""Remove these items from the playlist; says how many were there."""
|
||||
if not item_ids:
|
||||
return 0
|
||||
result = db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id.in_(item_ids)
|
||||
),
|
||||
execution_options={"synchronize_session": False},
|
||||
)
|
||||
removed = result.rowcount if isinstance(result, CursorResult) else 0
|
||||
if removed:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return removed
|
||||
|
||||
|
||||
def reorder_playlist(db: Session, pl: PlexPlaylist, rating_keys: list[str]) -> None:
|
||||
"""Set the play order from an explicit list of item keys (0-based positions)."""
|
||||
id_by_rk = {
|
||||
it.rating_key: it.id
|
||||
for it in db.execute(
|
||||
select(PlexItem).where(PlexItem.rating_key.in_(rating_keys or [""]))
|
||||
).scalars()
|
||||
}
|
||||
rows = {
|
||||
r.item_id: r
|
||||
for r in db.execute(
|
||||
select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[int] = set()
|
||||
for rk in rating_keys:
|
||||
row = rows.get(id_by_rk.get(rk, -1))
|
||||
if row is not None and row.item_id not in seen:
|
||||
row.position = pos
|
||||
pos += 1
|
||||
seen.add(row.item_id)
|
||||
# Rows the payload omitted (added in another tab meanwhile) keep their relative order after
|
||||
# the given ones, so no two rows share a position (the YouTube reorder does the same).
|
||||
for row in sorted(
|
||||
(r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
|
||||
):
|
||||
row.position = pos
|
||||
pos += 1
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- a user's watch state --------------------------------------------------------------------
|
||||
|
||||
|
||||
def mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
|
||||
"""Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
|
||||
checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
|
||||
read-then-write let both see "not watched" and both scrobble. The transition is decided by the
|
||||
write itself: an insert that did not conflict, or an update conditional on the old status —
|
||||
under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
|
||||
values = {
|
||||
"status": "watched",
|
||||
"watched_at": now,
|
||||
"position_seconds": 0,
|
||||
"progress_updated_at": now,
|
||||
"synced_to_plex": False,
|
||||
}
|
||||
inserted = db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(user_id=user_id, item_id=item_id, **values)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
.returning(PlexState.id)
|
||||
).first()
|
||||
if inserted is not None:
|
||||
return True
|
||||
mine = owned_by(PlexState.user_id, user_id) & (PlexState.item_id == item_id)
|
||||
result = db.execute(update(PlexState).where(mine, PlexState.status != "watched").values(values))
|
||||
if isinstance(result, CursorResult) and result.rowcount:
|
||||
return True
|
||||
# Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
|
||||
db.execute(update(PlexState).where(mine).values(values))
|
||||
return False
|
||||
|
||||
|
||||
def state_row(
|
||||
db: Session,
|
||||
user: User | int,
|
||||
item_id: int,
|
||||
st: PlexState | None = None,
|
||||
*,
|
||||
synced: bool = False,
|
||||
) -> PlexState:
|
||||
"""The user's state row on this item, made if the read before found none. Made by the insert
|
||||
itself (ON CONFLICT DO NOTHING) and then read: the first checkpoint of an item and the `final`
|
||||
flush beside it routinely both read "none", and a plain add let the second one fail on
|
||||
`uq_plex_user_item` — a 500, and the push it was carrying lost. Every first row goes through
|
||||
here: a checkpoint, a status, a season marked at once, a pull from Plex. `synced` is what a row
|
||||
made now says about Plex; a row another writer made first keeps its own."""
|
||||
if st is not None:
|
||||
return st
|
||||
db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(
|
||||
user_id=user if isinstance(user, int) else user.id,
|
||||
item_id=item_id,
|
||||
synced_to_plex=synced,
|
||||
)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
)
|
||||
made = queries.state(db, user, item_id)
|
||||
assert made is not None # inserted now, or committed by the request that won
|
||||
return made
|
||||
|
||||
|
||||
def clear_progress(db: Session, st: PlexState | None, now: datetime) -> None:
|
||||
"""A checkpoint too early to keep: a state that held nothing else goes, any other forgets its
|
||||
position."""
|
||||
if st is not None and st.status == "new":
|
||||
db.delete(st)
|
||||
elif st is not None:
|
||||
st.position_seconds = 0
|
||||
st.progress_updated_at = now
|
||||
db.flush()
|
||||
|
||||
|
||||
def checkpoint(
|
||||
db: Session,
|
||||
user: User,
|
||||
item_id: int,
|
||||
st: PlexState | None,
|
||||
position: int,
|
||||
now: datetime,
|
||||
*,
|
||||
dirty: bool,
|
||||
) -> PlexState:
|
||||
"""Record a resume position. `dirty` marks it not yet mirrored to Plex — set before a push, so
|
||||
the reconcile re-pushes it if the push fails and the push can settle it once Plex confirms."""
|
||||
st = state_row(db, user, item_id, st)
|
||||
st.position_seconds = position
|
||||
st.progress_updated_at = now
|
||||
if dirty:
|
||||
st.synced_to_plex = False
|
||||
db.flush()
|
||||
return st
|
||||
|
||||
|
||||
def forget_state(db: Session, st: PlexState) -> None:
|
||||
db.delete(st)
|
||||
db.flush()
|
||||
|
||||
|
||||
def set_status(
|
||||
db: Session, user: User, item_id: int, st: PlexState | None, status: str
|
||||
) -> PlexState:
|
||||
"""Set a watched or hidden status; watched also clears the resume point."""
|
||||
st = state_row(db, user, item_id, st)
|
||||
st.status = status
|
||||
if status == "watched":
|
||||
st.watched_at = datetime.now(timezone.utc)
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
db.flush()
|
||||
return st
|
||||
|
||||
|
||||
def bulk_state(
|
||||
db: Session, user_id: int, eps: list[PlexItem], watched: bool
|
||||
) -> list[tuple[int, str, str]]:
|
||||
"""Mark every episode in `eps` watched (or unwatched) for a user and say what changed, as the
|
||||
(item id, rating key, action) pushes a linked Plex account needs. Unwatching keeps the user's
|
||||
private "hidden" marks."""
|
||||
now = datetime.now(timezone.utc)
|
||||
existing = rollup.item_state_map(db, user_id, [e.id for e in eps])
|
||||
to_push: list[tuple[int, str, str]] = []
|
||||
for e in eps:
|
||||
st = existing.get(e.id)
|
||||
prev = st.status if st is not None else "new"
|
||||
if watched:
|
||||
st = state_row(db, user_id, e.id, st)
|
||||
prev = st.status # a first checkpoint beside this may have made it meanwhile
|
||||
st.status = "watched"
|
||||
st.watched_at = now
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
if prev != "watched":
|
||||
to_push.append((e.id, e.rating_key, "watched"))
|
||||
else: # → new: drop any state (but keep the user's private "hidden" marks)
|
||||
if st is not None and prev != "hidden":
|
||||
db.delete(st)
|
||||
if prev != "new":
|
||||
to_push.append((e.id, e.rating_key, "unwatched"))
|
||||
db.flush()
|
||||
return to_push
|
||||
@@ -33,7 +33,7 @@ from sqlalchemy.orm import Session as DbSession
|
||||
from app import logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.downloads.edit import encode_thread_cap
|
||||
from app.models import PlexItem
|
||||
from app.plex.models import PlexItem
|
||||
from app.plex import paths
|
||||
from app.plex.sync import HDR_TRANSFERS, HW_DECODABLE_CODECS
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.db import SessionLocal
|
||||
from app.models import PlexItem
|
||||
from app.plex.models import PlexItem
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex.cache import CACHE_DIR
|
||||
from app.plex.client import PlexClient, PlexError
|
||||
|
||||
@@ -21,7 +21,7 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
|
||||
from app.plex import detail, overlay, people
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.genres import hu_genres
|
||||
|
||||
@@ -22,7 +22,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.models import PlexItem, PlexPerson, PlexShow
|
||||
from app.plex.models import PlexItem, PlexPerson, PlexShow
|
||||
from app.plex.cards import display_title
|
||||
from app.plex.people import people_by_credits, warm_portraits
|
||||
|
||||
@@ -619,9 +619,9 @@ def enrich_person(db: Session, person: PlexPerson) -> bool:
|
||||
if not person.tmdb_person_id or not sysconfig.get_str(db, "tmdb_api_key"):
|
||||
return False
|
||||
try:
|
||||
# Called from a background task (off the request path — see people._enrich_person_bg), so it no
|
||||
# longer blocks a serve; still cap the outbound TMDB call with a short timeout so a hanging TMDB
|
||||
# doesn't tie up a threadpool worker for the default 30s.
|
||||
# Called from a background task (off the request path — see `plex.service.enrich_person_bg`),
|
||||
# so it no longer blocks a serve; still cap the outbound TMDB call with a short timeout so a
|
||||
# hanging TMDB doesn't tie up a threadpool worker for the default 30s.
|
||||
with TmdbClient(db, timeout=8.0) as tmdb:
|
||||
_fetch_person_into(tmdb, person)
|
||||
except TmdbError as exc:
|
||||
|
||||
@@ -29,8 +29,11 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import PlexItem, PlexLibrary, PlexLink, PlexState
|
||||
from app.userscope import owned_by
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexLink, PlexState
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex import queries
|
||||
from app.plex.service import state_row
|
||||
from app.plex.sync import (
|
||||
_TYPE_EPISODE,
|
||||
_TYPE_MOVIE,
|
||||
@@ -196,7 +199,7 @@ def link_for_push(db: Session, user_id: int) -> PlexLink | None:
|
||||
"""The active owner watch-push link for this user, or None when push is off / not the owner.
|
||||
Push is gated on `initial_import_done` so we never push a half-known local state over Plex's
|
||||
authoritative one before the one-time "Plex is master" import has reconciled them."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user_id)).scalars().first()
|
||||
link = db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user_id))).scalars().first()
|
||||
if link and link.uses_admin and link.sync_enabled and link.initial_import_done:
|
||||
return link
|
||||
return None
|
||||
@@ -251,13 +254,7 @@ def push_state_to_plex(
|
||||
# Flag the row synced — but an "unwatched" via status=new deletes the row, so there may be
|
||||
# nothing to flag; that's fine (unscrobble still happened). Skip the flag if the row was
|
||||
# re-edited since we captured `expect_ts` (leave it dirty for its own push / the reconcile).
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user_id, item_id)
|
||||
if st is not None and _row_matches(st, expect_ts):
|
||||
st.synced_to_plex = True
|
||||
db.commit()
|
||||
@@ -318,7 +315,7 @@ def push_bulk_state_to_plex(user_id: int, changes: list[tuple[int, str, str]]) -
|
||||
if synced_ids:
|
||||
db.execute(
|
||||
update(PlexState)
|
||||
.where(PlexState.user_id == user_id, PlexState.item_id.in_(synced_ids))
|
||||
.where(owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(synced_ids))
|
||||
.values(synced_to_plex=True)
|
||||
)
|
||||
db.commit()
|
||||
@@ -391,13 +388,7 @@ def _pull_apply(
|
||||
"""Bring one Plex-side positive signal into Siftlode under last-write-wins. Returns
|
||||
"watched"/"in_progress" if it changed the row, else None. Never clears a row (un-watch is the
|
||||
full reconcile's job) — this only pulls in watched/resume."""
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user_id, item_id)
|
||||
if _same_state(st, status, pos):
|
||||
if st is not None and not st.synced_to_plex:
|
||||
st.synced_to_plex = True # already matches Plex → just settle the flag, no churn
|
||||
@@ -418,8 +409,9 @@ def _pull_apply(
|
||||
# "almost always already pushed" assumption above excludes.
|
||||
return None
|
||||
if st is None:
|
||||
st = PlexState(user_id=user_id, item_id=item_id)
|
||||
db.add(st)
|
||||
st = state_row(db, user_id, item_id, synced=True)
|
||||
if not st.synced_to_plex: # a local change made it first, beside this pull: the guard above
|
||||
return None
|
||||
if status == "watched":
|
||||
st.status, st.position_seconds, st.watched_at, st.progress_updated_at = (
|
||||
"watched",
|
||||
@@ -449,7 +441,7 @@ def _repush_dirty(db: Session, plex: PlexClient, link: PlexLink) -> int:
|
||||
dirty = (
|
||||
db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == link.user_id,
|
||||
owned_by(PlexState.user_id, link.user_id),
|
||||
PlexState.synced_to_plex.is_(False),
|
||||
PlexState.status != "hidden",
|
||||
)
|
||||
@@ -505,7 +497,7 @@ def _reconcile_inprogress_movies(
|
||||
and aren't affected). Note: like the rest of this job, `metadata()` reads the TOKEN account (the
|
||||
owner) — a HTPC playing under a managed Plex user (accountID>1) is out of scope here as elsewhere."""
|
||||
conds = [
|
||||
PlexState.user_id == link.user_id,
|
||||
owned_by(PlexState.user_id, link.user_id),
|
||||
PlexState.status.notin_(("watched", "hidden")),
|
||||
PlexState.position_seconds > _PROGRESS_MIN_S,
|
||||
PlexItem.kind == "movie",
|
||||
@@ -627,7 +619,7 @@ def _clear_unwatched(db: Session, user_id: int, plex_states: dict, settled: set[
|
||||
select(PlexState)
|
||||
.join(PlexItem, PlexItem.id == PlexState.item_id)
|
||||
.join(PlexLibrary, PlexLibrary.id == PlexItem.library_id)
|
||||
.where(PlexState.user_id == user_id, PlexLibrary.plex_key.in_(settled))
|
||||
.where(owned_by(PlexState.user_id, user_id), PlexLibrary.plex_key.in_(settled))
|
||||
)
|
||||
for st in rows.scalars().all():
|
||||
if st.item_id in plex_states or st.status == "hidden" or not st.synced_to_plex:
|
||||
|
||||
@@ -16,6 +16,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import ApiQuotaUsage, QuotaEvent
|
||||
@@ -173,7 +174,7 @@ def actions_today(db: Session, user_id: int, action: str) -> int:
|
||||
select(func.count())
|
||||
.select_from(QuotaEvent)
|
||||
.where(
|
||||
QuotaEvent.user_id == user_id,
|
||||
owned_by(QuotaEvent.user_id, user_id),
|
||||
QuotaEvent.action == action,
|
||||
QuotaEvent.created_at >= pacific_day_start_utc(),
|
||||
)
|
||||
|
||||
@@ -11,12 +11,14 @@ from sqlalchemy import (
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
@@ -93,6 +95,15 @@ class Playlist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
sync phases; the foundation phase only uses local playlists.)"""
|
||||
|
||||
__tablename__ = "playlists"
|
||||
__table_args__ = (
|
||||
# One Watch later per user, held by the database: two first saves at once each made one.
|
||||
Index(
|
||||
"uq_playlist_watch_later",
|
||||
"user_id",
|
||||
unique=True,
|
||||
postgresql_where=text("kind = 'watch_later'"),
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
@@ -10,8 +10,10 @@ than joining Reader's tables itself.
|
||||
from sqlalchemy import ColumnElement, and_, func, or_, select
|
||||
from sqlalchemy.orm import InstrumentedAttribute, Session, aliased
|
||||
|
||||
from app.feedquery import feed_columns
|
||||
from app.models import Channel, User, Video
|
||||
from app.collation import collated
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
|
||||
from app.userscope import owned_by, owned_row
|
||||
|
||||
@@ -268,3 +270,92 @@ def tags_in_feed_views(db: Session) -> set[int]:
|
||||
for (filters,) in db.execute(select(SavedView.filters).where(SavedView.source == "feed"))
|
||||
for t in view_tag_ids(filters)
|
||||
}
|
||||
|
||||
|
||||
def held_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
|
||||
"""Which of these videos the user already holds: watched, in progress or hidden, or in one of
|
||||
their playlists (a live search sinks them below the new finds)."""
|
||||
if not video_ids:
|
||||
return set()
|
||||
has_state = (
|
||||
select(VideoState.id)
|
||||
.where(VideoState.video_id == Video.id, owned_by(VideoState.user_id, user))
|
||||
.exists()
|
||||
)
|
||||
in_playlist = (
|
||||
select(PlaylistItem.id)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(PlaylistItem.video_id == Video.id, owned_by(Playlist.user_id, user))
|
||||
.exists()
|
||||
)
|
||||
return set(
|
||||
db.scalars(select(Video.id).where(Video.id.in_(video_ids), or_(has_state, in_playlist)))
|
||||
)
|
||||
|
||||
|
||||
def feed_rows_by_ids(db: Session, user: User, video_ids: list[str]) -> list[dict]:
|
||||
"""These videos serialized with the feed's projection (their channel, this user's watch state,
|
||||
saved), in the order given."""
|
||||
if not video_ids:
|
||||
return []
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
.where(Video.id.in_(video_ids))
|
||||
).all()
|
||||
by_id = {r.id: serialize(r) for r in rows}
|
||||
return [by_id[i] for i in video_ids if i in by_id]
|
||||
|
||||
|
||||
def playlist_channels(db: Session, user: User, exclude: str | None, lang: str) -> list:
|
||||
"""The channels of the videos in the user's playlists that they don't subscribe to (`exclude`:
|
||||
their own channel), each with how many of those videos it has and across how many playlists;
|
||||
most present first, then by title in the reader's alphabet."""
|
||||
subscribed = select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
|
||||
q = (
|
||||
select(
|
||||
Channel,
|
||||
func.count(func.distinct(PlaylistItem.video_id)),
|
||||
func.count(func.distinct(Playlist.id)),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.where(owned_by(Playlist.user_id, user), Channel.id.not_in(subscribed))
|
||||
)
|
||||
if exclude:
|
||||
q = q.where(Channel.id != exclude)
|
||||
return list(
|
||||
db.execute(
|
||||
q.group_by(Channel.id).order_by(
|
||||
func.count(func.distinct(PlaylistItem.video_id)).desc(),
|
||||
collated(Channel.title, lang),
|
||||
)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def user_tag_links(db: Session, user: User, channel_id: str | None = None) -> dict[str, list[int]]:
|
||||
"""The user's own tag links by channel (one channel's, when given)."""
|
||||
stmt = select(ChannelTag.channel_id, ChannelTag.tag_id).where(
|
||||
owned_by(ChannelTag.user_id, user)
|
||||
)
|
||||
if channel_id is not None:
|
||||
stmt = stmt.where(ChannelTag.channel_id == channel_id)
|
||||
out: dict[str, list[int]] = {}
|
||||
for cid, tag_id in db.execute(stmt).tuples():
|
||||
out.setdefault(cid, []).append(tag_id)
|
||||
return out
|
||||
|
||||
|
||||
def user_tag_link(db: Session, user: User, channel_id: str, tag_id: int) -> ChannelTag | None:
|
||||
return db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == tag_id,
|
||||
owned_by(ChannelTag.user_id, user),
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
@@ -9,7 +9,7 @@ unique key, a row lock taken before a presence check. The comfort shelf's writes
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import exists, func, select, update
|
||||
from sqlalchemy import exists, func, select, text, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -239,24 +239,33 @@ def clear_state(db: Session, user: User, video_id: str) -> None:
|
||||
|
||||
def set_progress(db: Session, user: User, video_id: str, position: int) -> None:
|
||||
"""The player's resume point. 0 clears it: a row holding nothing else goes, one with a status
|
||||
(watched, hidden) keeps it with the position zeroed. The status itself is never changed here."""
|
||||
row = queries.video_state(db, user, video_id)
|
||||
(watched, hidden) keeps it with the position zeroed. The status itself is never changed here.
|
||||
A position is one upsert on (user, video): the first checkpoint and the end-of-playback mark
|
||||
beside it both read "no row", and a plain add let the second one fail on `uq_user_video`."""
|
||||
now = datetime.now(timezone.utc)
|
||||
if not position:
|
||||
if row is None:
|
||||
return
|
||||
if row.status == "new":
|
||||
db.delete(row)
|
||||
else:
|
||||
row.position_seconds = 0
|
||||
row.progress_updated_at = now
|
||||
db.flush()
|
||||
if position:
|
||||
db.execute(
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
position_seconds=position,
|
||||
progress_updated_at=now,
|
||||
)
|
||||
.on_conflict_do_update(
|
||||
constraint="uq_user_video",
|
||||
set_={"position_seconds": position, "progress_updated_at": now, "updated_at": now},
|
||||
)
|
||||
)
|
||||
return
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is None:
|
||||
row = VideoState(user_id=user.id, video_id=video_id)
|
||||
db.add(row)
|
||||
row.position_seconds = position
|
||||
row.progress_updated_at = now
|
||||
return
|
||||
if row.status == "new":
|
||||
db.delete(row)
|
||||
else:
|
||||
row.position_seconds = 0
|
||||
row.progress_updated_at = now
|
||||
db.flush()
|
||||
|
||||
|
||||
@@ -319,17 +328,25 @@ def create_playlist(db: Session, user: User, name: str) -> Playlist:
|
||||
|
||||
|
||||
def watch_later(db: Session, user: User) -> Playlist:
|
||||
"""The user's built-in Watch later playlist, created on first use."""
|
||||
"""The user's built-in Watch later playlist, created on first use. Created by the insert itself
|
||||
(ON CONFLICT DO NOTHING on the one-per-user index) and then read: the first two saves at once
|
||||
both read "none", and each used to make a list of its own."""
|
||||
pl = queries.watch_later(db, user)
|
||||
if pl is None:
|
||||
pl = Playlist(
|
||||
user_id=user.id,
|
||||
name="Watch later",
|
||||
kind="watch_later",
|
||||
position=_next_playlist_position(db, user.id),
|
||||
db.execute(
|
||||
pg_insert(Playlist)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
name="Watch later",
|
||||
kind="watch_later",
|
||||
position=_next_playlist_position(db, user.id),
|
||||
)
|
||||
.on_conflict_do_nothing(
|
||||
index_elements=["user_id"], index_where=text("kind = 'watch_later'")
|
||||
)
|
||||
)
|
||||
db.add(pl)
|
||||
db.flush() # the id, for the item the caller adds next
|
||||
pl = queries.watch_later(db, user)
|
||||
assert pl is not None # inserted now, or committed by the request that won
|
||||
return pl
|
||||
|
||||
|
||||
@@ -464,3 +481,17 @@ def reorder_items(db: Session, pl: Playlist, order: list[str]) -> int:
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return pos
|
||||
|
||||
|
||||
def attach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
|
||||
"""The user's own label on a channel (idempotent). The caller has checked the tag is theirs."""
|
||||
if queries.user_tag_link(db, user, channel_id, tag_id) is None:
|
||||
db.add(ChannelTag(channel_id=channel_id, tag_id=tag_id, user_id=user.id, source="user"))
|
||||
db.flush()
|
||||
|
||||
|
||||
def detach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
|
||||
row = queries.user_tag_link(db, user, channel_id, tag_id)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
|
||||
@@ -89,8 +89,9 @@ def owned_or_404(
|
||||
) -> M:
|
||||
"""Fetch a row by primary key that `user` owns, or raise 404 (`lock`: see `get_or_404`). A row owned by someone
|
||||
else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger.
|
||||
It replaced the near-identical per-route ownership helpers, and is called directly — a thin
|
||||
`_own_x` wrapper around it is the ad-hoc shape CLAUDE.md's patterns table rules out."""
|
||||
A core route calls it directly. A unit's route cannot (it may not name a unit's model as a value):
|
||||
it asks its unit's `queries.own_x`, which goes through the same `owned_row`, and answers
|
||||
`not_found` itself."""
|
||||
obj = owned_row(db, model, id_, user, owner_attr=owner_attr, lock=lock)
|
||||
if obj is None:
|
||||
raise not_found(what)
|
||||
|
||||
@@ -16,6 +16,7 @@ from sqlalchemy import select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -1124,7 +1125,7 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
|
||||
# Burn any other outstanding reset tokens for this user.
|
||||
for row in db.execute(
|
||||
select(AuthToken).where(
|
||||
AuthToken.user_id == user.id,
|
||||
owned_by(AuthToken.user_id, user),
|
||||
AuthToken.kind.in_(_PASSWORD_TOKEN_KINDS),
|
||||
AuthToken.used_at.is_(None),
|
||||
)
|
||||
|
||||
+70
-258
@@ -4,28 +4,22 @@ hidden, tags) and per-channel sync state. Local reads/writes plus YouTube-side w
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from functools import partial
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import Select, func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app.mailtext import lang_for_user
|
||||
from app import logs, quota, scheduler
|
||||
from app.auth import admin_user, current_user, require_write_scope
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
Subscription,
|
||||
User,
|
||||
Video,
|
||||
)
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, Tag
|
||||
from app.routes._common import get_or_404, not_found, owned_or_404, youtube_refusal
|
||||
from app.models import User
|
||||
from app.catalog import queries, service
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
from app.routes._common import not_found, youtube_refusal
|
||||
from app.sync.explore import explore_ingest_page
|
||||
from app.sync.runner import backfill_channel_now, run_recent_backfill
|
||||
from app.sync.subscriptions import apply_channel_details, details_stale
|
||||
@@ -33,35 +27,17 @@ from app.utils import iso
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.catalog.models import Channel, Subscription
|
||||
|
||||
router = APIRouter(prefix="/api/channels", tags=["channels"])
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def channel_aggregates(channel_ids: list[str]) -> Select:
|
||||
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
|
||||
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
|
||||
alone; a test holds that (test_channel_list_index_only)."""
|
||||
return (
|
||||
select(
|
||||
Video.channel_id,
|
||||
func.count(Video.id),
|
||||
func.max(Video.published_at),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.where(Video.channel_id.in_(channel_ids))
|
||||
.group_by(Video.channel_id)
|
||||
)
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_channels(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
rows = db.execute(
|
||||
select(Subscription, Channel)
|
||||
.join(Channel, Channel.id == Subscription.channel_id)
|
||||
.where(Subscription.user_id == user.id)
|
||||
.order_by(Subscription.priority.desc(), collated(Channel.title, lang_for_user(user)))
|
||||
).all()
|
||||
rows = queries.subscriptions(db, user, lang_for_user(user))
|
||||
|
||||
# Per-channel aggregates over the shared catalog (channel_aggregates), computed on read rather than
|
||||
# denormalised: an Index Only Scan over a ~20 MB index, where it used to read every video's heap page
|
||||
@@ -70,36 +46,21 @@ def list_channels(user: User = Depends(current_user), db: Session = Depends(get_
|
||||
# computed here — the manager table dropped its N/S/L column.)
|
||||
channel_ids = [c.id for _, c in rows]
|
||||
agg: dict[str, dict] = {}
|
||||
if channel_ids:
|
||||
for cid, n, last_up, total_dur in db.execute(channel_aggregates(channel_ids)).all():
|
||||
agg[cid] = {
|
||||
"stored": n,
|
||||
"last_video_at": iso(last_up),
|
||||
"total_duration_seconds": int(total_dur or 0),
|
||||
}
|
||||
for cid, n, last_up, total_dur in queries.aggregates(db, channel_ids):
|
||||
agg[cid] = {
|
||||
"stored": n,
|
||||
"last_video_at": iso(last_up),
|
||||
"total_duration_seconds": int(total_dur or 0),
|
||||
}
|
||||
|
||||
# Channels already in the *global* deep-backfill queue — i.e. at least one user (any
|
||||
# user) has requested full history and it isn't done yet. Their whole back-catalogue is
|
||||
# coming for everyone (video data is shared), so the UI shouldn't offer "get full
|
||||
# history" to a user who simply hasn't personally opted in.
|
||||
deep_queued: set[str] = set()
|
||||
if channel_ids:
|
||||
for (cid,) in db.execute(
|
||||
select(Subscription.channel_id)
|
||||
.where(
|
||||
Subscription.channel_id.in_(channel_ids),
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.distinct()
|
||||
).all():
|
||||
deep_queued.add(cid)
|
||||
deep_queued = queries.deep_queued(db, channel_ids)
|
||||
|
||||
# The user's personal tag links, grouped by channel.
|
||||
tags_by_channel: dict[str, list[int]] = {}
|
||||
for cid, tag_id in db.execute(
|
||||
select(ChannelTag.channel_id, ChannelTag.tag_id).where(ChannelTag.user_id == user.id)
|
||||
).all():
|
||||
tags_by_channel.setdefault(cid, []).append(tag_id)
|
||||
tags_by_channel = reader_queries.user_tag_links(db, user)
|
||||
|
||||
return [
|
||||
{
|
||||
@@ -128,27 +89,7 @@ DISCOVERY_ENRICH_CAP = 200
|
||||
def _discovery_rows(db: Session, user: User) -> list:
|
||||
"""The user's playlist channels they don't subscribe to (and that aren't their own),
|
||||
each with how many playlist videos come from it and across how many playlists."""
|
||||
subscribed = select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
q = (
|
||||
select(
|
||||
Channel,
|
||||
func.count(func.distinct(PlaylistItem.video_id)),
|
||||
func.count(func.distinct(Playlist.id)),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.where(Playlist.user_id == user.id, Channel.id.not_in(subscribed))
|
||||
)
|
||||
if user.yt_channel_id:
|
||||
q = q.where(Channel.id != user.yt_channel_id)
|
||||
return db.execute(
|
||||
q.group_by(Channel.id).order_by(
|
||||
func.count(func.distinct(PlaylistItem.video_id)).desc(),
|
||||
collated(Channel.title, lang_for_user(user)),
|
||||
)
|
||||
).all()
|
||||
return reader_queries.playlist_channels(db, user, user.yt_channel_id, lang_for_user(user))
|
||||
|
||||
|
||||
@router.get("/discovery")
|
||||
@@ -170,7 +111,7 @@ def discover_channels(
|
||||
):
|
||||
own = yt.get_my_channel_id()
|
||||
if own:
|
||||
user.yt_channel_id = own
|
||||
service.remember_own_channel(user, own)
|
||||
db.commit()
|
||||
except YouTubeError as exc:
|
||||
# No traceback anywhere in this file's YouTube handlers: for an upstream answer
|
||||
@@ -226,7 +167,7 @@ def discover_channels(
|
||||
]
|
||||
|
||||
|
||||
def _channel_summary(ch: Channel) -> dict:
|
||||
def _channel_summary(ch: "Channel") -> dict:
|
||||
"""The channel fields common to the list and discovery projections."""
|
||||
return {
|
||||
"id": ch.id,
|
||||
@@ -240,7 +181,7 @@ def _channel_summary(ch: Channel) -> dict:
|
||||
|
||||
|
||||
def _channel_detail_dict(
|
||||
channel: Channel,
|
||||
channel: "Channel",
|
||||
*,
|
||||
subscribed: bool,
|
||||
explored: bool,
|
||||
@@ -309,7 +250,7 @@ def channel_detail(
|
||||
needed (a stub) and refreshes it once it is stale (`DETAILS_MAX_AGE`), which is how a channel nobody subscribes to (an explored one, outside the
|
||||
subscription import) gets a banner that still resolves. The enrich uses the API key (1 unit,
|
||||
cached after)."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
# No `published_at is None` sentinel: a terminated channel never gets one, so it re-fetched on
|
||||
# every view; rows enriched before that field existed are long past DETAILS_MAX_AGE anyway.
|
||||
stale = details_stale(channel, datetime.now(timezone.utc))
|
||||
@@ -321,7 +262,7 @@ def channel_detail(
|
||||
):
|
||||
apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
|
||||
db.commit()
|
||||
channel = db.get(Channel, channel_id)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
except YouTubeError as exc:
|
||||
log.warning(
|
||||
"channel detail enrich failed: %s",
|
||||
@@ -331,41 +272,15 @@ def channel_detail(
|
||||
),
|
||||
)
|
||||
db.rollback()
|
||||
sub = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
sub = queries.subscription(db, user, channel_id)
|
||||
subscribed = sub is not None
|
||||
explored = (
|
||||
db.execute(
|
||||
select(ExploredChannel.id).where(
|
||||
ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
blocked = _is_blocked(db, user, channel_id)
|
||||
explored = queries.exploration(db, user, channel_id) is not None
|
||||
blocked = queries.is_blocked(db, user, channel_id)
|
||||
# Stored total + the Normal/Shorts/Live split in one grouped pass (same filters as the manager
|
||||
# list), so the channel page can show the per-type breakdown that moved off the table.
|
||||
n_all, n_short, n_live = db.execute(
|
||||
select(
|
||||
func.count(Video.id),
|
||||
func.count(Video.id).filter(Video.is_short.is_(True)),
|
||||
func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
|
||||
).where(Video.channel_id == channel_id)
|
||||
).one()
|
||||
stored = int(n_all or 0)
|
||||
count_short = int(n_short or 0)
|
||||
count_live = int(n_live or 0)
|
||||
stored, count_short, count_live = queries.video_counts(db, channel_id)
|
||||
count_normal = max(0, stored - count_short - count_live)
|
||||
tag_ids = list(
|
||||
db.scalars(
|
||||
select(ChannelTag.tag_id).where(
|
||||
ChannelTag.user_id == user.id, ChannelTag.channel_id == channel_id
|
||||
)
|
||||
).all()
|
||||
)
|
||||
tag_ids = reader_queries.user_tag_links(db, user, channel_id).get(channel_id, [])
|
||||
return _channel_detail_dict(
|
||||
channel,
|
||||
subscribed=subscribed,
|
||||
@@ -395,28 +310,15 @@ def explore_channel(
|
||||
it, then ingests one page of its uploads (newest-first) marked via_explore + enriched.
|
||||
Returns the next page token so the page can "load more". Guarded against the shared
|
||||
backfill quota reserve."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
if _is_blocked(db, user, channel_id):
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
if queries.is_blocked(db, user, channel_id):
|
||||
raise refused(403, "channel_blocked")
|
||||
if quota.backfill_reserve_reached():
|
||||
raise refused(429, "quota_reserve")
|
||||
|
||||
# Record/refresh the exploration (DB only) before ingesting, so the explorer can see the
|
||||
# videos as soon as they land. Flag the channel ephemeral unless it's organically followed.
|
||||
existing = db.execute(
|
||||
select(ExploredChannel).where(
|
||||
ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is None:
|
||||
db.add(ExploredChannel(user_id=user.id, channel_id=channel_id))
|
||||
else:
|
||||
existing.created_at = datetime.now(timezone.utc) # refresh the grace clock
|
||||
followed = db.execute(
|
||||
select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
|
||||
).first()
|
||||
if followed is None and not channel.from_explore:
|
||||
channel.from_explore = True
|
||||
service.explore(db, user, channel)
|
||||
db.commit()
|
||||
|
||||
try:
|
||||
@@ -430,7 +332,7 @@ def explore_channel(
|
||||
if details_stale(channel, datetime.now(timezone.utc)):
|
||||
apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
|
||||
db.commit()
|
||||
channel = db.get(Channel, channel_id)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
result = explore_ingest_page(db, yt, channel, page_token)
|
||||
except YouTubeError as exc:
|
||||
raise youtube_refusal(exc) from exc
|
||||
@@ -442,30 +344,20 @@ def explore_channel(
|
||||
}
|
||||
|
||||
|
||||
def _user_subscription(db: Session, user: User, channel_id: str) -> Subscription:
|
||||
sub = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
def _channel_or_404(db: Session, channel_id: str) -> "Channel":
|
||||
channel = queries.channel(db, channel_id)
|
||||
if channel is None:
|
||||
raise not_found("channel")
|
||||
return channel
|
||||
|
||||
|
||||
def _user_subscription(db: Session, user: User, channel_id: str) -> "Subscription":
|
||||
sub = queries.subscription(db, user, channel_id)
|
||||
if sub is None:
|
||||
raise not_found("subscription")
|
||||
return sub
|
||||
|
||||
|
||||
def _is_blocked(db: Session, user: User, channel_id: str) -> bool:
|
||||
"""Whether this user has blocked this channel."""
|
||||
return (
|
||||
db.execute(
|
||||
select(BlockedChannel.id).where(
|
||||
BlockedChannel.user_id == user.id,
|
||||
BlockedChannel.channel_id == channel_id,
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
class ChannelUpdateIn(BaseModel):
|
||||
# PATCH semantics: a field left unset (None) is untouched. Every field is optional so the
|
||||
# UI can send just the one control it changed. int/bool coercion + rejection of garbage
|
||||
@@ -483,19 +375,14 @@ def update_channel(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
sub = _user_subscription(db, user, channel_id)
|
||||
channel = db.get(Channel, channel_id)
|
||||
deep_turned_on = False
|
||||
if body.priority is not None:
|
||||
sub.priority = body.priority
|
||||
if body.hidden is not None:
|
||||
sub.hidden = body.hidden
|
||||
if body.deep_requested is not None:
|
||||
# Opt this channel into (or out of) full-history backfill. The deep scheduler
|
||||
# picks the flag up on its next run; turning it off won't undo already-fetched
|
||||
# videos, it just stops further deep paging if nobody else still wants it.
|
||||
want = body.deep_requested
|
||||
deep_turned_on = want and not sub.deep_requested
|
||||
sub.deep_requested = want
|
||||
channel = queries.channel(db, channel_id)
|
||||
deep_turned_on = service.update_subscription(
|
||||
db,
|
||||
sub,
|
||||
priority=body.priority,
|
||||
hidden=body.hidden,
|
||||
deep_requested=body.deep_requested,
|
||||
)
|
||||
db.commit()
|
||||
|
||||
# Opting a channel into full history should give an immediate feed: if we haven't even
|
||||
@@ -539,19 +426,8 @@ def reset_backfill(
|
||||
state (a "reset" trigger). Clears the channel's backfill markers, opts it back into deep
|
||||
backfill, and re-runs its recent pull immediately; the deep scheduler re-pages the full
|
||||
back-catalog on its next run. Idempotent — videos upsert by id, so nothing duplicates."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
channel.backfill_done = False
|
||||
channel.backfill_cursor = None
|
||||
channel.recent_synced_at = None
|
||||
# Re-arm the demand-driven deep scheduler for this channel. run_deep_backfill gates on an
|
||||
# EXISTS over subscriptions with deep_requested=True, so flip it on EVERY subscriber rather
|
||||
# than requiring the admin to be personally subscribed (they reset from the manager, not
|
||||
# their own feed). No-op row count if the channel has no subscribers.
|
||||
db.execute(
|
||||
update(Subscription)
|
||||
.where(Subscription.channel_id == channel_id)
|
||||
.values(deep_requested=True)
|
||||
)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
service.reset_backfill(db, channel)
|
||||
db.commit()
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_BACKFILL_RECENT):
|
||||
run_recent_backfill(db, [channel], max_channels=1)
|
||||
@@ -569,13 +445,8 @@ def subscribe(
|
||||
local subscription with YouTube's returned resource id and enriches the channel's metadata
|
||||
if it's still a stub, so the new row renders properly in the manager. It does NOT pull the
|
||||
channel's videos — the scheduler picks those up on its next run, like any subscribed channel."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
existing = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
if queries.subscription(db, user, channel_id) is not None:
|
||||
raise refused(409, "already_subscribed")
|
||||
with (
|
||||
quota.attribute(user.id, quota.QuotaAction.CHANNELS_SUBSCRIBE),
|
||||
@@ -615,24 +486,7 @@ def subscribe(
|
||||
reason=exc.reason,
|
||||
),
|
||||
)
|
||||
db.add(
|
||||
Subscription(
|
||||
user_id=user.id,
|
||||
channel_id=channel_id,
|
||||
yt_subscription_id=yt_sub_id or None,
|
||||
subscribed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
# Subscribing is the "keep" signal: a previously-explored channel becomes a normal organic
|
||||
# one. Drop its ephemeral flags so the cleanup job leaves it alone and its videos stop being
|
||||
# per-user private (they join the shared Library like any subscribed channel's uploads).
|
||||
if channel.from_explore:
|
||||
channel.from_explore = False
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.channel_id == channel_id, Video.via_explore.is_(True))
|
||||
.values(via_explore=False)
|
||||
)
|
||||
service.record_subscription(db, user, channel, yt_sub_id)
|
||||
db.commit()
|
||||
return {"ok": True, "subscribed": channel_id}
|
||||
|
||||
@@ -659,7 +513,7 @@ def unsubscribe(
|
||||
# 422 (a real, explainable error → speaking modal), not 502 which the client treats as
|
||||
# a transient "connection lost" gateway blip.
|
||||
raise youtube_refusal(exc) from exc
|
||||
db.delete(sub)
|
||||
service.drop_subscription(db, sub)
|
||||
db.commit()
|
||||
return {"ok": True, "unsubscribed": channel_id}
|
||||
|
||||
@@ -667,16 +521,7 @@ def unsubscribe(
|
||||
@router.get("/blocked/list")
|
||||
def list_blocked(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
"""The user's blocked channels (videos from these are excluded from their search/feed/explore)."""
|
||||
rows = (
|
||||
db.execute(
|
||||
select(Channel)
|
||||
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
|
||||
.where(BlockedChannel.user_id == user.id)
|
||||
.order_by(collated(Channel.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = queries.blocked_channels(db, user, lang_for_user(user))
|
||||
return [
|
||||
{"id": c.id, "title": c.title, "handle": c.handle, "thumbnail_url": c.thumbnail_url}
|
||||
for c in rows
|
||||
@@ -692,16 +537,9 @@ def block_channel(
|
||||
"""Block a channel for this user: its videos won't appear in their live search (and won't be
|
||||
ingested for them) and are hidden from their feed / explore. Idempotent. The now-hidden
|
||||
un-kept search/explore videos are reclaimed by the discovery-cleanup job in due course."""
|
||||
get_or_404(db, Channel, channel_id, what="channel")
|
||||
if not _is_blocked(db, user, channel_id):
|
||||
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
|
||||
# Stop any active exploration of it so it can be cleaned up.
|
||||
db.execute(
|
||||
ExploredChannel.__table__.delete().where(
|
||||
(ExploredChannel.user_id == user.id) & (ExploredChannel.channel_id == channel_id)
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
_channel_or_404(db, channel_id)
|
||||
service.block(db, user, channel_id)
|
||||
db.commit()
|
||||
return {"ok": True, "blocked": channel_id}
|
||||
|
||||
|
||||
@@ -711,11 +549,7 @@ def unblock_channel(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
db.execute(
|
||||
BlockedChannel.__table__.delete().where(
|
||||
(BlockedChannel.user_id == user.id) & (BlockedChannel.channel_id == channel_id)
|
||||
)
|
||||
)
|
||||
service.unblock(db, user, channel_id)
|
||||
db.commit()
|
||||
return {"ok": True, "unblocked": channel_id}
|
||||
|
||||
@@ -732,26 +566,12 @@ def attach_tag(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
_user_subscription(db, user, channel_id)
|
||||
# Only the user's own tags may be attached as personal links (owned_or_404 → 404 if the tag
|
||||
# is missing OR belongs to someone else, without leaking which).
|
||||
owned_or_404(db, Tag, body.tag_id, user, what="tag")
|
||||
exists = db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == body.tag_id,
|
||||
ChannelTag.user_id == user.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if exists is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel_id,
|
||||
tag_id=body.tag_id,
|
||||
user_id=user.id,
|
||||
source="user",
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
# Only the user's own tags may be attached as personal links (a 404 if the tag is missing OR
|
||||
# belongs to someone else, without leaking which).
|
||||
if reader_queries.own_tag(db, user, body.tag_id) is None:
|
||||
raise not_found("tag")
|
||||
reader_service.attach_user_tag(db, user, channel_id, body.tag_id)
|
||||
db.commit()
|
||||
return {"ok": True, "channel_id": channel_id, "tag_id": body.tag_id}
|
||||
|
||||
|
||||
@@ -762,14 +582,6 @@ def detach_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
row = db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == tag_id,
|
||||
ChannelTag.user_id == user.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.commit()
|
||||
reader_service.detach_user_tag(db, user, channel_id, tag_id)
|
||||
db.commit()
|
||||
return {"ok": True, "channel_id": channel_id, "tag_id": tag_id}
|
||||
|
||||
@@ -18,6 +18,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs, logsink
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -95,7 +96,7 @@ def stop_session(
|
||||
def list_sessions(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sessions = db.scalars(
|
||||
select(DebugSession)
|
||||
.where(DebugSession.user_id == user.id)
|
||||
.where(owned_by(DebugSession.user_id, user))
|
||||
.order_by(DebugSession.started_at.desc())
|
||||
.limit(LIST_MAX)
|
||||
).all()
|
||||
@@ -219,6 +220,7 @@ def send_issue(sid: str, user: User = Depends(current_user), db: Session = Depen
|
||||
raise refused(409, "issue_sending")
|
||||
new = session.issue_url is None
|
||||
if not new and not debugsession.unfinished(session):
|
||||
db.commit() # nothing to send: the lock ends here, not when the request's session closes
|
||||
return {"ok": True, "session": _view(db, session)}
|
||||
key = str(user.id)
|
||||
if new:
|
||||
|
||||
@@ -375,6 +375,9 @@ def update_folder(
|
||||
folder = own_folder(db, user, folder_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
# Which folder may be renamed is answered before what the name is, as it always was.
|
||||
if not folders.renamable(folder):
|
||||
raise refused(409, "file_folder_not_own")
|
||||
try:
|
||||
folders.rename(db, folder, _folder_name(body.name or ""))
|
||||
except folders.FolderError as e:
|
||||
|
||||
@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import Notification, User
|
||||
@@ -36,7 +37,7 @@ def list_notifications(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
where = [Notification.user_id == user.id]
|
||||
where = [owned_by(Notification.user_id, user)]
|
||||
if not include_dismissed:
|
||||
where.append(Notification.dismissed.is_(False))
|
||||
total = db.scalar(select(func.count()).select_from(Notification).where(*where)) or 0
|
||||
@@ -61,7 +62,7 @@ def unread_count(user: User = Depends(current_user), db: Session = Depends(get_d
|
||||
select(func.count())
|
||||
.select_from(Notification)
|
||||
.where(
|
||||
Notification.user_id == user.id,
|
||||
owned_by(Notification.user_id, user),
|
||||
Notification.read.is_(False),
|
||||
Notification.dismissed.is_(False),
|
||||
)
|
||||
@@ -77,8 +78,8 @@ def mark_read(
|
||||
) -> dict:
|
||||
n = owned_or_404(db, Notification, notification_id, user, what="notification")
|
||||
n.read = True
|
||||
db.commit()
|
||||
trim_read(db, user.id)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -86,11 +87,11 @@ def mark_read(
|
||||
def mark_all_read(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.read.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.read.is_(False))
|
||||
.values(read=True)
|
||||
)
|
||||
db.commit()
|
||||
trim_read(db, user.id)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -113,7 +114,7 @@ def clear_all(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"""Dismiss every (still-visible) notification for this user."""
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.dismissed.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.dismissed.is_(False))
|
||||
.values(dismissed=True, read=True)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
@@ -1,18 +1,22 @@
|
||||
"""Shared helpers, serializers and constants for the plex route package."""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import PlexItem
|
||||
from app.plex import queries
|
||||
from app.plex.cards import norm_lang as _norm_lang # noqa: F401 — the package's historic name
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def item_or_404(db: Session, rating_key: str) -> PlexItem:
|
||||
it = by_rating_key(db, PlexItem, rating_key)
|
||||
def item_or_404(db: Session, rating_key: str) -> "PlexItem":
|
||||
it = queries.item(db, rating_key)
|
||||
if it is None:
|
||||
raise not_found("plex_item")
|
||||
return it
|
||||
|
||||
@@ -1,27 +1,26 @@
|
||||
"""Plex admin: connection test, catalog sync, and owner watch-link config."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexLink,
|
||||
User,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex import queries, service
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex import watch_sync as plex_watch
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.utils import iso
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexLink
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
@@ -69,7 +68,7 @@ def _sample_plex_path(db: Session, plex: PlexClient, sections: list[dict]) -> st
|
||||
"""A real Plex-side media file path to validate the mount + path-map against. Prefers the
|
||||
already-mirrored catalog (cheap); before the first sync, pulls one movie leaf from an enabled
|
||||
section straight from Plex."""
|
||||
fp = db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
|
||||
fp = queries.sample_file_path(db)
|
||||
if fp:
|
||||
return fp
|
||||
wanted = plex_sync._enabled_section_keys(db)
|
||||
@@ -106,7 +105,7 @@ def trigger_sync(_: User = Depends(admin_user), db: Session = Depends(get_db)) -
|
||||
return {"ok": True, **plex_sync.sync(db)}
|
||||
|
||||
|
||||
def _link_dict(link: PlexLink | None) -> dict:
|
||||
def _link_dict(link: "PlexLink | None") -> dict:
|
||||
return {
|
||||
"linked": link is not None,
|
||||
"sync_enabled": bool(link and link.sync_enabled),
|
||||
@@ -119,8 +118,7 @@ def _link_dict(link: PlexLink | None) -> dict:
|
||||
@router.get("/watch/link")
|
||||
def watch_link_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""This user's Plex watch-sync link status (drives the Settings toggle)."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
return _link_dict(link)
|
||||
return _link_dict(queries.watch_link(db, user))
|
||||
|
||||
|
||||
class WatchLinkIn(BaseModel):
|
||||
@@ -139,11 +137,7 @@ def watch_link_set(
|
||||
if not sysconfig.get_bool(db, "plex_enabled"):
|
||||
raise refused(409, "plex_disabled")
|
||||
enabled = body.enabled
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
if link is None:
|
||||
link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
|
||||
db.add(link)
|
||||
link.sync_enabled = enabled
|
||||
link = service.set_watch_link(db, user, enabled)
|
||||
db.commit()
|
||||
|
||||
result: dict = {}
|
||||
@@ -156,7 +150,7 @@ def watch_link_set(
|
||||
def watch_import_now(user: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""Admin: (re-)run the "Plex is master" import for this account — Plex re-wins on the
|
||||
intersection; Siftlode-only states are preserved. Idempotent."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
link = queries.watch_link(db, user)
|
||||
if link is None or not link.sync_enabled:
|
||||
raise refused(409, "watch_sync_off")
|
||||
result = plex_watch.import_owner_watch_state(db, link)
|
||||
|
||||
@@ -1,110 +1,30 @@
|
||||
"""Plex browse: the library list, the unified cross-library browse feed, and its faceted filters."""
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
Integer,
|
||||
String,
|
||||
and_,
|
||||
cast,
|
||||
func,
|
||||
literal,
|
||||
null,
|
||||
or_,
|
||||
select,
|
||||
)
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated, sorted_words
|
||||
from app import sysconfig
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.mailtext import lang_for_user
|
||||
from app.models import User
|
||||
from app.plex import library as plex_library
|
||||
from app.plex import queries
|
||||
from app.plex.cards import display_title, episode_card
|
||||
from app.plex.query import (
|
||||
FIELD_COLUMNS,
|
||||
PLAYBACK_CLASSES,
|
||||
TS_CONFIG,
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_movie_state_filter,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
field_recheck,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
movie_status_case,
|
||||
playable_conds,
|
||||
playback_class_expr,
|
||||
show_status_case,
|
||||
split_csv,
|
||||
summary_col,
|
||||
title_col,
|
||||
vector_col,
|
||||
vector_match,
|
||||
)
|
||||
from app.plex.query import LibraryFilters
|
||||
from app.routes.plex._shared import _norm_lang
|
||||
from app.utils import to_tsquery_str
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def sort_column(u, sort: str, lang: str = "en"):
|
||||
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
|
||||
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
|
||||
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
|
||||
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
|
||||
in the reader's alphabet (`collation`)."""
|
||||
return {
|
||||
"added": u.c.added_at,
|
||||
"title": collated(u.c.title, lang),
|
||||
"year": u.c.year,
|
||||
"rating": u.c.rating,
|
||||
"duration": u.c.duration_s,
|
||||
"release": u.c.rel,
|
||||
"lastwatched": u.c.last_watched,
|
||||
}.get(sort, u.c.added_at)
|
||||
|
||||
|
||||
@router.get("/libraries")
|
||||
def list_libraries(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""The mirrored libraries the user can browse (drives the Plex scope selector). HU overlay
|
||||
libraries are excluded — they're not browsable, they only feed the EN twin's Hungarian metadata."""
|
||||
libs = (
|
||||
db.execute(
|
||||
select(PlexLibrary)
|
||||
.where(PlexLibrary.overlay_of.is_(None))
|
||||
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
out = []
|
||||
for lib in libs:
|
||||
if lib.kind == "movie":
|
||||
count = db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.library_id == lib.id, PlexItem.kind == "movie"
|
||||
)
|
||||
)
|
||||
else:
|
||||
count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
|
||||
out.append(
|
||||
{
|
||||
"key": lib.plex_key,
|
||||
"title": lib.title,
|
||||
"kind": lib.kind,
|
||||
"count": count or 0,
|
||||
}
|
||||
)
|
||||
out = [
|
||||
{"key": lib.plex_key, "title": lib.title, "kind": lib.kind, "count": count}
|
||||
for lib, count in queries.browsable_libraries(db, lang_for_user(user))
|
||||
]
|
||||
return {"enabled": sysconfig.get_bool(db, "plex_enabled"), "libraries": out}
|
||||
|
||||
|
||||
@@ -129,193 +49,60 @@ def unified_library(
|
||||
episodes come back in a separate `episodes` list (grouped result — the 'Episodes' section)."""
|
||||
offset = max(0, offset)
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
|
||||
ts = to_tsquery_str(q) if q else None
|
||||
tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
|
||||
if field not in FIELD_COLUMNS and field != "all":
|
||||
field = "all"
|
||||
# `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
|
||||
# the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
|
||||
grid_field = field != "episode"
|
||||
recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
|
||||
|
||||
selects = []
|
||||
if want_movies and grid_field:
|
||||
st = aliased(PlexState)
|
||||
m_status = movie_status_case(st)
|
||||
mq = (
|
||||
select(
|
||||
PlexItem.rating_key.label("rk"),
|
||||
literal("movie").label("kind"),
|
||||
title_col(PlexItem, lang).label("title"),
|
||||
PlexItem.year.label("year"),
|
||||
PlexItem.rating.label("rating"),
|
||||
# Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
|
||||
func.left(summary_col(PlexItem, lang), 300).label("summary"),
|
||||
PlexItem.added_at.label("added_at"),
|
||||
PlexItem.originally_available_at.label("rel"),
|
||||
PlexItem.duration_s.label("duration_s"),
|
||||
PlexItem.playable.label("playable"),
|
||||
PlexItem.transcode_hw.label("transcode_hw"),
|
||||
cast(null(), Integer).label("season_count"),
|
||||
func.coalesce(st.position_seconds, 0).label("position_seconds"),
|
||||
m_status.label("status"),
|
||||
literal(False).label("resumable"),
|
||||
# "last watched" sort key: when the movie was last resumed, else when it was marked
|
||||
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
|
||||
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexItem, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(st, and_(st.item_id == PlexItem.id, st.user_id == user.id))
|
||||
.where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
|
||||
)
|
||||
mq = apply_movie_state_filter(mq, st, show)
|
||||
mq = apply_meta_filters(mq, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
mq = mq.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
mq = mq.where(PlexItem.duration_s <= f.duration_max)
|
||||
# Playback-class filter (movie-only — a show is an aggregate with no single class).
|
||||
mq = mq.where(*playable_conds(split_csv(f.playable)))
|
||||
if tsq is not None:
|
||||
mq = mq.where(vector_match(PlexItem, lang, tsq))
|
||||
if recheck is not None:
|
||||
mq = mq.where(recheck(PlexItem, field, tsq, lang))
|
||||
selects.append(mq)
|
||||
if want_shows and grid_field:
|
||||
agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
|
||||
tot = func.coalesce(agg.c.total, 0)
|
||||
wat = func.coalesce(agg.c.watched, 0)
|
||||
inp = func.coalesce(agg.c.inprog, 0)
|
||||
s_status = show_status_case(tot, wat, inp)
|
||||
sq = (
|
||||
select(
|
||||
PlexShow.rating_key.label("rk"),
|
||||
literal("show").label("kind"),
|
||||
title_col(PlexShow, lang).label("title"),
|
||||
PlexShow.year.label("year"),
|
||||
PlexShow.rating.label("rating"),
|
||||
func.left(summary_col(PlexShow, lang), 300).label("summary"),
|
||||
PlexShow.added_at.label("added_at"),
|
||||
PlexShow.originally_available_at.label("rel"),
|
||||
cast(null(), Integer).label("duration_s"),
|
||||
cast(null(), String).label("playable"),
|
||||
cast(null(), Boolean).label("transcode_hw"),
|
||||
PlexShow.child_count.label("season_count"),
|
||||
literal(0).label("position_seconds"),
|
||||
s_status.label("status"),
|
||||
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
|
||||
(inp > 0).label("resumable"),
|
||||
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
|
||||
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
|
||||
agg.c.resumed_at.label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexShow, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_lib_ids))
|
||||
)
|
||||
sq = apply_show_state_filter(sq, tot, wat, inp, show)
|
||||
sq = apply_meta_filters(sq, PlexShow, p, lang)
|
||||
if tsq is not None:
|
||||
sq = sq.where(vector_match(PlexShow, lang, tsq))
|
||||
if recheck is not None:
|
||||
sq = sq.where(recheck(PlexShow, field, tsq, lang))
|
||||
selects.append(sq)
|
||||
|
||||
total, rows, eps = plex_library.library_page(
|
||||
db,
|
||||
user,
|
||||
scope=scope,
|
||||
q=q,
|
||||
field=field,
|
||||
sort=sort,
|
||||
sort_dir=sort_dir,
|
||||
show=show,
|
||||
lang=lang,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
f=f,
|
||||
)
|
||||
items = []
|
||||
total = 0
|
||||
if selects:
|
||||
union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
|
||||
u = union.subquery()
|
||||
total = db.scalar(select(func.count()).select_from(u)) or 0
|
||||
|
||||
if tsq is not None:
|
||||
order = [u.c.rank.desc()]
|
||||
else:
|
||||
col = sort_column(u, sort, lang)
|
||||
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
|
||||
order.append(u.c.rk.desc())
|
||||
rows = db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all()
|
||||
|
||||
for r in rows:
|
||||
if r.kind == "movie":
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "movie",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"duration_seconds": r.duration_s,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"playable": r.playable,
|
||||
"transcode_hw": r.transcode_hw,
|
||||
"status": r.status,
|
||||
"position_seconds": r.position_seconds,
|
||||
}
|
||||
)
|
||||
else:
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "show",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"season_count": r.season_count,
|
||||
"status": r.status,
|
||||
"resumable": r.resumable,
|
||||
}
|
||||
)
|
||||
|
||||
# Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
|
||||
# and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
|
||||
# target the movie/show grid, so they suppress episodes.
|
||||
episodes = []
|
||||
if tsq is not None and want_shows and field in ("all", "episode"):
|
||||
ep_st = aliased(PlexState)
|
||||
# In HU an episode is found by its Hungarian title too (the overlay fills it) — and
|
||||
# still by its English one, like `vector_match` does for the grid.
|
||||
if field == "episode":
|
||||
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
|
||||
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
|
||||
else:
|
||||
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
|
||||
hit = ep_vec.op("@@")(tsq)
|
||||
if lang == "hu":
|
||||
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
|
||||
ep_vec = ep_vec_hu
|
||||
eps = db.execute(
|
||||
select(PlexItem, ep_st, PlexShow)
|
||||
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, ep_st.user_id == user.id))
|
||||
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_lib_ids),
|
||||
hit,
|
||||
for r in rows:
|
||||
if r.kind == "movie":
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "movie",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"duration_seconds": r.duration_s,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"playable": r.playable,
|
||||
"transcode_hw": r.transcode_hw,
|
||||
"status": r.status,
|
||||
"position_seconds": r.position_seconds,
|
||||
}
|
||||
)
|
||||
.order_by(func.ts_rank(ep_vec, tsq).desc())
|
||||
.limit(24)
|
||||
).all()
|
||||
for e, s, sh in eps:
|
||||
card = episode_card(e, s, lang=lang)
|
||||
card["show_title"] = display_title(sh, lang) if sh is not None else None
|
||||
episodes.append(card)
|
||||
else:
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "show",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"season_count": r.season_count,
|
||||
"status": r.status,
|
||||
"resumable": r.resumable,
|
||||
}
|
||||
)
|
||||
|
||||
episodes = []
|
||||
for e, s, sh in eps:
|
||||
card = episode_card(e, s, lang=lang)
|
||||
card["show_title"] = display_title(sh, lang) if sh is not None else None
|
||||
episodes.append(card)
|
||||
|
||||
return {
|
||||
"scope": scope,
|
||||
@@ -341,237 +128,4 @@ def facets(
|
||||
filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
|
||||
a multi-select group still lets you add more. Merged across the scope's libraries; duration is
|
||||
movie-only. (Search text is not factored in — facets track the sidebar filters.)"""
|
||||
empty = {
|
||||
"genres": [],
|
||||
"content_ratings": [],
|
||||
"studios": [],
|
||||
"playable": [],
|
||||
"year_min": None,
|
||||
"year_max": None,
|
||||
"rating_max": None,
|
||||
"duration_min": None,
|
||||
"duration_max": None,
|
||||
}
|
||||
all_movie_ids, all_show_ids = content_libs(db)
|
||||
movie_ids = all_movie_ids if scope in ("movie", "both") else []
|
||||
show_ids = all_show_ids if scope in ("show", "both") else []
|
||||
if not movie_ids and not show_ids:
|
||||
return empty
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
|
||||
uid = user.id
|
||||
|
||||
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
|
||||
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
|
||||
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
|
||||
# The same filter on the same rows, so the counts do not change
|
||||
# (test_plex_facets_grid_parity).
|
||||
shows_in_scope: list[int] | None = None
|
||||
if show_ids and show in ("watched", "in_progress", "unwatched"):
|
||||
agg = rollup.agg_subquery(db, uid).subquery()
|
||||
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
in_scope = apply_show_state_filter(
|
||||
in_scope,
|
||||
func.coalesce(agg.c.total, 0),
|
||||
func.coalesce(agg.c.watched, 0),
|
||||
func.coalesce(agg.c.inprog, 0),
|
||||
show,
|
||||
)
|
||||
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
|
||||
|
||||
def wsq(query, model, ids, pp, kind_movie, with_dur):
|
||||
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
|
||||
facet aggregation query, so the facets reflect exactly the visible result set."""
|
||||
query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
|
||||
if kind_movie:
|
||||
query = query.where(model.kind == "movie")
|
||||
if with_dur and pp.get("duration_min") is not None:
|
||||
query = query.where(model.duration_s >= pp["duration_min"])
|
||||
if with_dur and pp.get("duration_max") is not None:
|
||||
query = query.where(model.duration_s <= pp["duration_max"])
|
||||
# Playback-class filter (movie-only) — applied for the OTHER groups so an active class
|
||||
# narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
|
||||
query = query.where(*playable_conds(split_csv(pp.get("playable"))))
|
||||
# Join watch-state and reuse the grid's exact movie state filter (unified_library) —
|
||||
# hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
|
||||
st = aliased(PlexState)
|
||||
query = query.outerjoin(st, and_(st.item_id == model.id, st.user_id == uid))
|
||||
query = apply_movie_state_filter(query, st, show)
|
||||
elif shows_in_scope is not None:
|
||||
query = query.where(model.id.in_(shows_in_scope))
|
||||
return query
|
||||
|
||||
genre_counts: dict[str, int] = {}
|
||||
cr_counts: dict[str, int] = {}
|
||||
studio_counts: dict[str, int] = {}
|
||||
playable_counts: dict[str, int] = {}
|
||||
years: list[int] = []
|
||||
ratings: list[float] = []
|
||||
durs: list[int] = []
|
||||
|
||||
def per_table(model, ids, kind_movie: bool) -> None:
|
||||
if not ids:
|
||||
return
|
||||
# Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
|
||||
# OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
|
||||
# offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
|
||||
# the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
|
||||
# every matching title carries them, so they stay togglable.)
|
||||
genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
|
||||
# Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
|
||||
# `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
|
||||
# jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
|
||||
# so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
|
||||
# whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
|
||||
# In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
|
||||
gcol = genres_col(model, lang)
|
||||
expanded = (
|
||||
wsq(
|
||||
select(func.jsonb_array_elements_text(gcol).label("g")),
|
||||
model,
|
||||
ids,
|
||||
genre_pp,
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(func.jsonb_typeof(gcol) == "array")
|
||||
.subquery()
|
||||
)
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
genre_counts[g] = genre_counts.get(g, 0) + c
|
||||
# Content ratings — exclude its own.
|
||||
crq = (
|
||||
wsq(
|
||||
select(model.content_rating, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "content_ratings": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.content_rating.isnot(None))
|
||||
.group_by(model.content_rating)
|
||||
)
|
||||
for cr, c in db.execute(crq).all():
|
||||
cr_counts[cr] = cr_counts.get(cr, 0) + c
|
||||
# Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
|
||||
stq = (
|
||||
wsq(
|
||||
select(model.studio, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "studios": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.studio.isnot(None))
|
||||
.group_by(model.studio)
|
||||
)
|
||||
for s, c in db.execute(stq).all():
|
||||
studio_counts[s] = studio_counts.get(s, 0) + c
|
||||
# Playback class (movie-only) — exclude its own so each class stays an addable OR option.
|
||||
if kind_movie:
|
||||
pcls = playback_class_expr()
|
||||
pcq = (
|
||||
wsq(
|
||||
select(pcls.label("cls"), func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "playable": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(pcls.isnot(None))
|
||||
.group_by(pcls)
|
||||
)
|
||||
for cls, c in db.execute(pcq).all():
|
||||
playable_counts[cls] = playable_counts.get(cls, 0) + c
|
||||
# Year bounds — exclude the year filter.
|
||||
by = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.year), func.max(model.year)),
|
||||
model,
|
||||
ids,
|
||||
{**p, "year_min": None, "year_max": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
).first()
|
||||
if by[0] is not None:
|
||||
years.append(by[0])
|
||||
if by[1] is not None:
|
||||
years.append(by[1])
|
||||
# Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
|
||||
# rating filter and the UI badge use (IMDb preferred, audience fallback).
|
||||
br = db.scalar(
|
||||
wsq(
|
||||
select(func.max(func.coalesce(model.imdb_rating, model.rating))),
|
||||
model,
|
||||
ids,
|
||||
{**p, "rating_min": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
)
|
||||
if br is not None:
|
||||
ratings.append(float(br))
|
||||
# Duration bounds (movies) — exclude the duration filter.
|
||||
if kind_movie:
|
||||
bd = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.duration_s), func.max(model.duration_s)),
|
||||
model,
|
||||
ids,
|
||||
p,
|
||||
kind_movie,
|
||||
False,
|
||||
)
|
||||
).first()
|
||||
if bd[0] is not None:
|
||||
durs.append(bd[0])
|
||||
if bd[1] is not None:
|
||||
durs.append(bd[1])
|
||||
|
||||
per_table(PlexItem, movie_ids, True)
|
||||
per_table(PlexShow, show_ids, False)
|
||||
# Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
|
||||
# be absent): the sidebar renders genres solely from this list and hides the whole genre section —
|
||||
# chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
|
||||
# undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
|
||||
for g in split_csv(f.genres):
|
||||
genre_counts.setdefault(g, 0)
|
||||
# Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
|
||||
for s in split_csv(f.studios):
|
||||
studio_counts.setdefault(s, 0)
|
||||
# Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
|
||||
# section when its facet list is empty, which would trap a zero-result selection with no way to undo).
|
||||
for c in split_csv(f.playable):
|
||||
if c in PLAYBACK_CLASSES:
|
||||
playable_counts.setdefault(c, 0)
|
||||
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
|
||||
return {
|
||||
"genres": [
|
||||
{"value": g, "count": c}
|
||||
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
|
||||
],
|
||||
"content_ratings": [
|
||||
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
|
||||
],
|
||||
"studios": [
|
||||
{"value": s, "count": c}
|
||||
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
|
||||
],
|
||||
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
|
||||
"playable": [
|
||||
{"value": c, "count": playable_counts[c]}
|
||||
for c in PLAYBACK_CLASSES
|
||||
if c in playable_counts
|
||||
],
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"rating_max": max(ratings) if ratings else None,
|
||||
"duration_min": min(durs) if durs else None,
|
||||
"duration_max": max(durs) if durs else None,
|
||||
}
|
||||
return plex_library.facets(db, user, scope=scope, show=show, lang=lang, f=f)
|
||||
|
||||
@@ -1,96 +1,41 @@
|
||||
"""Plex collections: browse mirrored collections + the admin manual-collection editor."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime
|
||||
from typing import Any, NamedTuple
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app import logs
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex.cards import SHOW_NEW, display_title, movie_card, norm_lang, show_card
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
|
||||
from app.utils import like_escape
|
||||
from app.plex.collections import (
|
||||
DEFAULT_MEMBER_SORT,
|
||||
Member,
|
||||
collection_card,
|
||||
collection_cert,
|
||||
collection_editable,
|
||||
collection_source,
|
||||
sort_members,
|
||||
)
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexCollection, PlexState
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
|
||||
# members (Plex shows one cert on the collection; we don't store it, so derive it).
|
||||
_CERT_RANK = {
|
||||
"G": 1,
|
||||
"TV-Y": 1,
|
||||
"TV-Y7": 1,
|
||||
"TV-G": 1,
|
||||
"PG": 2,
|
||||
"TV-PG": 2,
|
||||
"PG-13": 3,
|
||||
"TV-14": 3,
|
||||
"R": 4,
|
||||
"TV-MA": 4,
|
||||
"NC-17": 5,
|
||||
}
|
||||
|
||||
|
||||
def _collection_cert(certs: list[str | None]) -> str | None:
|
||||
ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
|
||||
return max(ranked, key=lambda x: x[1])[0] if ranked else None
|
||||
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
|
||||
|
||||
|
||||
def _collection_source(col: PlexCollection) -> str:
|
||||
first = (col.title or "").strip().split(" ", 1)[0].lower()
|
||||
if first in _STRIP_PROVIDERS:
|
||||
return first
|
||||
if col.smart:
|
||||
return "smart"
|
||||
return "collection"
|
||||
|
||||
|
||||
def _collection_editable(c: PlexCollection) -> bool:
|
||||
"""Effective editability: the admin marked it editable AND it's a plain manual collection — never
|
||||
smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
|
||||
return bool(c.editable) and not c.smart and _collection_source(c) == "collection"
|
||||
|
||||
|
||||
def _collection_card(c: PlexCollection) -> dict:
|
||||
return {
|
||||
"id": c.rating_key,
|
||||
"title": c.title,
|
||||
"summary": c.summary,
|
||||
"thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
|
||||
"child_count": c.child_count,
|
||||
"smart": c.smart,
|
||||
"source": _collection_source(c),
|
||||
"editable": c.editable,
|
||||
"can_edit": _collection_editable(
|
||||
c
|
||||
), # whether the edit controls should be offered (admin UI)
|
||||
}
|
||||
|
||||
|
||||
@router.get("/collections")
|
||||
def collections(
|
||||
@@ -110,64 +55,22 @@ def collections(
|
||||
sidebar token and the applied-filter chip had nothing to render but the raw id. Resolving them
|
||||
through this endpoint costs one small row each; pulling the whole list to find two titles is
|
||||
~200 KB."""
|
||||
if library is not None:
|
||||
lib = (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(library)))
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if lib is None:
|
||||
return {"collections": []}
|
||||
query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
|
||||
else:
|
||||
enabled_ids = list(
|
||||
db.execute(
|
||||
select(PlexLibrary.id).where(
|
||||
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
if not enabled_ids:
|
||||
return {"collections": []}
|
||||
query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
|
||||
term = (q or "").strip()
|
||||
if term:
|
||||
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
|
||||
if keys is not None:
|
||||
wanted = [k for k in (keys or "").split(",") if k]
|
||||
# An explicit but empty `keys=` asks for nothing — answer nothing, rather than falling through
|
||||
# to "every collection" (which is what an unguarded empty IN () would have to become).
|
||||
if not wanted:
|
||||
return {"collections": []}
|
||||
query = query.where(PlexCollection.rating_key.in_(wanted))
|
||||
rows = (
|
||||
db.execute(
|
||||
query.order_by(
|
||||
PlexCollection.child_count.desc().nullslast(),
|
||||
collated(PlexCollection.title, lang_for_user(user)),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
wanted = [k for k in keys.split(",") if k] if keys is not None else None
|
||||
rows = queries.collections_page(db, library, q, wanted, lang_for_user(user))
|
||||
# A montage of the top titles per tile + the collection's total runtime. Art unions MOVIE and SHOW
|
||||
# members (a collection lives in one library, so exactly one side has art) — a show collection would
|
||||
# otherwise render blank; the collection's own poster is the next fallback, then the colour hash.
|
||||
# Runtime is movie-only (shows have no single duration).
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
keys = [c.rating_key for c in rows]
|
||||
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
|
||||
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
|
||||
runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
|
||||
art_m, art_s, runtime = queries.collection_tiles(db, [c.rating_key for c in rows])
|
||||
|
||||
def _art(c: PlexCollection) -> list[str]:
|
||||
def _art(c: "PlexCollection") -> list[str]:
|
||||
member = art_m.get(c.rating_key) or art_s.get(c.rating_key)
|
||||
return member or ([c.rating_key] if c.thumb_key else [])
|
||||
|
||||
return {
|
||||
"collections": [
|
||||
{
|
||||
**_collection_card(c),
|
||||
**collection_card(c),
|
||||
"art": _art(c),
|
||||
"total_runtime_s": runtime.get(c.rating_key, 0),
|
||||
}
|
||||
@@ -185,89 +88,21 @@ def collections_for_item(
|
||||
"""What a movie or show card's "Add to collection" lists: the editable collections of the title's
|
||||
library (a collection lives in one library, and only a manual one can take a title — the same
|
||||
`can_edit` the editor offers), each saying whether the title is in it already, by name."""
|
||||
movie = by_rating_key(db, PlexItem, item_rating_key)
|
||||
title: PlexItem | PlexShow | None = (
|
||||
movie if movie is not None and movie.kind == "movie" else None
|
||||
) or by_rating_key(db, PlexShow, item_rating_key)
|
||||
movie = queries.movie(db, item_rating_key)
|
||||
title = movie or queries.show(db, item_rating_key)
|
||||
if title is None:
|
||||
raise not_found("item")
|
||||
member_of = set(title.collection_keys or [])
|
||||
rows = (
|
||||
db.execute(
|
||||
select(PlexCollection)
|
||||
.where(PlexCollection.library_id == title.library_id)
|
||||
.order_by(collated(PlexCollection.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = queries.library_collections(db, title.library_id, lang_for_user(user))
|
||||
return {
|
||||
"collections": [
|
||||
{**_collection_card(c), "contains": c.rating_key in member_of}
|
||||
{**collection_card(c), "contains": c.rating_key in member_of}
|
||||
for c in rows
|
||||
if _collection_editable(c)
|
||||
if collection_editable(c)
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
class Member(NamedTuple):
|
||||
"""One collection member flattened to everything the Sort control can order by. The page's two
|
||||
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
|
||||
ordering is a pure list sort over these, which is also what makes it unit-testable."""
|
||||
|
||||
kind: str # "movie" | "show"
|
||||
obj: Any # PlexItem | PlexShow, carried through to the card serializer
|
||||
title: str # lower-cased; never NULL (the column is not nullable)
|
||||
year: int | None
|
||||
rating: float | None
|
||||
added: datetime | None
|
||||
release: date | None
|
||||
duration: int | None # movies only — a show has no single runtime
|
||||
last_watched: datetime | None
|
||||
|
||||
|
||||
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
|
||||
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
|
||||
# that silently does nothing here.
|
||||
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
|
||||
"added": lambda m: m.added,
|
||||
"lastwatched": lambda m: m.last_watched,
|
||||
"release": lambda m: m.release,
|
||||
"year": lambda m: m.year,
|
||||
"rating": lambda m: m.rating,
|
||||
"duration": lambda m: m.duration,
|
||||
"title": lambda m: m.title,
|
||||
}
|
||||
|
||||
# What the page ordered by before it had a control, and still its default: a film series wants to be
|
||||
# read in release order, not by whenever Plex happened to ingest it.
|
||||
DEFAULT_MEMBER_SORT = "year"
|
||||
|
||||
|
||||
def sort_members(
|
||||
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
|
||||
) -> list[Member]:
|
||||
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
|
||||
|
||||
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
|
||||
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
|
||||
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
|
||||
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
|
||||
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
|
||||
|
||||
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
|
||||
compare as Python strings, which puts every accented initial after "z"."""
|
||||
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
|
||||
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
|
||||
if key is MEMBER_SORT_KEYS["title"]:
|
||||
key = by_title
|
||||
ordered = sorted(members, key=by_title) # stable secondary, applied first
|
||||
have = [m for m in ordered if key(m) is not None]
|
||||
missing = [m for m in ordered if key(m) is None]
|
||||
have.sort(key=key, reverse=sort_dir != "asc")
|
||||
return have + missing
|
||||
|
||||
|
||||
@router.get("/collections/{rating_key}")
|
||||
def collection_detail(
|
||||
rating_key: str,
|
||||
@@ -283,45 +118,20 @@ def collection_detail(
|
||||
what this page did before it had a control. Read-only here; edits go through the admin mutation
|
||||
endpoints below."""
|
||||
lang = norm_lang(lang)
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow)
|
||||
.where(PlexShow.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
movies, shows = queries.collection_members(db, rating_key, lang)
|
||||
mstate = rollup.item_state_map(db, user.id, [m.id for m in movies])
|
||||
# ONE rollup pass over the show members, for both the aggregate badge status and `resumed_at` —
|
||||
# the browse grid's "last watched" for a show (the newest resume/watched across its episodes). The
|
||||
# cheaper `rollup.state_map` gives only the status, which would leave `lastwatched` unorderable on
|
||||
# a show collection while the control still offered it.
|
||||
show_rows = (
|
||||
db.execute(
|
||||
rollup.agg_subquery(db, user.id, include_resumed=True).where(
|
||||
PlexItem.show_id.in_([s.id for s in shows])
|
||||
)
|
||||
).all()
|
||||
if shows
|
||||
else []
|
||||
)
|
||||
show_rows = queries.show_rollups(db, user.id, [s.id for s in shows])
|
||||
sstate = {r.show_id: rollup.show_state(r.total, r.watched, r.inprog) for r in show_rows}
|
||||
sresumed = {r.show_id: r.resumed_at for r in show_rows}
|
||||
|
||||
def _movie_last_watched(st: PlexState | None) -> datetime | None:
|
||||
def _movie_last_watched(st: "PlexState | None") -> datetime | None:
|
||||
# Same definition as the browse grid's: when it was last resumed, else when it was marked
|
||||
# watched (coalesce(progress_updated_at, watched_at)).
|
||||
return (st.progress_updated_at or st.watched_at) if st else None
|
||||
@@ -365,8 +175,8 @@ def collection_detail(
|
||||
years = [m.year for m in members if m.year is not None]
|
||||
certs = [m.content_rating for m in movies] + [s.content_rating for s in shows]
|
||||
return {
|
||||
**_collection_card(col),
|
||||
"content_rating": _collection_cert(certs),
|
||||
**collection_card(col),
|
||||
"content_rating": collection_cert(certs),
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"total_runtime_s": sum(m.duration_s or 0 for m in movies),
|
||||
@@ -374,11 +184,11 @@ def collection_detail(
|
||||
}
|
||||
|
||||
|
||||
def _editable_collection_or_403(db: Session, rating_key: str) -> PlexCollection:
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
def _editable_collection_or_403(db: Session, rating_key: str) -> "PlexCollection":
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
if not _collection_editable(col):
|
||||
if not collection_editable(col):
|
||||
raise refused(403, "collection_read_only")
|
||||
return col
|
||||
|
||||
@@ -389,16 +199,7 @@ def _reject_duplicate_name(
|
||||
"""No two collections in a section may share a name: Plex CONFLATES same-named collections'
|
||||
membership (both rating_keys resolve to one merged member list — a real footgun), so forbid it up
|
||||
front on create + rename rather than mirror the mess."""
|
||||
dup = (
|
||||
db.execute(
|
||||
select(PlexCollection).where(
|
||||
PlexCollection.library_id == library_id,
|
||||
func.lower(PlexCollection.title) == title.lower(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
dup = queries.collection_named(db, library_id, title)
|
||||
if dup is not None and dup.rating_key != exclude_rk:
|
||||
# Structured, so the bilingual UI says it in the reader's language (it was toasted in
|
||||
# Hungarian beside an English error dialog saying the same thing).
|
||||
@@ -441,15 +242,13 @@ def create_collection(
|
||||
seed = body.item_rating_key
|
||||
if not title or not seed:
|
||||
raise refused(400, "collection_fields_required")
|
||||
seed_show = by_rating_key(db, PlexShow, seed)
|
||||
seed_show = queries.show(db, seed)
|
||||
if library:
|
||||
lib = (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == library)).scalars().first()
|
||||
)
|
||||
lib = queries.library_by_key(db, library)
|
||||
else:
|
||||
# The "New collection" flow only knows the seed title → derive its library.
|
||||
seed_item = by_rating_key(db, PlexItem, seed) or seed_show
|
||||
lib = db.get(PlexLibrary, seed_item.library_id) if seed_item else None
|
||||
seed_item = queries.item(db, seed) or seed_show
|
||||
lib = queries.library(db, seed_item.library_id) if seed_item else None
|
||||
if lib is None:
|
||||
raise not_found("library")
|
||||
_reject_duplicate_name(db, lib.id, title)
|
||||
@@ -462,28 +261,16 @@ def create_collection(
|
||||
wrote = True
|
||||
new_rk = str(new.get("ratingKey") or "")
|
||||
col = plex_sync.resync_collection(db, plex, lib, new_rk)
|
||||
col.editable = True
|
||||
service.set_collection_editable(col, True)
|
||||
db.commit()
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
if wrote and new_rk:
|
||||
# It is on Plex now, and only the follow-up read failed. Its row is kept, editable: the
|
||||
# full sync preserves `editable` but creates a missing row read-only, so the collection
|
||||
# would drop out of every "add to" list; and the duplicate-name check reads these rows,
|
||||
# so a retry of the same name would make a second one on Plex. Plex made it with exactly
|
||||
# the seed in it, so the seed's row says so too — otherwise the title showed unticked in
|
||||
# the collection it was just added to, and the collection empty, until that sync.
|
||||
db.add(
|
||||
PlexCollection(
|
||||
rating_key=new_rk, library_id=lib.id, title=title, editable=True, child_count=1
|
||||
)
|
||||
)
|
||||
Model = PlexItem if lib.kind == "movie" else PlexShow
|
||||
if row := by_rating_key(db, Model, seed):
|
||||
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), new_rk]))
|
||||
# It is on Plex now, and only the follow-up read failed: keep its row (see the service).
|
||||
service.keep_created_collection(db, lib, new_rk, title, seed)
|
||||
db.commit()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.post("/collections/{rating_key}/items/{item_rating_key}")
|
||||
@@ -494,7 +281,7 @@ def collection_add_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -505,7 +292,7 @@ def collection_add_item(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.delete("/collections/{rating_key}/items/{item_rating_key}")
|
||||
@@ -516,7 +303,7 @@ def collection_remove_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -527,7 +314,7 @@ def collection_remove_item(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
class CollectionRenameIn(BaseModel):
|
||||
@@ -546,7 +333,7 @@ def collection_rename(
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
_reject_duplicate_name(db, col.library_id, title, exclude_rk=rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -557,7 +344,7 @@ def collection_rename(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.delete("/collections/{rating_key}")
|
||||
@@ -565,7 +352,7 @@ def collection_delete(
|
||||
rating_key: str, _: User = Depends(admin_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
plex.delete_collection(rating_key)
|
||||
@@ -590,12 +377,12 @@ def collection_set_editable(
|
||||
) -> dict:
|
||||
"""Mark an existing Plex collection as 'mine to edit' (or unmark). Smart / auto-list collections
|
||||
(IMDb/TMDb/…) can never be made editable — they're regenerated outside Plex."""
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
want = body.editable
|
||||
if want and (col.smart or _collection_source(col) != "collection"):
|
||||
if want and (col.smart or collection_source(col) != "collection"):
|
||||
raise refused(400, "collection_smart")
|
||||
col.editable = want
|
||||
service.set_collection_editable(col, want)
|
||||
db.commit()
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
@@ -4,52 +4,34 @@ import re
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app import logs
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.discover import queries
|
||||
from app.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.discover import queries as discover_queries
|
||||
from app.models import User
|
||||
from app.plex import detail as plex_detail
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import rollup, subtitles
|
||||
from app.plex.query import title_col
|
||||
from app.plex import queries, rollup, service, subtitles
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.collections import collection_source
|
||||
from app.plex.cards import SHOW_NEW, episode_card, movie_card, show_card
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes.plex._shared import _norm_lang, item_or_404
|
||||
from app.routes.plex.collections import _collection_source
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem, PlexSeason
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def _lib_key(db: Session, library_id: int | None) -> str | None:
|
||||
"""The Plex section key for a library id — the info/show page passes it to the admin collection
|
||||
editor (in the unified cross-library view there's no single library prop otherwise)."""
|
||||
if library_id is None:
|
||||
return None
|
||||
lib = db.get(PlexLibrary, library_id)
|
||||
return lib.plex_key if lib else None
|
||||
|
||||
|
||||
_SHOW_ENRICH_TTL_S = 300
|
||||
|
||||
_show_enrich_cache: dict[str, tuple[float, dict, list]] = {}
|
||||
@@ -113,39 +95,12 @@ def show_detail(
|
||||
"""A show's seasons + episodes (the drill-down view) with per-user watch state."""
|
||||
lang = _norm_lang(lang)
|
||||
hu = lang == "hu"
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
seasons = (
|
||||
db.execute(
|
||||
select(PlexSeason)
|
||||
.where(PlexSeason.show_id == sh.id)
|
||||
.order_by(PlexSeason.season_number.asc().nullsfirst())
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.item_id.in_([e.id for e in eps] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
seasons = queries.seasons(db, sh)
|
||||
eps = queries.episodes(db, sh)
|
||||
states = rollup.item_state_map(db, user, [e.id for e in eps])
|
||||
by_season: dict[int | None, list] = {}
|
||||
all_cards: list[dict] = [] # every episode, in season/episode order → show-level rollup
|
||||
for e in eps:
|
||||
@@ -163,9 +118,7 @@ def show_detail(
|
||||
meta, related_raw = _show_enrichment(db, sh.rating_key)
|
||||
related = _relatedshow_cards(db, user.id, related_raw, exclude=sh.id, lang=lang)
|
||||
if sh.people_enriched_at is None and meta:
|
||||
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
|
||||
plex_detail.apply_rich_detail(sh, meta) # imdb/tagline/full_cast for future views
|
||||
sh.people_enriched_at = datetime.now(timezone.utc)
|
||||
service.enrich_show(db, sh, meta) # imdb/tagline/full_cast for future views
|
||||
try:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — a failed enrich must not fail the page
|
||||
@@ -198,7 +151,7 @@ def show_detail(
|
||||
"year": sh.year,
|
||||
"thumb": f"/api/plex/image/{sh.rating_key}",
|
||||
"art": f"/api/plex/image/{sh.rating_key}?variant=art",
|
||||
"library": _lib_key(db, sh.library_id),
|
||||
"library": queries.library_key(db, sh.library_id),
|
||||
"content_rating": sh.content_rating,
|
||||
"rating": sh.rating,
|
||||
"genres": (sh.genres_hu or sh.genres if hu else sh.genres) or [],
|
||||
@@ -234,13 +187,13 @@ def show_next(
|
||||
Asked one show at a time, when a card is hovered or played, rather than carried on every card of a
|
||||
grid: three probes a show would cost a 60-card page 180 queries for the few cards anyone plays.
|
||||
`next` is null for a show with no episodes."""
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
return {"next": queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
|
||||
return {"next": discover_queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
|
||||
|
||||
|
||||
def _season_block(se: PlexSeason, cards: list[dict]) -> dict:
|
||||
def _season_block(se: "PlexSeason", cards: list[dict]) -> dict:
|
||||
"""A season's card for the show page (with its aggregate rollup) plus its episodes (used by the
|
||||
season subpage, read from the same cached show-detail payload)."""
|
||||
roll = rollup.rollup(cards)
|
||||
@@ -265,12 +218,7 @@ def _relatedshow_cards(
|
||||
rks = [str(m.get("ratingKey")) for m in related if m.get("ratingKey")]
|
||||
if not rks:
|
||||
return []
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks[:60]), PlexShow.id != exclude)
|
||||
).scalars()
|
||||
}
|
||||
shows = queries.shows_by_keys(db, rks[:60], exclude)
|
||||
statuses = rollup.state_map(db, user_id, [s.id for s in shows.values()])
|
||||
out: list[dict] = []
|
||||
seen: set[str] = set()
|
||||
@@ -285,73 +233,36 @@ def _relatedshow_cards(
|
||||
return out
|
||||
|
||||
|
||||
def _collection_strips(db: Session, it: PlexItem, user_id: int, lang: str) -> list[dict]:
|
||||
def _collection_strips(db: Session, it: "PlexItem", user_id: int, lang: str) -> list[dict]:
|
||||
"""The collections this movie belongs to, each with its member titles (playable cards) — the
|
||||
'Avatar Collection' strip on the info page. Pure local lookup (no Plex call)."""
|
||||
keys = it.collection_keys or []
|
||||
if not keys or it.kind != "movie":
|
||||
return []
|
||||
cols = {
|
||||
c.rating_key: c
|
||||
for c in db.execute(
|
||||
select(PlexCollection).where(PlexCollection.rating_key.in_(keys))
|
||||
).scalars()
|
||||
}
|
||||
cols = {c.rating_key: c for c in queries.collections_by_keys(db, keys)}
|
||||
# Smallest (most specific — e.g. a franchise) first; the big auto-lists (IMDb/TMDb Popular) last.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [c.title for c in cols.values()], lang))}
|
||||
ordered = sorted(cols.values(), key=lambda c: (c.child_count or 0, alpha[c.title]))
|
||||
out = []
|
||||
for col in ordered:
|
||||
rk = col.rating_key
|
||||
members = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.library_id == it.library_id,
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.collection_keys.contains([rk]),
|
||||
)
|
||||
.order_by(
|
||||
PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang)
|
||||
)
|
||||
.limit(60)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
ids = [m.id for m in members]
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user_id, PlexState.item_id.in_(ids or [0])
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
members = queries.library_movies_in(db, it.library_id, rk, lang, limit=60)
|
||||
states = rollup.item_state_map(db, user_id, [m.id for m in members])
|
||||
out.append(
|
||||
{
|
||||
"id": rk,
|
||||
"title": col.title,
|
||||
"source": _collection_source(col),
|
||||
"source": collection_source(col),
|
||||
"items": [movie_card(m, states.get(m.id), lang=lang) for m in members],
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _episode_nav(db: Session, it: PlexItem) -> tuple[str | None, str | None]:
|
||||
def _episode_nav(db: Session, it: "PlexItem") -> tuple[str | None, str | None]:
|
||||
if it.kind != "episode" or it.show_id is None:
|
||||
return None, None
|
||||
keys = list(
|
||||
db.execute(
|
||||
select(PlexItem.rating_key)
|
||||
.where(PlexItem.show_id == it.show_id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
keys = queries.episode_keys(db, it.show_id)
|
||||
try:
|
||||
i = keys.index(it.rating_key)
|
||||
except ValueError:
|
||||
@@ -398,13 +309,7 @@ def item_detail(
|
||||
prev/next, and the per-user resume position + watch status."""
|
||||
lang = _norm_lang(lang)
|
||||
it = item_or_404(db, rating_key)
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
|
||||
# Markers + audio/subtitle streams are file-specific, so they stay LIVE; the rich detail (cast/IMDb/
|
||||
# tagline/genres) is served from the DB row for movies (persisted below / by the backfill) and only
|
||||
@@ -485,11 +390,7 @@ def item_detail(
|
||||
# only stored the listing's top ~3), so fill a not-yet-enriched movie — its cast becomes filterable
|
||||
# and future views serve from the row. Episodes aren't enriched; a show's cast is filled from /show.
|
||||
if it.kind == "movie" and it.people_enriched_at is None and meta:
|
||||
cast = plex_people.cast_from_meta(meta)
|
||||
if cast:
|
||||
plex_people.enrich_row_people(db, it, cast)
|
||||
plex_detail.apply_rich_detail(it, meta)
|
||||
it.people_enriched_at = datetime.now(timezone.utc)
|
||||
service.enrich_movie(db, it, meta)
|
||||
try:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — a failed enrich must not fail the page
|
||||
@@ -522,8 +423,8 @@ def item_detail(
|
||||
rich = plex_detail.rich_detail(meta)
|
||||
|
||||
prev_id, next_id = _episode_nav(db, it)
|
||||
show = db.get(PlexShow, it.show_id) if it.kind == "episode" and it.show_id else None
|
||||
season = db.get(PlexSeason, it.season_id) if show and it.season_id else None
|
||||
show = queries.show_by_id(db, it.show_id) if it.kind == "episode" and it.show_id else None
|
||||
season = queries.season_by_id(db, it.season_id) if show and it.season_id else None
|
||||
return {
|
||||
"id": it.rating_key,
|
||||
"kind": it.kind,
|
||||
@@ -538,7 +439,7 @@ def item_detail(
|
||||
"video_bit_depth": it.video_bit_depth,
|
||||
"thumb": f"/api/plex/image/{it.rating_key}",
|
||||
"art": f"/api/plex/image/{it.rating_key}?variant=art",
|
||||
"library": _lib_key(db, it.library_id),
|
||||
"library": queries.library_key(db, it.library_id),
|
||||
"cast": rich.get("cast", []),
|
||||
"imdb_rating": rich.get("imdb_rating"),
|
||||
"imdb_id": rich.get("imdb_id"),
|
||||
|
||||
@@ -1,391 +1,18 @@
|
||||
"""Plex Discover: the rail-surface read-model.
|
||||
|
||||
A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
|
||||
Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
|
||||
separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
|
||||
recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
|
||||
every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
|
||||
their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
|
||||
card paths); the HU overlay coalesces only in the main grid.
|
||||
|
||||
The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
|
||||
Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
|
||||
grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
|
||||
the client hides). Genre-language matches the grid; duration is movie-only.
|
||||
"""
|
||||
"""Plex Discover: the rail-surface landing and its Surprise-me draw (`plex.discover`)."""
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import sorted_words
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import PlexItem, PlexShow, PlexState, User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import (
|
||||
SHOW_NEW,
|
||||
display_title,
|
||||
leaf_card,
|
||||
movie_card,
|
||||
show_card,
|
||||
)
|
||||
from app.plex.query import (
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
top_art_keys,
|
||||
)
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.plex.tmdb import resolve_recs
|
||||
from app.models import User
|
||||
from app.plex import discover as plex_discover
|
||||
from app.plex.discover import SURPRISE_PER_KIND
|
||||
from app.plex.query import LibraryFilters
|
||||
from app.routes.plex._shared import _norm_lang
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
GENRE_TILES = 12
|
||||
SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows (6 total)
|
||||
|
||||
|
||||
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
|
||||
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
return (
|
||||
movie_ids if scope in ("movie", "both") else [],
|
||||
show_ids if scope in ("show", "both") else [],
|
||||
)
|
||||
|
||||
|
||||
def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
|
||||
"""Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
|
||||
q = apply_meta_filters(q, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
q = q.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
q = q.where(PlexItem.duration_s <= f.duration_max)
|
||||
return q
|
||||
|
||||
|
||||
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
|
||||
sts = rollup.item_state_map(db, uid, [it.id for it in items])
|
||||
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
|
||||
|
||||
|
||||
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
|
||||
statuses = rollup.state_map(db, uid, [s.id for s in shows])
|
||||
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
|
||||
|
||||
|
||||
def _recently_added(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Newest movies + shows, interleaved by `added_at` desc into one rail."""
|
||||
movies = (
|
||||
db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
.order_by(PlexItem.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
|
||||
)
|
||||
.order_by(PlexShow.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
|
||||
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
|
||||
# review found this one still on the older `(is not None, value)` form, i.e. one rule in two
|
||||
# spellings, with the two comments explaining it differently.
|
||||
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
|
||||
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
|
||||
top = tagged[:RAIL_LIMIT]
|
||||
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
|
||||
# share one), so the merge doesn't depend on object identity staying alive.
|
||||
top_movies = [o for _, kind, o in top if kind == "movie"]
|
||||
top_shows = [o for _, kind, o in top if kind == "show"]
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
|
||||
}
|
||||
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
|
||||
|
||||
|
||||
def _recently_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
|
||||
card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
|
||||
sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
|
||||
rows: list[tuple[object, dict]] = [] # (watched_at, card)
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
)
|
||||
for it, st in db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all():
|
||||
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
|
||||
if show_ids:
|
||||
eq = (
|
||||
select(PlexItem, PlexState, PlexShow)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.join(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
for ep, st, sh in db.execute(eq).all():
|
||||
card = leaf_card(
|
||||
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
|
||||
)
|
||||
rows.append((st.watched_at, card))
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _continue_watching(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
|
||||
A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
|
||||
rows: list[tuple[object, dict]] = [] # (resumed_at, card)
|
||||
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.position_seconds > 0,
|
||||
# A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
|
||||
# `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
|
||||
or_(
|
||||
PlexState.status.is_(None),
|
||||
PlexState.status.notin_(["watched", "hidden"]),
|
||||
),
|
||||
)
|
||||
)
|
||||
mrows = db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.progress_updated_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for it, st in mrows:
|
||||
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
|
||||
|
||||
if show_ids:
|
||||
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
|
||||
# its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
|
||||
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
|
||||
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
|
||||
sq = apply_show_state_filter(
|
||||
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
|
||||
prog, prog.c.show_id == PlexShow.id
|
||||
),
|
||||
prog.c.total,
|
||||
prog.c.watched,
|
||||
prog.c.inprog,
|
||||
"in_progress",
|
||||
).where(PlexShow.library_id.in_(show_ids))
|
||||
srows = db.execute(
|
||||
apply_meta_filters(sq, PlexShow, p, lang)
|
||||
.order_by(prog.c.resumed_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for sh, resumed_at, total, watched, inprog in srows:
|
||||
state = rollup.show_state(total, watched, inprog)
|
||||
rows.append((resumed_at, show_card(sh, state, lang=lang)))
|
||||
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _genre_tiles(
|
||||
db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
|
||||
) -> list[dict]:
|
||||
"""The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
|
||||
The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
|
||||
The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
|
||||
counts: dict[str, int] = {}
|
||||
for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
|
||||
if not ids:
|
||||
continue
|
||||
gcol = genres_col(model, lang)
|
||||
base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
|
||||
model.library_id.in_(ids),
|
||||
func.jsonb_typeof(gcol) == "array",
|
||||
*meta_filter_conds(model, p, lang),
|
||||
)
|
||||
if model is PlexItem:
|
||||
base = base.where(model.kind == "movie")
|
||||
expanded = base.subquery()
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
counts[g] = counts.get(g, 0) + c
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
|
||||
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
|
||||
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
|
||||
art = top_art_keys(
|
||||
db,
|
||||
PlexItem,
|
||||
movie_ids,
|
||||
genres_col(PlexItem, lang),
|
||||
[g for g, _ in top],
|
||||
extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
|
||||
)
|
||||
return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
|
||||
|
||||
|
||||
def _because_you_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> dict | None:
|
||||
"""Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
|
||||
recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
|
||||
None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
|
||||
The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
|
||||
if not movie_ids:
|
||||
return None
|
||||
anchor = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
PlexItem.recommendations.isnot(None),
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(1)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if anchor is None:
|
||||
return None
|
||||
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
|
||||
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
|
||||
rks = [
|
||||
r["rating_key"]
|
||||
for r in resolved
|
||||
if r.get("in_library") and r.get("rating_key") != anchor.rating_key
|
||||
]
|
||||
if not rks:
|
||||
return None
|
||||
movies = {
|
||||
m.rating_key: m
|
||||
for m in db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
ordered_movies = [movies[rk] for rk in rks if rk in movies]
|
||||
ordered_shows = [shows[rk] for rk in rks if rk in shows]
|
||||
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
|
||||
}
|
||||
items: list[dict] = []
|
||||
for rk in rks:
|
||||
if rk in mcards:
|
||||
items.append(mcards[rk])
|
||||
elif rk in scards:
|
||||
items.append(scards[rk])
|
||||
if not items:
|
||||
return None
|
||||
return {
|
||||
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/discover")
|
||||
def discover(
|
||||
@@ -398,72 +25,7 @@ def discover(
|
||||
"""The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
|
||||
sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
|
||||
is null when there's no anchor."""
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
return {
|
||||
"scope": scope,
|
||||
"continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
|
||||
"because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
}
|
||||
|
||||
|
||||
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
|
||||
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
|
||||
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
|
||||
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
|
||||
fresh = (
|
||||
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
|
||||
.scalars()
|
||||
.all()
|
||||
if exclude
|
||||
else []
|
||||
)
|
||||
if len(fresh) == per:
|
||||
return list(fresh)
|
||||
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
|
||||
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
|
||||
return [*fresh, *rest.all()]
|
||||
|
||||
|
||||
def _kept(
|
||||
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
|
||||
) -> list[dict]:
|
||||
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
|
||||
stays in it, marked watched, and one gone from the scope's libraries drops out."""
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexItem.rating_key.in_(keep),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
cards = {
|
||||
c["id"]: c
|
||||
for c in [
|
||||
*_movie_cards(db, uid, list(movies), lang),
|
||||
*_show_cards(db, uid, list(shows), lang),
|
||||
]
|
||||
}
|
||||
return [cards[k] for k in keep if k in cards]
|
||||
return plex_discover.discover_page(db, user, scope, _norm_lang(lang), f)
|
||||
|
||||
|
||||
@router.get("/surprise")
|
||||
@@ -485,69 +47,4 @@ def surprise(
|
||||
excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
|
||||
SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
|
||||
one type."""
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
if keep:
|
||||
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
|
||||
per = min(limit, SURPRISE_PER_KIND)
|
||||
movies = []
|
||||
if movie_ids:
|
||||
# Exclude movies this user has watched.
|
||||
watched_movie = (
|
||||
select(PlexState.id)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
movies = _draw(
|
||||
db,
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
~watched_movie,
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
),
|
||||
PlexItem.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
shows = []
|
||||
if show_ids:
|
||||
# Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
|
||||
# as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
|
||||
prog = rollup.agg_subquery(db, user.id).subquery()
|
||||
not_fully_watched = or_(
|
||||
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
|
||||
)
|
||||
shows = _draw(
|
||||
db,
|
||||
apply_meta_filters(
|
||||
select(PlexShow)
|
||||
.outerjoin(prog, prog.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
|
||||
PlexShow,
|
||||
p,
|
||||
lang,
|
||||
),
|
||||
PlexShow.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
mcards = _movie_cards(db, user.id, movies, lang)
|
||||
scards = _show_cards(db, user.id, shows, lang)
|
||||
# Interleave movie/show cards, then trim to `limit`.
|
||||
items: list[dict] = []
|
||||
for i in range(max(len(mcards), len(scards))):
|
||||
if i < len(mcards):
|
||||
items.append(mcards[i])
|
||||
if i < len(scards):
|
||||
items.append(scards[i])
|
||||
return {"scope": scope, "items": items[:limit]}
|
||||
return plex_discover.surprise(db, user, scope, _norm_lang(lang), limit, exclude, keep, f)
|
||||
|
||||
@@ -4,26 +4,18 @@ import threading
|
||||
import time
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated
|
||||
from app import logs, sysconfig
|
||||
from app import sysconfig
|
||||
from app.auth import current_user
|
||||
from app.db import SessionLocal, get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexPerson,
|
||||
PlexShow,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.db import get_db
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.cards import SHOW_NEW, movie_card, norm_lang, show_card
|
||||
from app.plex.query import content_libs, split_csv, title_col
|
||||
from app.plex.query import content_libs, split_csv
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
# How many "Known for" titles the person page shows (the stored pool is deeper; see plex.tmdb).
|
||||
@@ -80,35 +72,13 @@ def people_cards(
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
photos = {
|
||||
p.name: p.photo_thumb
|
||||
for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(wanted))).scalars()
|
||||
}
|
||||
out = []
|
||||
for nm in wanted:
|
||||
count = 0
|
||||
if want_movies:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
PlexItem.cast_names.contains([nm]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
if want_shows:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexShow.id)).where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
PlexShow.cast_names.contains([nm]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
out.append({"name": nm, "photo": photos.get(nm), "count": count})
|
||||
photos = queries.person_photos(db, wanted)
|
||||
movies = movie_lib_ids if want_movies else []
|
||||
shows = show_lib_ids if want_shows else []
|
||||
out = [
|
||||
{"name": nm, "photo": photos.get(nm), "count": queries.cast_count(db, nm, movies, shows)}
|
||||
for nm in wanted
|
||||
]
|
||||
return {"people": out}
|
||||
|
||||
|
||||
@@ -144,24 +114,6 @@ def _should_schedule_enrich(person_id: int) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _enrich_person_bg(person_id: int) -> None:
|
||||
"""Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
|
||||
AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
|
||||
"no live TMDB call at serve time" invariant). The request's session is already closed by then, so
|
||||
open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
|
||||
next open, and the bio lands from the row by then."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
person = db.get(PlexPerson, person_id)
|
||||
if person is not None:
|
||||
plex_tmdb.enrich_person(db, person)
|
||||
except Exception: # noqa: BLE001 — background best-effort; never surfaces to a user
|
||||
db.rollback()
|
||||
log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@router.get("/person/{name}")
|
||||
def person_detail(
|
||||
name: str,
|
||||
@@ -183,7 +135,7 @@ def person_detail(
|
||||
fetches everyone ahead of time, so this lazy path is only for a person the job hasn't reached yet.
|
||||
"""
|
||||
lang = norm_lang(lang)
|
||||
person = db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
|
||||
person = queries.person(db, name)
|
||||
# Lazy one-time bio enrich, scheduled AFTER the response so the TMDB call is never on the serve path.
|
||||
# Gated by a per-person cooldown so concurrent/rapid opens (or repeated opens while TMDB is degraded)
|
||||
# can't pile background tasks onto the shared threadpool.
|
||||
@@ -197,53 +149,18 @@ def person_detail(
|
||||
and bool(sysconfig.get_str(db, "tmdb_api_key"))
|
||||
)
|
||||
if enriching and person is not None and _should_schedule_enrich(person.id):
|
||||
background_tasks.add_task(_enrich_person_bg, person.id)
|
||||
background_tasks.add_task(service.enrich_person_bg, person.id)
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
|
||||
movies: list[dict] = []
|
||||
if movie_lib_ids:
|
||||
items = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
or_(
|
||||
PlexItem.cast_names.contains([name]),
|
||||
PlexItem.directors.contains([name]),
|
||||
),
|
||||
)
|
||||
.order_by(
|
||||
PlexItem.year.desc().nullslast(), collated(title_col(PlexItem, lang), lang)
|
||||
)
|
||||
.limit(_PERSON_SECTION_MAX)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
items = queries.filmography(db, "movie", name, movie_lib_ids, lang, _PERSON_SECTION_MAX)
|
||||
states = rollup.item_state_map(db, user.id, [it.id for it in items])
|
||||
movies = [movie_card(it, states.get(it.id), lang=lang) for it in items]
|
||||
|
||||
shows: list[dict] = []
|
||||
if show_lib_ids:
|
||||
show_rows = (
|
||||
db.execute(
|
||||
select(PlexShow)
|
||||
.where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
or_(
|
||||
PlexShow.cast_names.contains([name]),
|
||||
PlexShow.directors.contains([name]),
|
||||
),
|
||||
)
|
||||
.order_by(
|
||||
PlexShow.year.desc().nullslast(), collated(title_col(PlexShow, lang), lang)
|
||||
)
|
||||
.limit(_PERSON_SECTION_MAX)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
show_rows = queries.filmography(db, "show", name, show_lib_ids, lang, _PERSON_SECTION_MAX)
|
||||
statuses = rollup.state_map(db, user.id, [s.id for s in show_rows])
|
||||
shows = [show_card(sh, statuses.get(sh.id, SHOW_NEW), lang=lang) for sh in show_rows]
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Plex playback: image/subtitle proxies, HLS streaming, and watch-state writes."""
|
||||
|
||||
import threading
|
||||
from typing import TYPE_CHECKING
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import httpx
|
||||
@@ -16,22 +17,13 @@ from fastapi import (
|
||||
)
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user, resolved_user_id
|
||||
from app.db import SessionLocal, get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import imgcache, subtitles
|
||||
from app.models import User
|
||||
from app.plex import imgcache, queries, service, subtitles
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import stream as plex_stream
|
||||
from app.plex import watch_sync as plex_watch
|
||||
@@ -40,6 +32,9 @@ from app.routes.plex._shared import item_or_404
|
||||
from app.refusal import refused
|
||||
from app.routes._common import ProgressIn, not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
@@ -280,14 +275,7 @@ def _push_watch(
|
||||
return
|
||||
# Capture the row's freshest timestamp NOW so the background push can tell if a newer local edit
|
||||
# lands before it settles synced_to_plex (which would otherwise strand that newer value).
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
marker = plex_watch._state_marker(st)
|
||||
marker = plex_watch._state_marker(queries.state(db, user_id, item_id))
|
||||
if action == "resume" and intent is not None:
|
||||
background.add_task(
|
||||
_push_resume_guarded, user_id, item_id, rating_key, time_ms, duration_ms, marker, intent
|
||||
@@ -305,37 +293,6 @@ def _push_watch(
|
||||
)
|
||||
|
||||
|
||||
def _mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
|
||||
"""Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
|
||||
checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
|
||||
read-then-write let both see "not watched" and both scrobble. The transition is decided by the
|
||||
write itself: an insert that did not conflict, or an update conditional on the old status —
|
||||
under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
|
||||
values = {
|
||||
"status": "watched",
|
||||
"watched_at": now,
|
||||
"position_seconds": 0,
|
||||
"progress_updated_at": now,
|
||||
"synced_to_plex": False,
|
||||
}
|
||||
inserted = db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(user_id=user_id, item_id=item_id, **values)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
.returning(PlexState.id)
|
||||
).first()
|
||||
if inserted is not None:
|
||||
return True
|
||||
mine = (PlexState.user_id == user_id) & (PlexState.item_id == item_id)
|
||||
if db.execute(
|
||||
update(PlexState).where(mine, PlexState.status != "watched").values(values)
|
||||
).rowcount:
|
||||
return True
|
||||
# Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
|
||||
db.execute(update(PlexState).where(mine).values(values))
|
||||
return False
|
||||
|
||||
|
||||
class ItemProgressIn(ProgressIn):
|
||||
"""The shared checkpoint (its bounds included), plus whether the player is leaving the item."""
|
||||
|
||||
@@ -371,7 +328,7 @@ def item_progress(
|
||||
now = datetime.now(timezone.utc)
|
||||
near_end = duration > 0 and position > duration - _FINISH_MARGIN_S
|
||||
if near_end:
|
||||
flipped = _mark_finished(db, user.id, it.id, now)
|
||||
flipped = service.mark_finished(db, user.id, it.id, now)
|
||||
# The finish supersedes every resume push still in flight for this item: stamping it as the
|
||||
# latest push makes `_resume_push_superseded` drop them. Popping the entry instead (to keep
|
||||
# the map small) removed the very guard, and an older resume push could then land at Plex
|
||||
@@ -385,33 +342,17 @@ def item_progress(
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "watched")
|
||||
return {"ok": True, "status": "watched", "position_seconds": 0}
|
||||
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
|
||||
if position < _PROGRESS_MIN_S:
|
||||
if st is not None and st.status == "new":
|
||||
db.delete(st)
|
||||
elif st is not None:
|
||||
st.position_seconds = 0
|
||||
st.progress_updated_at = now
|
||||
service.clear_progress(db, st, now)
|
||||
db.commit()
|
||||
return {"ok": True, "position_seconds": 0}
|
||||
|
||||
if st is None:
|
||||
st = PlexState(user_id=user.id, item_id=it.id)
|
||||
db.add(st)
|
||||
st.position_seconds = position
|
||||
st.progress_updated_at = now
|
||||
push_now = _should_push_resume(user.id, it.id, now, final)
|
||||
if push_now:
|
||||
# Mark dirty before the (background) push so the reconcile knows to re-push if it fails, and so
|
||||
# `push_state_to_plex` can settle synced_to_plex once Plex confirms.
|
||||
st.synced_to_plex = False
|
||||
# Marked dirty before the (background) push so the reconcile knows to re-push if it fails, and so
|
||||
# `push_state_to_plex` can settle synced_to_plex once Plex confirms.
|
||||
service.checkpoint(db, user, it.id, st, position, now, dirty=push_now)
|
||||
db.commit()
|
||||
if push_now:
|
||||
_push_watch(
|
||||
@@ -447,13 +388,7 @@ def item_state(
|
||||
status = body.status
|
||||
if status not in ("new", "watched", "hidden"):
|
||||
raise refused(400, "invalid_status")
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
prev_status = st.status if st is not None else "new"
|
||||
if status == "new":
|
||||
# Does Plex hold anything for this item that the un-mark has to clear? Two things qualify: a
|
||||
@@ -467,19 +402,12 @@ def item_state(
|
||||
st.status == "watched" or (st.position_seconds or 0) > 0
|
||||
)
|
||||
if st is not None:
|
||||
db.delete(st)
|
||||
service.forget_state(db, st)
|
||||
db.commit()
|
||||
if mirrored_on_plex:
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "unwatched")
|
||||
return {"ok": True, "status": "new"}
|
||||
if st is None:
|
||||
st = PlexState(user_id=user.id, item_id=it.id)
|
||||
db.add(st)
|
||||
st.status = status
|
||||
if status == "watched":
|
||||
st.watched_at = datetime.now(timezone.utc)
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
service.set_status(db, user, it.id, st, status)
|
||||
db.commit()
|
||||
if status == "watched" and prev_status != "watched":
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "watched")
|
||||
@@ -490,44 +418,15 @@ def _bulk_state(
|
||||
background: BackgroundTasks,
|
||||
db: Session,
|
||||
user_id: int,
|
||||
eps: list[PlexItem],
|
||||
eps: "list[PlexItem]",
|
||||
watched: bool,
|
||||
) -> int:
|
||||
"""Mark every episode in `eps` watched (or unwatched) for a user, mirroring each change to a
|
||||
linked Plex account in the background (best-effort). Returns how many rows actually changed."""
|
||||
now = datetime.now(timezone.utc)
|
||||
existing = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user_id,
|
||||
PlexState.item_id.in_([e.id for e in eps] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
linked Plex account in the background (best-effort). Returns how many rows actually changed.
|
||||
The handler commits before the response — which is when the background push runs, so the push
|
||||
always reads the committed marks."""
|
||||
pushable = plex_watch.link_for_push(db, user_id) is not None
|
||||
to_push: list[tuple[int, str, str]] = []
|
||||
for e in eps:
|
||||
st = existing.get(e.id)
|
||||
prev = st.status if st is not None else "new"
|
||||
if watched:
|
||||
if st is None:
|
||||
st = PlexState(user_id=user_id, item_id=e.id)
|
||||
db.add(st)
|
||||
st.status = "watched"
|
||||
st.watched_at = now
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
if prev != "watched":
|
||||
to_push.append((e.id, e.rating_key, "watched"))
|
||||
else: # → new: drop any state (but keep the user's private "hidden" marks)
|
||||
if st is not None and prev != "hidden":
|
||||
db.delete(st)
|
||||
if prev != "new":
|
||||
to_push.append((e.id, e.rating_key, "unwatched"))
|
||||
# Flush, not commit: the handler commits, and it does so before the response — which is when
|
||||
# the background push below runs, so the push always reads the committed marks.
|
||||
db.flush()
|
||||
to_push = service.bulk_state(db, user_id, eps, watched)
|
||||
if pushable and to_push:
|
||||
# One coalesced background task for the whole batch (reuses a single session + Plex client),
|
||||
# not one task per episode — a 200-ep show mark is 1 task, not 200.
|
||||
@@ -548,15 +447,11 @@ def show_state(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Mark a whole show watched/unwatched (all its episodes) for the current user + push to Plex."""
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
watched = body.watched
|
||||
eps = (
|
||||
db.execute(select(PlexItem).where(PlexItem.show_id == sh.id, PlexItem.kind == "episode"))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = queries.episodes(db, sh)
|
||||
changed = _bulk_state(background, db, user.id, eps, watched)
|
||||
db.commit()
|
||||
return {"ok": True, "changed": changed, "watched": watched}
|
||||
@@ -571,15 +466,11 @@ def season_state(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Mark a whole season watched/unwatched (all its episodes) for the current user + push to Plex."""
|
||||
se = by_rating_key(db, PlexSeason, rating_key)
|
||||
se = queries.season(db, rating_key)
|
||||
if se is None:
|
||||
raise not_found("season")
|
||||
watched = body.watched
|
||||
eps = (
|
||||
db.execute(select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode"))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = queries.season_episodes(db, se)
|
||||
changed = _bulk_state(background, db, user.id, eps, watched)
|
||||
db.commit()
|
||||
return {"ok": True, "changed": changed, "watched": watched}
|
||||
|
||||
@@ -1,33 +1,26 @@
|
||||
"""Plex playlists: the current user's personal, orderable playlists of playable items."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING, Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex.cards import display_title, leaf_card, norm_lang
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes._common import owned_or_404, not_found
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexPlaylist
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
|
||||
def _playlist_card(pl: "PlexPlaylist", count: int, thumb_rk: str | None) -> dict:
|
||||
return {
|
||||
"id": pl.id,
|
||||
"title": pl.title,
|
||||
@@ -36,13 +29,15 @@ def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _playlist_item_query(pid: int):
|
||||
return (
|
||||
select(PlexPlaylistItem, PlexItem)
|
||||
.join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
|
||||
.where(PlexPlaylistItem.playlist_id == pid)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
)
|
||||
def _card_of(db: Session, pl: "PlexPlaylist") -> dict:
|
||||
return _playlist_card(pl, queries.playlist_count(db, pl.id), queries.playlist_cover(db, pl.id))
|
||||
|
||||
|
||||
def _own_playlist(db: Session, user: User, pid: int, *, lock: bool = False) -> "PlexPlaylist":
|
||||
pl = queries.own_playlist(db, user, pid, lock=lock)
|
||||
if pl is None:
|
||||
raise not_found("playlist")
|
||||
return pl
|
||||
|
||||
|
||||
@router.get("/playlists")
|
||||
@@ -56,49 +51,17 @@ def playlists(
|
||||
`contains`=an item rating_key adds `has_item` per playlist (for the single 'add to playlist' dialog).
|
||||
`contains_group`=a comma-separated list of rating_keys adds `group_in` per playlist (how many of the
|
||||
group it already holds) + a top-level `group_size`, for the 'add a whole season/show' dialog."""
|
||||
contains_id = None
|
||||
if contains:
|
||||
contains_id = db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(contains)))
|
||||
contains_id = queries.item_id(db, contains) if contains else None
|
||||
group_ids: set[int] | None = None
|
||||
if contains_group is not None:
|
||||
rks = [s for s in (contains_group or "").split(",") if s]
|
||||
group_ids = set(
|
||||
db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rks or [""]))).scalars()
|
||||
)
|
||||
group_ids = queries.item_ids(db, [s for s in (contains_group or "").split(",") if s])
|
||||
out = []
|
||||
for pl in db.execute(
|
||||
select(PlexPlaylist)
|
||||
.where(PlexPlaylist.user_id == user.id)
|
||||
.order_by(PlexPlaylist.updated_at.desc())
|
||||
).scalars():
|
||||
count = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
first = db.execute(_playlist_item_query(pl.id)).first()
|
||||
card = _playlist_card(pl, count, first[1].rating_key if first else None)
|
||||
for pl in queries.playlists_of(db, user):
|
||||
card = _card_of(db, pl)
|
||||
if contains_id is not None:
|
||||
card["has_item"] = (
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.id).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id,
|
||||
PlexPlaylistItem.item_id == contains_id,
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
card["has_item"] = queries.playlist_holds(db, pl.id, {contains_id}) > 0
|
||||
if group_ids is not None:
|
||||
card["group_in"] = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id,
|
||||
PlexPlaylistItem.item_id.in_(group_ids or [0]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
card["group_in"] = queries.playlist_holds(db, pl.id, group_ids)
|
||||
out.append(card)
|
||||
resp: dict = {"playlists": out}
|
||||
if group_ids is not None:
|
||||
@@ -120,19 +83,10 @@ def create_playlist(
|
||||
title = body.title.strip()
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
pl = PlexPlaylist(user_id=user.id, title=title)
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
seed = body.item_rating_key
|
||||
count = 0
|
||||
thumb_rk = None
|
||||
if seed:
|
||||
it = by_rating_key(db, PlexItem, seed)
|
||||
if it is not None:
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0))
|
||||
count, thumb_rk = 1, it.rating_key
|
||||
seed = queries.item(db, body.item_rating_key) if body.item_rating_key else None
|
||||
pl = service.create_playlist(db, user, title, seed)
|
||||
db.commit()
|
||||
return _playlist_card(pl, count, thumb_rk)
|
||||
return _playlist_card(pl, 1 if seed else 0, seed.rating_key if seed else None)
|
||||
|
||||
|
||||
@router.get("/playlists/{pid}")
|
||||
@@ -143,26 +97,10 @@ def playlist_detail(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
lang = norm_lang(lang)
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
rows = db.execute(_playlist_item_query(pid)).all()
|
||||
items = [it for (_pi, it) in rows]
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.item_id.in_([it.id for it in items] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
shows = {
|
||||
sh.id: sh
|
||||
for sh in db.execute(
|
||||
select(PlexShow).where(
|
||||
PlexShow.id.in_([it.show_id for it in items if it.show_id] or [0])
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
pl = _own_playlist(db, user, pid)
|
||||
items = queries.playlist_items(db, pl.id)
|
||||
states = rollup.item_state_map(db, user, [it.id for it in items])
|
||||
shows = queries.shows_by_ids(db, {it.show_id for it in items if it.show_id})
|
||||
cards = [
|
||||
leaf_card(
|
||||
it,
|
||||
@@ -187,28 +125,21 @@ def rename_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
pl = _own_playlist(db, user, pid)
|
||||
title = body.title.strip()
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
pl.title = title
|
||||
service.rename_playlist(pl, title)
|
||||
db.commit()
|
||||
count = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
first = db.execute(_playlist_item_query(pl.id)).first()
|
||||
return _playlist_card(pl, count, first[1].rating_key if first else None)
|
||||
return _playlist_card(pl, queries.playlist_count(db, pl.id), queries.playlist_cover(db, pl.id))
|
||||
|
||||
|
||||
@router.delete("/playlists/{pid}")
|
||||
def delete_playlist(
|
||||
pid: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
db.delete(pl)
|
||||
pl = _own_playlist(db, user, pid)
|
||||
service.delete_playlist(db, pl)
|
||||
db.commit()
|
||||
return {"deleted": pid}
|
||||
|
||||
@@ -225,27 +156,11 @@ def playlist_add_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked (like every add / reorder here): two adds otherwise read the same max position.
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
it = by_rating_key(db, PlexItem, body.item_rating_key)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
it = queries.item(db, body.item_rating_key)
|
||||
if it is None:
|
||||
raise not_found("item")
|
||||
exists = (
|
||||
db.execute(
|
||||
select(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if exists is None:
|
||||
maxpos = db.scalar(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == pid
|
||||
)
|
||||
)
|
||||
db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=int(maxpos) + 1))
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
service.add_item(db, pl, it)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -259,15 +174,10 @@ def playlist_remove_item(
|
||||
) -> dict:
|
||||
# Locked like the adds and the reorder: the list row first, then its items — a remove that
|
||||
# locked an item row before a reorder locked the list deadlocked against it.
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
it = by_rating_key(db, PlexItem, item_rating_key)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
it = queries.item(db, item_rating_key)
|
||||
if it is not None:
|
||||
db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
service.remove_item(db, pl, it)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -276,25 +186,6 @@ class PlexItemKeysIn(BaseModel):
|
||||
item_rating_keys: list[str] = []
|
||||
|
||||
|
||||
def _items_by_rks(db: Session, rks: list) -> list[PlexItem]:
|
||||
"""Resolve rating_keys to PlexItems, preserving the input order (drops unknown keys)."""
|
||||
order = [str(k) for k in rks if str(k)]
|
||||
if not order:
|
||||
return []
|
||||
by_rk = {
|
||||
it.rating_key: it
|
||||
for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
|
||||
}
|
||||
seen: set[str] = set()
|
||||
out: list[PlexItem] = []
|
||||
for rk in order:
|
||||
it = by_rk.get(rk)
|
||||
if it is not None and rk not in seen:
|
||||
seen.add(rk)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
|
||||
class PlexItemAddBulkIn(PlexItemKeysIn):
|
||||
# Where each key goes back (the list page's "Undo" of a removal): an index over the
|
||||
# ORDERED rows, one per key. The two-call shape (append, then send the whole order) leaves the rows
|
||||
@@ -309,32 +200,6 @@ class PlexItemAddBulkIn(PlexItemKeysIn):
|
||||
return self
|
||||
|
||||
|
||||
def _insert_at(db: Session, pid: int, placed: list[tuple[int, PlexItem]]) -> int:
|
||||
"""Insert new rows at their indices (ascending, so each index is where the row stood when the
|
||||
rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
|
||||
already in the list does not move: the add stays idempotent."""
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem)
|
||||
.where(PlexPlaylistItem.playlist_id == pid)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
present = {r.item_id for r in rows}
|
||||
added = 0
|
||||
for at, it in sorted(placed, key=lambda p: p[0]):
|
||||
if it.id in present:
|
||||
continue
|
||||
row = PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=0)
|
||||
db.add(row)
|
||||
rows.insert(min(at, len(rows)), row)
|
||||
present.add(it.id)
|
||||
added += 1
|
||||
for i, r in enumerate(rows):
|
||||
r.position = i
|
||||
return added
|
||||
|
||||
|
||||
@router.post("/playlists/{pid}/items/bulk")
|
||||
def playlist_add_bulk(
|
||||
pid: int,
|
||||
@@ -345,56 +210,20 @@ def playlist_add_bulk(
|
||||
"""Append many items at once (a whole season/show). Already-present items are skipped; the input
|
||||
order is preserved for the newly-added ones. With `positions`, each key is put back at its index
|
||||
instead of appended. Returns how many were added + the new total."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
items = queries.items_by_keys(db, body.item_rating_keys)
|
||||
if body.positions is not None:
|
||||
by_rk = {it.rating_key: it for it in _items_by_rks(db, body.item_rating_keys)}
|
||||
by_rk = {it.rating_key: it for it in items}
|
||||
placed = [
|
||||
(at, by_rk[str(rk)])
|
||||
for rk, at in zip(body.item_rating_keys, body.positions, strict=True)
|
||||
if str(rk) in by_rk
|
||||
]
|
||||
added = _insert_at(db, pid, placed)
|
||||
if added:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "added": added, "item_count": total}
|
||||
items = _items_by_rks(db, body.item_rating_keys)
|
||||
existing = set(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pid)
|
||||
).scalars()
|
||||
)
|
||||
pos = int(
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == pid
|
||||
)
|
||||
)
|
||||
)
|
||||
added = 0
|
||||
for it in items:
|
||||
if it.id in existing:
|
||||
continue
|
||||
pos += 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=pos))
|
||||
existing.add(it.id)
|
||||
added += 1
|
||||
if added:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
added = service.insert_items_at(db, pl, placed)
|
||||
else:
|
||||
added = service.add_items(db, pl, items)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "added": added, "item_count": total}
|
||||
return {"ok": True, "added": added, "item_count": queries.playlist_count(db, pid)}
|
||||
|
||||
|
||||
@router.post("/playlists/{pid}/items/remove-bulk")
|
||||
@@ -406,26 +235,11 @@ def playlist_remove_bulk(
|
||||
) -> dict:
|
||||
"""Remove many items at once (a whole season/show, or a multi-select). Returns how many were
|
||||
removed + the new total."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
ids = [it.id for it in _items_by_rks(db, body.item_rating_keys)]
|
||||
removed = 0
|
||||
if ids:
|
||||
removed = db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id.in_(ids)
|
||||
),
|
||||
execution_options={"synchronize_session": False},
|
||||
).rowcount
|
||||
if removed:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
ids = [it.id for it in queries.items_by_keys(db, body.item_rating_keys)]
|
||||
removed = service.remove_items(db, pl, ids)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "removed": removed, "item_count": total}
|
||||
return {"ok": True, "removed": removed, "item_count": queries.playlist_count(db, pid)}
|
||||
|
||||
|
||||
@router.put("/playlists/{pid}/order")
|
||||
@@ -436,35 +250,7 @@ def reorder_playlist(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Set the playlist order from an explicit list of item rating_keys (0-based positions)."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
order = body.item_rating_keys
|
||||
id_by_rk = {
|
||||
it.rating_key: it.id
|
||||
for it in db.execute(
|
||||
select(PlexItem).where(PlexItem.rating_key.in_(order or [""]))
|
||||
).scalars()
|
||||
}
|
||||
rows = {
|
||||
r.item_id: r
|
||||
for r in db.execute(
|
||||
select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pid)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[int] = set()
|
||||
for rk in order:
|
||||
row = rows.get(id_by_rk.get(rk, -1))
|
||||
if row is not None and row.item_id not in seen:
|
||||
row.position = pos
|
||||
pos += 1
|
||||
seen.add(row.item_id)
|
||||
# Rows the payload omitted (added in another tab meanwhile) keep their relative order after
|
||||
# the given ones, so no two rows share a position (the YouTube reorder does the same).
|
||||
for row in sorted(
|
||||
(r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
|
||||
):
|
||||
row.position = pos
|
||||
pos += 1
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
service.reorder_playlist(db, pl, body.item_rating_keys)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -10,6 +10,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import quota
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
@@ -27,7 +28,7 @@ def _sum(db: Session, *conds) -> int:
|
||||
@router.get("/my-usage")
|
||||
def my_usage(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
now = datetime.now(timezone.utc)
|
||||
mine = QuotaEvent.user_id == user.id
|
||||
mine = owned_by(QuotaEvent.user_id, user)
|
||||
by_action = {
|
||||
action: int(units)
|
||||
for action, units in db.execute(
|
||||
|
||||
@@ -18,7 +18,8 @@ from sqlalchemy.orm import Session
|
||||
from app import scheduler
|
||||
from app.auth import admin_user
|
||||
from app.db import get_db
|
||||
from app.models import Channel, JobRun, LogEvent, User
|
||||
from app.models import JobRun, LogEvent, User
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.routes import _account as account
|
||||
from app.routes._keyset import PAGE_MAX, accounts, older_than, page, utc
|
||||
from app.runstatus import RunStatus
|
||||
@@ -116,10 +117,7 @@ def list_runs(
|
||||
lambda r: r.started_at,
|
||||
)
|
||||
channels = {r.target for r in runs if r.job in scheduler.AD_HOC_JOBS and r.target}
|
||||
titles: dict[str, str] = {}
|
||||
if channels:
|
||||
rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(channels)))
|
||||
titles = {cid: title for cid, title in rows.tuples() if title is not None}
|
||||
titles = catalog_queries.channel_titles(db, channels)
|
||||
lines: dict[int, int] = {}
|
||||
if runs:
|
||||
counted = db.execute(
|
||||
|
||||
@@ -4,7 +4,6 @@ and the shared quota picture."""
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, quota, state, sysconfig
|
||||
@@ -12,7 +11,8 @@ from app.audit import AuditAction
|
||||
from app.auth import admin_user
|
||||
from app.config import settings
|
||||
from app.db import get_db
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.scheduler import (
|
||||
JOB_INTERVALS,
|
||||
MAX_INTERVAL,
|
||||
@@ -23,7 +23,6 @@ from app.scheduler import (
|
||||
trigger_all,
|
||||
trigger_job,
|
||||
)
|
||||
from app.sync.explore import followed_channel
|
||||
from app.sync.runner import estimate_deep_backfill
|
||||
from app.routes._common import not_found
|
||||
from app.refusal import refused
|
||||
@@ -122,58 +121,14 @@ def get_scheduler(_: User = Depends(admin_user), db: Session = Depends(get_db))
|
||||
snap = scheduler_snapshot(db)
|
||||
|
||||
# Work still queued (shared catalog, so these are instance-wide, not per-user).
|
||||
deep_requested = (
|
||||
select(Subscription.id)
|
||||
.where(
|
||||
Subscription.channel_id == Channel.id,
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
deep_channels = (
|
||||
db.execute(select(Channel).where(Channel.backfill_done.is_(False), deep_requested))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
deep_eta = estimate_deep_backfill(db, deep_channels)
|
||||
|
||||
queue = {
|
||||
"channels_recent_pending": db.scalar(
|
||||
# The same selection run_recent_backfill works through, or the card counts channels
|
||||
# the job will never touch and never reaches zero.
|
||||
select(func.count())
|
||||
.select_from(Channel)
|
||||
.where(Channel.recent_synced_at.is_(None), followed_channel())
|
||||
)
|
||||
or 0,
|
||||
# The whole opted-in BACKLOG, deliberately not `run_deep_backfill`'s next-pass selection
|
||||
# (which also wants `recent_synced_at`): the same figure /api/sync/my-status shows the user,
|
||||
# and a channel missing its recent pull is one the same job tick fetches first, not one the
|
||||
# job will never touch. Narrowing it here would under-report work that is genuinely queued.
|
||||
"channels_deep_pending": deep_eta["channels_pending"],
|
||||
"deep_eta_seconds": deep_eta["eta_seconds"],
|
||||
"videos_pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_shorts": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_live_refresh": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(
|
||||
or_(
|
||||
Video.live_status.in_(("live", "upcoming")),
|
||||
and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
|
||||
)
|
||||
)
|
||||
)
|
||||
or 0,
|
||||
}
|
||||
deep_eta = estimate_deep_backfill(db, catalog_queries.deep_backlog(db))
|
||||
queue = catalog_queries.work_queue(db)
|
||||
# The whole opted-in BACKLOG, deliberately not `run_deep_backfill`'s next-pass selection
|
||||
# (which also wants `recent_synced_at`): the same figure /api/sync/my-status shows the user,
|
||||
# and a channel missing its recent pull is one the same job tick fetches first, not one the
|
||||
# job will never touch. Narrowing it here would under-report work that is genuinely queued.
|
||||
queue["channels_deep_pending"] = deep_eta["channels_pending"]
|
||||
queue["deep_eta_seconds"] = deep_eta["eta_seconds"]
|
||||
|
||||
used = quota.units_used_today(db)
|
||||
quota_info = {
|
||||
|
||||
+14
-243
@@ -11,37 +11,24 @@ regardless of source.
|
||||
Ingested results are flagged (videos.via_search / channels.from_search) so the Library can hide
|
||||
this search-discovered "noise" by default. Shorts and currently-live/upcoming videos are never
|
||||
ingested this way (Shorts are confirmed by the youtube.com/shorts probe; live/upcoming are
|
||||
dropped from the search response up front)."""
|
||||
dropped from the search response up front).
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
This is the HTTP side — the paging, the caps and the refusals; what a page of results leaves in the
|
||||
catalog is `catalog/search`."""
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, delete, func, or_, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, quota, sysconfig
|
||||
from app.auth import current_user
|
||||
from app.catalog import queries
|
||||
from app.catalog import search as catalog_search
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
User,
|
||||
Video,
|
||||
)
|
||||
from app.reader.models import Playlist, PlaylistItem, VideoState
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.utils import like_escape
|
||||
from app.sync.subscriptions import apply_channel_details
|
||||
from app.sync.videos import insert_stubs, apply_video_details, parse_dt
|
||||
from app.models import User
|
||||
from app.reader import queries as reader_queries
|
||||
from app.youtube import search_scrape
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
from app.refusal import refused
|
||||
from app.routes._common import youtube_refusal
|
||||
|
||||
@@ -52,198 +39,6 @@ router = APIRouter(prefix="/api/search", tags=["search"])
|
||||
SEARCH_COST = 100
|
||||
|
||||
|
||||
def _classify_shorts(db: Session, videos: list[Video]) -> None:
|
||||
"""Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
|
||||
|
||||
Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
|
||||
synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
|
||||
duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
|
||||
quota). Undetermined probes are left for the scheduler to retry."""
|
||||
probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
|
||||
to_probe: list[Video] = []
|
||||
for v in videos:
|
||||
if v.shorts_probed:
|
||||
continue
|
||||
if v.enriched_at is None:
|
||||
# Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
|
||||
# so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
|
||||
# (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
|
||||
continue
|
||||
if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
|
||||
v.is_short = False
|
||||
v.shorts_probed = True
|
||||
else:
|
||||
to_probe.append(v)
|
||||
if to_probe:
|
||||
with make_client() as client:
|
||||
|
||||
def work(v: Video):
|
||||
return v, probe_is_short(client, v.id)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
for v, result in pool.map(work, to_probe):
|
||||
if result is None:
|
||||
continue
|
||||
v.is_short = result
|
||||
v.shorts_probed = True
|
||||
db.commit()
|
||||
|
||||
|
||||
def _serialize_ids(db: Session, user: User, ids: list[str]) -> list[dict]:
|
||||
"""Serialize the given video ids with the shared feed projection (joined channel + this
|
||||
user's watch state + saved), preserving the input (search relevance) order."""
|
||||
if not ids:
|
||||
return []
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
.where(Video.id.in_(ids))
|
||||
).all()
|
||||
by_id = {r.id: serialize(r) for r in rows}
|
||||
return [by_id[i] for i in ids if i in by_id]
|
||||
|
||||
|
||||
def _ingest_candidates(
|
||||
db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
|
||||
) -> list[str]:
|
||||
"""Materialise one page of search candidates into the catalog (channel stubs + enrich,
|
||||
video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
|
||||
video ids — non-Short, non-live — in the input relevance order."""
|
||||
ids = [it["id"] for it in candidates]
|
||||
if not ids:
|
||||
return []
|
||||
|
||||
# 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
|
||||
# then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
|
||||
channel_ids = list({it["channel_id"] for it in candidates})
|
||||
existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
|
||||
new_channel_ids = [c for c in channel_ids if c not in existing]
|
||||
if new_channel_ids:
|
||||
titles = {it["channel_id"]: it["channel_title"] for it in candidates}
|
||||
for cid in new_channel_ids:
|
||||
db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
|
||||
db.commit()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
|
||||
details = yt.get_channels(new_channel_ids)
|
||||
apply_channel_details(db, details, new_channel_ids)
|
||||
db.commit()
|
||||
except YouTubeError:
|
||||
# The stubs remain and the details can be backfilled later — but until they are, those
|
||||
# channels sit in the results with no thumbnail and no subscriber count, looking like
|
||||
# channels that have neither.
|
||||
log.warning(
|
||||
"the channels found by a search could not be enriched",
|
||||
extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
|
||||
)
|
||||
db.rollback() # details can be backfilled later; the stubs remain
|
||||
|
||||
# 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
|
||||
# NOTHING), so an organically-synced video stays organic.
|
||||
# published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
|
||||
# (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
|
||||
# enrich in step 3 overwrites it with the video's real snippet date.
|
||||
now = datetime.now(timezone.utc)
|
||||
insert_stubs(
|
||||
db,
|
||||
[
|
||||
{
|
||||
"id": it["id"],
|
||||
"channel_id": it["channel_id"],
|
||||
"title": it["title"],
|
||||
"published_at": parse_dt(it["published_at"]) or now,
|
||||
"thumbnail_url": it["thumbnail_url"],
|
||||
"via_search": True,
|
||||
}
|
||||
for it in candidates
|
||||
],
|
||||
)
|
||||
|
||||
# 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
|
||||
videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
|
||||
detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
|
||||
except YouTubeError:
|
||||
# Every candidate then stays unenriched, and an unenriched video cannot be judged live or
|
||||
# Short — so it is dropped. A working search returns nothing and reads as "no results".
|
||||
log.warning(
|
||||
"the videos found by a search could not be enriched",
|
||||
extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
|
||||
)
|
||||
detail_by_id = {}
|
||||
enriched_at = datetime.now(
|
||||
timezone.utc
|
||||
) # when THIS enrich ran (distinct from the stub placeholder)
|
||||
for v in videos:
|
||||
item = detail_by_id.get(v.id)
|
||||
if item is None:
|
||||
continue
|
||||
apply_video_details(v, item)
|
||||
v.enriched_at = enriched_at
|
||||
db.commit()
|
||||
|
||||
# 4) Confirm/deny Shorts (no quota) so none enter via search.
|
||||
_classify_shorts(db, videos)
|
||||
|
||||
keep = {
|
||||
v.id
|
||||
for v in videos
|
||||
if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
|
||||
}
|
||||
return [vid for vid in ids if vid in keep]
|
||||
|
||||
|
||||
def _new_first(db: Session, user: User, ids: list[str]) -> list[str]:
|
||||
"""Reorder results so genuinely-new discoveries come first and things the user already has
|
||||
(a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
|
||||
to the back — keeping the relevance order within each group."""
|
||||
if not ids:
|
||||
return ids
|
||||
sub_ch = select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
has_state = (
|
||||
select(VideoState.id)
|
||||
.where(VideoState.video_id == Video.id, VideoState.user_id == user.id)
|
||||
.exists()
|
||||
)
|
||||
in_playlist = (
|
||||
select(PlaylistItem.id)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(PlaylistItem.video_id == Video.id, Playlist.user_id == user.id)
|
||||
.exists()
|
||||
)
|
||||
have = set(
|
||||
db.scalars(
|
||||
select(Video.id).where(
|
||||
Video.id.in_(ids),
|
||||
or_(Video.channel_id.in_(sub_ch), has_state, in_playlist),
|
||||
)
|
||||
).all()
|
||||
)
|
||||
return [i for i in ids if i not in have] + [i for i in ids if i in have]
|
||||
|
||||
|
||||
def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
|
||||
"""Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
|
||||
|
||||
"Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
|
||||
video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
|
||||
wildcard — either way the term was never folded in, and local search lost that relevance."""
|
||||
padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.id.in_(video_ids))
|
||||
.where(padded.notilike(f"% {like_escape(term)} %"))
|
||||
.values(
|
||||
search_terms=func.trim(
|
||||
func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@router.get("/youtube")
|
||||
def search_youtube(
|
||||
q: str | None = None,
|
||||
@@ -285,9 +80,7 @@ def search_youtube(
|
||||
raise refused(429, "quota_spent")
|
||||
|
||||
# Channels this user has blocked are dropped before ingestion — never shown, never stored.
|
||||
blocked = set(
|
||||
db.scalars(select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)).all()
|
||||
)
|
||||
blocked = queries.blocked_ids(db, user)
|
||||
|
||||
collected: list[str] = []
|
||||
next_cursor = cursor
|
||||
@@ -328,7 +121,7 @@ def search_youtube(
|
||||
# write/shape failure degrades to a 422 with a message on this JSON endpoint instead of a
|
||||
# bare 500 hidden behind the FE's generic "search failed" (same rationale as B-core-1).
|
||||
try:
|
||||
collected.extend(_ingest_candidates(db, yt, user, candidates))
|
||||
collected.extend(catalog_search.ingest_candidates(db, yt, user, candidates))
|
||||
except Exception as exc: # any ingest failure, not just YouTubeError
|
||||
db.rollback()
|
||||
# The term is NOT in this line. A search string is exactly what the privacy
|
||||
@@ -376,17 +169,12 @@ def search_youtube(
|
||||
# Record these as THIS user's search finds (Mine feed / Source filter), and fold the
|
||||
# query into each result's search_terms so local full-text search inherits YouTube's
|
||||
# relevance. Idempotent; the generated search_vector updates automatically.
|
||||
db.execute(
|
||||
pg_insert(SearchFind)
|
||||
.values([{"user_id": user.id, "video_id": vid} for vid in ordered])
|
||||
.on_conflict_do_nothing(index_elements=["user_id", "video_id"])
|
||||
)
|
||||
_fold_term(db, ordered, term)
|
||||
catalog_search.record_finds(db, user, ordered, term)
|
||||
db.commit()
|
||||
|
||||
ordered = _new_first(db, user, ordered)
|
||||
ordered = catalog_search.new_first(db, user, ordered)
|
||||
return {
|
||||
"items": _serialize_ids(db, user, ordered),
|
||||
"items": reader_queries.feed_rows_by_ids(db, user, ordered),
|
||||
"next_cursor": next_cursor,
|
||||
"has_more": bool(next_cursor),
|
||||
"source": source,
|
||||
@@ -410,23 +198,6 @@ def clear_search(
|
||||
ids = body.video_ids
|
||||
if not ids:
|
||||
return {"ok": True, "removed": 0}
|
||||
db.execute(
|
||||
delete(SearchFind).where(SearchFind.user_id == user.id, SearchFind.video_id.in_(ids))
|
||||
)
|
||||
db.commit()
|
||||
has_state = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
in_playlist = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
removed = db.execute(
|
||||
delete(Video).where(
|
||||
Video.id.in_(ids),
|
||||
Video.via_search.is_(True),
|
||||
~has_state,
|
||||
~in_playlist,
|
||||
~subscribed,
|
||||
~other_find,
|
||||
)
|
||||
).rowcount
|
||||
removed = catalog_search.clear(db, user, ids)
|
||||
db.commit()
|
||||
return {"ok": True, "removed": removed}
|
||||
|
||||
+11
-54
@@ -1,12 +1,12 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy import and_, case, func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, quota, state
|
||||
from app.audit import AuditAction
|
||||
from app.auth import admin_user, current_user, has_read_scope
|
||||
from app.db import get_db
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog import queries, service
|
||||
from app.routes._common import youtube_refusal
|
||||
from app.ratelimit import RateLimiter
|
||||
from app.sync.runner import estimate_deep_backfill
|
||||
@@ -47,19 +47,8 @@ def sync_subscriptions(user: User = Depends(current_user), db: Session = Depends
|
||||
|
||||
@router.get("/status")
|
||||
def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sub_count = db.scalar(
|
||||
select(func.count()).select_from(Subscription).where(Subscription.user_id == user.id)
|
||||
)
|
||||
return {
|
||||
"subscriptions": sub_count,
|
||||
"channels_total": db.scalar(select(func.count()).select_from(Channel)),
|
||||
"channels_backfilling": db.scalar(
|
||||
select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
|
||||
),
|
||||
"videos_total": db.scalar(select(func.count()).select_from(Video)),
|
||||
"pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
),
|
||||
**queries.catalog_totals(db, user),
|
||||
"quota_used_today": quota.units_used_today(db),
|
||||
"quota_remaining_today": quota.remaining_today(db),
|
||||
"paused": state.is_sync_paused(db),
|
||||
@@ -70,51 +59,21 @@ def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db
|
||||
@router.get("/my-status")
|
||||
def my_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""Per-user sync progress: how far the channels *this* user subscribes to have synced."""
|
||||
sub_join = and_(Subscription.channel_id == Channel.id, Subscription.user_id == user.id)
|
||||
total, details_synced, recent_synced, all_deep_done = db.execute(
|
||||
select(
|
||||
func.count(Channel.id),
|
||||
func.count(Channel.details_synced_at),
|
||||
func.count(Channel.recent_synced_at),
|
||||
func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
|
||||
)
|
||||
.select_from(Channel)
|
||||
.join(Subscription, sub_join)
|
||||
).one()
|
||||
total = total or 0
|
||||
all_deep_done = int(all_deep_done or 0)
|
||||
total, details_synced, recent_synced, all_deep_done = queries.my_progress(db, user)
|
||||
|
||||
# Per-user full-history (deep) backfill: channels this user has opted in, and an ETA
|
||||
# for the still-pending ones (quota-bound estimate).
|
||||
deep_requested_channels = (
|
||||
db.execute(
|
||||
select(Channel).join(
|
||||
Subscription,
|
||||
and_(sub_join, Subscription.deep_requested.is_(True)),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
deep_requested_channels = queries.my_deep_requested(db, user)
|
||||
deep_eta = estimate_deep_backfill(db, deep_requested_channels)
|
||||
# "done / requested" is shown as one ratio, so both halves count the SAME channels: the ones this
|
||||
# user asked full history for. Counting every subscribed channel whose back-catalog happens to be
|
||||
# complete (another user asked, or it was short) gave "342/329".
|
||||
deep_done = sum(1 for c in deep_requested_channels if c.backfill_done)
|
||||
|
||||
my_videos = db.scalar(
|
||||
select(func.count(Video.id)).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
)
|
||||
)
|
||||
my_videos = queries.my_video_count(db, user)
|
||||
# Whole-catalog total (every video any user's subscriptions have pulled in), so the
|
||||
# header can show "yours" against the shared library size.
|
||||
total_videos = db.scalar(select(func.count(Video.id)))
|
||||
total_videos = queries.video_total(db)
|
||||
return {
|
||||
"channels_total": total,
|
||||
"channels_details_synced": details_synced,
|
||||
@@ -125,8 +84,8 @@ def my_status(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"channels_deep_requested": len(deep_requested_channels),
|
||||
"deep_pending_count": deep_eta["channels_pending"],
|
||||
"deep_eta_seconds": deep_eta["eta_seconds"],
|
||||
"my_videos": my_videos or 0,
|
||||
"total_videos": total_videos or 0,
|
||||
"my_videos": my_videos,
|
||||
"total_videos": total_videos,
|
||||
"quota_used_today": quota.units_used_today(db),
|
||||
"quota_remaining_today": quota.remaining_today(db),
|
||||
"paused": state.is_sync_paused(db),
|
||||
@@ -143,11 +102,9 @@ def deep_all(
|
||||
on: bool = True,
|
||||
) -> dict:
|
||||
"""Opt every one of the user's channels into (or out of) full-history backfill."""
|
||||
result = db.execute(
|
||||
update(Subscription).where(Subscription.user_id == user.id).values(deep_requested=on)
|
||||
)
|
||||
updated = service.set_deep_all(db, user, on)
|
||||
db.commit()
|
||||
return {"ok": True, "updated": result.rowcount, "deep_requested": on}
|
||||
return {"ok": True, "updated": updated, "deep_requested": on}
|
||||
|
||||
|
||||
@router.post("/pause")
|
||||
|
||||
@@ -16,6 +16,7 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens, audit
|
||||
from app.apitokens import GRANT_COLUMNS, Right
|
||||
from app.audit import AuditAction
|
||||
@@ -86,7 +87,7 @@ def _serialize(db: Session, token: ApiToken) -> dict:
|
||||
@router.get("")
|
||||
def list_tokens(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
tokens = (
|
||||
db.execute(select(ApiToken).where(ApiToken.user_id == user.id).order_by(ApiToken.id))
|
||||
db.execute(select(ApiToken).where(owned_by(ApiToken.user_id, user)).order_by(ApiToken.id))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
@@ -134,7 +135,7 @@ def create_token(
|
||||
if (epoch or 0) != (user.session_epoch or 0):
|
||||
raise refused(401, "not_authenticated")
|
||||
count = db.execute(
|
||||
select(func.count(ApiToken.id)).where(ApiToken.user_id == user.id)
|
||||
select(func.count(ApiToken.id)).where(owned_by(ApiToken.user_id, user))
|
||||
).scalar_one()
|
||||
if count >= MAX_TOKENS:
|
||||
raise refused(409, "api_token_limit", limit=MAX_TOKENS)
|
||||
|
||||
@@ -9,13 +9,13 @@ from typing import Callable
|
||||
from apscheduler.executors.pool import ThreadPoolExecutor
|
||||
from apscheduler.jobstores.base import ConflictingIdError
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from sqlalchemy import select
|
||||
|
||||
from app import audit, debugsession, jobruns, logs, logsink, progress, quota, sysconfig
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal
|
||||
from app.downloads.gc import run_download_gc
|
||||
from app.models import Channel, JobRun
|
||||
from app.models import JobRun
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.plex.sync import sync as run_plex_sync
|
||||
from app.plex.tmdb import run_enrich as run_tmdb_enrich
|
||||
from app.plex.tmdb import run_people_enrich
|
||||
@@ -305,7 +305,7 @@ def scheduler_snapshot(db) -> dict:
|
||||
"running": going is not None,
|
||||
"target": shown.target if shown else None,
|
||||
"target_title": (
|
||||
db.scalar(select(Channel.title).where(Channel.id == shown.target))
|
||||
catalog_queries.channel_title(db, shown.target)
|
||||
if shown is not None and shown.target
|
||||
else None
|
||||
),
|
||||
|
||||
@@ -12,7 +12,7 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, sysconfig
|
||||
from app.models import Channel, Video
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from sqlalchemy import delete, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import Channel, ExploredChannel, SearchFind, Subscription, Video
|
||||
from app.catalog.models import Channel, ExploredChannel, SearchFind, Subscription, Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.sync.videos import insert_stubs, stub_from_playlist_item, apply_video_details
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, is_playlist_gone
|
||||
|
||||
@@ -27,7 +27,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.config import settings
|
||||
from app.models import Video
|
||||
from app.catalog.models import Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.notifications import create_notification
|
||||
from app.sync.runner import get_service_user
|
||||
|
||||
@@ -20,7 +20,8 @@ from sqlalchemy.pool import NullPool
|
||||
|
||||
from app import logs, progress, quota
|
||||
from app.auth import has_read_scope
|
||||
from app.models import User, Video
|
||||
from app.models import User
|
||||
from app.catalog.models import Video
|
||||
from app.reader.models import Playlist, PlaylistItem
|
||||
from app.sync.playlist_merge import merge_plan
|
||||
from app.sync.runner import token_users
|
||||
|
||||
@@ -16,7 +16,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import Channel, OAuthToken, Subscription, User
|
||||
from app.models import OAuthToken, User
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.sync.explore import followed_channel
|
||||
from app.sync.subscriptions import import_subscriptions
|
||||
from app.sync.videos import (
|
||||
|
||||
@@ -7,9 +7,11 @@ from sqlalchemy import or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import Channel, Subscription, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.utils import normalize_description
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -172,7 +174,7 @@ def import_subscriptions(db: Session, user: User) -> dict:
|
||||
new_channels += 1
|
||||
sub_row = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == cid
|
||||
owned_by(Subscription.user_id, user.id), Subscription.channel_id == cid
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if sub_row is None:
|
||||
@@ -185,7 +187,9 @@ def import_subscriptions(db: Session, user: User) -> dict:
|
||||
# (see `should_prune`).
|
||||
removed = 0
|
||||
existing = (
|
||||
db.execute(select(Subscription).where(Subscription.user_id == user.id)).scalars().all()
|
||||
db.execute(select(Subscription).where(owned_by(Subscription.user_id, user.id)))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
prune_skipped = not should_prune(len(fetched), len(existing))
|
||||
if prune_skipped:
|
||||
|
||||
@@ -23,7 +23,8 @@ from collections import Counter
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import Subscription, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Subscription
|
||||
from app.reader.models import ChannelTag, Tag
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.models import LIVE_OR_UPCOMING, Channel, Video
|
||||
from app.catalog.models import LIVE_OR_UPCOMING, Channel, Video
|
||||
from app.titles import normalize_title
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail, is_playlist_gone
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
|
||||
@@ -5,6 +5,8 @@ names the whole schema. Whatever needs all of it — the migrations, the test da
|
||||
every mapper — imports this module.
|
||||
"""
|
||||
|
||||
import app.catalog.models # Catalog's tables
|
||||
import app.downloads.models # Capture's tables
|
||||
import app.models # the core tables
|
||||
import app.plex.models # Plex's tables
|
||||
import app.reader.models # noqa: F401 — Reader's tables
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
"""Whose a row is, and who an account is to the others — one place for both.
|
||||
|
||||
**Ownership.** A user-owned table says whose a row is in one column (`user_id`, or `to_user_id` for
|
||||
something handed to them). A unit's reads and writes name that column through `owned_by` and fetch a
|
||||
something handed to them). Every read and write names that column through `owned_by` and fetches a
|
||||
row by id through `owned_row`, never with an inline `Model.user_id == …`: the rule "a row owned by
|
||||
someone else is answered like a missing one" then lives here, and a contract test
|
||||
(`test_unit_fences.py`) holds a switched-on unit to it.
|
||||
someone else is answered like a missing one" then lives here. A contract test
|
||||
(`test_unit_fences.py`) holds every module, core included, to the filter's form; a rule this module
|
||||
cannot say (own or shared, built-in or own) is written out where it applies, with its reason.
|
||||
|
||||
**Recipients.** "An active, non-suspended account" — the download share-by-email and the recipient
|
||||
picker filter on it. Kept here so the two forms (SQL clauses for a query, a Python check for a loaded
|
||||
|
||||
@@ -31,6 +31,7 @@ from functools import partial
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
import app.tables # noqa: F401 — every table, before the first query
|
||||
from app import logs
|
||||
from app import logsink
|
||||
from app import sysconfig
|
||||
|
||||
@@ -105,6 +105,11 @@ def pytest_configure(config):
|
||||
)
|
||||
|
||||
|
||||
def _write_xact(db) -> str | None:
|
||||
"""The id of the open transaction if it has written anything, else None (a read never takes one)."""
|
||||
return db.scalar(text("SELECT pg_current_xact_id_if_assigned()::text"))
|
||||
|
||||
|
||||
def _request_dep(db, unfinished: list[str]):
|
||||
"""`get_db` for a test request: the test's own session, plus the check a shared session hides.
|
||||
|
||||
@@ -114,15 +119,16 @@ def _request_dep(db, unfinished: list[str]):
|
||||
and the suite stays green over a handler that forgot to commit (deep review, debt burn). So a
|
||||
request that ends normally with a write it did not commit — flushed, or still only staged — is
|
||||
recorded, and the test fails on it at teardown. Only the ROOT transaction's end clears it: a
|
||||
savepoint released inside the handler commits nothing. The test's own staged rows are flushed before the request starts — they are the
|
||||
test's business, not the handler's. A request that raises is rolled back, as production's
|
||||
savepoint released inside the handler commits nothing. What the test itself staged or wrote is
|
||||
committed before the request starts — it is the test's business, not the handler's, and a
|
||||
request then begins with no write open, so any write left at its end is the handler's. A request that raises is rolled back, as production's
|
||||
`close()` rolls it back: without that, what a failing handler flushed but never committed stayed
|
||||
readable to the rest of the test — the one state production cannot be in (an issue created
|
||||
on the tracker before a failed upload was kept only by an explicit commit, and nothing saw it
|
||||
missing)."""
|
||||
|
||||
def dep(conn: HTTPConnection):
|
||||
db.flush()
|
||||
db.commit()
|
||||
state = {"dirty": False}
|
||||
|
||||
def flushed(session, ctx):
|
||||
@@ -138,7 +144,12 @@ def _request_dep(db, unfinished: list[str]):
|
||||
try:
|
||||
yield db
|
||||
staged = db.new or db.deleted or any(db.is_modified(o) for o in db.dirty)
|
||||
if state["dirty"] or staged:
|
||||
# Core DML (`db.execute(update(...))`) fires no flush and stages nothing; the database
|
||||
# itself says whether a write is open — the request began with none, and a transaction
|
||||
# that wrote and was not committed holds an id (see `release_read_connection`). A row
|
||||
# lock (`FOR UPDATE`) takes one too: the handler ends what it locked.
|
||||
core_dml = _write_xact(db) is not None
|
||||
if state["dirty"] or staged or core_dml:
|
||||
unfinished.append(f"{conn.scope.get('method', 'WS')} {conn.url.path}")
|
||||
except Exception:
|
||||
db.rollback()
|
||||
|
||||
@@ -9,7 +9,8 @@ from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.auth import admin_user
|
||||
from app.main import app
|
||||
from app.models import AuditLog, Channel, JobRun, LogEvent, QuotaEvent
|
||||
from app.models import AuditLog, JobRun, LogEvent, QuotaEvent
|
||||
from app.catalog.models import Channel
|
||||
from tests.test_body_limit import _route_objects
|
||||
|
||||
T0 = datetime(2026, 10, 1, 12, 0, tzinfo=timezone.utc)
|
||||
|
||||
@@ -7,7 +7,9 @@ drops an unknown id, so the view silently showed the whole feed under its old na
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models import Channel
|
||||
from app.catalog.models import Channel
|
||||
from app.db import SessionLocal
|
||||
from app.sync.autotag import run_autotag_all
|
||||
from app.reader.models import ChannelTag, SavedView, Tag
|
||||
from app.reader import service as reader_service
|
||||
|
||||
@@ -66,3 +68,15 @@ def test_a_view_kept_tag_list_larger_than_a_bind_parameter_limit_still_cleans_up
|
||||
db.commit()
|
||||
assert reader_service.prune_orphan_system_tags(db) == 1
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_the_job_commits_its_cleanup(db):
|
||||
"""The prune only flushes; the job ends its own transaction. Read from a session of its own,
|
||||
the orphan is gone only once the job committed — the job's session would see its own delete
|
||||
either way."""
|
||||
orphan = Tag(user_id=None, name="Cooking", category="topic")
|
||||
db.add(orphan)
|
||||
db.commit()
|
||||
assert run_autotag_all(db, only_missing=True)["orphan_tags_removed"] == 1
|
||||
with SessionLocal() as fresh:
|
||||
assert fresh.execute(select(Tag).where(Tag.name == "Cooking")).first() is None
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
"""The channel manager's local writes, through HTTP: each one lands and is committed.
|
||||
|
||||
The service tests (`test_catalog_service.py`) hold what each write does; these hold that the route
|
||||
calls it and commits — the `db` fixture fails a test whose handler left a write uncommitted — and
|
||||
what a client sees back. None of these reach YouTube. Needs the DB.
|
||||
"""
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.catalog.models import BlockedChannel, Channel, Subscription
|
||||
from app.reader.models import ChannelTag, Tag
|
||||
|
||||
|
||||
def _seed(db, user):
|
||||
db.add(Channel(id="UCr", title="R"))
|
||||
db.flush()
|
||||
db.add(Subscription(user_id=user.id, channel_id="UCr"))
|
||||
tag = Tag(user_id=user.id, name="Mine")
|
||||
db.add(tag)
|
||||
db.commit()
|
||||
return tag
|
||||
|
||||
|
||||
def test_a_subscriptions_settings_change_and_list_back(db, make_user, as_user):
|
||||
user = make_user()
|
||||
_seed(db, user)
|
||||
client = as_user(user)
|
||||
r = client.patch("/api/channels/UCr", json={"priority": 2, "hidden": True})
|
||||
assert r.status_code == 200, r.text
|
||||
assert (r.json()["priority"], r.json()["hidden"]) == (2, True)
|
||||
listed = client.get("/api/channels").json()
|
||||
assert [(c["id"], c["priority"], c["hidden"]) for c in listed] == [("UCr", 2, True)]
|
||||
r = client.post("/api/sync/deep-all?on=false")
|
||||
assert r.json() == {"ok": True, "updated": 1, "deep_requested": False}
|
||||
|
||||
|
||||
def test_a_block_and_an_unblock(db, make_user, as_user):
|
||||
user = make_user()
|
||||
_seed(db, user)
|
||||
client = as_user(user)
|
||||
assert client.post("/api/channels/UCr/block").json() == {"ok": True, "blocked": "UCr"}
|
||||
assert client.post("/api/channels/UCr/block").status_code == 200 # idempotent
|
||||
assert [c["id"] for c in client.get("/api/channels/blocked/list").json()] == ["UCr"]
|
||||
assert client.get("/api/channels/UCr").json()["blocked"] is True
|
||||
assert client.delete("/api/channels/UCr/block").json() == {"ok": True, "unblocked": "UCr"}
|
||||
db.expire_all()
|
||||
assert db.execute(select(BlockedChannel)).first() is None
|
||||
|
||||
|
||||
def test_a_tag_attached_to_a_channel_and_detached(db, make_user, as_user):
|
||||
user, stranger = make_user(), make_user()
|
||||
tag = _seed(db, user)
|
||||
client = as_user(user)
|
||||
r = client.post("/api/channels/UCr/tags", json={"tag_id": tag.id})
|
||||
assert r.status_code == 200, r.text
|
||||
assert client.post("/api/channels/UCr/tags", json={"tag_id": tag.id}).status_code == 200
|
||||
assert client.get("/api/channels/UCr").json()["tag_ids"] == [tag.id]
|
||||
db.add(Subscription(user_id=stranger.id, channel_id="UCr"))
|
||||
db.commit()
|
||||
# Another user's tag is not theirs to attach: the same 404 as a missing one.
|
||||
r = as_user(stranger).post("/api/channels/UCr/tags", json={"tag_id": tag.id})
|
||||
assert r.json()["detail"] == {"code": "not_found", "reason": "tag"}
|
||||
assert client.delete(f"/api/channels/UCr/tags/{tag.id}").status_code == 200
|
||||
db.expire_all()
|
||||
assert db.execute(select(ChannelTag)).first() is None
|
||||
|
||||
|
||||
def test_the_sync_status_counts_the_callers_subscriptions(db, make_user, as_user):
|
||||
user = make_user()
|
||||
_seed(db, user)
|
||||
body = as_user(user).get("/api/sync/status").json()
|
||||
assert (body["subscriptions"], body["channels_total"]) == (1, 1)
|
||||
mine = as_user(user).get("/api/sync/my-status").json()
|
||||
assert (mine["channels_total"], mine["channels_recent_pending"]) == (1, 1)
|
||||
@@ -0,0 +1,295 @@
|
||||
"""Catalog's service and queries, one use case each, below the HTTP layer.
|
||||
|
||||
The routes are thin callers now, and their own tests stay the contract a client sees. These hold
|
||||
what each write does and leaves on its own: that it flushes and leaves the commit to its caller,
|
||||
that a row of someone else's reads as a missing one, and the rule each write carries (an explored
|
||||
channel flagged only while nobody follows it, a subscription that keeps it, a cleared search that
|
||||
spares what anyone holds). DB-backed (real Postgres).
|
||||
"""
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.catalog import queries, search, service
|
||||
from app.catalog.models import (
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
Video,
|
||||
)
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
|
||||
|
||||
def _channel(db, cid="UCc", **kw):
|
||||
ch = Channel(id=cid, title=kw.pop("title", cid), **kw)
|
||||
db.add(ch)
|
||||
db.commit()
|
||||
return ch
|
||||
|
||||
|
||||
def _video(db, vid, cid="UCc", **kw):
|
||||
v = Video(id=vid, channel_id=cid, published_at=datetime.now(timezone.utc), **kw)
|
||||
db.add(v)
|
||||
db.commit()
|
||||
return v
|
||||
|
||||
|
||||
def _sub(db, user, cid="UCc", **kw):
|
||||
s = Subscription(user_id=user.id, channel_id=cid, **kw)
|
||||
db.add(s)
|
||||
db.commit()
|
||||
return s
|
||||
|
||||
|
||||
def _uncommitted(db):
|
||||
return bool(db.new or db.dirty or db.deleted) or db.in_transaction()
|
||||
|
||||
|
||||
# --- whose a row is ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_users_subscriptions_blocks_and_explorations_are_theirs_alone(db, make_user):
|
||||
owner, stranger = make_user(), make_user()
|
||||
ch = _channel(db)
|
||||
_sub(db, owner)
|
||||
service.explore(db, owner, ch)
|
||||
service.block(db, owner, "UCc")
|
||||
db.commit()
|
||||
assert queries.subscription(db, owner, "UCc") is not None
|
||||
assert queries.subscription(db, stranger, "UCc") is None
|
||||
assert queries.exploration(db, stranger, "UCc") is None
|
||||
assert queries.is_blocked(db, stranger, "UCc") is False
|
||||
assert queries.blocked_ids(db, stranger) == set()
|
||||
assert [s.channel_id for s, _ in queries.subscriptions(db, stranger, "en")] == []
|
||||
|
||||
|
||||
# --- subscription settings -------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_full_history_reports_only_the_switch_on(db, make_user):
|
||||
user = make_user()
|
||||
_channel(db)
|
||||
sub = _sub(db, user)
|
||||
assert service.update_subscription(db, sub, priority=3) is False
|
||||
assert service.update_subscription(db, sub, deep_requested=True) is True
|
||||
assert service.update_subscription(db, sub, deep_requested=True) is False # already on
|
||||
assert _uncommitted(db)
|
||||
db.commit()
|
||||
assert (sub.priority, sub.hidden, sub.deep_requested) == (3, False, True)
|
||||
|
||||
|
||||
def test_full_history_for_all_moves_only_the_callers_rows(db, make_user):
|
||||
user, other = make_user(), make_user()
|
||||
_channel(db)
|
||||
_sub(db, user)
|
||||
theirs = _sub(db, other)
|
||||
assert service.set_deep_all(db, user, True) == 1
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
assert theirs.deep_requested is False
|
||||
|
||||
|
||||
def test_a_reset_asks_every_subscriber_for_full_history(db, make_user):
|
||||
a, b = make_user(), make_user()
|
||||
ch = _channel(db, backfill_done=True, recent_synced_at=datetime.now(timezone.utc))
|
||||
_sub(db, a)
|
||||
_sub(db, b)
|
||||
service.reset_backfill(db, ch)
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
assert (ch.backfill_done, ch.recent_synced_at) == (False, None)
|
||||
assert set(db.scalars(select(Subscription.deep_requested))) == {True}
|
||||
|
||||
|
||||
# --- explore, subscribe, block ---------------------------------------------------------------
|
||||
|
||||
|
||||
def test_an_exploration_is_one_row_and_flags_the_channel_only_while_nobody_follows_it(
|
||||
db, make_user
|
||||
):
|
||||
explorer, follower = make_user(), make_user()
|
||||
lone = _channel(db, "UClone")
|
||||
kept = _channel(db, "UCkept")
|
||||
_sub(db, follower, "UCkept")
|
||||
service.explore(db, explorer, lone)
|
||||
service.explore(db, explorer, kept)
|
||||
db.commit()
|
||||
first = queries.exploration(db, explorer, "UClone").created_at
|
||||
service.explore(db, explorer, lone) # refreshes the grace clock
|
||||
db.commit()
|
||||
assert db.scalar(select(ExploredChannel.id).where(ExploredChannel.channel_id == "UClone"))
|
||||
assert len(db.scalars(select(ExploredChannel.id)).all()) == 2
|
||||
assert queries.exploration(db, explorer, "UClone").created_at >= first
|
||||
assert (lone.from_explore, kept.from_explore) == (True, False)
|
||||
|
||||
|
||||
def test_subscribing_keeps_an_explored_channel_and_its_videos(db, make_user):
|
||||
user = make_user()
|
||||
ch = _channel(db, from_explore=True)
|
||||
v = _video(db, "v1", via_explore=True)
|
||||
service.record_subscription(db, user, ch, "")
|
||||
db.commit()
|
||||
db.expire_all()
|
||||
sub = queries.subscription(db, user, "UCc")
|
||||
assert (sub.yt_subscription_id, ch.from_explore, v.via_explore) == (None, False, False)
|
||||
service.drop_subscription(db, sub)
|
||||
db.commit()
|
||||
assert queries.subscription(db, user, "UCc") is None
|
||||
|
||||
|
||||
def test_a_block_is_once_and_ends_the_exploration(db, make_user):
|
||||
user = make_user()
|
||||
ch = _channel(db)
|
||||
service.explore(db, user, ch)
|
||||
db.commit()
|
||||
service.block(db, user, "UCc")
|
||||
service.block(db, user, "UCc")
|
||||
db.commit()
|
||||
assert len(db.scalars(select(BlockedChannel.id)).all()) == 1
|
||||
assert queries.exploration(db, user, "UCc") is None
|
||||
assert [c.id for c in queries.blocked_channels(db, user, "en")] == ["UCc"]
|
||||
service.unblock(db, user, "UCc")
|
||||
db.commit()
|
||||
assert queries.is_blocked(db, user, "UCc") is False
|
||||
|
||||
|
||||
# --- what the status cards count -------------------------------------------------------------
|
||||
|
||||
|
||||
def test_my_progress_counts_only_my_channels_and_my_unhidden_videos(db, make_user):
|
||||
user, other = make_user(), make_user()
|
||||
now = datetime.now(timezone.utc)
|
||||
_channel(db, "UCa", details_synced_at=now, recent_synced_at=now, backfill_done=True)
|
||||
_channel(db, "UCb")
|
||||
_channel(db, "UCtheirs")
|
||||
_video(db, "a1", "UCa")
|
||||
_video(db, "b1", "UCb")
|
||||
_video(db, "t1", "UCtheirs")
|
||||
_sub(db, user, "UCa", deep_requested=True)
|
||||
_sub(db, user, "UCb", hidden=True)
|
||||
_sub(db, other, "UCtheirs")
|
||||
assert queries.my_progress(db, user) == (2, 1, 1, 1)
|
||||
assert [c.id for c in queries.my_deep_requested(db, user)] == ["UCa"]
|
||||
assert queries.my_video_count(db, user) == 1
|
||||
assert queries.video_total(db) == 3
|
||||
assert queries.catalog_totals(db, user)["subscriptions"] == 2
|
||||
|
||||
|
||||
def test_a_channel_with_no_title_is_left_out_of_the_titles(db):
|
||||
_channel(db, "UCt", title="Titled")
|
||||
db.add(Channel(id="UCn", title=None))
|
||||
db.commit()
|
||||
assert queries.channel_titles(db, {"UCt", "UCn", "UCgone"}) == {"UCt": "Titled"}
|
||||
assert queries.channel_title(db, "UCt") == "Titled"
|
||||
|
||||
|
||||
def test_the_deep_backlog_is_the_unfinished_channels_someone_asked_for(db, make_user):
|
||||
user = make_user()
|
||||
_channel(db, "UCwant")
|
||||
_channel(db, "UCdone", backfill_done=True)
|
||||
_channel(db, "UCnobody")
|
||||
_sub(db, user, "UCwant", deep_requested=True)
|
||||
_sub(db, user, "UCdone", deep_requested=True)
|
||||
_sub(db, user, "UCnobody")
|
||||
assert [c.id for c in queries.deep_backlog(db)] == ["UCwant"]
|
||||
assert queries.deep_queued(db, ["UCwant", "UCdone", "UCnobody"]) == {"UCwant", "UCdone"}
|
||||
|
||||
|
||||
# --- live search -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_cleared_search_spares_what_anyone_holds(db, make_user):
|
||||
user, other, follower = make_user(), make_user(), make_user()
|
||||
_channel(db, "UCs")
|
||||
_channel(db, "UCfollowed")
|
||||
_sub(db, follower, "UCfollowed")
|
||||
for vid in ("loose", "watched", "found_by_other"):
|
||||
_video(db, vid, "UCs", via_search=True)
|
||||
_video(db, "followed", "UCfollowed", via_search=True)
|
||||
_video(db, "organic", "UCs")
|
||||
ids = ["loose", "watched", "found_by_other", "followed", "organic"]
|
||||
search.record_finds(db, user, ids, "cat")
|
||||
search.record_finds(db, user, ids, "cat") # idempotent
|
||||
search.record_finds(db, other, ["found_by_other"], "dog")
|
||||
reader_service.mark(db, other, "watched", "watched")
|
||||
db.commit()
|
||||
assert search.clear(db, user, ids) == 1
|
||||
db.commit()
|
||||
left = set(db.scalars(select(Video.id)))
|
||||
assert left == {"watched", "found_by_other", "followed", "organic"}
|
||||
assert db.scalars(select(SearchFind.user_id)).all() == [other.id]
|
||||
|
||||
|
||||
def test_a_search_puts_what_the_user_already_has_last(db, make_user):
|
||||
user = make_user()
|
||||
_channel(db, "UCs")
|
||||
_channel(db, "UCmine")
|
||||
_sub(db, user, "UCmine")
|
||||
for vid, cid in (("new1", "UCs"), ("mine", "UCmine"), ("seen", "UCs"), ("new2", "UCs")):
|
||||
_video(db, vid, cid)
|
||||
reader_service.mark(db, user, "seen", "hidden")
|
||||
db.commit()
|
||||
assert search.new_first(db, user, ["mine", "new1", "seen", "new2"]) == [
|
||||
"new1",
|
||||
"new2",
|
||||
"mine",
|
||||
"seen",
|
||||
]
|
||||
|
||||
|
||||
# --- what Catalog asks Reader ----------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_users_tag_links_are_made_once_and_only_theirs_are_read(db, make_user):
|
||||
user, other = make_user(), make_user()
|
||||
_channel(db)
|
||||
mine = reader_service.create_tag(db, user, "Mine", "other", None)
|
||||
theirs = reader_service.create_tag(db, other, "Theirs", "other", None)
|
||||
reader_service.attach_user_tag(db, user, "UCc", mine.id)
|
||||
reader_service.attach_user_tag(db, user, "UCc", mine.id)
|
||||
reader_service.attach_user_tag(db, other, "UCc", theirs.id)
|
||||
db.commit()
|
||||
assert reader_queries.user_tag_links(db, user) == {"UCc": [mine.id]}
|
||||
assert reader_queries.user_tag_links(db, user, "UCelse") == {}
|
||||
reader_service.detach_user_tag(db, user, "UCc", mine.id)
|
||||
db.commit()
|
||||
assert reader_queries.user_tag_links(db, user) == {}
|
||||
assert reader_queries.user_tag_links(db, other) == {"UCc": [theirs.id]}
|
||||
|
||||
|
||||
def test_discovery_offers_the_playlist_channels_not_followed_and_not_ones_own(db, make_user):
|
||||
user = make_user()
|
||||
for cid in ("UCnew", "UCfollowed", "UCown"):
|
||||
_channel(db, cid)
|
||||
for vid, cid in (("n1", "UCnew"), ("n2", "UCnew"), ("f1", "UCfollowed"), ("o1", "UCown")):
|
||||
_video(db, vid, cid)
|
||||
_sub(db, user, "UCfollowed")
|
||||
pl = reader_service.create_playlist(db, user, "L")
|
||||
for vid in ("n1", "n2", "f1", "o1"):
|
||||
reader_service.add_item(db, pl, vid, mark_dirty=False)
|
||||
db.commit()
|
||||
rows = reader_queries.playlist_channels(db, user, "UCown", "en")
|
||||
assert [(ch.id, n, lists) for ch, n, lists in rows] == [("UCnew", 2, 1)]
|
||||
|
||||
|
||||
def test_an_exploration_older_than_its_grace_is_still_one_row(db, make_user):
|
||||
"""The grace clock is refreshed in place: re-exploring after the cutoff does not add a row."""
|
||||
user = make_user()
|
||||
ch = _channel(db)
|
||||
db.add(
|
||||
ExploredChannel(
|
||||
user_id=user.id,
|
||||
channel_id="UCc",
|
||||
created_at=datetime.now(timezone.utc) - timedelta(days=30),
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
service.explore(db, user, ch)
|
||||
db.commit()
|
||||
rows = db.scalars(select(ExploredChannel)).all()
|
||||
assert len(rows) == 1
|
||||
assert rows[0].created_at > datetime.now(timezone.utc) - timedelta(minutes=1)
|
||||
@@ -12,7 +12,8 @@ import app.state as state
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.main import app
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
from app.sync import runner, videos
|
||||
|
||||
_PUB = datetime(2020, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
@@ -10,7 +10,7 @@ from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from app.models import Channel, Subscription
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.routes import channels
|
||||
from app.sync import subscriptions
|
||||
from app.sync.subscriptions import DETAILS_MAX_AGE, import_subscriptions
|
||||
|
||||
@@ -14,8 +14,8 @@ from sqlalchemy import text
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
from app.feedquery import filtered_query
|
||||
from app.models import Video
|
||||
from app.routes.channels import channel_aggregates
|
||||
from app.catalog.models import Video
|
||||
from app.catalog.queries import channel_aggregates
|
||||
|
||||
INDEX = "ix_videos_channel_pub_cover"
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ import pytest
|
||||
from fastapi import HTTPException
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models import Channel, Subscription
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.routes import channels
|
||||
from app.youtube.client import YouTubeError
|
||||
|
||||
|
||||
@@ -14,7 +14,8 @@ import app.state as state
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.main import app
|
||||
from app.models import Channel, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel
|
||||
from app.reader.models import ChannelTag, Tag
|
||||
|
||||
|
||||
|
||||
@@ -18,7 +18,8 @@ from app.downloads import errors as dlerrors
|
||||
from app.downloads import formats, pipeline, quota, runner, service
|
||||
from app.downloads import probe as probemod
|
||||
from app.downloads.edit import INPUT_GUARD
|
||||
from app.models import Channel, PlexItem, PlexLibrary, PlexShow, Video
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
from app.plex import paths as plex_paths
|
||||
|
||||
|
||||
@@ -9,7 +9,9 @@ from app.auth import current_user
|
||||
from app.collation import sorted_words
|
||||
from app.db import get_db
|
||||
from app.main import app
|
||||
from app.models import Channel, PlexCollection, PlexItem, PlexLibrary, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexLibrary
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
from app.reader.models import Playlist, PlaylistItem
|
||||
from datetime import datetime, timezone
|
||||
|
||||
@@ -222,7 +224,7 @@ def test_the_share_recipients_are_in_the_readers_alphabet(db, monkeypatch, overr
|
||||
|
||||
|
||||
def test_the_blocked_channels_are_in_the_readers_alphabet(db, monkeypatch, override_dep):
|
||||
from app.models import BlockedChannel
|
||||
from app.catalog.models import BlockedChannel
|
||||
|
||||
user = _user(db, "hu")
|
||||
db.add_all(Channel(id=f"UC{i}", title=w) for i, w in enumerate(WORDS))
|
||||
|
||||
@@ -17,17 +17,9 @@ from app.discover import hero, pins, queries, rails
|
||||
from app.discover.budget import BANDS, fits, remaining_seconds
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import leaf_card
|
||||
from app.models import (
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
Subscription,
|
||||
User,
|
||||
Video,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexShow, PlexState
|
||||
from app.catalog.models import BlockedChannel, Channel, Subscription, Video
|
||||
from app.reader.models import ComfortPin, Playlist, PlaylistItem, VideoState
|
||||
|
||||
NOW = datetime(2026, 6, 1, tzinfo=timezone.utc)
|
||||
|
||||
@@ -12,7 +12,8 @@ contradicts the remote's own count is not believed either.
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models import Channel, PlexCollection, PlexItem, PlexLibrary, Video
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexLibrary
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import Playlist, PlaylistItem
|
||||
from app.plex import sync as plex_sync
|
||||
from app.sync import playlists as plsync
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Every process the app runs as sees every table before its first query.
|
||||
|
||||
The units declare their tables in their own `models` modules, and a core model names a unit's by
|
||||
string (`User.subscriptions` → `Subscription`). A fresh interpreter that never imported the unit's
|
||||
module fails to configure its mappers on its first query: the download child once did exactly that,
|
||||
and every job failed, while this suite stayed green — it imports `app.tables` in-process, which hides
|
||||
the gap. So each entry point is imported here in a clean interpreter, and must register the whole
|
||||
schema on its own.
|
||||
"""
|
||||
|
||||
import ast
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import app.tables # noqa: F401
|
||||
from app.db import Base
|
||||
|
||||
BACKEND = Path(__file__).parent.parent
|
||||
APP = BACKEND / "app"
|
||||
|
||||
# Each `python -m` module the deployment starts, and the API's app module.
|
||||
ENTRY_POINTS = ("app.main", "app.worker", "app.downloads.runner")
|
||||
|
||||
# The table list goes out on one marked line: the API's logging writes to the same stdout, and a
|
||||
# line it logs while importing is not the probe's answer.
|
||||
_MARK = "entry-point-tables:"
|
||||
_PROBE = (
|
||||
"import importlib, json, sys\n"
|
||||
"importlib.import_module(sys.argv[1])\n"
|
||||
"from sqlalchemy.orm import configure_mappers\n"
|
||||
"from app.db import Base\n"
|
||||
"configure_mappers()\n"
|
||||
f"print({_MARK!r} + json.dumps(sorted(Base.metadata.tables)))\n"
|
||||
)
|
||||
|
||||
|
||||
def test_each_entry_point_registers_the_whole_schema_on_its_own():
|
||||
whole = sorted(Base.metadata.tables)
|
||||
for module in ENTRY_POINTS:
|
||||
r = subprocess.run(
|
||||
[sys.executable, "-c", _PROBE, module],
|
||||
cwd=BACKEND,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
check=False,
|
||||
)
|
||||
assert r.returncode == 0, (module, r.stderr[-2000:])
|
||||
answer = [ln for ln in r.stdout.splitlines() if ln.startswith(_MARK)]
|
||||
assert len(answer) == 1, (module, r.stdout[-2000:])
|
||||
assert json.loads(answer[0].removeprefix(_MARK)) == whole, module
|
||||
|
||||
|
||||
def test_every_module_that_runs_as_a_process_is_listed():
|
||||
mains = set()
|
||||
for path in APP.rglob("*.py"):
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||
for node in tree.body:
|
||||
if (
|
||||
isinstance(node, ast.If)
|
||||
and isinstance(node.test, ast.Compare)
|
||||
and isinstance(node.test.left, ast.Name)
|
||||
and node.test.left.id == "__name__"
|
||||
):
|
||||
rel = path.relative_to(BACKEND).with_suffix("").as_posix()
|
||||
mains.add(rel.replace("/", "."))
|
||||
assert mains <= set(ENTRY_POINTS), mains - set(ENTRY_POINTS)
|
||||
@@ -6,7 +6,7 @@ from datetime import datetime, timedelta, timezone
|
||||
import pytest
|
||||
|
||||
from app import sysconfig
|
||||
from app.models import Channel, Video
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.sync import runner
|
||||
from app.youtube.client import YouTubeError
|
||||
|
||||
|
||||
@@ -16,7 +16,8 @@ from datetime import datetime, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
|
||||
# Every sort feedquery.sort_keys defines except "relevance" (which needs a search query to rank).
|
||||
_SORTS = [
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user