Compare commits
37
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fdf8076013 | ||
|
|
6287fc9cf8 | ||
|
|
3c6c23507b | ||
|
|
ef99f7b5c7 | ||
|
|
94418a087e | ||
|
|
28b77fed1e | ||
|
|
50764a916d | ||
|
|
f21cdfa156 | ||
|
|
d6dada7fe5 | ||
|
|
6645caa157 | ||
|
|
59ec4a28ba | ||
|
|
dc4f4f5143 | ||
|
|
909732b3c1 | ||
|
|
215f022fe3 | ||
|
|
6ea750a698 | ||
|
|
c5f121139b | ||
|
|
b04b2cbd62 | ||
|
|
3a5212cd8a | ||
|
|
b376373af5 | ||
|
|
8206384c91 | ||
|
|
e27656ee7e | ||
|
|
d73cdcee97 | ||
|
|
5ce292cd28 | ||
|
|
ad6ccbac05 | ||
|
|
531eb2eef1 | ||
|
|
2cf1b688d9 | ||
|
|
3b0e365e26 | ||
|
|
533571f572 | ||
|
|
4b6db47083 | ||
|
|
5229fd96be | ||
|
|
238e6f8a61 | ||
|
|
a35eece652 | ||
|
|
b703234c17 | ||
|
|
dbb4065e7b | ||
|
|
e82f87acb0 | ||
|
|
e9c60f72cd | ||
|
|
3a688fa0f5 |
@@ -7,6 +7,32 @@ edit the release notes and re-run the generator.
|
||||
|
||||
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
|
||||
|
||||
## 0.109.0 — 2026-10-21
|
||||
|
||||
### New
|
||||
|
||||
- Accounts: a browser is signed in to one account at a time. The account menu's "Sign in as someone else" replaces it; the account list and switcher are gone. After this update everyone signs in once more.
|
||||
- Settings follow you: the Home start page, the channel manager and Plex playlist layouts, the discovery sorts and a dismissed onboarding or release banner are saved to your account, on every device. Performance mode stays per device.
|
||||
- Development: every key the browser stores declares its kind, held by a contract test; one Google connect route serves the sign-in and the YouTube access grants.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Accounts: a tab still showing another account than the one signed in now reloads as that one and says why; a failed "Sign in as someone else" says what happened.
|
||||
- A link whose path starts with // no longer opens the error screen.
|
||||
- A member following a link to an admin page lands on the feed, with the rail and the address bar in step.
|
||||
- Google: an unknown access request is refused instead of quietly asking for read access, and a YouTube grant on an instance without Google says so.
|
||||
|
||||
## 0.108.1 — 2026-10-20
|
||||
|
||||
### New
|
||||
|
||||
- Development: the backend is split into four units (downloads, reader, catalog, Plex), each owning its tables. A contract test holds what each one reads and writes, and that every query names whose rows it asks for.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Watch later: saving the first two videos at once no longer makes two lists. An existing second list is merged into the first, its videos and its comfort-shelf pin with it.
|
||||
- Playback: the first resume point of a video or a Plex item no longer fails when it is saved together with the watched mark, a whole season marked at once, or a Plex sync.
|
||||
|
||||
## 0.108.0 — 2026-10-18
|
||||
|
||||
### New
|
||||
|
||||
@@ -63,9 +63,16 @@ how it gets built. Concretely:
|
||||
## Architecture shape (don't fight it)
|
||||
|
||||
`routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
|
||||
`downloads/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`downloads/`, `reader/`, `discover/`, `catalog/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `ratelimit`, `bodylimit`, `progress`, `collation`,
|
||||
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
|
||||
The domain is four **units**, each owning its tables: Capture (`downloads/`), Reader (`reader/`, `discover/`, the playlist
|
||||
sync), Catalog (`catalog/`, the `sync/` ingests) and Plex (`plex/`). A unit's route does no database work: it reads through
|
||||
the unit's `queries`, writes through its `service` (which flushes) and commits. A unit's code reads another unit's tables
|
||||
only along a declared edge and writes them only through that unit's service; core reaches a unit's tables through its
|
||||
`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`) — as far as an AST sees: what it
|
||||
cannot (an ORM write on another unit's loaded row, SQL in `text()`, a commit by `begin()`) is review's to catch, listed in
|
||||
[`backend/app/README.md`](backend/app/README.md) with the conventions.
|
||||
Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
|
||||
entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
|
||||
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
|
||||
@@ -74,44 +81,44 @@ boundary is one-way and **lint-enforced**, not just described: a platform compon
|
||||
feature component or a domain API slice, and one layer down `lib/` may not import from `components/` at
|
||||
all, not even a type (`no-restricted-imports` in `eslint.config.js`, with its own test). A new generic
|
||||
component goes in `ui/`; see [`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
|
||||
Data via react-query (`qk` factory), per-account `localStorage`, and server prefs; API through
|
||||
`lib/api/*` slices over `core.req<T>`.
|
||||
Data via react-query (`qk` factory), the signed-in account's `localStorage` (`storage.ts`), and server
|
||||
prefs; API through `lib/api/*` slices over `core.req<T>`.
|
||||
|
||||
**Not wanted** (reviewed and rejected as over-engineering for this size): microservices, event bus,
|
||||
CQRS, a repository/DI framework, a router library, a generic CRUD-router factory or base serializer.
|
||||
|
||||
## Canonical patterns — when you have a choice, pick THIS
|
||||
|
||||
| Area | Use | Not |
|
||||
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; helpers `flush` (test-held list) | helpers that commit (only `quota` and `jobruns` own their session) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
|
||||
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
|
||||
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
|
||||
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
|
||||
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
|
||||
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
|
||||
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect | a bare `catch {}` or `notify()` with no error attached |
|
||||
| FE floating or full-screen layer | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers") | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]` |
|
||||
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
|
||||
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
|
||||
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
|
||||
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
|
||||
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback` | a render-phase `ref.current = x` — the React Compiler skips the component |
|
||||
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
|
||||
| FE popover / scroll-fade ref | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}` | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips |
|
||||
| FE async handler with cleanup | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
|
||||
| Area | Use | Not |
|
||||
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404`. A composite rule (own or shared, built-in or own) says so in its `queries` | an inline owner filter — `Model.user_id == …`, an alias's, `.in_()`, `filter_by(user_id=…)` (fence-held, every module) — or a `db.get` and a compare where `owned_row` fits |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; a unit's `service` and every helper `flush` (fence-held, every module) | a helper that commits — only a listed one (what owns its session, or a core step that must land whatever its caller does next), each with its reason: the fence's `UNIT_COMMITS` / `CORE_COMMITS`, a route's `COMMITTING_HELPERS` (`test_route_conventions`) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
| FE browser storage | a key in `storage.ts` with its kind (contract-held): a `pref` through `usePref` (the server's, follows the account); `device` / `view` / `cache` this browser's; `public` kept for the /watch page | a `"siftlode.x"` literal; a preference kept only in `localStorage` (it stays on one device) |
|
||||
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
|
||||
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
|
||||
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
|
||||
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
|
||||
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
|
||||
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)`; a YouTube write action (subscribe, sync, …): `notifyYouTubeActionError` — a 403 offers Connect | a bare `catch {}` or `notify()` with no error attached |
|
||||
| FE floating or full-screen layer | `ui/Overlay` (portals to `<body>`) or the popover engine (`AnchoredPopover` portals itself); stacking by the z-index scale (`z-menu` … `z-popover`, docs/DESIGN.md "Layers") | one rendered in place inside a virtualized row or a masked/transformed scroller — clipped or painted over; a magic `z-[n]` |
|
||||
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
|
||||
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
|
||||
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
|
||||
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
|
||||
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+); a handler or timer: `useLatest` / `useStableCallback` | a render-phase `ref.current = x` — the React Compiler skips the component |
|
||||
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
|
||||
| FE popover / scroll-fade ref | destructure it: `const { setReference } = pop.refs`, then `ref={setReference}` | `ref={pop.refs.setReference}` / `ref={fade.ref}` — the compiler then treats the whole object as a ref and skips |
|
||||
| FE async handler with cleanup | `try { … } catch { … }`, then the cleanup after it (steps that `return` early go in their own function); a conditional, loop or `?.` the `try` needs goes before it or into a module-level helper | `try/finally`, or a `?:` / `??` / loop / `?.` inside `try` — the React Compiler skips the whole component (and `react-hooks/todo` misses most of these) |
|
||||
|
||||
Not repeated here, because a lint rule refuses the wrong one and its message names the right one:
|
||||
buttons (`ui/Button`, `IconButton`, `Pressable`, `Chip`, `DragHandle`), text fields and native
|
||||
|
||||
@@ -640,48 +640,6 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/channels.py": [
|
||||
{
|
||||
"code": "reportReturnType",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 11,
|
||||
"lineCount": 6
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 49,
|
||||
"endColumn": 56,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 38,
|
||||
"endColumn": 44,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 33,
|
||||
"endColumn": 39,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/discover.py": [
|
||||
{
|
||||
"code": "reportInvalidTypeForm",
|
||||
@@ -692,277 +650,7 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/downloads.py": [
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
"startColumn": 17,
|
||||
"endColumn": 19,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
"startColumn": 82,
|
||||
"endColumn": 84,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
"startColumn": 87,
|
||||
"endColumn": 89,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
"startColumn": 30,
|
||||
"endColumn": 32,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
"startColumn": 49,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/playlists.py": [
|
||||
{
|
||||
"code": "reportOptionalOperand",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalOperand",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 13,
|
||||
"endColumn": 5,
|
||||
"lineCount": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 16,
|
||||
"endColumn": 5,
|
||||
"lineCount": 11
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 9
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 13,
|
||||
"endColumn": 5,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 15,
|
||||
"lineCount": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 12,
|
||||
"endColumn": 31,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 23,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 12,
|
||||
"endColumn": 34,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 12,
|
||||
"endColumn": 28,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 23,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/browse.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 61,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 26,
|
||||
"endColumn": 61,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 13,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 13,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 27,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 15,
|
||||
"endColumn": 17,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 28,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 15,
|
||||
"endColumn": 17,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalSubscript",
|
||||
"range": {
|
||||
"startColumn": 28,
|
||||
"endColumn": 30,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/collections.py": [
|
||||
{
|
||||
"code": "reportAssignmentType",
|
||||
"range": {
|
||||
"startColumn": 11,
|
||||
"endColumn": 39,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 76,
|
||||
"endColumn": 80,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 75,
|
||||
"endColumn": 79,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 80,
|
||||
"endColumn": 84,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
@@ -1020,117 +708,7 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/discover.py": [
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 4,
|
||||
"endColumn": 63,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 18,
|
||||
"endColumn": 48,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportCallIssue",
|
||||
"range": {
|
||||
"startColumn": 4,
|
||||
"endColumn": 63,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 18,
|
||||
"endColumn": 48,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/playback.py": [
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 6,
|
||||
"endColumn": 14,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 51,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 51,
|
||||
"endColumn": 54,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/plex/playlists.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 77,
|
||||
"endColumn": 83,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 77,
|
||||
"endColumn": 83,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 8,
|
||||
"endColumn": 9,
|
||||
"lineCount": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 10,
|
||||
"endColumn": 18,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/scheduler.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 55,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportOptionalMemberAccess",
|
||||
"range": {
|
||||
@@ -1148,42 +726,6 @@
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/search.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 25,
|
||||
"endColumn": 31,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 6,
|
||||
"endColumn": 14,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/routes/sync.py": [
|
||||
{
|
||||
"code": "reportArgumentType",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 65,
|
||||
"lineCount": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
"range": {
|
||||
"startColumn": 42,
|
||||
"endColumn": 50,
|
||||
"lineCount": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"./app/sync/explore.py": [
|
||||
{
|
||||
"code": "reportAttributeAccessIssue",
|
||||
|
||||
@@ -4,7 +4,7 @@ from sqlalchemy import engine_from_config, pool
|
||||
from app.config import settings
|
||||
from app.db import Base
|
||||
from app.logs import configure_logging
|
||||
import app.models # noqa: F401 (import so models register on Base.metadata)
|
||||
import app.tables # noqa: F401 (import so every table registers on Base.metadata)
|
||||
|
||||
config = context.config
|
||||
config.set_main_option("sqlalchemy.url", settings.database_url)
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""One Watch later list per user, held by the database
|
||||
|
||||
Revision ID: 0112_one_watch_later
|
||||
Revises: 0111_audit_request_id
|
||||
Create Date: 2026-10-20
|
||||
|
||||
Watch later was made on a user's first save by a read then an insert, so two first saves at once
|
||||
each made a list, and the reader then picked between them at random. A partial unique index holds
|
||||
"one per user" now. An instance that already has a second list keeps the oldest: the others' videos
|
||||
move into it (after its own, in their order; one it already holds is not added twice), a Home pin
|
||||
of one of them moves to it (unless it is pinned itself; of several, the first on the shelf), then
|
||||
they go — a pin left on them goes with them, as the kept list already holds one.
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0112_one_watch_later"
|
||||
down_revision: str | None = "0111_audit_request_id"
|
||||
branch_labels: str | None = None
|
||||
depends_on: str | None = None
|
||||
|
||||
_EXTRA = """
|
||||
SELECT p.id, k.keep_id
|
||||
FROM playlists p
|
||||
JOIN (
|
||||
SELECT user_id, min(id) AS keep_id FROM playlists WHERE kind = 'watch_later' GROUP BY user_id
|
||||
) k ON k.user_id = p.user_id
|
||||
WHERE p.kind = 'watch_later' AND p.id <> k.keep_id
|
||||
"""
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
f"""
|
||||
INSERT INTO playlist_items (playlist_id, video_id, position, added_at)
|
||||
SELECT e.keep_id, i.video_id,
|
||||
(SELECT coalesce(max(position), -1) FROM playlist_items WHERE playlist_id = e.keep_id)
|
||||
+ row_number() OVER (PARTITION BY e.keep_id ORDER BY i.playlist_id, i.position, i.id),
|
||||
i.added_at
|
||||
FROM playlist_items i
|
||||
JOIN ({_EXTRA}) e ON e.id = i.playlist_id
|
||||
ON CONFLICT ON CONSTRAINT uq_playlist_video DO NOTHING
|
||||
"""
|
||||
)
|
||||
op.execute(
|
||||
f"""
|
||||
UPDATE comfort_pins c SET playlist_id = m.keep_id
|
||||
FROM (
|
||||
SELECT DISTINCT ON (e.keep_id) p.id AS pin_id, e.keep_id
|
||||
FROM comfort_pins p
|
||||
JOIN ({_EXTRA}) e ON e.id = p.playlist_id
|
||||
WHERE NOT EXISTS (SELECT 1 FROM comfort_pins k WHERE k.playlist_id = e.keep_id)
|
||||
ORDER BY e.keep_id, p.position, p.id
|
||||
) m
|
||||
WHERE c.id = m.pin_id
|
||||
"""
|
||||
)
|
||||
op.execute(f"DELETE FROM playlists WHERE id IN (SELECT id FROM ({_EXTRA}) e)")
|
||||
op.create_index(
|
||||
"uq_playlist_watch_later",
|
||||
"playlists",
|
||||
["user_id"],
|
||||
unique=True,
|
||||
postgresql_where=sa.text("kind = 'watch_later'"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("uq_playlist_watch_later", table_name="playlists")
|
||||
+56
-16
@@ -1,9 +1,8 @@
|
||||
# Backend conventions
|
||||
|
||||
The rules below are already followed across `backend/app` — they lived only in a few module
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
|
||||
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
|
||||
I copy".
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them. When you add code,
|
||||
match these so the next reader doesn't have to guess "which pattern do I copy".
|
||||
|
||||
## Module layering
|
||||
|
||||
@@ -13,12 +12,13 @@ Three layers, by *who* runs the code — not by feature:
|
||||
response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
|
||||
Business logic that a request happens to trigger does **not** live here if anything else runs it too.
|
||||
|
||||
2. **`plex/`, `downloads/`, `sync/`, `issues/` — domain logic shared by more than one caller.** Anything the
|
||||
**scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
|
||||
route, the worker, and the scheduler share one definition. Examples from this codebase:
|
||||
`downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
|
||||
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline;
|
||||
`issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
|
||||
2. **The units and the other domain packages — logic shared by more than one caller.** A unit's
|
||||
database work (below), and anything the **scheduler or the worker** runs (not just a request), lives
|
||||
in a domain package, so the route, the worker, and the scheduler share one definition. Examples from
|
||||
this codebase: `downloads/service.py` owns enqueue + the job state transitions (routes call it);
|
||||
`plex/sync.py` is run by both the admin route and the scheduler; `downloads/pipeline.py` is the
|
||||
worker's child pipeline; `issues/bundle.py` is the one definition of a bug report's files, for the
|
||||
zip export and the tracker.
|
||||
|
||||
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's
|
||||
folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every
|
||||
@@ -34,19 +34,59 @@ Three layers, by *who* runs the code — not by feature:
|
||||
a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
|
||||
pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
|
||||
|
||||
## Units
|
||||
|
||||
The domain is four units, each owning its tables (`models.py` in its package):
|
||||
|
||||
| Unit | Package | Its routes |
|
||||
| --- | --- | --- |
|
||||
| Capture | `downloads/`, the worker | `downloads.py`, `files.py`, `public.py` |
|
||||
| Reader | `reader/`, `discover/`, `feedquery.py`, the playlist sync and tag suggestions in `sync/` | `feed.py`, `playlists.py`, `saved_views.py`, `discover.py`, `tags.py` |
|
||||
| Catalog | `catalog/`, the ingests in `sync/` | `channels.py`, `search.py`, `sync.py` |
|
||||
| Plex | `plex/` | `plex/` |
|
||||
|
||||
- **A unit's route does no database work.** It validates, reads through the unit's `queries.py`,
|
||||
writes through its `service.py`, serializes, and commits. It may name a unit's model as a type
|
||||
(under `TYPE_CHECKING`), never use one as a value, and it passes a kind (`"movie"`), not a model, to
|
||||
a query.
|
||||
- **A unit's code keeps to its edges.** It reads another unit's tables only along a declared edge
|
||||
(Capture and Reader read Catalog's and Plex's: the feed joins `Video` with `VideoState` in one
|
||||
query) and writes them only through the owner's service. Core code reaches a unit's tables through
|
||||
that unit's `queries`.
|
||||
- **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id,
|
||||
user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the
|
||||
same answer as a missing one). A unit's `queries.own_x` returns None and its route answers
|
||||
`not_found`; a core route calls `owned_or_404`. A rule `owned_row` cannot say — a job its owner
|
||||
shared, a built-in profile nobody owns, a folder read under `FOR SHARE` — is written out in the
|
||||
unit's own code, beside the reason.
|
||||
|
||||
All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's
|
||||
packages and edges (`UNIT_PACKAGES`, `READS`) and what may commit (`UNIT_COMMITS`, `CORE_COMMITS`)
|
||||
are listed there. It reads the code's AST, so it holds the forms written here: an owner column
|
||||
compared inline (a model's, an import alias's or an `aliased()` one's, by `==`, `!=`, `.in_()`, or a
|
||||
`filter_by(user_id=…)`), a model built or an `insert`/`update`/`delete` (under any import name, of a
|
||||
model or its `__table__`) across units, and a `.commit()` call off the lists all fail it. What an AST
|
||||
cannot tell is review's to catch: an ORM write on another unit's loaded row (an attribute, a
|
||||
`db.delete`), SQL in `text()`, a column or model reached another way (bound to a variable, through
|
||||
`__table__.c` or a module path), and a commit made by a `begin()` block or a `commit` handed in.
|
||||
|
||||
## Transaction ownership
|
||||
|
||||
This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
|
||||
(only `quota` and `jobruns` own their session)"** — so read it that way, not as an exception to it. The commit belongs
|
||||
to the **operation owner**; anything called *inside* another transaction only stages its change
|
||||
This follows CLAUDE.md's canonical rule — **the handler commits; a unit's `service` and every helper
|
||||
flush** — so read it that way, not as an exception to it. The commit belongs to the **operation
|
||||
owner**; anything called *inside* another transaction only stages its change
|
||||
(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
|
||||
`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
|
||||
mutation it records, so an audit entry can never exist without its change (or vice-versa).
|
||||
|
||||
- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
|
||||
whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
|
||||
a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
|
||||
`jobruns`' run rows — the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
|
||||
- The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it
|
||||
makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the
|
||||
request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, the
|
||||
scheduler's jobs, a unit's jobs (the worker's `jobdb` writes, the syncs and ingests that commit per
|
||||
page or batch so a later failure loses nothing already stored), and a few core steps that must land
|
||||
whatever their caller does next (a token's last-used stamp, a refreshed YouTube grant) — the fence's
|
||||
`UNIT_COMMITS` and `CORE_COMMITS` name each with its reason, `test_route_conventions` a route's own
|
||||
committing helpers. Everything else flushes.
|
||||
- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
|
||||
loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
|
||||
attribute — see `service.decr_ref_count`.
|
||||
|
||||
@@ -26,6 +26,7 @@ from fastapi import Depends, Request
|
||||
from sqlalchemy import delete, or_, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -223,7 +224,9 @@ def revoke_all(db: Session, user_id: int) -> int:
|
||||
change, "log out everywhere", a revoked untrusted password — ends its tokens too: one minted
|
||||
from a stolen session must not outlive the owner's answer to that theft. Returns how many."""
|
||||
gone = (
|
||||
db.execute(delete(ApiToken).where(ApiToken.user_id == user_id).returning(ApiToken.id))
|
||||
db.execute(
|
||||
delete(ApiToken).where(owned_by(ApiToken.user_id, user_id)).returning(ApiToken.id)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
@@ -110,10 +110,9 @@ def record(
|
||||
|
||||
def gc(db: Session, retention_days: int) -> int:
|
||||
"""Delete audit rows older than `retention_days`. Returns the number removed. A
|
||||
retention_days <= 0 disables pruning (keep forever)."""
|
||||
retention_days <= 0 disables pruning (keep forever). Flushes; the job commits."""
|
||||
if retention_days <= 0:
|
||||
return 0
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
|
||||
result = db.execute(delete(AuditLog).where(AuditLog.created_at < cutoff))
|
||||
db.commit()
|
||||
return result.rowcount or 0
|
||||
|
||||
+49
-89
@@ -4,6 +4,7 @@ from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.requests import HTTPConnection
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import debugsession
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -24,8 +25,6 @@ def operator_contact() -> str | None:
|
||||
return next(iter(sorted(settings.admin_email_set)), None)
|
||||
|
||||
|
||||
MAX_SESSION_ACCOUNTS = 6
|
||||
|
||||
# OAuth scope vocabulary — the atoms + the request bundles, kept together as core. has_*_scope check
|
||||
# the atoms against a user's stored grant; the *_SCOPES bundles are what the login/callback endpoints
|
||||
# request (imported by app/routes/auth.py).
|
||||
@@ -230,7 +229,7 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
|
||||
# request's access line, a queued row, a later background task would otherwise put rows back
|
||||
# under the id after this DELETE (logs.forget_user).
|
||||
logs.forget_user(user_id)
|
||||
db.execute(delete(LogEvent).where(LogEvent.user_id == user_id))
|
||||
db.execute(delete(LogEvent).where(owned_by(LogEvent.user_id, user_id)))
|
||||
# Every canonically-equal invite goes, not just the literal spelling: is_allowed matches on the
|
||||
# canonical form, so leaving a sibling row behind would keep granting access to a mailbox whose
|
||||
# account was just erased — and prod really does carry approved pairs for one mailbox.
|
||||
@@ -252,94 +251,63 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
|
||||
background.add_task(email_mod.send_account_deleted, email, lang, operator_contact())
|
||||
|
||||
|
||||
ACTIVE_ACCOUNT_HEADER = "x-siftlode-account"
|
||||
def session_user_id(conn: HTTPConnection) -> int | None:
|
||||
"""The account this browser session signed in as, read from the signed cookie alone (no DB).
|
||||
One account per session: signing in as another replaces it (`routes.auth._establish_session`)."""
|
||||
return conn.session.get("user_id")
|
||||
|
||||
|
||||
def resolved_user_id(conn: HTTPConnection, *, strict: bool = True) -> tuple[int | None, bool]:
|
||||
"""Which account this connection acts as, and whether that's the session's default account.
|
||||
Takes an HTTPConnection rather than a Request: it reads only the session and the headers.
|
||||
|
||||
The signed cookie holds the *wallet* (`account_ids`, every account signed into this browser)
|
||||
plus a default `user_id`. A caller may name its account with the X-Siftlode-Account header (or
|
||||
`?account=`), but ONLY an account in the wallet — so a tab can't impersonate an account that
|
||||
never authenticated here. A connection that names no account acts as the cookie default.
|
||||
|
||||
A connection naming an account NOT in the wallet acts as nobody (`None`) — typically a tab still
|
||||
pinned to an account that was signed out in another tab. It must not fall back to the default:
|
||||
that tab would go on acting as an account it never showed, its player checkpoints and pref saves
|
||||
landing there and its Sign out signing THAT account out. Only the bootstrap /api/me probe passes
|
||||
`strict=False` and gets the default, which is how such a tab learns who it now is (the frontend
|
||||
re-pins it and reloads). An account whose session ended while it stayed in the wallet (deleted,
|
||||
suspended, revoked) leaves the wallet when it is refused (`_session_user`), so a tab pinned to it
|
||||
takes the same way back.
|
||||
"""
|
||||
default_id = conn.session.get("user_id")
|
||||
wallet = conn.session.get("account_ids") or []
|
||||
# Header for normal XHR; ?account= for a context that can't set headers (a plain <a> file
|
||||
# download). Both are wallet-gated below.
|
||||
hdr = conn.headers.get(ACTIVE_ACCOUNT_HEADER) or conn.query_params.get("account")
|
||||
if hdr:
|
||||
try:
|
||||
hid = int(hdr)
|
||||
except ValueError:
|
||||
# Under `strict` this becomes a 401 and the tab authenticates as nobody, which reads
|
||||
# in the browser exactly like a session that expired. The value itself stays out —
|
||||
# it came from a request, and `wallet` says which accounts were actually on offer.
|
||||
log.debug(
|
||||
"the active-account header could not be read as an id",
|
||||
extra=logs.event("auth.account_header.invalid", accounts=len(wallet)),
|
||||
)
|
||||
hid = None
|
||||
if hid is not None and hid in wallet:
|
||||
return hid, hid == default_id
|
||||
if strict:
|
||||
return None, False
|
||||
return default_id, True
|
||||
def stamp_session(request: Request, user: User) -> None:
|
||||
"""Record the account's current `session_epoch` in the signed cookie, so `current_user` can later
|
||||
reject the cookie once the epoch moves (password reset/change, or "log out everywhere")."""
|
||||
request.session["epoch"] = user.session_epoch or 0
|
||||
|
||||
|
||||
def drop_from_wallet(session: dict, user_id: int) -> None:
|
||||
"""Take one account out of this browser's wallet (`account_ids`); the session default is untouched."""
|
||||
session["account_ids"] = [a for a in (session.get("account_ids") or []) if a != user_id]
|
||||
# The account a tab was built for, sent with every request it makes (`identityHeaders`, the
|
||||
# frontend). The cookie alone says who is asking, and one browser holds one account: a sign-in as
|
||||
# someone else in another tab replaces it under every open tab. A tab still showing the previous
|
||||
# account would otherwise act as the new one — its player checkpoints, pref saves, Sign out and
|
||||
# Delete account landing there — until it noticed. It never selects an account; a mismatch is
|
||||
# refused, and the tab reloads as whoever is signed in now.
|
||||
EXPECTED_ACCOUNT_HEADER = "x-siftlode-expect-account"
|
||||
|
||||
|
||||
def _end_account_session(request: Request, user_id: int, is_default: bool) -> None:
|
||||
"""An account whose session here is no longer valid. Only nuke the whole session when it is the
|
||||
SESSION DEFAULT; a per-tab account just leaves the wallet (other tabs' default may still be valid).
|
||||
Leaving is what lets a tab pinned to it recover: its next /api/me names an account no longer signed
|
||||
in here and gets the default. Left in, it answered "signed out" — the landing page, with the default
|
||||
still signed in, and no way off it but closing the tab."""
|
||||
if is_default:
|
||||
request.session.clear()
|
||||
else:
|
||||
drop_from_wallet(request.session, user_id)
|
||||
def built_for_another_account(user_id: int, expected: str | None) -> bool:
|
||||
"""Whether a tab naming `expected` is showing another account than the one signed in. No name is
|
||||
no claim (a caller from before this check, a script): only a mismatch is refused, and the
|
||||
session stays — it is the other tab's sign-in, and it holds. The one comparison for every way a
|
||||
tab names its account: this header on a request, `expect` on a navigation."""
|
||||
if expected is None or expected == str(user_id):
|
||||
return False
|
||||
log.info(
|
||||
"a request from a tab built for another account was refused",
|
||||
extra=logs.event("auth.session.account_changed", user_id=user_id),
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def current_user(request: Request, db: Session = Depends(get_db)) -> User:
|
||||
return _session_user(request, db, *resolved_user_id(request))
|
||||
user = _session_user(request, db, session_user_id(request))
|
||||
if built_for_another_account(user.id, request.headers.get(EXPECTED_ACCOUNT_HEADER)):
|
||||
raise refused(401, "account_changed")
|
||||
return user
|
||||
|
||||
|
||||
def _session_user(request: Request, db: Session, user_id: int | None, is_default: bool) -> User:
|
||||
def _session_user(request: Request, db: Session, user_id: int | None) -> User:
|
||||
if not user_id:
|
||||
raise refused(401, "not_authenticated")
|
||||
user = db.get(User, user_id)
|
||||
if user is None or not user.is_active or user.is_suspended:
|
||||
# Suspended/deactivated mid-session → end it so the block takes effect at once.
|
||||
_end_account_session(request, user_id, is_default)
|
||||
request.session.clear()
|
||||
raise refused(401, "not_authenticated")
|
||||
# SA4: reject a cookie whose recorded epoch is behind the account's current one (revoked by a
|
||||
# password reset/change or "log out everywhere"). A cookie with no recorded epoch (pre-SA4, or an
|
||||
# account signed in before this shipped) is treated as epoch 0 — so the FIRST bump revokes it too.
|
||||
epochs = request.session.get("epochs") or {}
|
||||
if int(epochs.get(str(user_id), 0)) != (user.session_epoch or 0):
|
||||
_end_account_session(request, user_id, is_default)
|
||||
# A cookie whose epoch is behind the account's (revoked by a password reset/change or "log out
|
||||
# everywhere") is refused — and so is one with no epoch at all, which only a build before the
|
||||
# one-account session wrote: its holder signs in once more.
|
||||
epoch = request.session.get("epoch")
|
||||
if epoch is None or int(epoch) != (user.session_epoch or 0):
|
||||
request.session.clear()
|
||||
raise refused(401, "not_authenticated")
|
||||
# Always keep the session default in the switchable list — covers sessions created before
|
||||
# multi-session existed (their account_ids was never seeded), so the switcher isn't empty.
|
||||
default_id = request.session.get("user_id")
|
||||
accounts = request.session.get("account_ids") or []
|
||||
if default_id and default_id not in accounts:
|
||||
accounts.append(default_id)
|
||||
request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
|
||||
# Attribute this request's log lines to the account. Here, not in the request middleware:
|
||||
# this is the one place a session becomes an account, and the middleware runs outside both the
|
||||
# session middleware and this task's context. `request.state` is shared across middleware layers
|
||||
@@ -357,25 +325,17 @@ def _session_user(request: Request, db: Session, user_id: int | None, is_default
|
||||
def optional_current_user(request: Request, db: Session = Depends(get_db)) -> User | None:
|
||||
"""Like `current_user` but returns None instead of raising 401 when there's no valid session.
|
||||
Used by the bootstrap /api/me probe so the logged-out landing doesn't log a failed request to
|
||||
the browser console (a non-2xx fetch is a console error no matter how the app handles it).
|
||||
The one lenient resolution (`strict=False`): a tab naming an account no longer signed in here gets
|
||||
the default account, which is how it learns who it now is — see `resolved_user_id`. That includes an
|
||||
account refused right here, which `_session_user` has just taken out of the wallet."""
|
||||
user_id, is_default = resolved_user_id(request, strict=False)
|
||||
the browser console (a non-2xx fetch is a console error no matter how the app handles it)."""
|
||||
user_id = session_user_id(request)
|
||||
if user_id is None:
|
||||
return None
|
||||
try:
|
||||
return _session_user(request, db, user_id, is_default)
|
||||
# silent-ok: this is the anonymous case as much as the refused one — every landing page hit
|
||||
# comes through here. The second attempt below is the handling, and it reports.
|
||||
return _session_user(request, db, user_id)
|
||||
except HTTPException:
|
||||
if is_default:
|
||||
return None
|
||||
try:
|
||||
return _session_user(request, db, *resolved_user_id(request, strict=False))
|
||||
except HTTPException:
|
||||
# Both the pinned account AND the default one were refused, so the tab is signed out
|
||||
# without being told why — and it got here from a session that named a real account.
|
||||
# A session that named an account, refused: the browser is signed out without being told
|
||||
# why, so say it here.
|
||||
log.info(
|
||||
"a session was resolved to nobody: neither the named nor the default account held",
|
||||
"a session was refused: its account no longer holds",
|
||||
extra=logs.event("auth.session.probe_rejected", user_id=user_id),
|
||||
)
|
||||
return None
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
"""Catalog's tables: the shared YouTube world every user reads — channels and their videos — and
|
||||
each user's footprint in it: what they subscribe to, block, explored or found by search. Only this
|
||||
unit writes them; another unit reads them along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
Computed,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.models import User
|
||||
|
||||
|
||||
class Channel(Base, TimestampMixin):
|
||||
"""A YouTube channel. Shared across all users (one channel's videos are the same
|
||||
for everyone), so its expensive metadata is fetched and stored only once."""
|
||||
|
||||
__tablename__ = "channels"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(32), primary_key=True) # "UC..." channel id
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
handle: Mapped[str | None] = mapped_column(String(255))
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
uploads_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
subscriber_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
video_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Channel "About" metadata (fetched with the same channels.list call, no extra quota):
|
||||
# lifetime view count, the channel's join date, banner image, and external links.
|
||||
total_view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
banner_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
external_links: Mapped[list | None] = mapped_column(JSON)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
country: Mapped[str | None] = mapped_column(String(8))
|
||||
# Creator keyword tags (brandingSettings.channel.keywords) — a single space-separated string,
|
||||
# multi-word tags quoted; the client parses it into chips for the About tab.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
# Sync bookkeeping.
|
||||
details_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
recent_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_rss_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# When the newest uploads were last read through the Data API because the RSS feed would not
|
||||
# answer (`sync/runner.py` `run_feed_fallback`): the once-per-`rss_fallback_hours` clock.
|
||||
feed_fallback_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
backfill_cursor: Mapped[str | None] = mapped_column(String(128))
|
||||
backfill_done: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this channel only entered the catalog because a live YouTube search
|
||||
# surfaced one of its videos (nobody subscribes to it / it has no organic upload sync).
|
||||
# Lets the Library hide search-only "noise" by default.
|
||||
from_search: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
# True when this channel exists only because a user EXPLORED it (opened its page without
|
||||
# subscribing) — an ephemeral, per-explorer addition. The cleanup job hard-deletes such a
|
||||
# channel (and its via_explore videos) once nobody keeps it (no subscription, no live
|
||||
# ExploredChannel row, no per-user state) after a grace period. Cleared when someone
|
||||
# subscribes (the "keep" signal promotes it to organic).
|
||||
from_explore: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false", index=True
|
||||
)
|
||||
|
||||
videos: Mapped[list["Video"]] = relationship(back_populates="channel")
|
||||
subscriptions: Mapped[list["Subscription"]] = relationship(back_populates="channel")
|
||||
|
||||
|
||||
class Subscription(Base, TimestampMixin):
|
||||
"""Per-user link to a channel, with the user's own priority/visibility overrides."""
|
||||
|
||||
__tablename__ = "subscriptions"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# YouTube's subscription resource id, needed to unsubscribe via the API.
|
||||
yt_subscription_id: Mapped[str | None] = mapped_column(String(128))
|
||||
priority: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
hidden: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Per-user opt-in to full-history (deep) backfill of this channel. Default off so a
|
||||
# new user's unique channels don't trigger a big shared-quota burst; the deep
|
||||
# scheduler only picks up channels at least one user has requested.
|
||||
deep_requested: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
subscribed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
user: Mapped["User"] = relationship(back_populates="subscriptions")
|
||||
channel: Mapped["Channel"] = relationship(back_populates="subscriptions")
|
||||
|
||||
|
||||
# Video.live_status values that mean "currently a live/upcoming broadcast" — transient states
|
||||
# hidden from feeds/search and revisited by the refresh job until they settle into a VOD.
|
||||
LIVE_OR_UPCOMING = ("live", "upcoming")
|
||||
|
||||
|
||||
class Video(Base, TimestampMixin):
|
||||
"""A YouTube video. Shared across users; per-user state lives elsewhere."""
|
||||
|
||||
__tablename__ = "videos"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(16), primary_key=True) # video id
|
||||
# No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
|
||||
# (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
|
||||
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
|
||||
# sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
|
||||
# Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
|
||||
# Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
|
||||
channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
|
||||
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
# NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
|
||||
# forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
|
||||
# that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
|
||||
# PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
|
||||
# the real date); the ix_videos_published_at index stays for the `oldest` sort.
|
||||
published_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
|
||||
)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
|
||||
# The feed's views / duration / title sorts each ride a migration-defined index in their exact
|
||||
# ORDER BY, one per direction (and per ICU collation for title): migration 0096, held to
|
||||
# `feedquery.order_by` by test_feed_sort_indexes.
|
||||
duration_seconds: Mapped[int | None] = mapped_column(Integer)
|
||||
view_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
like_count: Mapped[int | None] = mapped_column(BigInteger)
|
||||
category_id: Mapped[int | None] = mapped_column(Integer)
|
||||
topic_categories: Mapped[list | None] = mapped_column(JSON)
|
||||
default_language: Mapped[str | None] = mapped_column(String(16))
|
||||
detected_language: Mapped[str | None] = mapped_column(String(16))
|
||||
# Creator-supplied keyword tags (snippet.tags joined by spaces) — fetched for free with the
|
||||
# snippet we already request. Indexed into search_vector so the feed search matches the
|
||||
# uploader's own keywords, not just words in the title.
|
||||
keywords: Mapped[str | None] = mapped_column(Text)
|
||||
# Accumulated distinct live-search queries that surfaced this video (across all users) —
|
||||
# folds YouTube's own relevance judgement into the local search: a video YouTube returned
|
||||
# for "ti amo magyarul" becomes findable by that query locally even if its title lacks it.
|
||||
search_terms: Mapped[str | None] = mapped_column(Text)
|
||||
# No btree here or on the flag columns below: migration 0069 (PERF-3) dropped the never-picked
|
||||
# boolean btrees and replaced the useful ones with partial indexes over the target subset
|
||||
# (via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — migration-defined, like the
|
||||
# search_vector GIN.
|
||||
is_short: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Whether the youtube.com/shorts/<id> probe has run for this video.
|
||||
shorts_probed: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video first entered the catalog via a live YouTube search (not an
|
||||
# organic channel/playlist sync). Used to hide search-discovered "noise" from the
|
||||
# Library by default; left False by the organic ingest paths.
|
||||
via_search: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# True when this video entered the catalog because a user EXPLORED its (un-subscribed)
|
||||
# channel. Like via_search it's hidden from the shared Library by default, but it's
|
||||
# additionally PER-USER private: a via_explore video is only visible to users who
|
||||
# explored or subscribe to its channel (a gate in the feed query), so one user's
|
||||
# curiosity never floods everyone else's catalog. Left False by the organic ingest paths.
|
||||
via_explore: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# none | live | upcoming | premiere | was_live
|
||||
live_status: Mapped[str] = mapped_column(
|
||||
String(16), default="none", server_default="none", index=True
|
||||
)
|
||||
enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# videos.list?part=status fields (populated during enrichment); used by the
|
||||
# maintenance/validation job to decide whether a video is playable anywhere.
|
||||
embeddable: Mapped[bool | None] = mapped_column(Boolean)
|
||||
privacy_status: Mapped[str | None] = mapped_column(String(16)) # public|unlisted|private
|
||||
upload_status: Mapped[str | None] = mapped_column(
|
||||
String(16)
|
||||
) # processed|failed|rejected|deleted
|
||||
# Maintenance/validation lifecycle. last_checked_at drives the rolling re-validation
|
||||
# (least-recently-checked first); unavailable_since marks a video as currently
|
||||
# unplayable (hidden from the feed immediately, hard-deleted after a grace period);
|
||||
# unavailable_reason records why (deleted|private|paywalled|abandoned|stuck_live).
|
||||
# last_checked_at: no plain btree — migration 0069 (PERF-18) replaced it with the partial
|
||||
# ix_videos_recheck_order (last_checked_at NULLS FIRST WHERE unavailable_since IS NULL) that the
|
||||
# revalidate job's exact ORDER BY rides; unavailable_since is a partial WHERE IS NOT NULL (PERF-3).
|
||||
last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
unavailable_reason: Mapped[str | None] = mapped_column(String(24))
|
||||
|
||||
# Weighted full-text search document (DB-generated, never written by the app): title (A) >
|
||||
# creator keywords + search queries (B) > truncated description (C). ts_rank honours the
|
||||
# weights so title matches outrank description matches. Backed by a GIN index (migration
|
||||
# 0032). The accent-insensitive `unaccent_simple` config comes from migration 0031.
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(keywords, '') || ' ' || "
|
||||
"coalesce(search_terms, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(description, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
channel: Mapped["Channel"] = relationship(back_populates="videos")
|
||||
|
||||
|
||||
class SearchFind(Base, TimestampMixin):
|
||||
"""Per-user record of a video the user surfaced via their own live YouTube search.
|
||||
|
||||
Lets "videos I searched for" appear in that user's own (Mine) feed and the Mine Source
|
||||
filter — distinct from the GLOBAL `Video.via_search` provenance, which marks a video as
|
||||
search-discovered by *anyone* and drives the shared Library's Source filter."""
|
||||
|
||||
__tablename__ = "search_finds"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_search_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
|
||||
class ExploredChannel(Base, TimestampMixin):
|
||||
"""Per-user record of a channel the user opened the page of WITHOUT subscribing.
|
||||
|
||||
Exploring an un-subscribed channel ingests its recent uploads (flagged via_explore) so the
|
||||
user can browse them, but keeps them out of everyone else's catalog. This row (a) gates the
|
||||
explorer's access to those ephemeral videos in the feed query and (b) drives the grace-period
|
||||
purge: a from_explore channel with no live ExploredChannel row (and no subscription / no
|
||||
per-user state on its videos) is hard-deleted by the cleanup job. Subscribing is the "keep"
|
||||
signal — it promotes the channel to organic and these rows become irrelevant. `created_at`
|
||||
(from TimestampMixin) is the grace clock; a re-visit refreshes it."""
|
||||
|
||||
__tablename__ = "explored_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_explored_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
|
||||
|
||||
class BlockedChannel(Base, TimestampMixin):
|
||||
"""Per-user channel blocklist. Videos from a blocked channel are excluded from this user's
|
||||
live YouTube search (and not ingested on their behalf) and hidden from their feed / explore
|
||||
views — a personal "never show me this channel again" that doesn't affect other users."""
|
||||
|
||||
__tablename__ = "blocked_channels"
|
||||
__table_args__ = (UniqueConstraint("user_id", "channel_id", name="uq_user_blocked_channel"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
@@ -0,0 +1,296 @@
|
||||
"""Catalog's reads: channels and videos, and each user's footprint in them (subscriptions, blocks,
|
||||
explorations), for the HTTP layer and for the core code that asks about them.
|
||||
|
||||
A user's row is read through `userscope`, so someone else's is the same answer as a missing one.
|
||||
Catalog does not read another unit's tables: what it needs from Reader (a playlist's channels, a
|
||||
user's tag links) it asks `reader.queries`.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Select, and_, case, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
Subscription,
|
||||
Video,
|
||||
)
|
||||
from app.collation import collated
|
||||
from app.models import User
|
||||
from app.sync.explore import followed_channel
|
||||
from app.userscope import owned_by
|
||||
|
||||
# --- channels --------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def channel(db: Session, channel_id: str) -> Channel | None:
|
||||
return db.get(Channel, channel_id)
|
||||
|
||||
|
||||
def channel_title(db: Session, channel_id: str) -> str | None:
|
||||
return db.scalar(select(Channel.title).where(Channel.id == channel_id))
|
||||
|
||||
|
||||
def channel_titles(db: Session, ids: set[str]) -> dict[str, str]:
|
||||
"""Each channel's title by id; a channel with no title yet is left out."""
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(ids)))
|
||||
return {cid: title for cid, title in rows.tuples() if title is not None}
|
||||
|
||||
|
||||
def channel_aggregates(channel_ids: list[str]) -> Select:
|
||||
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
|
||||
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
|
||||
alone; a test holds that (test_channel_list_index_only)."""
|
||||
return (
|
||||
select(
|
||||
Video.channel_id,
|
||||
func.count(Video.id),
|
||||
func.max(Video.published_at),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.where(Video.channel_id.in_(channel_ids))
|
||||
.group_by(Video.channel_id)
|
||||
)
|
||||
|
||||
|
||||
def aggregates(db: Session, channel_ids: list[str]) -> list[tuple[str, int, datetime | None, int]]:
|
||||
"""`channel_aggregates`, run: (channel id, stored videos, last upload, total seconds)."""
|
||||
if not channel_ids:
|
||||
return []
|
||||
return list(db.execute(channel_aggregates(channel_ids)).tuples())
|
||||
|
||||
|
||||
def video_counts(db: Session, channel_id: str) -> tuple[int, int, int]:
|
||||
"""A channel's stored videos, its Shorts and its live/upcoming ones, in one grouped pass."""
|
||||
n_all, n_short, n_live = db.execute(
|
||||
select(
|
||||
func.count(Video.id),
|
||||
func.count(Video.id).filter(Video.is_short.is_(True)),
|
||||
func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
|
||||
).where(Video.channel_id == channel_id)
|
||||
).one()
|
||||
return int(n_all or 0), int(n_short or 0), int(n_live or 0)
|
||||
|
||||
|
||||
def deep_queued(db: Session, channel_ids: list[str]) -> set[str]:
|
||||
"""Which of these channels at least one user (any user) has asked full history for."""
|
||||
if not channel_ids:
|
||||
return set()
|
||||
return set(
|
||||
db.execute(
|
||||
select(Subscription.channel_id)
|
||||
.where(
|
||||
Subscription.channel_id.in_(channel_ids),
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.distinct()
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def followed(db: Session, channel_id: str) -> bool:
|
||||
"""Whether anyone subscribes to the channel."""
|
||||
return (
|
||||
db.execute(
|
||||
select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
# --- a user's subscriptions, blocks and explorations ----------------------------------------
|
||||
|
||||
|
||||
def subscriptions(db: Session, user: User, lang: str) -> list[tuple[Subscription, Channel]]:
|
||||
"""The user's subscriptions with their channels, highest priority first, then by title in the
|
||||
reader's alphabet."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Subscription, Channel)
|
||||
.join(Channel, Channel.id == Subscription.channel_id)
|
||||
.where(owned_by(Subscription.user_id, user))
|
||||
.order_by(Subscription.priority.desc(), collated(Channel.title, lang))
|
||||
).tuples()
|
||||
)
|
||||
|
||||
|
||||
def subscription(db: Session, user: User, channel_id: str) -> Subscription | None:
|
||||
return db.execute(
|
||||
select(Subscription).where(
|
||||
owned_by(Subscription.user_id, user), Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def subscribed_channels(user: User):
|
||||
"""The channel ids the user subscribes to, as a subquery."""
|
||||
return select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
|
||||
|
||||
|
||||
def exploration(db: Session, user: User, channel_id: str) -> ExploredChannel | None:
|
||||
return db.execute(
|
||||
select(ExploredChannel).where(
|
||||
owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def is_blocked(db: Session, user: User, channel_id: str) -> bool:
|
||||
return (
|
||||
db.execute(
|
||||
select(BlockedChannel.id).where(
|
||||
owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def blocked_ids(db: Session, user: User) -> set[str]:
|
||||
return set(
|
||||
db.scalars(select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user)))
|
||||
)
|
||||
|
||||
|
||||
def blocked_channels(db: Session, user: User, lang: str) -> list[Channel]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel)
|
||||
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
|
||||
.where(owned_by(BlockedChannel.user_id, user))
|
||||
.order_by(collated(Channel.title, lang))
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- sync progress ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def catalog_totals(db: Session, user: User) -> dict:
|
||||
"""The instance's catalog size and this user's subscription count (the sync status card)."""
|
||||
return {
|
||||
"subscriptions": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Subscription)
|
||||
.where(owned_by(Subscription.user_id, user))
|
||||
),
|
||||
"channels_total": db.scalar(select(func.count()).select_from(Channel)),
|
||||
"channels_backfilling": db.scalar(
|
||||
select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
|
||||
),
|
||||
"videos_total": db.scalar(select(func.count()).select_from(Video)),
|
||||
"pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def my_progress(db: Session, user: User) -> tuple[int, int, int, int]:
|
||||
"""Over the channels this user subscribes to: how many, how many have details, how many their
|
||||
recent uploads, how many their whole history."""
|
||||
total, details_synced, recent_synced, all_deep_done = db.execute(
|
||||
select(
|
||||
func.count(Channel.id),
|
||||
func.count(Channel.details_synced_at),
|
||||
func.count(Channel.recent_synced_at),
|
||||
func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
|
||||
)
|
||||
.select_from(Channel)
|
||||
.join(
|
||||
Subscription,
|
||||
and_(Subscription.channel_id == Channel.id, owned_by(Subscription.user_id, user)),
|
||||
)
|
||||
).one()
|
||||
return total or 0, details_synced or 0, recent_synced or 0, int(all_deep_done or 0)
|
||||
|
||||
|
||||
def my_deep_requested(db: Session, user: User) -> list[Channel]:
|
||||
"""The channels this user asked full history for."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Channel.id,
|
||||
owned_by(Subscription.user_id, user),
|
||||
Subscription.deep_requested.is_(True),
|
||||
),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def my_video_count(db: Session, user: User) -> int:
|
||||
"""The videos of the channels this user subscribes to and has not hidden."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(Video.id)).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
owned_by(Subscription.user_id, user),
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def video_total(db: Session) -> int:
|
||||
return db.scalar(select(func.count(Video.id))) or 0
|
||||
|
||||
|
||||
def deep_backlog(db: Session) -> list[Channel]:
|
||||
"""Every channel someone asked full history for that does not have it yet."""
|
||||
deep_requested = (
|
||||
select(Subscription.id)
|
||||
.where(Subscription.channel_id == Channel.id, Subscription.deep_requested.is_(True))
|
||||
.exists()
|
||||
)
|
||||
return list(
|
||||
db.execute(
|
||||
select(Channel).where(Channel.backfill_done.is_(False), deep_requested)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def work_queue(db: Session) -> dict:
|
||||
"""What the catalog's jobs still have queued, instance-wide (the scheduler card)."""
|
||||
return {
|
||||
# The same selection run_recent_backfill works through, or the card counts channels the
|
||||
# job will never touch and never reaches zero.
|
||||
"channels_recent_pending": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Channel)
|
||||
.where(Channel.recent_synced_at.is_(None), followed_channel())
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_shorts": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_live_refresh": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(
|
||||
or_(
|
||||
Video.live_status.in_(("live", "upcoming")),
|
||||
and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
|
||||
)
|
||||
)
|
||||
)
|
||||
or 0,
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Live YouTube search, the catalog's side: what a page of results leaves in the shared catalog.
|
||||
|
||||
A result becomes a catalog row so it renders with the normal feed cards and the in-app player and
|
||||
gains per-user state. The ingest runs in steps with a YouTube call between each, and each step's
|
||||
rows are committed before the next call: an enrich that fails rolls back to the stubs, which stay
|
||||
(the FK targets for what follows, backfilled later). Results are flagged (`videos.via_search`,
|
||||
`channels.from_search`) so the Library can hide this search-discovered noise; Shorts and
|
||||
live/upcoming videos are never kept. The HTTP layer (`routes/search.py`) runs the paging, the caps
|
||||
and the refusals.
|
||||
"""
|
||||
|
||||
from collections.abc import Sequence
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete, func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, quota, sysconfig
|
||||
from app.catalog import queries
|
||||
from app.catalog.models import LIVE_OR_UPCOMING, Channel, SearchFind, Subscription, Video
|
||||
from app.models import User
|
||||
from app.reader import queries as reader_queries
|
||||
from app.sync.subscriptions import apply_channel_details
|
||||
from app.sync.videos import apply_video_details, insert_stubs, parse_dt
|
||||
from app.userscope import owned_by
|
||||
from app.utils import like_escape
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def classify_shorts(db: Session, videos: Sequence[Video]) -> None:
|
||||
"""Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
|
||||
|
||||
Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
|
||||
synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
|
||||
duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
|
||||
quota). Undetermined probes are left for the scheduler to retry."""
|
||||
probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
|
||||
to_probe: list[Video] = []
|
||||
for v in videos:
|
||||
if v.shorts_probed:
|
||||
continue
|
||||
if v.enriched_at is None:
|
||||
# Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
|
||||
# so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
|
||||
# (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
|
||||
continue
|
||||
if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
|
||||
v.is_short = False
|
||||
v.shorts_probed = True
|
||||
else:
|
||||
to_probe.append(v)
|
||||
if to_probe:
|
||||
with make_client() as client:
|
||||
|
||||
def work(v: Video):
|
||||
return v, probe_is_short(client, v.id)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
for v, result in pool.map(work, to_probe):
|
||||
if result is None:
|
||||
continue
|
||||
v.is_short = result
|
||||
v.shorts_probed = True
|
||||
db.commit()
|
||||
|
||||
|
||||
def ingest_candidates(
|
||||
db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
|
||||
) -> list[str]:
|
||||
"""Materialise one page of search candidates into the catalog (channel stubs + enrich,
|
||||
video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
|
||||
video ids — non-Short, non-live — in the input relevance order."""
|
||||
ids = [it["id"] for it in candidates]
|
||||
if not ids:
|
||||
return []
|
||||
|
||||
# 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
|
||||
# then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
|
||||
channel_ids = list({it["channel_id"] for it in candidates})
|
||||
existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
|
||||
new_channel_ids = [c for c in channel_ids if c not in existing]
|
||||
if new_channel_ids:
|
||||
titles = {it["channel_id"]: it["channel_title"] for it in candidates}
|
||||
for cid in new_channel_ids:
|
||||
db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
|
||||
db.commit()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
|
||||
details = yt.get_channels(new_channel_ids)
|
||||
apply_channel_details(db, details, new_channel_ids)
|
||||
db.commit()
|
||||
except YouTubeError:
|
||||
# The stubs remain and the details can be backfilled later — but until they are, those
|
||||
# channels sit in the results with no thumbnail and no subscriber count, looking like
|
||||
# channels that have neither.
|
||||
log.warning(
|
||||
"the channels found by a search could not be enriched",
|
||||
extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
|
||||
)
|
||||
db.rollback() # details can be backfilled later; the stubs remain
|
||||
|
||||
# 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
|
||||
# NOTHING), so an organically-synced video stays organic.
|
||||
# published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
|
||||
# (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
|
||||
# enrich in step 3 overwrites it with the video's real snippet date.
|
||||
now = datetime.now(timezone.utc)
|
||||
insert_stubs(
|
||||
db,
|
||||
[
|
||||
{
|
||||
"id": it["id"],
|
||||
"channel_id": it["channel_id"],
|
||||
"title": it["title"],
|
||||
"published_at": parse_dt(it["published_at"]) or now,
|
||||
"thumbnail_url": it["thumbnail_url"],
|
||||
"via_search": True,
|
||||
}
|
||||
for it in candidates
|
||||
],
|
||||
)
|
||||
|
||||
# 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
|
||||
videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
|
||||
detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
|
||||
except YouTubeError:
|
||||
# Every candidate then stays unenriched, and an unenriched video cannot be judged live or
|
||||
# Short — so it is dropped. A working search returns nothing and reads as "no results".
|
||||
log.warning(
|
||||
"the videos found by a search could not be enriched",
|
||||
extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
|
||||
)
|
||||
detail_by_id = {}
|
||||
enriched_at = datetime.now(
|
||||
timezone.utc
|
||||
) # when THIS enrich ran (distinct from the stub placeholder)
|
||||
for v in videos:
|
||||
item = detail_by_id.get(v.id)
|
||||
if item is None:
|
||||
continue
|
||||
apply_video_details(v, item)
|
||||
v.enriched_at = enriched_at
|
||||
db.commit()
|
||||
|
||||
# 4) Confirm/deny Shorts (no quota) so none enter via search.
|
||||
classify_shorts(db, videos)
|
||||
|
||||
keep = {
|
||||
v.id
|
||||
for v in videos
|
||||
if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
|
||||
}
|
||||
return [vid for vid in ids if vid in keep]
|
||||
|
||||
|
||||
def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
|
||||
"""Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
|
||||
|
||||
"Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
|
||||
video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
|
||||
wildcard — either way the term was never folded in, and local search lost that relevance."""
|
||||
padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.id.in_(video_ids))
|
||||
.where(padded.notilike(f"% {like_escape(term)} %"))
|
||||
.values(
|
||||
search_terms=func.trim(
|
||||
func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def new_first(db: Session, user: User, ids: list[str]) -> list[str]:
|
||||
"""Reorder results so genuinely-new discoveries come first and things the user already has
|
||||
(a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
|
||||
to the back — keeping the relevance order within each group."""
|
||||
if not ids:
|
||||
return ids
|
||||
have = set(
|
||||
db.scalars(
|
||||
select(Video.id).where(
|
||||
Video.id.in_(ids), Video.channel_id.in_(queries.subscribed_channels(user))
|
||||
)
|
||||
)
|
||||
) | reader_queries.held_by(db, user, ids)
|
||||
return [i for i in ids if i not in have] + [i for i in ids if i in have]
|
||||
|
||||
|
||||
def record_finds(db: Session, user: User, video_ids: list[str], term: str) -> None:
|
||||
"""These results as THIS user's search finds (the Mine feed, the Source filter), and the term
|
||||
folded into each one's `search_terms` so local full-text search inherits YouTube's relevance.
|
||||
Idempotent."""
|
||||
db.execute(
|
||||
pg_insert(SearchFind)
|
||||
.values([{"user_id": user.id, "video_id": vid} for vid in video_ids])
|
||||
.on_conflict_do_nothing(index_elements=["user_id", "video_id"])
|
||||
)
|
||||
_fold_term(db, video_ids, term)
|
||||
|
||||
|
||||
def clear(db: Session, user: User, video_ids: list[str]) -> int:
|
||||
"""Discard these results "as if never added": drop this user's finds of them, then delete the
|
||||
ones nobody kept — a `via_search` video no reader holds (`reader.queries.kept_by_a_reader`),
|
||||
whose channel nobody subscribes to, and that no other user still has as a find. Returns how
|
||||
many videos went."""
|
||||
db.execute(
|
||||
delete(SearchFind).where(
|
||||
owned_by(SearchFind.user_id, user), SearchFind.video_id.in_(video_ids)
|
||||
)
|
||||
)
|
||||
subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
result = db.execute(
|
||||
delete(Video).where(
|
||||
Video.id.in_(video_ids),
|
||||
Video.via_search.is_(True),
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~subscribed,
|
||||
~other_find,
|
||||
)
|
||||
)
|
||||
return result.rowcount if isinstance(result, CursorResult) else 0
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Catalog's writes a request makes: a user's subscription settings, what they explore or block, a
|
||||
subscription recorded or dropped, a channel's sync markers reset.
|
||||
|
||||
Every function here flushes and leaves the commit to its caller (the handler). The YouTube side of a
|
||||
subscribe or an unsubscribe is the handler's (it needs the user's client and its refusals); what
|
||||
lands here is the local record of it. The ingest jobs (`sync/`, `catalog/search`) write the
|
||||
catalog's shared rows themselves.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete, update
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.catalog import queries
|
||||
from app.catalog.models import BlockedChannel, Channel, ExploredChannel, Subscription, Video
|
||||
from app.models import User
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
def remember_own_channel(user: User, channel_id: str) -> None:
|
||||
"""The user's own YouTube channel, so discovery never offers it to them."""
|
||||
user.yt_channel_id = channel_id
|
||||
|
||||
|
||||
def update_subscription(
|
||||
db: Session,
|
||||
sub: Subscription,
|
||||
*,
|
||||
priority: int | None = None,
|
||||
hidden: bool | None = None,
|
||||
deep_requested: bool | None = None,
|
||||
) -> bool:
|
||||
"""Change the settings given (None leaves one as it is). Returns whether full history was just
|
||||
turned ON — the caller then pulls the channel now rather than at the scheduler's next tick."""
|
||||
turned_on = False
|
||||
if priority is not None:
|
||||
sub.priority = priority
|
||||
if hidden is not None:
|
||||
sub.hidden = hidden
|
||||
if deep_requested is not None:
|
||||
# Turning it off won't undo already-fetched videos; it stops further deep paging if nobody
|
||||
# else still wants it.
|
||||
turned_on = deep_requested and not sub.deep_requested
|
||||
sub.deep_requested = deep_requested
|
||||
db.flush()
|
||||
return turned_on
|
||||
|
||||
|
||||
def set_deep_all(db: Session, user: User, on: bool) -> int:
|
||||
"""Every one of the user's channels in (or out of) full-history backfill; how many rows moved."""
|
||||
result = db.execute(
|
||||
update(Subscription).where(owned_by(Subscription.user_id, user)).values(deep_requested=on)
|
||||
)
|
||||
return result.rowcount if isinstance(result, CursorResult) else 0
|
||||
|
||||
|
||||
def explore(db: Session, user: User, channel: Channel) -> None:
|
||||
"""Record (or refresh the grace clock of) the user's exploration of a channel, and flag the
|
||||
channel ephemeral unless someone follows it — the cleanup job's keep rule reads both."""
|
||||
existing = queries.exploration(db, user, channel.id)
|
||||
if existing is None:
|
||||
db.add(ExploredChannel(user_id=user.id, channel_id=channel.id))
|
||||
else:
|
||||
existing.created_at = datetime.now(timezone.utc)
|
||||
if not channel.from_explore and not queries.followed(db, channel.id):
|
||||
channel.from_explore = True
|
||||
db.flush()
|
||||
|
||||
|
||||
def reset_backfill(db: Session, channel: Channel) -> None:
|
||||
"""Fetch the channel again from scratch: its backfill markers cleared, and full history asked
|
||||
for on EVERY subscription to it — the deep scheduler gates on an EXISTS over those, so the admin
|
||||
who resets it need not subscribe themselves."""
|
||||
channel.backfill_done = False
|
||||
channel.backfill_cursor = None
|
||||
channel.recent_synced_at = None
|
||||
db.execute(
|
||||
update(Subscription)
|
||||
.where(Subscription.channel_id == channel.id)
|
||||
.values(deep_requested=True)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def record_subscription(db: Session, user: User, channel: Channel, yt_subscription_id: str) -> None:
|
||||
"""The subscription YouTube now holds, recorded here. Subscribing is the "keep" signal: an
|
||||
explored channel becomes an organic one, so the cleanup job leaves it alone and its videos join
|
||||
the shared Library like any subscribed channel's uploads."""
|
||||
db.add(
|
||||
Subscription(
|
||||
user_id=user.id,
|
||||
channel_id=channel.id,
|
||||
yt_subscription_id=yt_subscription_id or None,
|
||||
subscribed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
if channel.from_explore:
|
||||
channel.from_explore = False
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.channel_id == channel.id, Video.via_explore.is_(True))
|
||||
.values(via_explore=False)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def drop_subscription(db: Session, sub: Subscription) -> None:
|
||||
db.delete(sub)
|
||||
db.flush()
|
||||
|
||||
|
||||
def block(db: Session, user: User, channel_id: str) -> None:
|
||||
"""Block a channel for this user (idempotent), and end any exploration of it so the cleanup job
|
||||
can reclaim it."""
|
||||
if queries.is_blocked(db, user, channel_id):
|
||||
return
|
||||
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
|
||||
db.execute(
|
||||
delete(ExploredChannel).where(
|
||||
owned_by(ExploredChannel.user_id, user), ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def unblock(db: Session, user: User, channel_id: str) -> None:
|
||||
db.execute(
|
||||
delete(BlockedChannel).where(
|
||||
owned_by(BlockedChannel.user_id, user), BlockedChannel.channel_id == channel_id
|
||||
)
|
||||
)
|
||||
@@ -26,6 +26,7 @@ from uuid import uuid4
|
||||
from sqlalchemy import and_, delete, func, not_, select, update
|
||||
from sqlalchemy.orm import Session, defer
|
||||
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app import diagnostics
|
||||
from app.models import DebugScreenshot, DebugSession, LogEvent, User
|
||||
|
||||
@@ -76,7 +77,9 @@ def open_session(db: Session, user: User) -> DebugSession | None:
|
||||
|
||||
def _unclosed(db: Session, user: User) -> DebugSession | None:
|
||||
return db.scalar(
|
||||
select(DebugSession).where(DebugSession.user_id == user.id, DebugSession.ended_at.is_(None))
|
||||
select(DebugSession).where(
|
||||
owned_by(DebugSession.user_id, user), DebugSession.ended_at.is_(None)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -126,12 +129,8 @@ def from_header(db: Session, user: User, value: str | None) -> DebugSession | No
|
||||
"""The session a request names, if it is the caller's own and still live; else None."""
|
||||
if not value or len(value) != _ID_LEN or not all(c in "0123456789abcdef" for c in value):
|
||||
return None
|
||||
session = db.get(DebugSession, value)
|
||||
if (
|
||||
session is None
|
||||
or session.user_id != user.id
|
||||
or not is_live(session, datetime.now(timezone.utc))
|
||||
):
|
||||
session = owned_row(db, DebugSession, value, user)
|
||||
if session is None or not is_live(session, datetime.now(timezone.utc)):
|
||||
return None
|
||||
return session
|
||||
|
||||
@@ -354,7 +353,7 @@ def stored_screenshot_bytes(db: Session, user_id: int) -> int:
|
||||
db.scalar(
|
||||
select(func.coalesce(func.sum(DebugScreenshot.size), 0))
|
||||
.join(DebugSession, DebugSession.id == DebugScreenshot.session_id)
|
||||
.where(DebugSession.user_id == user_id)
|
||||
.where(owned_by(DebugSession.user_id, user_id))
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
@@ -22,7 +22,8 @@ to the client, which has both the native payload and the translations.
|
||||
"""
|
||||
|
||||
from app.discover.budget import remaining_seconds
|
||||
from app.models import Channel, Playlist
|
||||
from app.catalog.models import Channel
|
||||
from app.reader.models import Playlist
|
||||
|
||||
# Plex artwork is a poster for the things you browse and a landscape still for the things you play.
|
||||
_POSTER_KINDS = ("movie", "show")
|
||||
|
||||
@@ -22,10 +22,14 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.discover import cards, queries
|
||||
from app.models import Channel, ComfortPin, PlexItem, PlexShow, Playlist, User, Video
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import ComfortPin, Playlist
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import SHOW_NEW, leaf_card, show_card
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.userscope import owned_by, owned_row
|
||||
|
||||
# kind -> the ComfortPin column that holds it. The client's vocabulary and the schema's, in one place.
|
||||
PIN_COLUMNS: dict[str, str] = {
|
||||
@@ -68,11 +72,11 @@ def resolve_ref(db: Session, user: User, kind: str, ref: str) -> tuple[str, obje
|
||||
# such pin target", which is the same answer this function gives for one that does not exist.
|
||||
except ValueError:
|
||||
return None
|
||||
pl = db.get(Playlist, pid)
|
||||
# A playlist is per-user, unlike the other four (which are shared catalog rows whose PRIVACY
|
||||
# lives in the state tables). Someone else's list must read as "no such thing", not as a 403 —
|
||||
# the same reason `owned_or_404` 404s rather than 403s.
|
||||
return (col, pl.id) if pl is not None and pl.user_id == user.id else None
|
||||
pl = owned_row(db, Playlist, pid, user)
|
||||
return (col, pl.id) if pl is not None else None
|
||||
|
||||
|
||||
def reorder(db: Session, user: User, ids: list[int]) -> None:
|
||||
@@ -92,7 +96,7 @@ def reorder(db: Session, user: User, ids: list[int]) -> None:
|
||||
# old slots — and one global lock order means two reorders cannot deadlock.
|
||||
locked = db.execute(
|
||||
select(ComfortPin)
|
||||
.where(ComfortPin.user_id == user.id)
|
||||
.where(owned_by(ComfortPin.user_id, user.id))
|
||||
.order_by(ComfortPin.id)
|
||||
.with_for_update()
|
||||
.execution_options(populate_existing=True)
|
||||
@@ -117,7 +121,7 @@ def list_pins(db: Session, user: User) -> list[ComfortPin]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(ComfortPin)
|
||||
.where(ComfortPin.user_id == user.id)
|
||||
.where(owned_by(ComfortPin.user_id, user.id))
|
||||
.order_by(ComfortPin.position.asc(), ComfortPin.id.asc())
|
||||
)
|
||||
.scalars()
|
||||
@@ -132,17 +136,44 @@ def next_position(db: Session, user: User) -> int:
|
||||
deletes the row, and every FK cascade silently removes a pin when the thing it points at goes
|
||||
away — so past the first hole a count under-counts and would drop the new pin into the MIDDLE of
|
||||
the shelf, or onto a position another pin already holds (nothing in the schema forbids that).
|
||||
Same idiom as `routes/playlists._next_playlist_position`."""
|
||||
Same idiom as a new playlist's place in the rail (`reader.service`)."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(ComfortPin.position), -1)).where(
|
||||
ComfortPin.user_id == user.id
|
||||
owned_by(ComfortPin.user_id, user.id)
|
||||
)
|
||||
)
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def own_pin(db: Session, user: User, pin_id: int) -> ComfortPin | None:
|
||||
return owned_row(db, ComfortPin, pin_id, user)
|
||||
|
||||
|
||||
def pin_of(db: Session, user: User, column: str, value: object) -> ComfortPin | None:
|
||||
"""The user's pin of this thing (`column`/`value` from `resolve_ref`), if it is on the shelf."""
|
||||
return db.execute(
|
||||
select(ComfortPin).where(
|
||||
owned_by(ComfortPin.user_id, user), getattr(ComfortPin, column) == value
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def add(db: Session, user: User, column: str, value: object) -> ComfortPin:
|
||||
"""A new pin at the end of the shelf. A second pin of the same thing (another tab, at the same
|
||||
moment) raises `IntegrityError` at the flush: the unique constraint is the guard."""
|
||||
pin = ComfortPin(user_id=user.id, position=next_position(db, user), **{column: value})
|
||||
db.add(pin)
|
||||
db.flush()
|
||||
return pin
|
||||
|
||||
|
||||
def remove(db: Session, pin: ComfortPin) -> None:
|
||||
db.delete(pin)
|
||||
db.flush()
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Batch:
|
||||
"""What the shelf resolves for every pinned channel and playlist in one go."""
|
||||
|
||||
@@ -6,7 +6,7 @@ Two rules shape this module:
|
||||
**Reuse the query layer, not the route.** The YouTube half calls `feedquery.filtered_query` /
|
||||
`sort_keys`, so the Home surface inherits the feed's own definitions of subscribed, blocked,
|
||||
explore-visible and in-progress for free, and cannot drift from them. The Plex half calls
|
||||
`plex.rollup` and the `plex.cards` serializers rather than `routes/plex/discover.py`: that route
|
||||
`plex.rollup` and the `plex.cards` serializers rather than `plex/discover.py`: that read-model
|
||||
narrows EVERY rail by the Plex sidebar's `LibraryFilters`, which is right for a surface that HAS a
|
||||
sidebar and wrong for a landing page that does not.
|
||||
|
||||
@@ -29,22 +29,17 @@ from app.feedquery import (
|
||||
serialize,
|
||||
sort_keys,
|
||||
)
|
||||
from app.models import (
|
||||
Channel,
|
||||
PlaylistItem,
|
||||
PlexItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader.models import PlaylistItem, VideoState
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import display_title, leaf_card, movie_card, show_card
|
||||
from app.plex.query import apply_show_state_filter, content_libs
|
||||
from app.userscope import owned_by
|
||||
|
||||
# How many rows a single rail may carry — Home's rails and the Plex module's Discover rails alike
|
||||
# (`routes/plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
|
||||
# (`plex/discover.py` imports it): the two surfaces should feel like the same app, and a rail
|
||||
# is a rail. One definition, because a comment was all that held two copies in step.
|
||||
RAIL_LIMIT = 20
|
||||
|
||||
@@ -98,7 +93,7 @@ def yt_in_progress(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[tup
|
||||
q = (
|
||||
select(*feed_columns(user, st), st.progress_updated_at.label("progressed_at"))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.join(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.join(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(
|
||||
st.position_seconds > 0,
|
||||
st.status.notin_(("watched", "hidden")),
|
||||
@@ -141,7 +136,7 @@ def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
|
||||
row = db.execute(
|
||||
select(*feed_columns(user, st))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(Video.id == video_id)
|
||||
).first()
|
||||
return serialize(row) if row is not None else None
|
||||
@@ -184,7 +179,7 @@ def playlists_summary(
|
||||
select(*feed_columns(user, st), PlaylistItem.playlist_id.label("_playlist"))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
|
||||
.outerjoin(st, and_(st.video_id == Video.id, owned_by(st.user_id, user)))
|
||||
.where(PlaylistItem.playlist_id.in_(playlist_ids))
|
||||
.order_by(PlaylistItem.playlist_id, PlaylistItem.position.asc())
|
||||
).all()
|
||||
@@ -209,7 +204,7 @@ def _unseen_movie_conds(uid: int):
|
||||
.select_from(PlexState)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
PlexState.user_id == uid,
|
||||
owned_by(PlexState.user_id, uid),
|
||||
or_(
|
||||
PlexState.status.in_(("watched", "hidden")),
|
||||
# Started counts as "not a surprise" too — see the note in `yt_surprise`.
|
||||
@@ -226,7 +221,7 @@ def plex_in_progress(
|
||||
) -> list[tuple[object, dict]]:
|
||||
"""In-progress movies + partially-watched shows as (progressed_at, card).
|
||||
|
||||
Mirrors `routes/plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
|
||||
Mirrors `plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
|
||||
NULL status still counts as started, so `notin_` alone would drop it), and the same shared
|
||||
`rollup.agg_subquery` + `apply_show_state_filter` for shows, so the two rails cannot disagree on
|
||||
what "in progress" means. What it does NOT inherit is that route's `LibraryFilters` narrowing:
|
||||
@@ -237,7 +232,9 @@ def plex_in_progress(
|
||||
if movie_libs:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_libs),
|
||||
@@ -331,7 +328,9 @@ def next_episode(db: Session, uid: int, show_id: int, *, lang: str) -> dict | No
|
||||
)
|
||||
base = (
|
||||
select(PlexItem, PlexState)
|
||||
.outerjoin(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.outerjoin(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
@@ -11,7 +11,8 @@ from sqlalchemy.orm import Session
|
||||
from app import sysconfig
|
||||
from app.discover import cards, queries
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.models import PlexShow, User
|
||||
from app.models import User
|
||||
from app.plex.models import PlexShow
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.rollup import NO_TS
|
||||
|
||||
@@ -75,7 +76,7 @@ def surprise(
|
||||
) -> list[dict]:
|
||||
"""The hero's draw pool: unseen things that fit the budget, from both sources under ONE seed.
|
||||
|
||||
Interleaved rather than concatenated — the same alternation `routes/plex/discover.py` uses for its
|
||||
Interleaved rather than concatenated — the same alternation `plex/discover.py` uses for its
|
||||
movie/show split — so a short pool never comes back all-YouTube or all-Plex just because one side
|
||||
happened to be queried first.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ from app.db import SessionLocal
|
||||
from app.downloads import adapters
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import formats, service
|
||||
from app.models import MediaAsset
|
||||
from app.downloads.models import MediaAsset
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
@@ -33,8 +33,9 @@ from app import logs
|
||||
from app.downloads import clip as clipmod
|
||||
from app.downloads import formats
|
||||
from app.downloads.adapters import _host_is
|
||||
from app.models import DownloadCredential, MediaAsset
|
||||
from app.downloads.models import DownloadCredential, MediaAsset
|
||||
from app.security import decrypt_optional, encrypt
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -157,7 +158,7 @@ def salt_for_enqueue(db: Session, user_id: int, url: str | None) -> str | None:
|
||||
def get_credential(db: Session, user_id: int, host_key: str) -> DownloadCredential | None:
|
||||
return db.execute(
|
||||
select(DownloadCredential).where(
|
||||
DownloadCredential.user_id == user_id, DownloadCredential.host_key == host_key
|
||||
owned_by(DownloadCredential.user_id, user_id), DownloadCredential.host_key == host_key
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
@@ -214,7 +215,7 @@ def status_list(db: Session, user_id: int) -> list[dict]:
|
||||
have = {
|
||||
c.host_key: c
|
||||
for c in db.execute(
|
||||
select(DownloadCredential).where(DownloadCredential.user_id == user_id)
|
||||
select(DownloadCredential).where(owned_by(DownloadCredential.user_id, user_id))
|
||||
).scalars()
|
||||
}
|
||||
out = []
|
||||
|
||||
@@ -32,7 +32,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import sysconfig
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DownloadJob, FileFolder, MediaAsset
|
||||
from app.downloads.models import DownloadJob, FileFolder, MediaAsset
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
class FolderKind(StrEnum):
|
||||
@@ -106,7 +107,9 @@ def source_key_for(db: Session, job: DownloadJob) -> str:
|
||||
def source_folder(db: Session, user_id: int, key: str) -> FileFolder:
|
||||
"""The user's source folder for `key`, made on first use. A race on the unique (user, key) is
|
||||
settled by re-selecting, inside a SAVEPOINT so the caller's own pending writes survive."""
|
||||
stmt = select(FileFolder).where(FileFolder.user_id == user_id, FileFolder.source_key == key)
|
||||
stmt = select(FileFolder).where(
|
||||
owned_by(FileFolder.user_id, user_id), FileFolder.source_key == key
|
||||
)
|
||||
folder = db.execute(stmt).scalar_one_or_none()
|
||||
if folder is not None:
|
||||
return folder
|
||||
@@ -131,7 +134,7 @@ def hub_folder(db: Session, user_id: int) -> FileFolder:
|
||||
"""The user's Hub folder, made on first use; a race on its one-per-user index is settled like
|
||||
`source_folder`'s."""
|
||||
stmt = select(FileFolder).where(
|
||||
FileFolder.user_id == user_id, FileFolder.kind == FolderKind.HUB
|
||||
owned_by(FileFolder.user_id, user_id), FileFolder.kind == FolderKind.HUB
|
||||
)
|
||||
folder = db.execute(stmt).scalar_one_or_none()
|
||||
if folder is not None:
|
||||
@@ -310,3 +313,33 @@ def delete_own(db: Session, folder: FileFolder) -> int:
|
||||
db.flush() # the files leave before the folder goes: its foreign key RESTRICTs the delete
|
||||
db.delete(folder)
|
||||
return len(jobs)
|
||||
|
||||
|
||||
def create_own(db: Session, user_id: int, name: str) -> FileFolder:
|
||||
"""A new folder of the user's own; FolderError("file_folder_name_taken") when they already have
|
||||
one by that name (the unique (user, lower(name)))."""
|
||||
folder = FileFolder(user_id=user_id, kind=FolderKind.OWN, name=name)
|
||||
try:
|
||||
with db.begin_nested():
|
||||
db.add(folder)
|
||||
db.flush()
|
||||
except IntegrityError as e:
|
||||
raise FolderError("file_folder_name_taken") from e
|
||||
return folder
|
||||
|
||||
|
||||
def renamable(folder: FileFolder) -> bool:
|
||||
"""Only a folder of the user's own making takes a name: a source folder's is its origin's."""
|
||||
return folder.kind == FolderKind.OWN
|
||||
|
||||
|
||||
def rename(db: Session, folder: FileFolder, name: str) -> None:
|
||||
"""Rename one of the user's own folders (a source folder's name is its origin's)."""
|
||||
if not renamable(folder):
|
||||
raise FolderError("file_folder_not_own")
|
||||
folder.name = name
|
||||
try:
|
||||
with db.begin_nested():
|
||||
db.flush()
|
||||
except IntegrityError as e:
|
||||
raise FolderError("file_folder_name_taken") from e
|
||||
|
||||
@@ -31,7 +31,7 @@ from app import apitokens, logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.downloads import quota, service, storage
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DownloadJob, MediaAsset
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
from app.notifications import create_notification
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -6,7 +6,7 @@ from sqlalchemy import text
|
||||
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import DownloadJob, MediaAsset
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
|
||||
|
||||
# --- small DB helpers (short-lived sessions; the download itself holds no transaction) -------
|
||||
|
||||
@@ -15,20 +15,24 @@ grant appended to the file URL, so the raw password never rides in the media URL
|
||||
import hashlib
|
||||
import hmac
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import update as sql_update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import security
|
||||
from app import audit, security
|
||||
from app.config import settings
|
||||
from app.downloads import storage
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
|
||||
from app.downloads.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
|
||||
from app.models import User
|
||||
from app.utils import iso
|
||||
from app.userscope import owned_by
|
||||
|
||||
_GRANT_TTL = 6 * 3600 # a watch session's grant is good for 6h, then the viewer re-unlocks
|
||||
# The most files a folder link lists: as many as the owner's own Files list shows (`routes.files.
|
||||
# MAX_LIST`; a test holds the two equal — this layer may not import that one).
|
||||
# The most files a folder link lists: as many as the owner's own Files list shows
|
||||
# (`queries.MAX_LIST`; a test holds the two equal — `queries` imports this module).
|
||||
FOLDER_MAX = 500
|
||||
|
||||
|
||||
@@ -59,7 +63,7 @@ def in_folder(folder: FileFolder) -> tuple:
|
||||
moved out stops opening through the link at its next request; one moved in starts."""
|
||||
return (
|
||||
DownloadJob.folder_id == folder.id,
|
||||
DownloadJob.user_id == folder.user_id,
|
||||
owned_by(DownloadJob.user_id, folder.user_id),
|
||||
DownloadJob.status == JobStatus.DONE,
|
||||
)
|
||||
|
||||
@@ -170,3 +174,117 @@ def public_meta(
|
||||
"file_url": file_url,
|
||||
"poster_url": poster_url,
|
||||
}
|
||||
|
||||
|
||||
# --- the owner's writes --------------------------------------------------------------------
|
||||
# Each flushes and records its audit row; the handler commits.
|
||||
|
||||
|
||||
def expiry(days: int | None) -> datetime | None:
|
||||
"""None/0/negative → no expiry; otherwise N days out, capped at ~10 years."""
|
||||
if not days or days <= 0:
|
||||
return None
|
||||
return datetime.now(timezone.utc) + timedelta(days=min(days, 3650))
|
||||
|
||||
|
||||
def audit_view(link: DownloadLink) -> dict:
|
||||
"""A link's settings as the audit records them: whether it has a password, never which."""
|
||||
return {
|
||||
"allow_download": link.allow_download,
|
||||
"has_password": link.password_hash is not None,
|
||||
"expires_at": iso(link.expires_at),
|
||||
}
|
||||
|
||||
|
||||
def create(
|
||||
db: Session,
|
||||
user: User,
|
||||
*,
|
||||
password: str,
|
||||
allow_download: bool,
|
||||
expires_days: int | None,
|
||||
**target: int,
|
||||
) -> tuple[DownloadLink, str]:
|
||||
"""A link to `target` (`job_id=` or `folder_id=`, already checked to be the user's), and the
|
||||
token it is opened by — which is in hand only now."""
|
||||
token = new_token()
|
||||
link = DownloadLink(
|
||||
**target,
|
||||
created_by=user.id,
|
||||
allow_download=allow_download,
|
||||
password_hash=security.hash_password(password) if password else None,
|
||||
expires_at=expiry(expires_days),
|
||||
)
|
||||
seal(link, token)
|
||||
db.add(link)
|
||||
db.flush()
|
||||
audit.record(
|
||||
db,
|
||||
user.id,
|
||||
audit.AuditAction.DOWNLOAD_LINK_CREATE,
|
||||
target_type="download_link",
|
||||
target_id=link.id,
|
||||
after={**target, **audit_view(link)},
|
||||
)
|
||||
return link, token
|
||||
|
||||
|
||||
def update(
|
||||
db: Session,
|
||||
user: User,
|
||||
link: DownloadLink,
|
||||
*,
|
||||
allow_download: bool | None = None,
|
||||
expires_days: int | None = None,
|
||||
password: str | None = None,
|
||||
sent: frozenset[str] = frozenset(),
|
||||
) -> None:
|
||||
"""Change what the caller sent (`sent` names the fields, so `expires_days=None` can mean "never
|
||||
expires"). Setting, clearing or changing the password bumps its version, so any grant issued
|
||||
under the previous one stops working now rather than living out its TTL."""
|
||||
before = audit_view(link)
|
||||
if "allow_download" in sent:
|
||||
link.allow_download = bool(allow_download)
|
||||
if "expires_days" in sent:
|
||||
link.expires_at = expiry(expires_days)
|
||||
if "password" in sent:
|
||||
pw = (password or "").strip()
|
||||
link.password_hash = security.hash_password(pw) if pw else None
|
||||
link.password_version += 1
|
||||
after = audit_view(link)
|
||||
if after != before or "password" in sent:
|
||||
audit.record(
|
||||
db,
|
||||
user.id,
|
||||
audit.AuditAction.DOWNLOAD_LINK_UPDATE,
|
||||
target_type="download_link",
|
||||
target_id=link.id,
|
||||
before=before,
|
||||
after=after,
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def revoke(db: Session, user: User, link: DownloadLink) -> None:
|
||||
link_id = link.id
|
||||
db.delete(link)
|
||||
audit.record(
|
||||
db,
|
||||
user.id,
|
||||
audit.AuditAction.DOWNLOAD_LINK_REVOKE,
|
||||
target_type="download_link",
|
||||
target_id=link_id,
|
||||
after={"job_id": link.job_id} if link.job_id is not None else {"folder_id": link.folder_id},
|
||||
)
|
||||
db.flush()
|
||||
|
||||
|
||||
def count_view(db: Session, link: DownloadLink) -> None:
|
||||
"""One more opening of the link. In SQL, not `+= 1` on the loaded row: two opens at once would
|
||||
both write the same count."""
|
||||
db.execute(
|
||||
sql_update(DownloadLink)
|
||||
.where(DownloadLink.id == link.id)
|
||||
.values(view_count=DownloadLink.view_count + 1)
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
|
||||
@@ -0,0 +1,385 @@
|
||||
"""Capture's tables: the downloaded files (`MediaAsset`), each user's jobs over them and the folders
|
||||
they sit in, the shares and public links that hand them on, a user's quota, cookies and format
|
||||
profiles, and where they stopped playing one. Only this unit writes them; another unit reads them
|
||||
along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import date, datetime, timezone
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
Date,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, validates
|
||||
|
||||
from app.db import Base
|
||||
from app.models import DISPLAY_TEXT_MAX, TimestampMixin, UpdatedAtMixin
|
||||
|
||||
|
||||
class LibraryState(Base):
|
||||
"""Where a user stopped in a library item (their own download, or one shared with them) — the
|
||||
library player's resume point. Keyed by the JOB, not the file: a job is the user's library entry,
|
||||
a re-download keeps it, and deleting the job deletes the row; the shared file under it may outlive
|
||||
both. A revoked share leaves its recipient's row behind, unread — only a job the user can still
|
||||
reach is ever read or written — and it is back if the item is shared again. Rows exist only for a
|
||||
position worth keeping (see `routes._common.kept_position`); no row means "from the start"."""
|
||||
|
||||
__tablename__ = "library_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "job_id", name="uq_library_user_job"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
job_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("download_jobs.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
|
||||
position_seconds: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
progress_updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
|
||||
class MediaAsset(Base, TimestampMixin):
|
||||
"""A physical downloaded media file — the SHARED cache layer of the download center.
|
||||
|
||||
Mirrors the app's "shared catalog, private per-user state" model: one row per unique
|
||||
``(source_kind, source_ref, format_sig)`` is the global cache. When a second user requests
|
||||
the same video with a format that hashes to the same `format_sig`, their `download_job`
|
||||
links to this ready asset instantly — no re-download, no cost. Only the SOURCE download is
|
||||
shared; per-user edited derivatives (phase 2) are never cached here.
|
||||
|
||||
`rel_path` is relative to `settings.files_root` (the mount), so the physical filename is
|
||||
system-decided and bound to the source id — never renamed (the user's custom name is display
|
||||
metadata on `download_job`, applied only when downloading to their own machine). The naming
|
||||
fields (`title`/`uploader`/`upload_date`) are denormalized so the Plex-tree path can be built
|
||||
uniformly for both catalog videos and ad-hoc URLs. `ref_count` (active jobs pointing here) frees
|
||||
the file with its last holder, and `last_access_at` orders the GC's LRU eviction. How long a file
|
||||
stays is not the asset's: each holder's job carries its own `expires_at` (its library folder's)."""
|
||||
|
||||
__tablename__ = "media_assets"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"source_kind", "source_ref", "format_sig", name="uq_media_asset_cache_key"
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
source_kind: Mapped[str] = mapped_column(String(16)) # "youtube" | "url"
|
||||
source_ref: Mapped[str] = mapped_column(String(512)) # video id, or full URL for ad-hoc
|
||||
format_sig: Mapped[str] = mapped_column(String(64)) # hash of the output-affecting spec
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(16), default="pending", server_default="pending", index=True
|
||||
) # "pending" | "ready" | "error"
|
||||
rel_path: Mapped[str | None] = mapped_column(Text) # relative to files_root
|
||||
storyboard_path: Mapped[str | None] = mapped_column(Text) # phase-2 filmstrip mosaic
|
||||
# A locally-generated poster frame (ffmpeg), set only when the source had NO thumbnail so the
|
||||
# card / watch page / link preview can still show an image. Relative to files_root.
|
||||
poster_path: Mapped[str | None] = mapped_column(Text)
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
duration_s: Mapped[int | None] = mapped_column(Integer)
|
||||
width: Mapped[int | None] = mapped_column(Integer)
|
||||
height: Mapped[int | None] = mapped_column(Integer)
|
||||
container: Mapped[str | None] = mapped_column(String(16))
|
||||
vcodec: Mapped[str | None] = mapped_column(String(32))
|
||||
acodec: Mapped[str | None] = mapped_column(String(32))
|
||||
# "video" | "audio" | "file" — the media-kind discriminator that drives which library actions
|
||||
# apply (play/edit for media; download-only for a generic uploaded file). Null while an asset is
|
||||
# still pending (kind is known once the file is probed at finalize). It is the single stored home
|
||||
# of the invariant `downloads.mediakind.derive` used to compute — never re-derived in prose (an
|
||||
# earlier ad-hoc `vcodec is None and acodec` check lived only in links.py; consolidated here).
|
||||
media_kind: Mapped[str | None] = mapped_column(String(16))
|
||||
nfo_written: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# Denormalized naming/metadata for the Plex tree + ad-hoc display.
|
||||
title: Mapped[str | None] = mapped_column(Text)
|
||||
uploader: Mapped[str | None] = mapped_column(String(255))
|
||||
# yt-dlp's channel/uploader page URL (channel_url|uploader_url) — lets an auto-filled channel
|
||||
# render as a clickable link. Null for sources without one (e.g. reddit) or older rows.
|
||||
uploader_url: Mapped[str | None] = mapped_column(Text)
|
||||
upload_date: Mapped[date | None] = mapped_column(Date)
|
||||
thumbnail_url: Mapped[str | None] = mapped_column(Text)
|
||||
# yt-dlp's canonical page URL for the source (webpage_url) — the clean "downloaded from"
|
||||
# reference shown in the Files library. Null until the worker extracts it (older/queued rows
|
||||
# fall back to a URL derived from source_kind+source_ref).
|
||||
source_webpage_url: Mapped[str | None] = mapped_column(Text)
|
||||
error: Mapped[str | None] = mapped_column(Text)
|
||||
ref_count: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
last_access_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# The file's SHA-256 (hex), so a machine fetching it can check what it got. Hashed while an upload
|
||||
# streams in, otherwise on the first ask (`routes.files`); NULL = not hashed yet.
|
||||
sha256: Mapped[str | None] = mapped_column(String(64))
|
||||
|
||||
@validates("rel_path")
|
||||
def _forget_hash(self, _key: str, rel_path: str | None) -> str | None:
|
||||
# A hash names the bytes at `rel_path`: a new file there (a conversion, a retry) is not them.
|
||||
self.sha256 = None
|
||||
return rel_path
|
||||
|
||||
|
||||
class DownloadProfile(Base, TimestampMixin):
|
||||
"""A named download preset — a reusable format profile.
|
||||
|
||||
`user_id` NULL + `is_builtin` marks the shipped presets (Best MP4, 1080p, audio-only, …);
|
||||
a user's saved custom profiles carry their `user_id`. `spec` is the format definition
|
||||
(mode av/v/a, max_height, container, audio_format, codec prefs, embed subs/chapters/
|
||||
thumbnail, sponsorblock) that `downloads.formats` turns into a yt-dlp selector + a cache
|
||||
`format_sig`."""
|
||||
|
||||
__tablename__ = "download_profiles"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), index=True
|
||||
) # NULL for a builtin preset
|
||||
name: Mapped[str] = mapped_column(String(80))
|
||||
spec: Mapped[dict] = mapped_column(JSON)
|
||||
is_builtin: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
sort_order: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
|
||||
|
||||
class FileFolder(Base, TimestampMixin):
|
||||
"""A folder of the Files library (`app.downloads.folders`): a row, never a directory.
|
||||
|
||||
A SOURCE folder (`source_key` set: youtube / plex / upload / site:<host>) is made the first time a
|
||||
user gets a file from that origin; an OWN folder (`name` set) is one the user made; the HUB folder
|
||||
(neither set, one per user) is where they swap files with a machine of theirs. Every file —
|
||||
a `DownloadJob` — sits in exactly one (`DownloadJob.folder_id`). `retention_days` is how long a
|
||||
file stays once done (NULL = until deleted); a file's own `expires_at` is stamped from it."""
|
||||
|
||||
__tablename__ = "file_folders"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("user_id", "source_key", name="uq_file_folder_source"),
|
||||
Index(
|
||||
"uq_file_folder_own_name",
|
||||
"user_id",
|
||||
func.lower(text("name")),
|
||||
unique=True,
|
||||
postgresql_where=text("name IS NOT NULL"),
|
||||
),
|
||||
# One Hub folder per user.
|
||||
Index("uq_file_folder_hub", "user_id", unique=True, postgresql_where=text("kind = 'hub'")),
|
||||
CheckConstraint(
|
||||
"(kind = 'source' AND source_key IS NOT NULL AND name IS NULL)"
|
||||
" OR (kind = 'own' AND name IS NOT NULL AND source_key IS NULL)"
|
||||
" OR (kind = 'hub' AND name IS NULL AND source_key IS NULL)",
|
||||
name="ck_file_folder_kind",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
kind: Mapped[str] = mapped_column(String(16)) # folders.FolderKind
|
||||
# "site:" + a host name (DNS caps one at 253 characters).
|
||||
source_key: Mapped[str | None] = mapped_column(String(260))
|
||||
name: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
|
||||
retention_days: Mapped[int | None] = mapped_column(Integer)
|
||||
|
||||
|
||||
def _ready_on_insert(context) -> datetime | None:
|
||||
"""A job inserted already done (an upload kept as it came) was ready the moment it was made; one
|
||||
that becomes done later is stamped then (`folders.stamp_done`, `service.finalize_if_running`)."""
|
||||
done = context.get_current_parameters().get("status") == "done"
|
||||
return datetime.now(timezone.utc) if done else None
|
||||
|
||||
|
||||
class DownloadJob(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A user's request to download one source — the per-user layer over `MediaAsset`.
|
||||
|
||||
The worker claims `queued` jobs with ``FOR UPDATE SKIP LOCKED`` and writes live
|
||||
`progress`/`speed_bps`/`eta_s`/`phase` here (the frontend polls via useLiveQuery: there is
|
||||
no server push channel, and the worker is a separate process anyway). `profile_snapshot`
|
||||
freezes the spec at enqueue so later edits/deletion of the profile don't change this job.
|
||||
`display_name` is the user's own name for the file (display + the Content-Disposition on
|
||||
local download); the on-disk asset is never renamed. `asset_id` links to the shared file
|
||||
(SET NULL if the asset is GC'd — the job then shows as expired).
|
||||
|
||||
Phase 2 (editor): `job_kind='edit'` jobs derive a new per-user clip from an already-downloaded
|
||||
ready job. They still hang off a `MediaAsset` (with `source_kind='edit'`, a per-user cache key)
|
||||
so all the file-serve/ref-count/GC/quota machinery is reused; the worker branches on the
|
||||
asset's `source_kind` and runs ffmpeg instead of yt-dlp. `source_job_id`/`source_asset_id`
|
||||
point at the parent download the clip was cut from; `edit_spec` is the (trim/crop) recipe."""
|
||||
|
||||
__tablename__ = "download_jobs"
|
||||
__table_args__ = (
|
||||
Index("ix_download_jobs_claim", "status", "queue_pos"),
|
||||
# Every way a job becomes done stamps when (`folders.stamp_done`, `finalize_if_running`).
|
||||
CheckConstraint(
|
||||
"status <> 'done' OR ready_at IS NOT NULL", name="ck_download_jobs_ready_at"
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
asset_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("media_assets.id", ondelete="SET NULL"), index=True
|
||||
)
|
||||
source_kind: Mapped[str] = mapped_column(String(16)) # "youtube" | "url"
|
||||
source_ref: Mapped[str] = mapped_column(String(512))
|
||||
# "download" (default), "edit" (a per-user trim/crop derivative of another job), "upload" (a
|
||||
# user's own file, converted by the worker when it is not phone-safe — see `downloads.upload`) or
|
||||
# "clip" (a range cut straight from a YouTube video or a Plex item — see `downloads.clip`).
|
||||
job_kind: Mapped[str] = mapped_column(String(16), default="download", server_default="download")
|
||||
# For edit jobs: the parent download this clip is cut from (SET NULL if the parent is deleted).
|
||||
source_job_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("download_jobs.id", ondelete="SET NULL")
|
||||
)
|
||||
source_asset_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("media_assets.id", ondelete="SET NULL")
|
||||
)
|
||||
# For edit jobs: the trim/crop recipe {trim?:{start_s,end_s}, crop?:{x,y,w,h}} in pixels. For clip
|
||||
# jobs: the clip spec {source, ref, start_s, end_s, audio_track?, crop?:{x,y,w,h,aspect}}, its crop
|
||||
# in fractions of the box it was drawn in (`downloads.clip.normalize`).
|
||||
edit_spec: Mapped[dict | None] = mapped_column(JSON)
|
||||
profile_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("download_profiles.id", ondelete="SET NULL")
|
||||
)
|
||||
profile_snapshot: Mapped[dict] = mapped_column(JSON)
|
||||
display_name: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
|
||||
# Per-user display overrides + extras (like display_name — metadata only, never the file).
|
||||
# `display_uploader`/`display_uploader_url` override the asset's auto channel name/link;
|
||||
# `extra_links` is an optional list of extra reference URLs shown on the card + watch page.
|
||||
display_uploader: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
|
||||
display_uploader_url: Mapped[str | None] = mapped_column(Text)
|
||||
extra_links: Mapped[list | None] = mapped_column(JSON)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(16), default="queued", server_default="queued", index=True
|
||||
) # queued | running | paused | done | error | canceled
|
||||
progress: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
speed_bps: Mapped[int | None] = mapped_column(BigInteger)
|
||||
eta_s: Mapped[int | None] = mapped_column(Integer)
|
||||
phase: Mapped[str | None] = mapped_column(String(32))
|
||||
error: Mapped[str | None] = mapped_column(Text) # raw yt-dlp/ffmpeg text (technical detail)
|
||||
error_code: Mapped[str | None] = mapped_column(
|
||||
String(24)
|
||||
) # category, see downloads.errors.classify
|
||||
# The host-specific failure-catalog entry on top of the category (`facebook.login_wall`), or NULL.
|
||||
error_detail: Mapped[str | None] = mapped_column(String(48)) # see downloads.errors.diagnose
|
||||
queue_pos: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
# The Files library: the folder this file sits in (`downloads.folders.place` at creation; an own
|
||||
# folder is emptied back into the source folders before it is deleted, hence RESTRICT), whether
|
||||
# the user keeps it past its folder's retention, and when it goes (stamped once done; NULL =
|
||||
# never). `expiry_warned` marks the one warning the GC sends before it deletes the file.
|
||||
folder_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("file_folders.id", ondelete="RESTRICT"), index=True
|
||||
)
|
||||
# The origin the file came from (the source folder's key), recorded where it is placed: an edit
|
||||
# traces its origin through its parent, and that parent may be deleted or expire later — the
|
||||
# file must still know which source folder it may go back to. NULL only for a file placed before
|
||||
# the column existed outside a source folder; `folders.source_key_for` works those out.
|
||||
source_key: Mapped[str | None] = mapped_column(String(260))
|
||||
kept: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
expiry_warned: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# When the file became done — what the library's "Added" means: a download that failed for days
|
||||
# and then went through is new to its user, so it sorts as new. `created_at` stays when it was
|
||||
# asked for (the queue's order). Set again when a file is made anew (a retry, a re-download).
|
||||
ready_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), default=_ready_on_insert
|
||||
)
|
||||
|
||||
|
||||
class DownloadShare(Base, TimestampMixin):
|
||||
"""A user→user grant: `to_user` may access `from_user`'s downloaded file (private by
|
||||
default). Since assets are already deduped/shared on disk, a share is just an ACL row."""
|
||||
|
||||
__tablename__ = "download_shares"
|
||||
__table_args__ = (UniqueConstraint("job_id", "to_user_id", name="uq_download_share"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
job_id: Mapped[int] = mapped_column(ForeignKey("download_jobs.id", ondelete="CASCADE"))
|
||||
from_user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
to_user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
|
||||
class DownloadQuota(Base, UpdatedAtMixin):
|
||||
"""Per-user download limits (admin-managed). A row exists only when an admin customizes a
|
||||
user; otherwise the sysconfig `download_default_*` values apply. `unlimited` bypasses the
|
||||
byte cap entirely (e.g. for the admin's own account)."""
|
||||
|
||||
__tablename__ = "download_quotas"
|
||||
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), primary_key=True
|
||||
)
|
||||
max_bytes: Mapped[int] = mapped_column(BigInteger)
|
||||
max_concurrent: Mapped[int] = mapped_column(Integer)
|
||||
max_jobs: Mapped[int] = mapped_column(Integer)
|
||||
unlimited: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class DownloadLink(Base, TimestampMixin):
|
||||
"""A public capability link to one download, or to one of its owner's own folders — the "share
|
||||
by link" surface.
|
||||
|
||||
Anyone holding the unguessable token can watch the file on the login-free `/watch/{token}`
|
||||
player page (Google-Drive style); a folder link opens the folder's list there, with what is in
|
||||
the folder at the time it is opened. The token itself is never stored as written: a link is
|
||||
found by its SHA-256 (`token_hash`), and `token_enc` is the Fernet-encrypted copy its owner
|
||||
copies the link from again — absent without a key (`downloads/links.py`). Optional per-link
|
||||
controls: `expires_at`, `allow_download` (stream-only vs downloadable), and a `password_hash`
|
||||
(argon2, like a user password). Revoke = delete the row. Distinct from `DownloadShare`, which
|
||||
grants a *registered* user access via the in-app "Shared with me" list."""
|
||||
|
||||
__tablename__ = "download_links"
|
||||
__table_args__ = (
|
||||
# A link opens exactly one thing: a file or a folder.
|
||||
CheckConstraint("(job_id IS NULL) <> (folder_id IS NULL)", name="ck_download_link_target"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
job_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("download_jobs.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
folder_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("file_folders.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
token_hash: Mapped[str] = mapped_column(String(64), unique=True)
|
||||
token_enc: Mapped[str | None] = mapped_column(Text) # Fernet ciphertext — never returned/logged
|
||||
created_by: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
allow_download: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
password_hash: Mapped[str | None] = mapped_column(Text)
|
||||
# Bumped whenever the password changes; folded into the signed watch grant so rotating the
|
||||
# password immediately invalidates any grants issued under the old one (instead of them living
|
||||
# out the 6h grant TTL).
|
||||
password_version: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
view_count: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
|
||||
|
||||
class DownloadCredential(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A user's stored login cookies for ONE auth-walled download host (Facebook, Instagram, …).
|
||||
|
||||
Per-user, per-host: each user pastes their OWN browser cookies.txt (Netscape format) for a host
|
||||
where they're logged in (e.g. a closed Facebook group they belong to), and a download of a
|
||||
yt-dlp-supported, auth-walled link on that host is retried with the ACTING user's cookie. The
|
||||
blob is Fernet-encrypted at rest (`cookies_enc`) and is NEVER returned by any API/log/audit —
|
||||
the routes are write-only (host + updated_at only). See `downloads.credentials`.
|
||||
|
||||
Two load-bearing security controls live around this row, NOT on it:
|
||||
* **cache privacy** — a cookie-authed download SALTS its MediaAsset `format_sig` with the
|
||||
user id, so the authed asset is private to that user and can never be cache-served to
|
||||
another user (`downloads.credentials.cookie_salt`).
|
||||
* **host scope (SEC-12)** — a cookie is attached ONLY when the download URL's netloc matches
|
||||
this row's `host_key` (`downloads.credentials.host_key_for_url`), so an arbitrary `?url=`
|
||||
can't exfiltrate the user's session to a look-alike host.
|
||||
|
||||
`host_key` is a short canonical key from the supported-host registry (e.g. "facebook"), not the
|
||||
raw netloc — one credential can cover several domains of the same site (facebook.com/fb.watch)."""
|
||||
|
||||
__tablename__ = "download_credentials"
|
||||
__table_args__ = (UniqueConstraint("user_id", "host_key", name="uq_download_credential"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
host_key: Mapped[str] = mapped_column(String(64))
|
||||
cookies_enc: Mapped[str] = mapped_column(Text) # Fernet ciphertext — never returned/logged
|
||||
@@ -16,7 +16,7 @@ from sqlalchemy.orm import Session
|
||||
from app.config import settings
|
||||
from app.downloads import links as linksmod
|
||||
from app.downloads import storage
|
||||
from app.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
|
||||
from app.downloads.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
|
||||
|
||||
_OG_BLOCK = re.compile(r"<!--OG:START-->.*?<!--OG:END-->", re.DOTALL)
|
||||
|
||||
|
||||
@@ -28,7 +28,9 @@ from app.downloads.jobdb import (
|
||||
)
|
||||
from app.downloads.stated import StatedFailure
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset, PlexItem
|
||||
from app.models import DISPLAY_TEXT_MAX
|
||||
from app.plex.models import PlexItem
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
from app.plex import lookups as plex_lookups
|
||||
from app.plex import paths as plex_paths
|
||||
from app.notifications import create_notification
|
||||
|
||||
@@ -0,0 +1,404 @@
|
||||
"""Capture's reads: what the library, the download tray, the share dialogs and the public watch pages
|
||||
show. Each returns rows or plain values and writes nothing; whose a row is goes through
|
||||
`userscope.owned_by` / `owned_row`, so a row of someone else's reads as a missing one.
|
||||
|
||||
The HTTP layer (`routes.downloads`, `routes.files`, `routes.public`) asks here and serializes; the
|
||||
writes are the service's (`service`, `folders`, `links`, `quota`)."""
|
||||
|
||||
from collections.abc import Sequence
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.apitokens import FilesCaller, Right
|
||||
from app.collation import collated
|
||||
from app.downloads import folders, links, quota
|
||||
from app.downloads.models import (
|
||||
DownloadJob,
|
||||
DownloadLink,
|
||||
DownloadProfile,
|
||||
DownloadQuota,
|
||||
DownloadShare,
|
||||
FileFolder,
|
||||
LibraryState,
|
||||
MediaAsset,
|
||||
)
|
||||
from app.downloads.status import FINISHED, JobStatus
|
||||
from app.models import User
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app.utils import like_escape
|
||||
|
||||
# Cap a listing so a heavy library can't return an unbounded result set (the old Download Center's
|
||||
# own cap; far beyond any real library).
|
||||
MAX_LIST = 500
|
||||
|
||||
|
||||
# --- jobs and their files ------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_job(db: Session, user: User, job_id: int) -> DownloadJob | None:
|
||||
return owned_row(db, DownloadJob, job_id, user)
|
||||
|
||||
|
||||
def job(db: Session, job_id: int) -> DownloadJob | None:
|
||||
"""A job by id, whoever's — for a public link, which already names its job."""
|
||||
return db.get(DownloadJob, job_id)
|
||||
|
||||
|
||||
def accessible_job(db: Session, user: User, job_id: int) -> DownloadJob | None:
|
||||
"""A file the caller may read: their own, or one shared with them."""
|
||||
row = db.get(DownloadJob, job_id)
|
||||
if row is None:
|
||||
return None
|
||||
if row.user_id == user.id or shared_with(db, job_id, user) is not None:
|
||||
return row
|
||||
return None
|
||||
|
||||
|
||||
def readable_job(db: Session, caller: FilesCaller, job_id: int) -> DownloadJob | None:
|
||||
"""A file `caller` may read: as `accessible_job` for the user; for a token, one of the user's
|
||||
own in a folder it may read."""
|
||||
if caller.grants is None:
|
||||
return accessible_job(db, caller.user, job_id)
|
||||
row = own_job(db, caller.user, job_id)
|
||||
if row is None or not caller.may(row.folder_id, Right.READ):
|
||||
return None
|
||||
return row
|
||||
|
||||
|
||||
def asset_of(db: Session, job: DownloadJob | None) -> MediaAsset | None:
|
||||
"""The file under a job, if it has one yet."""
|
||||
return db.get(MediaAsset, job.asset_id) if job is not None and job.asset_id else None
|
||||
|
||||
|
||||
def assets_for(db: Session, jobs: Sequence[DownloadJob]) -> dict[int, MediaAsset]:
|
||||
ids = {j.asset_id for j in jobs if j.asset_id}
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(select(MediaAsset).where(MediaAsset.id.in_(ids))).scalars()
|
||||
return {a.id: a for a in rows}
|
||||
|
||||
|
||||
def positions_for(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, int]:
|
||||
"""`user`'s resume point in each of `jobs` that has one (the library player's `library_states`)."""
|
||||
ids = [j.id for j in jobs]
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(
|
||||
select(LibraryState.job_id, LibraryState.position_seconds).where(
|
||||
owned_by(LibraryState.user_id, user), LibraryState.job_id.in_(ids)
|
||||
)
|
||||
).all()
|
||||
return {job_id: pos for job_id, pos in rows}
|
||||
|
||||
|
||||
def source_keys(db: Session, jobs: Sequence[DownloadJob]) -> dict[int, str]:
|
||||
"""Where each file came from (its source folder's key), for the list's Source column, the
|
||||
"back to its source folder" move and the download tray's site: a file in a source folder has its
|
||||
folder's key; one moved into an own folder (or someone else's, shared) is traced back by
|
||||
`folders.source_key_for`."""
|
||||
ids = {j.folder_id for j in jobs if j.folder_id is not None}
|
||||
by_id = (
|
||||
{f.id: f for f in db.execute(select(FileFolder).where(FileFolder.id.in_(ids))).scalars()}
|
||||
if ids
|
||||
else {}
|
||||
)
|
||||
out: dict[int, str] = {}
|
||||
for j in jobs:
|
||||
folder = by_id.get(j.folder_id) if j.folder_id is not None else None
|
||||
if folder is not None and folder.kind == folders.FolderKind.SOURCE and folder.source_key:
|
||||
out[j.id] = folder.source_key
|
||||
else:
|
||||
out[j.id] = folders.source_key_for(db, j)
|
||||
return out
|
||||
|
||||
|
||||
def queue(db: Session, user: User, recent: datetime) -> list[DownloadJob]:
|
||||
"""The download tray: what is waiting, running, paused or failed, and what finished (done or
|
||||
canceled) since `recent`. A done job is timed from when it became done, never from its last
|
||||
write: a rename, a move or the GC's expiry warning touches an old file's row, and the tray would
|
||||
announce it as just finished."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(DownloadJob)
|
||||
.where(
|
||||
owned_by(DownloadJob.user_id, user),
|
||||
DownloadJob.status.not_in(FINISHED)
|
||||
| ((DownloadJob.status == JobStatus.DONE) & (DownloadJob.ready_at >= recent))
|
||||
| ((DownloadJob.status == JobStatus.CANCELED) & (DownloadJob.updated_at >= recent)),
|
||||
)
|
||||
.order_by(DownloadJob.created_at.desc(), DownloadJob.id.desc())
|
||||
.limit(MAX_LIST)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def statuses_by_source(db: Session, user: User, statuses: Sequence[str]) -> list[tuple[str, str]]:
|
||||
"""(source_ref, status) of each of `user`'s jobs in one of `statuses`."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(DownloadJob.source_ref, DownloadJob.status).where(
|
||||
owned_by(DownloadJob.user_id, user), DownloadJob.status.in_(tuple(statuses))
|
||||
)
|
||||
).tuples()
|
||||
)
|
||||
|
||||
|
||||
def library(
|
||||
db: Session,
|
||||
user: User,
|
||||
*,
|
||||
folder_id: int | None,
|
||||
listable: set[int] | None,
|
||||
shared: bool,
|
||||
term: str,
|
||||
sort: str,
|
||||
order: str,
|
||||
lang: str,
|
||||
) -> list[DownloadJob]:
|
||||
"""The caller's done files (optionally one folder's, or only in the folders a token may list),
|
||||
or the ones shared with them; searched by name and channel, sorted by when they were added, by
|
||||
name (in the reader's alphabet) or by size. A file is added to its owner's list when it is done,
|
||||
not when it was asked for (a download that failed for days and then went through is new); one
|
||||
shared with the caller, when it was shared."""
|
||||
stmt = (
|
||||
select(DownloadJob)
|
||||
.join(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
|
||||
.where(DownloadJob.status == JobStatus.DONE)
|
||||
)
|
||||
if shared:
|
||||
stmt = stmt.join(DownloadShare, DownloadShare.job_id == DownloadJob.id).where(
|
||||
owned_by(DownloadShare.to_user_id, user)
|
||||
)
|
||||
else:
|
||||
stmt = stmt.where(owned_by(DownloadJob.user_id, user))
|
||||
if folder_id is not None:
|
||||
stmt = stmt.where(DownloadJob.folder_id == folder_id)
|
||||
elif listable is not None:
|
||||
stmt = stmt.where(DownloadJob.folder_id.in_(listable))
|
||||
name = func.coalesce(DownloadJob.display_name, MediaAsset.title, "")
|
||||
if term:
|
||||
like = f"%{like_escape(term)}%"
|
||||
stmt = stmt.where(
|
||||
or_(
|
||||
name.ilike(like),
|
||||
func.coalesce(DownloadJob.display_uploader, MediaAsset.uploader, "").ilike(like),
|
||||
)
|
||||
)
|
||||
key = {
|
||||
"added": DownloadShare.created_at if shared else DownloadJob.ready_at,
|
||||
"name": collated(name, lang),
|
||||
"size": MediaAsset.size_bytes,
|
||||
}[sort]
|
||||
first = key.asc() if order == "asc" else key.desc()
|
||||
tie = DownloadJob.id.asc() if order == "asc" else DownloadJob.id.desc()
|
||||
return list(db.execute(stmt.order_by(first.nulls_last(), tie).limit(MAX_LIST)).scalars())
|
||||
|
||||
|
||||
def recent_jobs(db: Session) -> list[DownloadJob]:
|
||||
"""Everyone's newest jobs, for the admin's list."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(DownloadJob)
|
||||
.order_by(DownloadJob.created_at.desc(), DownloadJob.id.desc())
|
||||
.limit(MAX_LIST)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- shares --------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def shared_with(db: Session, job_id: int, user: User | int) -> DownloadShare | None:
|
||||
"""The grant that hands this job to `user`, if there is one."""
|
||||
return db.execute(
|
||||
select(DownloadShare).where(
|
||||
DownloadShare.job_id == job_id, owned_by(DownloadShare.to_user_id, user)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def shared_at(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, datetime]:
|
||||
rows = db.execute(
|
||||
select(DownloadShare.job_id, DownloadShare.created_at).where(
|
||||
owned_by(DownloadShare.to_user_id, user),
|
||||
DownloadShare.job_id.in_([j.id for j in jobs]),
|
||||
)
|
||||
).all()
|
||||
return {job_id: at for job_id, at in rows}
|
||||
|
||||
|
||||
# --- folders -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_folder(db: Session, user: User, folder_id: int) -> FileFolder | None:
|
||||
return owned_row(db, FileFolder, folder_id, user)
|
||||
|
||||
|
||||
def folders_by_id(db: Session, ids: set[int]) -> dict[int, FileFolder]:
|
||||
"""These folders, by id — for a reader that already knows which ones (a token's grants)."""
|
||||
if not ids:
|
||||
return {}
|
||||
return {f.id: f for f in db.execute(select(FileFolder).where(FileFolder.id.in_(ids))).scalars()}
|
||||
|
||||
|
||||
def folders_with_totals(
|
||||
db: Session, user: User, listable: set[int] | None
|
||||
) -> list[tuple[FileFolder, int, int]]:
|
||||
"""The user's folders (only `listable` ones, when given) with how many done files each holds and
|
||||
their size. A source folder its files have all left stays (count 0)."""
|
||||
done = (DownloadJob.folder_id == FileFolder.id) & (DownloadJob.status == JobStatus.DONE)
|
||||
stmt = select(
|
||||
FileFolder,
|
||||
func.count(DownloadJob.id),
|
||||
func.coalesce(func.sum(MediaAsset.size_bytes), 0),
|
||||
).where(owned_by(FileFolder.user_id, user))
|
||||
if listable is not None:
|
||||
stmt = stmt.where(FileFolder.id.in_(listable))
|
||||
rows = db.execute(
|
||||
stmt.outerjoin(DownloadJob, done)
|
||||
.outerjoin(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
|
||||
.group_by(FileFolder.id)
|
||||
.order_by(FileFolder.kind.desc(), FileFolder.source_key, FileFolder.name, FileFolder.id)
|
||||
).tuples()
|
||||
return [(f, count, int(size or 0)) for f, count, size in rows]
|
||||
|
||||
|
||||
def folder_totals(db: Session, folder_id: int) -> tuple[int, int]:
|
||||
count, size = db.execute(
|
||||
select(func.count(DownloadJob.id), func.coalesce(func.sum(MediaAsset.size_bytes), 0))
|
||||
.select_from(DownloadJob)
|
||||
.outerjoin(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
|
||||
.where(DownloadJob.folder_id == folder_id, DownloadJob.status == JobStatus.DONE)
|
||||
).one()
|
||||
return count, int(size)
|
||||
|
||||
|
||||
def linked_folders(db: Session, folder_ids: list[int]) -> set[int]:
|
||||
"""Which of these folders a live public link opens (one not past its expiry)."""
|
||||
now = datetime.now(timezone.utc)
|
||||
rows = db.execute(
|
||||
select(DownloadLink.folder_id).where(
|
||||
DownloadLink.folder_id.in_(folder_ids),
|
||||
or_(DownloadLink.expires_at.is_(None), DownloadLink.expires_at > now),
|
||||
)
|
||||
).scalars()
|
||||
return {folder_id for folder_id in rows if folder_id is not None}
|
||||
|
||||
|
||||
# --- links ---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_link(db: Session, user: User, link_id: int) -> DownloadLink | None:
|
||||
"""A link to one of the user's files or folders."""
|
||||
link = db.get(DownloadLink, link_id)
|
||||
if link is None:
|
||||
return None
|
||||
target = (
|
||||
owned_row(db, DownloadJob, link.job_id, user)
|
||||
if link.job_id is not None
|
||||
else owned_row(db, FileFolder, link.folder_id, user)
|
||||
)
|
||||
return link if target is not None else None
|
||||
|
||||
|
||||
def links_to(
|
||||
db: Session, *, job_id: int | None = None, folder_id: int | None = None
|
||||
) -> list[DownloadLink]:
|
||||
"""The links to one file or one folder, newest first."""
|
||||
target = (
|
||||
DownloadLink.job_id == job_id if job_id is not None else DownloadLink.folder_id == folder_id
|
||||
)
|
||||
return list(
|
||||
db.execute(select(DownloadLink).where(target).order_by(DownloadLink.id.desc())).scalars()
|
||||
)
|
||||
|
||||
|
||||
def link_folder(db: Session, link: DownloadLink) -> FileFolder | None:
|
||||
return db.get(FileFolder, link.folder_id) if link.folder_id is not None else None
|
||||
|
||||
|
||||
def folder_job(db: Session, folder: FileFolder, job_id: int) -> DownloadJob | None:
|
||||
"""One of the files a link to `folder` reaches (`links.in_folder`), while it is in it."""
|
||||
return db.execute(
|
||||
select(DownloadJob).where(DownloadJob.id == job_id, *links.in_folder(folder))
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def owner_suspended(db: Session, job: DownloadJob) -> bool:
|
||||
"""The job's owner is suspended (or gone): the worker leaves their queued jobs where they are."""
|
||||
owner = db.get(User, job.user_id)
|
||||
return owner is None or owner.is_suspended
|
||||
|
||||
|
||||
# --- profiles ------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def profiles_for(db: Session, user: User) -> list[DownloadProfile]:
|
||||
"""The built-in format profiles, then the user's own."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(DownloadProfile)
|
||||
.where(DownloadProfile.is_builtin.is_(True) | owned_by(DownloadProfile.user_id, user))
|
||||
.order_by(
|
||||
DownloadProfile.is_builtin.desc(), DownloadProfile.sort_order, DownloadProfile.id
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def own_profile(db: Session, user: User, profile_id: int) -> DownloadProfile | None:
|
||||
"""One of the user's own profiles (a built-in is no one's to change)."""
|
||||
p = owned_row(db, DownloadProfile, profile_id, user)
|
||||
return None if p is None or p.is_builtin else p
|
||||
|
||||
|
||||
def usable_profile(db: Session, user: User, profile_id: int) -> DownloadProfile | None:
|
||||
"""A profile the user may enqueue with: a built-in, or their own."""
|
||||
p = db.get(DownloadProfile, profile_id)
|
||||
if p is None or (not p.is_builtin and p.user_id != user.id):
|
||||
return None
|
||||
return p
|
||||
|
||||
|
||||
def default_profile(db: Session) -> DownloadProfile | None:
|
||||
"""The first built-in, for an enqueue that named no format."""
|
||||
return db.execute(
|
||||
select(DownloadProfile)
|
||||
.where(DownloadProfile.is_builtin.is_(True))
|
||||
.order_by(DownloadProfile.sort_order)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
# --- storage and quota (admin) -------------------------------------------------------------
|
||||
|
||||
|
||||
def storage_totals(db: Session) -> tuple[int, int, int]:
|
||||
"""(files on disk, their bytes, every asset row). On disk is `quota.ON_DISK`: ready files, and
|
||||
uploads waiting for or failed in their conversion — the same disk the footprint and the GC cap
|
||||
count."""
|
||||
on_disk = db.execute(
|
||||
select(func.count(), func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(quota.ON_DISK)
|
||||
).one()
|
||||
total_assets = db.execute(select(func.count()).select_from(MediaAsset)).scalar_one()
|
||||
return on_disk[0], int(on_disk[1]), total_assets
|
||||
|
||||
|
||||
def done_files_per_user(db: Session) -> dict[int, int]:
|
||||
"""How many finished files each user's Files lists."""
|
||||
return dict(
|
||||
db.execute(
|
||||
select(DownloadJob.user_id, func.count())
|
||||
.where(DownloadJob.status == JobStatus.DONE)
|
||||
.group_by(DownloadJob.user_id)
|
||||
)
|
||||
.tuples()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def custom_quota(db: Session, user_id: int) -> DownloadQuota | None:
|
||||
"""The admin's own limits for this user, if any were set."""
|
||||
return db.get(DownloadQuota, user_id)
|
||||
@@ -26,7 +26,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import sysconfig
|
||||
from app.downloads.status import JobKind, JobStatus
|
||||
from app.models import DownloadJob, DownloadQuota, MediaAsset
|
||||
from app.downloads.models import DownloadJob, DownloadQuota, MediaAsset
|
||||
from app.userscope import owned_by
|
||||
|
||||
# Jobs a user is considered to be "holding" (occupy a job slot / footprint). Terminal failures
|
||||
# (error, canceled) don't count.
|
||||
@@ -85,7 +86,7 @@ def footprint(db: Session, user_id: int) -> int:
|
||||
held_assets = (
|
||||
select(DownloadJob.asset_id)
|
||||
.where(
|
||||
DownloadJob.user_id == user_id,
|
||||
owned_by(DownloadJob.user_id, user_id),
|
||||
or_(
|
||||
DownloadJob.status.in_(_HOLDING),
|
||||
and_(DownloadJob.job_kind == JobKind.UPLOAD, DownloadJob.status == JobStatus.ERROR),
|
||||
@@ -109,7 +110,7 @@ def _count(db: Session, user_id: int, states) -> int:
|
||||
db.execute(
|
||||
select(func.count())
|
||||
.select_from(DownloadJob)
|
||||
.where(DownloadJob.user_id == user_id, DownloadJob.status.in_(states))
|
||||
.where(owned_by(DownloadJob.user_id, user_id), DownloadJob.status.in_(states))
|
||||
).scalar()
|
||||
or 0
|
||||
)
|
||||
@@ -201,3 +202,46 @@ def usage(db: Session, user_id: int) -> dict:
|
||||
"max_jobs": lim.max_jobs,
|
||||
"unlimited": lim.unlimited,
|
||||
}
|
||||
|
||||
|
||||
def set_custom(
|
||||
db: Session,
|
||||
user_id: int,
|
||||
*,
|
||||
max_bytes: int | None = None,
|
||||
max_concurrent: int | None = None,
|
||||
max_jobs: int | None = None,
|
||||
unlimited: bool | None = None,
|
||||
) -> None:
|
||||
"""The admin's own limits for a user: starts from what applies now, then each value given (a
|
||||
size floored at 0, a count at 1). The handler commits."""
|
||||
row = db.get(DownloadQuota, user_id)
|
||||
if row is None:
|
||||
cur = resolve(db, user_id)
|
||||
row = DownloadQuota(
|
||||
user_id=user_id,
|
||||
max_bytes=cur.max_bytes,
|
||||
max_concurrent=cur.max_concurrent,
|
||||
max_jobs=cur.max_jobs,
|
||||
unlimited=cur.unlimited,
|
||||
)
|
||||
db.add(row)
|
||||
if max_bytes is not None:
|
||||
row.max_bytes = max(0, max_bytes)
|
||||
if max_concurrent is not None:
|
||||
row.max_concurrent = max(1, max_concurrent)
|
||||
if max_jobs is not None:
|
||||
row.max_jobs = max(1, max_jobs)
|
||||
if unlimited is not None:
|
||||
row.unlimited = unlimited
|
||||
db.flush()
|
||||
|
||||
|
||||
def reset_custom(db: Session, user_id: int) -> bool:
|
||||
"""Back to the defaults; False when the user had no limits of their own."""
|
||||
row = db.get(DownloadQuota, user_id)
|
||||
if row is None:
|
||||
return False
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
@@ -16,6 +16,7 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import app.tables # noqa: F401 — a fresh interpreter: every table, before the first query
|
||||
from app import logs
|
||||
from app import logsink
|
||||
from app.config import settings
|
||||
@@ -23,7 +24,7 @@ from app.db import SessionLocal
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import formats
|
||||
from app.downloads.status import JobKind
|
||||
from app.models import DownloadJob
|
||||
from app.downloads.models import DownloadJob
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
@@ -1,16 +1,19 @@
|
||||
"""Enqueue + cache resolution for the download center (the per-user layer over MediaAsset).
|
||||
"""Capture's writes: enqueue + cache resolution, the job transitions, and what the library's
|
||||
dialogs change (profiles, resume points, shares, a file's display fields).
|
||||
|
||||
`enqueue` is the single entry point used by the API: it resolves (or creates) the shared
|
||||
MediaAsset for the requested source+format, links a per-user DownloadJob to it, and — on a
|
||||
cache hit (asset already ready) — marks the job done instantly with no re-download. The worker
|
||||
handles everything else. State transitions used by the routes (pause/resume/cancel/delete) and
|
||||
the worker live in `worker.py` / the routes; this module owns enqueue + asset bookkeeping.
|
||||
handles everything else. Every function here flushes and leaves the commit to its caller (the
|
||||
handler); the folder, link and quota writes are `folders`, `links` and `quota`.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import delete as delete_rows
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -21,17 +24,19 @@ from app.downloads import credentials
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import folders, formats, mediakind, quota, storage
|
||||
from app.downloads.status import HOLDING, JobKind, JobStatus
|
||||
from app.models import (
|
||||
DISPLAY_TEXT_MAX,
|
||||
Channel,
|
||||
from app.models import DISPLAY_TEXT_MAX, User
|
||||
from app.plex.models import PlexItem, PlexShow
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.downloads.models import (
|
||||
DownloadJob,
|
||||
DownloadProfile,
|
||||
DownloadShare,
|
||||
LibraryState,
|
||||
MediaAsset,
|
||||
PlexItem,
|
||||
PlexShow,
|
||||
Video,
|
||||
)
|
||||
from app.plex import lookups as plex_lookups
|
||||
from app.plex import paths as plex_paths
|
||||
from app.userscope import owned_by, owned_row
|
||||
|
||||
|
||||
class EditError(Exception):
|
||||
@@ -230,8 +235,7 @@ def enqueue(
|
||||
folders.stamp_done(db, job)
|
||||
asset.last_access_at = datetime.now(timezone.utc)
|
||||
|
||||
db.commit()
|
||||
db.refresh(job)
|
||||
db.flush()
|
||||
return job
|
||||
|
||||
|
||||
@@ -310,8 +314,7 @@ def enqueue_edit(
|
||||
folders.stamp_done(db, job)
|
||||
asset.last_access_at = datetime.now(timezone.utc)
|
||||
|
||||
db.commit()
|
||||
db.refresh(job)
|
||||
db.flush()
|
||||
return job
|
||||
|
||||
|
||||
@@ -399,8 +402,7 @@ def enqueue_clip(
|
||||
folders.stamp_done(db, job)
|
||||
asset.last_access_at = datetime.now(timezone.utc)
|
||||
|
||||
db.commit()
|
||||
db.refresh(job)
|
||||
db.flush()
|
||||
return job
|
||||
|
||||
|
||||
@@ -508,8 +510,8 @@ def release_asset(db: Session, job: DownloadJob) -> None:
|
||||
# --- Job state transitions ---------------------------------------------------------------------
|
||||
# The single home for the pause/resume/cancel/delete/redownload transitions + their asset
|
||||
# bookkeeping (release, failed-asset reset, fresh-asset rebuild), so the routes are thin callers and
|
||||
# the worker/GC could share the same definitions (B-sub-8). Each commits its own transition; the
|
||||
# route keeps only auth + HTTP mapping + serialization.
|
||||
# the worker/GC could share the same definitions (B-sub-8). Each flushes its transition and the caller
|
||||
# commits it; the route keeps only auth + HTTP mapping + serialization.
|
||||
|
||||
|
||||
def pause(db: Session, job: DownloadJob) -> None:
|
||||
@@ -517,7 +519,7 @@ def pause(db: Session, job: DownloadJob) -> None:
|
||||
in any other state."""
|
||||
if job.status in (JobStatus.QUEUED, JobStatus.RUNNING):
|
||||
job.status = JobStatus.PAUSED
|
||||
db.commit()
|
||||
db.flush()
|
||||
|
||||
|
||||
def resume(db: Session, job: DownloadJob) -> None:
|
||||
@@ -541,7 +543,7 @@ def resume(db: Session, job: DownloadJob) -> None:
|
||||
asset.status = "pending"
|
||||
asset.error = None
|
||||
job.status = JobStatus.QUEUED
|
||||
db.commit()
|
||||
db.flush()
|
||||
|
||||
|
||||
def _follow_the_cookie(db: Session, job: DownloadJob, asset: MediaAsset) -> MediaAsset:
|
||||
@@ -576,14 +578,14 @@ def cancel(db: Session, job: DownloadJob) -> None:
|
||||
if job.status not in (JobStatus.DONE, JobStatus.CANCELED):
|
||||
release_asset(db, job) # release while the job still counts as holding
|
||||
job.status = JobStatus.CANCELED
|
||||
db.commit()
|
||||
db.flush()
|
||||
|
||||
|
||||
def delete(db: Session, job: DownloadJob) -> None:
|
||||
"""Delete a job, releasing its asset hold first (which frees the file iff this was the last holder)."""
|
||||
release_asset(db, job)
|
||||
db.delete(job)
|
||||
db.commit()
|
||||
db.flush()
|
||||
|
||||
|
||||
def redownload(db: Session, job: DownloadJob) -> None:
|
||||
@@ -632,4 +634,181 @@ def redownload(db: Session, job: DownloadJob) -> None:
|
||||
job.queue_pos = next_queue_pos(db) # tail of the queue, not its stale original slot
|
||||
job.expires_at = None # stamped again when the rebuild is done
|
||||
job.expiry_warned = False
|
||||
db.commit()
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- the library's writes ------------------------------------------------------------------
|
||||
# What the HTTP layer changes for a user: each flushes, the handler commits.
|
||||
|
||||
|
||||
def lock_own_job(db: Session, user: User, job_id: int) -> DownloadJob | None:
|
||||
"""One of the user's jobs, re-read under a row lock, for a transition that moves its asset's ref
|
||||
count (resume, cancel, delete, redownload) or writes the status those decide on (pause). Two
|
||||
overlapping requests for one job run one after the other and the second decides on the first's
|
||||
committed result — never both releasing the same hold, and never a pause turning a job a cancel
|
||||
just released back into a holder; either can free a file another job still holds. None: not
|
||||
theirs, or gone.
|
||||
|
||||
The asset is locked BEFORE the job: the order a finishing worker child takes them in
|
||||
(`finalize_if_running`), so the two can never deadlock."""
|
||||
while True:
|
||||
owned = owned_row(db, DownloadJob, job_id, user)
|
||||
if owned is None:
|
||||
return None
|
||||
asset_id = owned.asset_id
|
||||
if asset_id is not None:
|
||||
db.execute(select(MediaAsset.id).where(MediaAsset.id == asset_id).with_for_update())
|
||||
job = db.execute(
|
||||
select(DownloadJob)
|
||||
.where(DownloadJob.id == job_id)
|
||||
.with_for_update()
|
||||
.execution_options(populate_existing=True)
|
||||
).scalar_one_or_none()
|
||||
if job is None:
|
||||
return None
|
||||
if job.asset_id == asset_id:
|
||||
return job
|
||||
db.rollback() # another request moved the job to another asset meanwhile: lock that one
|
||||
|
||||
|
||||
def hold_for_share(db: Session, user: User, job_id: int) -> DownloadJob | None:
|
||||
"""One of the user's files, held (FOR SHARE) for the rest of the transaction: a delete of it at
|
||||
the same moment (the user's, or the retention sweep's) waits and takes the new link with it,
|
||||
rather than failing the link's foreign key — or, done first, leaves None."""
|
||||
if owned_row(db, DownloadJob, job_id, user) is None:
|
||||
return None
|
||||
return db.execute(
|
||||
select(DownloadJob)
|
||||
.where(DownloadJob.id == job_id)
|
||||
.with_for_update(read=True)
|
||||
.execution_options(populate_existing=True)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def create_profile(db: Session, user: User, name: str, spec: dict) -> DownloadProfile:
|
||||
"""A format profile of the user's own, after their others."""
|
||||
maxpos = db.execute(
|
||||
select(func.coalesce(func.max(DownloadProfile.sort_order), 100)).where(
|
||||
owned_by(DownloadProfile.user_id, user)
|
||||
)
|
||||
).scalar_one()
|
||||
p = DownloadProfile(
|
||||
user_id=user.id, name=name[:80], spec=spec, is_builtin=False, sort_order=maxpos + 1
|
||||
)
|
||||
db.add(p)
|
||||
db.flush()
|
||||
return p
|
||||
|
||||
|
||||
def update_profile(profile: DownloadProfile, *, name: str | None, spec: dict | None) -> None:
|
||||
"""Rename a profile and/or replace its spec; None leaves that one as it is."""
|
||||
if name is not None:
|
||||
profile.name = name[:80]
|
||||
if spec is not None:
|
||||
profile.spec = spec
|
||||
|
||||
|
||||
def delete_profile(db: Session, profile: DownloadProfile) -> None:
|
||||
db.delete(profile)
|
||||
db.flush()
|
||||
|
||||
|
||||
def set_position(db: Session, user: User, job_id: int, position: int) -> None:
|
||||
"""The library player's resume point in a file the user can play; 0 clears it.
|
||||
|
||||
A clear is a statement, not a loaded row's delete: the periodic checkpoint and the pagehide beacon
|
||||
of one stop may both clear it, and the second then deletes nothing rather than failing. A set is
|
||||
an upsert, one row per (user, job) even when two checkpoints race, not a read-then-insert that
|
||||
the second would fail on the unique key."""
|
||||
if not position:
|
||||
db.execute(
|
||||
delete_rows(LibraryState).where(
|
||||
owned_by(LibraryState.user_id, user), LibraryState.job_id == job_id
|
||||
)
|
||||
)
|
||||
return
|
||||
now = datetime.now(timezone.utc)
|
||||
db.execute(
|
||||
pg_insert(LibraryState)
|
||||
.values(user_id=user.id, job_id=job_id, position_seconds=position, progress_updated_at=now)
|
||||
.on_conflict_do_update(
|
||||
constraint="uq_library_user_job",
|
||||
set_={"position_seconds": position, "progress_updated_at": now},
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def share(db: Session, job: DownloadJob, sender: User, recipient: User) -> bool:
|
||||
"""Hand a file to another user, with a notice for them; False when it was already theirs."""
|
||||
from app.notifications import create_notification
|
||||
|
||||
exists = db.execute(
|
||||
select(DownloadShare.id).where(
|
||||
DownloadShare.job_id == job.id, owned_by(DownloadShare.to_user_id, recipient)
|
||||
)
|
||||
).first()
|
||||
if exists is not None:
|
||||
return False
|
||||
db.add(DownloadShare(job_id=job.id, from_user_id=sender.id, to_user_id=recipient.id))
|
||||
create_notification(
|
||||
db,
|
||||
user_id=recipient.id,
|
||||
type="download_shared",
|
||||
title=(job.display_name or "A download") + " was shared with you",
|
||||
# The client names the notice in the reader's language from `data`; the title is the
|
||||
# English fallback only.
|
||||
data={
|
||||
"kind": "download_shared",
|
||||
"from": sender.email,
|
||||
"job_id": job.id,
|
||||
"title": job.display_name,
|
||||
},
|
||||
commit=False,
|
||||
)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
|
||||
def unshare(db: Session, job_id: int, recipient: User) -> bool:
|
||||
"""Take a file back from one user (its owner's act, or the recipient's own dismissal); False
|
||||
when it was not theirs."""
|
||||
row = db.execute(
|
||||
select(DownloadShare).where(
|
||||
DownloadShare.job_id == job_id, owned_by(DownloadShare.to_user_id, recipient)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is None:
|
||||
return False
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
|
||||
# The display fields a user may set on their file; the file on disk keeps its own name.
|
||||
DISPLAY_FIELDS = frozenset(
|
||||
{"display_name", "display_uploader", "display_uploader_url", "extra_links"}
|
||||
)
|
||||
|
||||
|
||||
def set_display(job: DownloadJob, **fields: object) -> None:
|
||||
"""Set a file's display fields (each already cleaned by the caller); a field not given is left."""
|
||||
unknown = set(fields) - DISPLAY_FIELDS
|
||||
if unknown:
|
||||
raise ValueError(f"not a display field: {sorted(unknown)}")
|
||||
for name, value in fields.items():
|
||||
setattr(job, name, value)
|
||||
|
||||
|
||||
def touch(asset: MediaAsset) -> None:
|
||||
"""The file was just served: the cache's idle clock starts again."""
|
||||
asset.last_access_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def store_sha256(db: Session, asset_id: int, digest: str) -> None:
|
||||
"""Keep a file's hash once it is known — only where none is, so a concurrent hash of the same
|
||||
file never overwrites the first."""
|
||||
db.execute(
|
||||
update(MediaAsset)
|
||||
.where(MediaAsset.id == asset_id, MediaAsset.sha256.is_(None))
|
||||
.values(sha256=digest)
|
||||
)
|
||||
|
||||
@@ -37,7 +37,8 @@ from app.config import settings
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import folders, mediakind, playable, probe, quota, service, storage
|
||||
from app.downloads.status import JobKind, JobStatus
|
||||
from app.models import DownloadJob, MediaAsset, display_text
|
||||
from app.models import display_text
|
||||
from app.downloads.models import DownloadJob, MediaAsset
|
||||
|
||||
|
||||
def _title_from_filename(filename: str) -> str:
|
||||
|
||||
+117
-24
@@ -1,10 +1,11 @@
|
||||
"""Feed read-model: the pure query-building and row-serialization layer behind the feed.
|
||||
"""Feed read-model: Reader's query-building, paging and row serialization behind the feed.
|
||||
|
||||
Extracted from `routes/feed.py` so the read-model has one home instead of living inside the
|
||||
router module — the feed route handlers and the playlist-detail query share these helpers (the
|
||||
projection, the filter builder, and keyset paging). Nothing here defines a route or reaches for
|
||||
request state; it's all `(db, user, params) -> query`, and a malformed cursor raises a plain
|
||||
`ValueError` for the route handler to translate (the tsquery sanitizer lives in `app.utils`).
|
||||
One home for the read-model rather than the router module: the feed routes, the playlist-detail
|
||||
query and Home share these helpers (the projection, the filter builder, and keyset paging), and the
|
||||
feed's routes only validate and answer — a page, its counts and its facets are built here. Nothing
|
||||
here defines a route or reaches for request state; it's all `(db, user, params) -> query`, and a
|
||||
malformed cursor raises `BadCursor` for the route handler to translate (the tsquery sanitizer lives
|
||||
in `app.utils`).
|
||||
"""
|
||||
|
||||
import base64
|
||||
@@ -16,22 +17,19 @@ from sqlalchemy import BigInteger, Select, and_, case, cast, false, func, or_, s
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.collation import collated
|
||||
from app.models import (
|
||||
from app.models import User
|
||||
from app.catalog.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ChannelTag,
|
||||
ExploredChannel,
|
||||
Playlist,
|
||||
PlaylistItem,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
Tag,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, Tag, VideoState
|
||||
from app.utils import iso, like_escape, to_tsquery_str
|
||||
from app.userscope import owned_by
|
||||
|
||||
|
||||
def _channel_url(channel_id: str, handle: str | None) -> str:
|
||||
@@ -48,7 +46,7 @@ def _saved_expr(user: User):
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(
|
||||
Playlist.user_id == user.id,
|
||||
owned_by(Playlist.user_id, user.id),
|
||||
Playlist.kind == "watch_later",
|
||||
PlaylistItem.video_id == Video.id,
|
||||
)
|
||||
@@ -103,7 +101,7 @@ def serialize(row) -> dict:
|
||||
|
||||
def _tag_visible(user: User):
|
||||
"""A ChannelTag row this user may see: a shared system tag (user_id NULL) or their own."""
|
||||
return or_(ChannelTag.user_id.is_(None), ChannelTag.user_id == user.id)
|
||||
return or_(ChannelTag.user_id.is_(None), owned_by(ChannelTag.user_id, user.id))
|
||||
|
||||
|
||||
# The pseudo-category of a selected tag id no Tag row answers (a saved view or the feed state outlives
|
||||
@@ -122,7 +120,7 @@ def tags_by_category(db: Session, user: User, tag_ids: list[int]) -> dict[str, l
|
||||
if not tag_ids:
|
||||
return {} # skip the pointless `WHERE id IN ()` round-trip on the no-tag-filter hot path
|
||||
out: dict[str, list[int]] = {}
|
||||
visible = or_(Tag.user_id.is_(None), Tag.user_id == user.id)
|
||||
visible = or_(Tag.user_id.is_(None), owned_by(Tag.user_id, user.id))
|
||||
for tid, cat in db.execute(
|
||||
select(Tag.id, Tag.category).where(Tag.id.in_(tag_ids), visible)
|
||||
).all():
|
||||
@@ -142,7 +140,7 @@ def not_blocked_cond(user: User):
|
||||
position — and the one query that copied only PART of this predicate set is exactly how that
|
||||
video came back as the landing page's suggestion after the user had said "never again"."""
|
||||
return Video.channel_id.notin_(
|
||||
select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)
|
||||
select(BlockedChannel.channel_id).where(owned_by(BlockedChannel.user_id, user.id))
|
||||
)
|
||||
|
||||
|
||||
@@ -187,7 +185,7 @@ def filtered_query(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
owned_by(Subscription.user_id, user.id),
|
||||
),
|
||||
)
|
||||
else:
|
||||
@@ -199,12 +197,12 @@ def filtered_query(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
owned_by(Subscription.user_id, user.id),
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
).outerjoin(
|
||||
SearchFind,
|
||||
and_(SearchFind.video_id == Video.id, SearchFind.user_id == user.id),
|
||||
and_(SearchFind.video_id == Video.id, owned_by(SearchFind.user_id, user.id)),
|
||||
)
|
||||
|
||||
# The state-gated views (watched/hidden/in_progress) only ever return videos that HAVE a
|
||||
@@ -214,7 +212,7 @@ def filtered_query(
|
||||
# exactly. 'unwatched'/'all' include row-less 'new' videos, so they keep the outer join.
|
||||
state_required = show in ("watched", "hidden", "in_progress")
|
||||
join_state = query.join if state_required else query.outerjoin
|
||||
query = join_state(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
query = join_state(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
|
||||
# Hide videos the maintenance job has flagged as currently unplayable (deleted/private/
|
||||
# abandoned). They're either deleted after a grace period or unhidden if they recover;
|
||||
@@ -227,8 +225,8 @@ def filtered_query(
|
||||
# catalog. Applied in BOTH scopes; the channel page reaches its ephemeral videos by
|
||||
# asking for scope=all + channel_ids (the channel is in this accessible set).
|
||||
accessible_explore = (
|
||||
select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
).union(select(ExploredChannel.channel_id).where(ExploredChannel.user_id == user.id))
|
||||
select(Subscription.channel_id).where(owned_by(Subscription.user_id, user.id))
|
||||
).union(select(ExploredChannel.channel_id).where(owned_by(ExploredChannel.user_id, user.id)))
|
||||
query = query.where(
|
||||
or_(
|
||||
Video.via_explore.is_(False),
|
||||
@@ -346,7 +344,7 @@ def filtered_query(
|
||||
if untagged:
|
||||
query = query.where(
|
||||
Video.channel_id.notin_(
|
||||
select(ChannelTag.channel_id).where(ChannelTag.user_id == user.id)
|
||||
select(ChannelTag.channel_id).where(owned_by(ChannelTag.user_id, user.id))
|
||||
)
|
||||
)
|
||||
|
||||
@@ -696,3 +694,98 @@ def decode_cursor(keys: list[dict], cursor: str) -> tuple[list, str, datetime |
|
||||
# Framework-free: the route handler translates this into an HTTP 400 so the
|
||||
# read-model stays web-layer-agnostic (see routes/feed.get_feed).
|
||||
raise ValueError("Invalid feed cursor") from exc
|
||||
|
||||
|
||||
class BadCursor(ValueError):
|
||||
"""A `cursor` this module did not write (or one from another sort): the caller's bad input."""
|
||||
|
||||
|
||||
def feed_page(
|
||||
db: Session,
|
||||
user: User,
|
||||
params: dict,
|
||||
*,
|
||||
sort: str,
|
||||
seed: int,
|
||||
limit: int,
|
||||
cursor: str | None,
|
||||
lang: str,
|
||||
) -> dict:
|
||||
"""One page of the feed, and the cursor for the next. A malformed cursor raises `BadCursor`."""
|
||||
query, rank_expr = filtered_query(db, user, **params)
|
||||
keys = sort_keys(sort, seed, rank_expr, lang)
|
||||
|
||||
vals, cursor_id, window = None, None, None
|
||||
if cursor:
|
||||
try:
|
||||
vals, cursor_id, window = decode_cursor(keys, cursor)
|
||||
except ValueError as e:
|
||||
raise BadCursor(str(e)) from e
|
||||
ranked = sort == "relevance" and rank_expr is not None
|
||||
if ranked and not cursor:
|
||||
window = rank_window_start(db, query)
|
||||
if ranked and window is not None:
|
||||
keys = sort_keys(sort, seed, windowed_rank(rank_expr, window), lang)
|
||||
|
||||
# Expose each key column on the row so we can build the next cursor from the last item.
|
||||
query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
|
||||
if vals is not None:
|
||||
for k, v in zip(keys, vals, strict=True):
|
||||
k["value"] = v
|
||||
if ranked and window is not None:
|
||||
rows = windowed_relevance_rows(db, query, keys, window, cursor_id, limit + 1)
|
||||
else:
|
||||
query = query.order_by(*order_by(keys))
|
||||
if cursor_id is not None:
|
||||
query = query.where(keyset_predicate(keys, cursor_id))
|
||||
rows = db.execute(query.limit(limit + 1)).all()
|
||||
has_more = len(rows) > limit
|
||||
page = rows[:limit]
|
||||
return {
|
||||
"items": [serialize(r) for r in page],
|
||||
"next_cursor": (
|
||||
encode_cursor(keys, page[-1], window if ranked else None)
|
||||
if (has_more and page)
|
||||
else None
|
||||
),
|
||||
"has_more": has_more,
|
||||
"limit": limit,
|
||||
}
|
||||
|
||||
|
||||
def feed_count(db: Session, user: User, **params) -> int:
|
||||
"""How many videos the filters match — the reference oracle `feed_summary`'s count is tested
|
||||
against."""
|
||||
query, _rank = filtered_query(db, user, **params)
|
||||
return db.scalar(select(func.count()).select_from(query.subquery())) or 0
|
||||
|
||||
|
||||
def facet_counts(db: Session, user: User, **params) -> dict[int, int]:
|
||||
"""Per-tag channel counts for the current filter context, one pass per category — the reference
|
||||
oracle `feed_summary`'s facets are tested against.
|
||||
|
||||
Each category is counted with every other filter applied but its own selections ignored
|
||||
(drill-down faceting: selecting one topic doesn't zero out the rest). A conjunctive (AND)
|
||||
category keeps them applied, so each remaining chip narrows to channels that ALSO have every
|
||||
selected tag, and tags that can't co-occur drop to zero."""
|
||||
visible = _tag_visible(user)
|
||||
counts: dict[int, int] = {}
|
||||
for category in ("language", "topic", "other"):
|
||||
conjunctive = params.get("tag_modes", {}).get(category) == "and"
|
||||
base, _rank = filtered_query(
|
||||
db,
|
||||
user,
|
||||
**{**params, "exclude_tag_category": None if conjunctive else category},
|
||||
)
|
||||
channels = base.with_only_columns(Video.channel_id).distinct().subquery()
|
||||
rows = db.execute(
|
||||
select(ChannelTag.tag_id, func.count(func.distinct(ChannelTag.channel_id)))
|
||||
.select_from(channels)
|
||||
.join(ChannelTag, ChannelTag.channel_id == channels.c.channel_id)
|
||||
.join(Tag, and_(Tag.id == ChannelTag.tag_id, Tag.category == category))
|
||||
.where(visible)
|
||||
.group_by(ChannelTag.tag_id)
|
||||
).all()
|
||||
for tag_id, count in rows:
|
||||
counts[tag_id] = count
|
||||
return counts
|
||||
|
||||
@@ -28,6 +28,7 @@ log = logs.get_logger(__name__)
|
||||
from app import bodylimit, collation, logsink, state, sysconfig # noqa: E402 — logging is configured first
|
||||
from app.config import settings # noqa: E402 — logging is configured first
|
||||
from app.db import SessionLocal, engine # noqa: E402 — logging is configured first
|
||||
import app.tables # noqa: E402 — every table, before the first query
|
||||
from app.downloads import service as download_service # noqa: E402 — logging is configured first
|
||||
from app.downloads import storage # noqa: E402 — logging is configured first
|
||||
from app.plex import stream as plex_stream # noqa: E402 — logging is configured first
|
||||
|
||||
+6
-1216
File diff suppressed because it is too large
Load Diff
@@ -9,6 +9,7 @@ table can't grow without bound for a heavy user.
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.models import Notification
|
||||
|
||||
# Keep at most this many READ notifications per user; older read ones are pruned. Unread
|
||||
@@ -48,11 +49,13 @@ def create_notification(
|
||||
|
||||
def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
"""Delete a user's oldest READ notifications beyond `cap`. Returns the number removed.
|
||||
Idempotent and cheap; safe to call after marking things read."""
|
||||
Idempotent and cheap; safe to call after marking things read. Flushes first, so a notice the
|
||||
caller has just marked read counts; the caller commits."""
|
||||
db.flush()
|
||||
ids = (
|
||||
db.execute(
|
||||
select(Notification.id)
|
||||
.where(Notification.user_id == user_id, Notification.read.is_(True))
|
||||
.where(owned_by(Notification.user_id, user_id), Notification.read.is_(True))
|
||||
.order_by(Notification.created_at.desc())
|
||||
.offset(cap)
|
||||
)
|
||||
@@ -63,5 +66,4 @@ def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
return 0
|
||||
# One bulk DELETE rather than a get+delete per row (the ids are already in hand).
|
||||
db.execute(delete(Notification).where(Notification.id.in_(ids)))
|
||||
db.commit()
|
||||
return len(ids)
|
||||
|
||||
@@ -5,7 +5,7 @@ router. Pure functions — no DB, no request state."""
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
from app.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
|
||||
|
||||
class ShowState(NamedTuple):
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
"""What a collection is to the app: where it comes from, whether it may be edited, its card, its
|
||||
effective content rating, and how its members order — shared by the collection pages, the info
|
||||
page's collection strips and the admin editor."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime
|
||||
from typing import Any, NamedTuple
|
||||
|
||||
from app.plex.models import PlexCollection
|
||||
|
||||
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
|
||||
# members (Plex shows one cert on the collection; we don't store it, so derive it).
|
||||
_CERT_RANK = {
|
||||
"G": 1,
|
||||
"TV-Y": 1,
|
||||
"TV-Y7": 1,
|
||||
"TV-G": 1,
|
||||
"PG": 2,
|
||||
"TV-PG": 2,
|
||||
"PG-13": 3,
|
||||
"TV-14": 3,
|
||||
"R": 4,
|
||||
"TV-MA": 4,
|
||||
"NC-17": 5,
|
||||
}
|
||||
|
||||
|
||||
def collection_cert(certs: list[str | None]) -> str | None:
|
||||
ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
|
||||
return max(ranked, key=lambda x: x[1])[0] if ranked else None
|
||||
|
||||
|
||||
_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
|
||||
|
||||
|
||||
def collection_source(col: PlexCollection) -> str:
|
||||
first = (col.title or "").strip().split(" ", 1)[0].lower()
|
||||
if first in _STRIP_PROVIDERS:
|
||||
return first
|
||||
if col.smart:
|
||||
return "smart"
|
||||
return "collection"
|
||||
|
||||
|
||||
def collection_editable(c: PlexCollection) -> bool:
|
||||
"""Effective editability: the admin marked it editable AND it's a plain manual collection — never
|
||||
smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
|
||||
return bool(c.editable) and not c.smart and collection_source(c) == "collection"
|
||||
|
||||
|
||||
def collection_card(c: PlexCollection) -> dict:
|
||||
return {
|
||||
"id": c.rating_key,
|
||||
"title": c.title,
|
||||
"summary": c.summary,
|
||||
"thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
|
||||
"child_count": c.child_count,
|
||||
"smart": c.smart,
|
||||
"source": collection_source(c),
|
||||
"editable": c.editable,
|
||||
"can_edit": collection_editable(
|
||||
c
|
||||
), # whether the edit controls should be offered (admin UI)
|
||||
}
|
||||
|
||||
|
||||
class Member(NamedTuple):
|
||||
"""One collection member flattened to everything the Sort control can order by. The page's two
|
||||
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
|
||||
ordering is a pure list sort over these, which is also what makes it unit-testable."""
|
||||
|
||||
kind: str # "movie" | "show"
|
||||
obj: Any # PlexItem | PlexShow, carried through to the card serializer
|
||||
title: str # lower-cased; never NULL (the column is not nullable)
|
||||
year: int | None
|
||||
rating: float | None
|
||||
added: datetime | None
|
||||
release: date | None
|
||||
duration: int | None # movies only — a show has no single runtime
|
||||
last_watched: datetime | None
|
||||
|
||||
|
||||
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
|
||||
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
|
||||
# that silently does nothing here.
|
||||
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
|
||||
"added": lambda m: m.added,
|
||||
"lastwatched": lambda m: m.last_watched,
|
||||
"release": lambda m: m.release,
|
||||
"year": lambda m: m.year,
|
||||
"rating": lambda m: m.rating,
|
||||
"duration": lambda m: m.duration,
|
||||
"title": lambda m: m.title,
|
||||
}
|
||||
|
||||
# What the page ordered by before it had a control, and still its default: a film series wants to be
|
||||
# read in release order, not by whenever Plex happened to ingest it.
|
||||
DEFAULT_MEMBER_SORT = "year"
|
||||
|
||||
|
||||
def sort_members(
|
||||
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
|
||||
) -> list[Member]:
|
||||
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
|
||||
|
||||
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
|
||||
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
|
||||
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
|
||||
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
|
||||
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
|
||||
|
||||
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
|
||||
compare as Python strings, which puts every accented initial after "z"."""
|
||||
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
|
||||
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
|
||||
if key is MEMBER_SORT_KEYS["title"]:
|
||||
key = by_title
|
||||
ordered = sorted(members, key=by_title) # stable secondary, applied first
|
||||
have = [m for m in ordered if key(m) is not None]
|
||||
missing = [m for m in ordered if key(m) is None]
|
||||
have.sort(key=key, reverse=sort_dir != "asc")
|
||||
return have + missing
|
||||
@@ -1,10 +1,11 @@
|
||||
"""Parse the "info page" rich metadata out of a full Plex metadata dict, and persist the subset that
|
||||
the detail endpoints serve from the DB row.
|
||||
|
||||
Shared by the live info-page path (routes.plex.browse) and the background rich re-sync (plex.sync) so
|
||||
the two never diverge — the same one-parser rule that keeps `people.py`'s cast handling consistent.
|
||||
`rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY fields onto a movie/show row
|
||||
(genres/directors/studio/content_rating already land from the cheap listing at `_apply_facet_fields`)."""
|
||||
Shared by the live info-page path (`plex.service.enrich_movie` / `enrich_show`) and the background
|
||||
rich re-sync (plex.sync) so the two never diverge — the same one-parser rule that keeps `people.py`'s
|
||||
cast handling consistent. `rich_detail` is the pure parser; `apply_rich_detail` writes the rich-ONLY
|
||||
fields onto a movie/show row (genres/directors/studio/content_rating already land from the cheap
|
||||
listing at `_apply_facet_fields`)."""
|
||||
|
||||
from app.plex import people
|
||||
from app.plex.plaintext import plain_text
|
||||
|
||||
@@ -0,0 +1,536 @@
|
||||
"""Plex Discover's reads: the rail-surface read-model.
|
||||
|
||||
A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
|
||||
Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
|
||||
separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
|
||||
recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
|
||||
every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
|
||||
their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
|
||||
card paths); the HU overlay coalesces only in the main grid.
|
||||
|
||||
The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
|
||||
Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
|
||||
grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
|
||||
the client hides). Genre-language matches the grid; duration is movie-only.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import sorted_words
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import SHOW_NEW, display_title, leaf_card, movie_card, show_card
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.query import (
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
top_art_keys,
|
||||
)
|
||||
from app.plex.tmdb import resolve_recs
|
||||
from app.userscope import owned_by
|
||||
|
||||
GENRE_TILES = 12
|
||||
SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows (6 total)
|
||||
|
||||
|
||||
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
|
||||
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
return (
|
||||
movie_ids if scope in ("movie", "both") else [],
|
||||
show_ids if scope in ("show", "both") else [],
|
||||
)
|
||||
|
||||
|
||||
def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
|
||||
"""Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
|
||||
q = apply_meta_filters(q, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
q = q.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
q = q.where(PlexItem.duration_s <= f.duration_max)
|
||||
return q
|
||||
|
||||
|
||||
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
|
||||
sts = rollup.item_state_map(db, uid, [it.id for it in items])
|
||||
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
|
||||
|
||||
|
||||
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
|
||||
statuses = rollup.state_map(db, uid, [s.id for s in shows])
|
||||
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
|
||||
|
||||
|
||||
def _recently_added(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Newest movies + shows, interleaved by `added_at` desc into one rail."""
|
||||
movies = (
|
||||
db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
.order_by(PlexItem.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
|
||||
)
|
||||
.order_by(PlexShow.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
|
||||
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
|
||||
# review found this one still on the older `(is not None, value)` form, i.e. one rule in two
|
||||
# spellings, with the two comments explaining it differently.
|
||||
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
|
||||
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
|
||||
top = tagged[:RAIL_LIMIT]
|
||||
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
|
||||
# share one), so the merge doesn't depend on object identity staying alive.
|
||||
top_movies = [o for _, kind, o in top if kind == "movie"]
|
||||
top_shows = [o for _, kind, o in top if kind == "show"]
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
|
||||
}
|
||||
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
|
||||
|
||||
|
||||
def _recently_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
|
||||
card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
|
||||
sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
|
||||
rows: list[tuple[datetime | None, dict]] = [] # (watched_at, card)
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
)
|
||||
for it, st in db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all():
|
||||
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
|
||||
if show_ids:
|
||||
eq = (
|
||||
select(PlexItem, PlexState, PlexShow)
|
||||
.join(
|
||||
PlexState, and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid))
|
||||
)
|
||||
.join(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
for ep, st, sh in db.execute(eq).all():
|
||||
card = leaf_card(
|
||||
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
|
||||
)
|
||||
rows.append((st.watched_at, card))
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _continue_watching(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
|
||||
A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
|
||||
rows: list[tuple[datetime | None, dict]] = [] # (resumed_at, card)
|
||||
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.position_seconds > 0,
|
||||
# A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
|
||||
# `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
|
||||
or_(
|
||||
PlexState.status.is_(None),
|
||||
PlexState.status.notin_(["watched", "hidden"]),
|
||||
),
|
||||
)
|
||||
)
|
||||
mrows = db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.progress_updated_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for it, st in mrows:
|
||||
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
|
||||
|
||||
if show_ids:
|
||||
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
|
||||
# its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
|
||||
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
|
||||
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
|
||||
sq = apply_show_state_filter(
|
||||
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
|
||||
prog, prog.c.show_id == PlexShow.id
|
||||
),
|
||||
prog.c.total,
|
||||
prog.c.watched,
|
||||
prog.c.inprog,
|
||||
"in_progress",
|
||||
).where(PlexShow.library_id.in_(show_ids))
|
||||
srows = db.execute(
|
||||
apply_meta_filters(sq, PlexShow, p, lang)
|
||||
.order_by(prog.c.resumed_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for sh, resumed_at, total, watched, inprog in srows:
|
||||
state = rollup.show_state(total, watched, inprog)
|
||||
rows.append((resumed_at, show_card(sh, state, lang=lang)))
|
||||
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _genre_tiles(
|
||||
db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
|
||||
) -> list[dict]:
|
||||
"""The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
|
||||
The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
|
||||
The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
|
||||
counts: dict[str, int] = {}
|
||||
for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
|
||||
if not ids:
|
||||
continue
|
||||
gcol = genres_col(model, lang)
|
||||
base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
|
||||
model.library_id.in_(ids),
|
||||
func.jsonb_typeof(gcol) == "array",
|
||||
*meta_filter_conds(model, p, lang),
|
||||
)
|
||||
if model is PlexItem:
|
||||
base = base.where(model.kind == "movie")
|
||||
expanded = base.subquery()
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
counts[g] = counts.get(g, 0) + c
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
|
||||
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
|
||||
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
|
||||
art = top_art_keys(
|
||||
db,
|
||||
PlexItem,
|
||||
movie_ids,
|
||||
genres_col(PlexItem, lang),
|
||||
[g for g, _ in top],
|
||||
extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
|
||||
)
|
||||
return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
|
||||
|
||||
|
||||
def _because_you_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> dict | None:
|
||||
"""Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
|
||||
recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
|
||||
None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
|
||||
The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
|
||||
if not movie_ids:
|
||||
return None
|
||||
anchor = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, owned_by(PlexState.user_id, uid)),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
PlexItem.recommendations.isnot(None),
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(1)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if anchor is None:
|
||||
return None
|
||||
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
|
||||
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
|
||||
rks = [
|
||||
r["rating_key"]
|
||||
for r in resolved
|
||||
if r.get("in_library") and r.get("rating_key") != anchor.rating_key
|
||||
]
|
||||
if not rks:
|
||||
return None
|
||||
movies = {
|
||||
m.rating_key: m
|
||||
for m in db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
ordered_movies = [movies[rk] for rk in rks if rk in movies]
|
||||
ordered_shows = [shows[rk] for rk in rks if rk in shows]
|
||||
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
|
||||
}
|
||||
items: list[dict] = []
|
||||
for rk in rks:
|
||||
if rk in mcards:
|
||||
items.append(mcards[rk])
|
||||
elif rk in scards:
|
||||
items.append(scards[rk])
|
||||
if not items:
|
||||
return None
|
||||
return {
|
||||
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
def discover_page(db: Session, user: User, scope: str, lang: str, f: LibraryFilters) -> dict:
|
||||
"""The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
|
||||
sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
|
||||
is null when there's no anchor. `lang` is already normalised."""
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
return {
|
||||
"scope": scope,
|
||||
"continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
|
||||
"because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
}
|
||||
|
||||
|
||||
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
|
||||
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
|
||||
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
|
||||
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
|
||||
fresh = (
|
||||
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
|
||||
.scalars()
|
||||
.all()
|
||||
if exclude
|
||||
else []
|
||||
)
|
||||
if len(fresh) == per:
|
||||
return list(fresh)
|
||||
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
|
||||
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
|
||||
return [*fresh, *rest.all()]
|
||||
|
||||
|
||||
def _kept(
|
||||
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
|
||||
) -> list[dict]:
|
||||
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
|
||||
stays in it, marked watched, and one gone from the scope's libraries drops out."""
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexItem.rating_key.in_(keep),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
cards = {
|
||||
c["id"]: c
|
||||
for c in [
|
||||
*_movie_cards(db, uid, list(movies), lang),
|
||||
*_show_cards(db, uid, list(shows), lang),
|
||||
]
|
||||
}
|
||||
return [cards[k] for k in keep if k in cards]
|
||||
|
||||
|
||||
def surprise(
|
||||
db: Session,
|
||||
user: User,
|
||||
scope: str,
|
||||
lang: str,
|
||||
limit: int,
|
||||
exclude: list[str],
|
||||
keep: list[str],
|
||||
f: LibraryFilters,
|
||||
) -> dict:
|
||||
"""A fresh random sample for the reshuffling Surprise-me rail (a random draw each call), narrowed by
|
||||
the active filters. Only titles the user has NOT seen — a watched movie / a fully-watched show is
|
||||
excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
|
||||
SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
|
||||
one type."""
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
if keep:
|
||||
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
|
||||
per = min(limit, SURPRISE_PER_KIND)
|
||||
movies = []
|
||||
if movie_ids:
|
||||
# Exclude movies this user has watched.
|
||||
watched_movie = (
|
||||
select(PlexState.id)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
owned_by(PlexState.user_id, user),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
movies = _draw(
|
||||
db,
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
~watched_movie,
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
),
|
||||
PlexItem.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
shows = []
|
||||
if show_ids:
|
||||
# Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
|
||||
# as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
|
||||
prog = rollup.agg_subquery(db, user.id).subquery()
|
||||
not_fully_watched = or_(
|
||||
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
|
||||
)
|
||||
shows = _draw(
|
||||
db,
|
||||
apply_meta_filters(
|
||||
select(PlexShow)
|
||||
.outerjoin(prog, prog.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
|
||||
PlexShow,
|
||||
p,
|
||||
lang,
|
||||
),
|
||||
PlexShow.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
mcards = _movie_cards(db, user.id, movies, lang)
|
||||
scards = _show_cards(db, user.id, shows, lang)
|
||||
# Interleave movie/show cards, then trim to `limit`.
|
||||
items: list[dict] = []
|
||||
for i in range(max(len(mcards), len(scards))):
|
||||
if i < len(mcards):
|
||||
items.append(mcards[i])
|
||||
if i < len(scards):
|
||||
items.append(scards[i])
|
||||
return {"scope": scope, "items": items[:limit]}
|
||||
@@ -15,7 +15,7 @@ from fastapi import Response
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import PlexCollection, PlexItem, PlexSeason, PlexShow
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexSeason, PlexShow
|
||||
from app.plex.cache import CACHE_DIR
|
||||
from app.plex.lookups import by_rating_key
|
||||
|
||||
|
||||
@@ -0,0 +1,473 @@
|
||||
"""The Plex browse grid's reads: the unified movies + shows feed, its search, and its facets.
|
||||
|
||||
A user's watch state is joined through `userscope`; the feed and its facets share every filter
|
||||
(`plex.query`), so a facet count is the size of what the grid would show.
|
||||
"""
|
||||
|
||||
from sqlalchemy import Boolean, Integer, String, and_, cast, func, literal, null, or_, select
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import norm_lang
|
||||
from app.plex.models import PlexItem, PlexShow, PlexState
|
||||
from app.plex.query import (
|
||||
FIELD_COLUMNS,
|
||||
PLAYBACK_CLASSES,
|
||||
TS_CONFIG,
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_movie_state_filter,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
field_recheck,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
movie_status_case,
|
||||
playable_conds,
|
||||
playback_class_expr,
|
||||
show_status_case,
|
||||
split_csv,
|
||||
summary_col,
|
||||
title_col,
|
||||
vector_col,
|
||||
vector_match,
|
||||
)
|
||||
from app.userscope import owned_by
|
||||
from app.utils import to_tsquery_str
|
||||
|
||||
|
||||
def sort_column(u, sort: str, lang: str = "en"):
|
||||
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
|
||||
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
|
||||
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
|
||||
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
|
||||
in the reader's alphabet (`collation`)."""
|
||||
return {
|
||||
"added": u.c.added_at,
|
||||
"title": collated(u.c.title, lang),
|
||||
"year": u.c.year,
|
||||
"rating": u.c.rating,
|
||||
"duration": u.c.duration_s,
|
||||
"release": u.c.rel,
|
||||
"lastwatched": u.c.last_watched,
|
||||
}.get(sort, u.c.added_at)
|
||||
|
||||
|
||||
def library_page(
|
||||
db: Session,
|
||||
user: User,
|
||||
*,
|
||||
scope: str,
|
||||
q: str | None,
|
||||
field: str,
|
||||
sort: str,
|
||||
sort_dir: str,
|
||||
show: str,
|
||||
lang: str,
|
||||
offset: int,
|
||||
limit: int,
|
||||
f: LibraryFilters,
|
||||
) -> tuple[int, list, list]:
|
||||
"""One page of the unified cross-library browse: (total, rows, episodes). `rows` mixes movies
|
||||
and shows (`kind` says which), scoped to movie|show|both, with the shared filters/search/watch
|
||||
state (a show's state is the aggregate of its episodes). On a search that also matches episodes
|
||||
(and shows are in scope), `episodes` holds the matching ones as (item, state, show) — the
|
||||
grouped 'Episodes' section. `lang` is already normalised."""
|
||||
p = f.as_dict()
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
|
||||
ts = to_tsquery_str(q) if q else None
|
||||
tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
|
||||
if field not in FIELD_COLUMNS and field != "all":
|
||||
field = "all"
|
||||
# `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
|
||||
# the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
|
||||
grid_field = field != "episode"
|
||||
recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
|
||||
|
||||
selects = []
|
||||
if want_movies and grid_field:
|
||||
st = aliased(PlexState)
|
||||
m_status = movie_status_case(st)
|
||||
mq = (
|
||||
select(
|
||||
PlexItem.rating_key.label("rk"),
|
||||
literal("movie").label("kind"),
|
||||
title_col(PlexItem, lang).label("title"),
|
||||
PlexItem.year.label("year"),
|
||||
PlexItem.rating.label("rating"),
|
||||
# Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
|
||||
func.left(summary_col(PlexItem, lang), 300).label("summary"),
|
||||
PlexItem.added_at.label("added_at"),
|
||||
PlexItem.originally_available_at.label("rel"),
|
||||
PlexItem.duration_s.label("duration_s"),
|
||||
PlexItem.playable.label("playable"),
|
||||
PlexItem.transcode_hw.label("transcode_hw"),
|
||||
cast(null(), Integer).label("season_count"),
|
||||
func.coalesce(st.position_seconds, 0).label("position_seconds"),
|
||||
m_status.label("status"),
|
||||
literal(False).label("resumable"),
|
||||
# "last watched" sort key: when the movie was last resumed, else when it was marked
|
||||
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
|
||||
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexItem, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(st, and_(st.item_id == PlexItem.id, owned_by(st.user_id, user)))
|
||||
.where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
|
||||
)
|
||||
mq = apply_movie_state_filter(mq, st, show)
|
||||
mq = apply_meta_filters(mq, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
mq = mq.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
mq = mq.where(PlexItem.duration_s <= f.duration_max)
|
||||
# Playback-class filter (movie-only — a show is an aggregate with no single class).
|
||||
mq = mq.where(*playable_conds(split_csv(f.playable)))
|
||||
if tsq is not None:
|
||||
mq = mq.where(vector_match(PlexItem, lang, tsq))
|
||||
if recheck is not None and (narrow := recheck(PlexItem, field, tsq, lang)) is not None:
|
||||
mq = mq.where(narrow)
|
||||
selects.append(mq)
|
||||
if want_shows and grid_field:
|
||||
agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
|
||||
tot = func.coalesce(agg.c.total, 0)
|
||||
wat = func.coalesce(agg.c.watched, 0)
|
||||
inp = func.coalesce(agg.c.inprog, 0)
|
||||
s_status = show_status_case(tot, wat, inp)
|
||||
sq = (
|
||||
select(
|
||||
PlexShow.rating_key.label("rk"),
|
||||
literal("show").label("kind"),
|
||||
title_col(PlexShow, lang).label("title"),
|
||||
PlexShow.year.label("year"),
|
||||
PlexShow.rating.label("rating"),
|
||||
func.left(summary_col(PlexShow, lang), 300).label("summary"),
|
||||
PlexShow.added_at.label("added_at"),
|
||||
PlexShow.originally_available_at.label("rel"),
|
||||
cast(null(), Integer).label("duration_s"),
|
||||
cast(null(), String).label("playable"),
|
||||
cast(null(), Boolean).label("transcode_hw"),
|
||||
PlexShow.child_count.label("season_count"),
|
||||
literal(0).label("position_seconds"),
|
||||
s_status.label("status"),
|
||||
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
|
||||
(inp > 0).label("resumable"),
|
||||
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
|
||||
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
|
||||
agg.c.resumed_at.label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexShow, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_lib_ids))
|
||||
)
|
||||
sq = apply_show_state_filter(sq, tot, wat, inp, show)
|
||||
sq = apply_meta_filters(sq, PlexShow, p, lang)
|
||||
if tsq is not None:
|
||||
sq = sq.where(vector_match(PlexShow, lang, tsq))
|
||||
if recheck is not None and (narrow := recheck(PlexShow, field, tsq, lang)) is not None:
|
||||
sq = sq.where(narrow)
|
||||
selects.append(sq)
|
||||
|
||||
total = 0
|
||||
rows: list = []
|
||||
if selects:
|
||||
union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
|
||||
u = union.subquery()
|
||||
total = db.scalar(select(func.count()).select_from(u)) or 0
|
||||
|
||||
if tsq is not None:
|
||||
order = [u.c.rank.desc()]
|
||||
else:
|
||||
col = sort_column(u, sort, lang)
|
||||
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
|
||||
order.append(u.c.rk.desc())
|
||||
rows = list(db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all())
|
||||
|
||||
# Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
|
||||
# and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
|
||||
# target the movie/show grid, so they suppress episodes.
|
||||
eps: list = []
|
||||
if tsq is not None and want_shows and field in ("all", "episode"):
|
||||
ep_st = aliased(PlexState)
|
||||
# In HU an episode is found by its Hungarian title too (the overlay fills it) — and
|
||||
# still by its English one, like `vector_match` does for the grid.
|
||||
if field == "episode":
|
||||
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
|
||||
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
|
||||
else:
|
||||
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
|
||||
hit = ep_vec.op("@@")(tsq)
|
||||
if lang == "hu":
|
||||
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
|
||||
ep_vec = ep_vec_hu
|
||||
eps = list(
|
||||
db.execute(
|
||||
select(PlexItem, ep_st, PlexShow)
|
||||
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, owned_by(ep_st.user_id, user)))
|
||||
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_lib_ids),
|
||||
hit,
|
||||
)
|
||||
.order_by(func.ts_rank(ep_vec, tsq).desc())
|
||||
.limit(24)
|
||||
).all()
|
||||
)
|
||||
|
||||
return total, rows, eps
|
||||
|
||||
|
||||
def facets(db: Session, user: User, *, scope: str, show: str, lang: str, f: LibraryFilters) -> dict:
|
||||
"""Available filter values for the unified library (scope movie|show|both), ADAPTED to the active
|
||||
filters (faceted search): each group's values/bounds are computed with all the OTHER active
|
||||
filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
|
||||
a multi-select group still lets you add more. Merged across the scope's libraries; duration is
|
||||
movie-only. (Search text is not factored in — facets track the sidebar filters.) `lang` is
|
||||
normalised here, after the empty answer."""
|
||||
empty = {
|
||||
"genres": [],
|
||||
"content_ratings": [],
|
||||
"studios": [],
|
||||
"playable": [],
|
||||
"year_min": None,
|
||||
"year_max": None,
|
||||
"rating_max": None,
|
||||
"duration_min": None,
|
||||
"duration_max": None,
|
||||
}
|
||||
all_movie_ids, all_show_ids = content_libs(db)
|
||||
movie_ids = all_movie_ids if scope in ("movie", "both") else []
|
||||
show_ids = all_show_ids if scope in ("show", "both") else []
|
||||
if not movie_ids and not show_ids:
|
||||
return empty
|
||||
lang = norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
|
||||
uid = user.id
|
||||
|
||||
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
|
||||
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
|
||||
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
|
||||
# The same filter on the same rows, so the counts do not change
|
||||
# (test_plex_facets_grid_parity).
|
||||
shows_in_scope: list[int] | None = None
|
||||
if show_ids and show in ("watched", "in_progress", "unwatched"):
|
||||
agg = rollup.agg_subquery(db, uid).subquery()
|
||||
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
in_scope = apply_show_state_filter(
|
||||
in_scope,
|
||||
func.coalesce(agg.c.total, 0),
|
||||
func.coalesce(agg.c.watched, 0),
|
||||
func.coalesce(agg.c.inprog, 0),
|
||||
show,
|
||||
)
|
||||
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
|
||||
|
||||
def wsq(query, model, ids, pp, kind_movie, with_dur):
|
||||
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
|
||||
facet aggregation query, so the facets reflect exactly the visible result set."""
|
||||
query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
|
||||
if kind_movie:
|
||||
query = query.where(model.kind == "movie")
|
||||
if with_dur and pp.get("duration_min") is not None:
|
||||
query = query.where(model.duration_s >= pp["duration_min"])
|
||||
if with_dur and pp.get("duration_max") is not None:
|
||||
query = query.where(model.duration_s <= pp["duration_max"])
|
||||
# Playback-class filter (movie-only) — applied for the OTHER groups so an active class
|
||||
# narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
|
||||
query = query.where(*playable_conds(split_csv(pp.get("playable"))))
|
||||
# Join watch-state and reuse the grid's exact movie state filter (unified_library) —
|
||||
# hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
|
||||
st = aliased(PlexState)
|
||||
query = query.outerjoin(st, and_(st.item_id == model.id, owned_by(st.user_id, uid)))
|
||||
query = apply_movie_state_filter(query, st, show)
|
||||
elif shows_in_scope is not None:
|
||||
query = query.where(model.id.in_(shows_in_scope))
|
||||
return query
|
||||
|
||||
genre_counts: dict[str, int] = {}
|
||||
cr_counts: dict[str, int] = {}
|
||||
studio_counts: dict[str, int] = {}
|
||||
playable_counts: dict[str, int] = {}
|
||||
years: list[int] = []
|
||||
ratings: list[float] = []
|
||||
durs: list[int] = []
|
||||
|
||||
def per_table(model, ids, kind_movie: bool) -> None:
|
||||
if not ids:
|
||||
return
|
||||
# Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
|
||||
# OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
|
||||
# offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
|
||||
# the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
|
||||
# every matching title carries them, so they stay togglable.)
|
||||
genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
|
||||
# Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
|
||||
# `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
|
||||
# jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
|
||||
# so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
|
||||
# whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
|
||||
# In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
|
||||
gcol = genres_col(model, lang)
|
||||
expanded = (
|
||||
wsq(
|
||||
select(func.jsonb_array_elements_text(gcol).label("g")),
|
||||
model,
|
||||
ids,
|
||||
genre_pp,
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(func.jsonb_typeof(gcol) == "array")
|
||||
.subquery()
|
||||
)
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
genre_counts[g] = genre_counts.get(g, 0) + c
|
||||
# Content ratings — exclude its own.
|
||||
crq = (
|
||||
wsq(
|
||||
select(model.content_rating, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "content_ratings": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.content_rating.isnot(None))
|
||||
.group_by(model.content_rating)
|
||||
)
|
||||
for cr, c in db.execute(crq).all():
|
||||
cr_counts[cr] = cr_counts.get(cr, 0) + c
|
||||
# Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
|
||||
stq = (
|
||||
wsq(
|
||||
select(model.studio, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "studios": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.studio.isnot(None))
|
||||
.group_by(model.studio)
|
||||
)
|
||||
for s, c in db.execute(stq).all():
|
||||
studio_counts[s] = studio_counts.get(s, 0) + c
|
||||
# Playback class (movie-only) — exclude its own so each class stays an addable OR option.
|
||||
if kind_movie:
|
||||
pcls = playback_class_expr()
|
||||
pcq = (
|
||||
wsq(
|
||||
select(pcls.label("cls"), func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "playable": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(pcls.isnot(None))
|
||||
.group_by(pcls)
|
||||
)
|
||||
for cls, c in db.execute(pcq).all():
|
||||
playable_counts[cls] = playable_counts.get(cls, 0) + c
|
||||
# Year bounds — exclude the year filter.
|
||||
by = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.year), func.max(model.year)),
|
||||
model,
|
||||
ids,
|
||||
{**p, "year_min": None, "year_max": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
).one()
|
||||
if by[0] is not None:
|
||||
years.append(by[0])
|
||||
if by[1] is not None:
|
||||
years.append(by[1])
|
||||
# Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
|
||||
# rating filter and the UI badge use (IMDb preferred, audience fallback).
|
||||
br = db.scalar(
|
||||
wsq(
|
||||
select(func.max(func.coalesce(model.imdb_rating, model.rating))),
|
||||
model,
|
||||
ids,
|
||||
{**p, "rating_min": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
)
|
||||
if br is not None:
|
||||
ratings.append(float(br))
|
||||
# Duration bounds (movies) — exclude the duration filter.
|
||||
if kind_movie:
|
||||
bd = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.duration_s), func.max(model.duration_s)),
|
||||
model,
|
||||
ids,
|
||||
p,
|
||||
kind_movie,
|
||||
False,
|
||||
)
|
||||
).one()
|
||||
if bd[0] is not None:
|
||||
durs.append(bd[0])
|
||||
if bd[1] is not None:
|
||||
durs.append(bd[1])
|
||||
|
||||
per_table(PlexItem, movie_ids, True)
|
||||
per_table(PlexShow, show_ids, False)
|
||||
# Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
|
||||
# be absent): the sidebar renders genres solely from this list and hides the whole genre section —
|
||||
# chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
|
||||
# undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
|
||||
for g in split_csv(f.genres):
|
||||
genre_counts.setdefault(g, 0)
|
||||
# Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
|
||||
for s in split_csv(f.studios):
|
||||
studio_counts.setdefault(s, 0)
|
||||
# Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
|
||||
# section when its facet list is empty, which would trap a zero-result selection with no way to undo).
|
||||
for c in split_csv(f.playable):
|
||||
if c in PLAYBACK_CLASSES:
|
||||
playable_counts.setdefault(c, 0)
|
||||
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
|
||||
return {
|
||||
"genres": [
|
||||
{"value": g, "count": c}
|
||||
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
|
||||
],
|
||||
"content_ratings": [
|
||||
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
|
||||
],
|
||||
"studios": [
|
||||
{"value": s, "count": c}
|
||||
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
|
||||
],
|
||||
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
|
||||
"playable": [
|
||||
{"value": c, "count": playable_counts[c]}
|
||||
for c in PLAYBACK_CLASSES
|
||||
if c in playable_counts
|
||||
],
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"rating_max": max(ratings) if ratings else None,
|
||||
"duration_min": min(durs) if durs else None,
|
||||
"duration_max": max(durs) if durs else None,
|
||||
}
|
||||
@@ -0,0 +1,463 @@
|
||||
"""Plex's tables: a self-contained catalog mirrored from a local Plex server — libraries, shows,
|
||||
seasons, items, people, collections, the server's playlists — and each user's private part of it:
|
||||
watch state, the account link, their own playlists. Only this unit writes them; another unit reads
|
||||
them along a declared edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
BigInteger,
|
||||
Boolean,
|
||||
Computed,
|
||||
Date,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin, UpdatedAtMixin
|
||||
|
||||
|
||||
# --- Plex integration (optional) -------------------------------------------------------------
|
||||
# A dedicated, self-contained catalog mirrored from a locally-reachable Plex server. Playback is
|
||||
# from the LOCAL physical file (Plex is metadata only); per-user watch state lives in plex_states
|
||||
# so it works for users without a Plex account. Mirrors the app's "shared catalog, private
|
||||
# per-user state" model. The weighted FTS search_vector uses the same unaccent_simple config as
|
||||
# `videos` (migrations 0031/0032).
|
||||
|
||||
|
||||
class PlexLibrary(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex library section. Only `enabled` sections are exposed in the feed.
|
||||
`plex_key` is the section id on the Plex server."""
|
||||
|
||||
__tablename__ = "plex_libraries"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
plex_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
title: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16)) # "movie" | "show"
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# HU-overlay support. `metadata_lang` tags the section's metadata language. An OVERLAY
|
||||
# library (overlay_of set) is the Hungarian-metadata twin of another section: it is never browsable
|
||||
# on its own — its titles' HU fields are written onto the matched EN rows (paired by `guid`).
|
||||
metadata_lang: Mapped[str] = mapped_column(String(8), default="en", server_default="en")
|
||||
overlay_of: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="SET NULL"), index=True
|
||||
)
|
||||
|
||||
|
||||
class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A TV show (grandparent of episodes) — holds poster + summary for the drill-down page, plus
|
||||
the same filterable metadata as movies (genres/rating/content_rating/studio/people) so the TV
|
||||
grid can be filtered/sorted like the movie grid. All mirrored cheaply from the show listing."""
|
||||
|
||||
__tablename__ = "plex_shows"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_shows_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_shows_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
# ix_plex_shows_collections_gin already created with collections (migration 0047).
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# Canonical external id (e.g. "tmdb:12345") — the key that pairs this EN row with its HU-overlay
|
||||
# twin during sync. Indexed for the guid->row overlay match.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(
|
||||
String(512)
|
||||
) # Plex thumb path (proxied, not stored)
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer) # seasons
|
||||
collection_keys: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # Plex collections this show is in
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the show section listing (parallels PlexItem).
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True) # Plex audienceRating (~IMDb)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True) # network/studio
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True) # first air date
|
||||
# none_as_null: an empty tag list must persist as SQL NULL, not JSON `null`. Without it a Python
|
||||
# None round-trips as a JSON scalar that jsonb_array_elements_text() can't unnest ("cannot extract
|
||||
# elements from a scalar") — the trap a metadata-poor library sprang on /facets. Same on plex_items.
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
people_text: Mapped[str | None] = mapped_column(
|
||||
Text
|
||||
) # cast+directors, folded into search_vector
|
||||
# Set once the show's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: see the identical block on plex_items for the full rationale. Additive;
|
||||
# served from the row; NULL until enriched. (`budget`/`revenue` stay NULL for shows — TV has none.) ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched,
|
||||
# so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexSeason(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A season under a show — used to order episodes for prev/next stepping."""
|
||||
|
||||
__tablename__ = "plex_seasons"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
show_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str | None] = mapped_column(String(255))
|
||||
season_number: Mapped[int | None] = mapped_column(Integer, index=True)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
|
||||
|
||||
class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A playable LEAF — a movie or a single episode. This is what the feed lists and the player
|
||||
plays. Movies carry `library_id` and no parents; episodes carry show/season refs +
|
||||
season/episode numbers. `file_path` is the Plex-side absolute path of the physical media,
|
||||
mapped through `plex_path_map` to a local mount at playback time. `playable` is the
|
||||
ffprobe-derived browser-compatibility class (direct | remux | transcode)."""
|
||||
|
||||
__tablename__ = "plex_items"
|
||||
__table_args__ = (
|
||||
Index("ix_plex_items_show_order", "show_id", "season_number", "episode_number"),
|
||||
Index("ix_plex_items_genres_gin", "genres", postgresql_using="gin"),
|
||||
Index("ix_plex_items_directors_gin", "directors", postgresql_using="gin"),
|
||||
Index("ix_plex_items_cast_gin", "cast_names", postgresql_using="gin"),
|
||||
Index("ix_plex_items_collections_gin", "collection_keys", postgresql_using="gin"),
|
||||
Index("ix_plex_items_genres_hu_gin", "genres_hu", postgresql_using="gin"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
kind: Mapped[str] = mapped_column(String(16), index=True) # "movie" | "episode"
|
||||
# Canonical external id (e.g. "tmdb:12345") — pairs a MOVIE row with its HU-overlay twin during
|
||||
# sync; indexed for the guid->row match. Episodes carry it too but it's unused for them.
|
||||
guid: Mapped[str | None] = mapped_column(String(64), index=True)
|
||||
show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_seasons.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
season_number: Mapped[int | None] = mapped_column(Integer)
|
||||
episode_number: Mapped[int | None] = mapped_column(Integer)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
year: Mapped[int | None] = mapped_column(Integer)
|
||||
duration_s: Mapped[int | None] = mapped_column(Integer)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
art_key: Mapped[str | None] = mapped_column(String(512))
|
||||
cast: Mapped[list | None] = mapped_column(JSON) # [role names]
|
||||
# Physical media (from the Plex Part).
|
||||
file_path: Mapped[str | None] = mapped_column(Text) # Plex-side absolute path
|
||||
part_key: Mapped[str | None] = mapped_column(String(255)) # /library/parts/... (fallback fetch)
|
||||
container: Mapped[str | None] = mapped_column(String(16))
|
||||
codec_video: Mapped[str | None] = mapped_column(String(32))
|
||||
codec_audio: Mapped[str | None] = mapped_column(String(32))
|
||||
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
|
||||
# Browser-playability class from ffprobe: "direct" | "remux" | "transcode" (None = unprobed).
|
||||
playable: Mapped[str | None] = mapped_column(String(12), index=True)
|
||||
probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Transcode HW-capability facet (the playback-class filter). NULL for non-transcode items; for a
|
||||
# `playable="transcode"` item: True = the prod iGPU can hardware-transcode it (codec is HW-decodable
|
||||
# AND the source is 8-bit SDR), False = software libx264 only (AV1/XviD, or a 10-bit/HDR source that
|
||||
# breaks the VAAPI nv12 pipeline — see plex/stream.py). Derived by `plex.sync.transcode_hw_capable`:
|
||||
# set codec-only at sync (bit depth unknown from the cheap listing), then refined by the bit-depth
|
||||
# backfill (`_probe_media_batch`) which downgrades a 10-bit/HDR HEVC to False. The one filterable
|
||||
# home of the invariant `_want_hw`'s source gate computes at play time — kept in sync via the shared
|
||||
# classifier so the catalog facet and the runtime encoder choice never disagree by codec.
|
||||
transcode_hw: Mapped[bool | None] = mapped_column(Boolean, index=True)
|
||||
# Video bit depth (8/10/12…) from the Plex media Stream, filled by the bit-depth backfill. NULL =
|
||||
# not yet probed. Surfaced on the info page (e.g. "HEVC · 10-bit") and feeds `transcode_hw`.
|
||||
video_bit_depth: Mapped[int | None] = mapped_column(Integer)
|
||||
# Watermark for the bit-depth backfill: when this item's media Stream was last probed for
|
||||
# bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
|
||||
# probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
|
||||
media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
|
||||
# Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
|
||||
# `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
|
||||
# filters (GIN-indexed) and double as content signals for a future watch-habit recommender.
|
||||
rating: Mapped[float | None] = mapped_column(Float, index=True)
|
||||
content_rating: Mapped[str | None] = mapped_column(String(16), index=True)
|
||||
studio: Mapped[str | None] = mapped_column(String(255), index=True)
|
||||
originally_available_at: Mapped[Date | None] = mapped_column(Date, index=True)
|
||||
genres: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
directors: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
cast_names: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# rating_keys of the Plex collections this movie belongs to (GIN — filter + card-strip siblings).
|
||||
collection_keys: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
# Cast + director names, space-joined — folded into search_vector (weight B) so the search box
|
||||
# finds titles by a person's name (and ranks them above summary-only mentions).
|
||||
people_text: Mapped[str | None] = mapped_column(Text)
|
||||
# Set once the movie's FULL cast has been rich-fetched by the backfill; NULL = still the listing top-3.
|
||||
people_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Rich-fetch-only detail fields, persisted by the same backfill that fills the full cast so
|
||||
# the info page serves from this row instead of a live plex.metadata() call. NULL until enriched.
|
||||
imdb_rating: Mapped[float | None] = mapped_column(
|
||||
Float
|
||||
) # IMDb-specific score (vs `rating` = audienceRating)
|
||||
imdb_id: Mapped[str | None] = mapped_column(String(16)) # tt… → the clickable IMDb link
|
||||
tagline: Mapped[str | None] = mapped_column(Text)
|
||||
full_cast: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # ordered [{name, role, thumb}]
|
||||
# --- TMDB enrichment: matched by the canonical `guid` ("tmdb:12345"), filled by the TMDB
|
||||
# backfill job, SERVED FROM THIS ROW (never a live TMDB call). Additive — imdb_rating/rating
|
||||
# above stay Plex-sourced; `tmdb_rating` is the distinct TMDb community score. NULL until enriched;
|
||||
# `tmdb_enriched_at` is the "tried" watermark (stamped on a definitive no-match too, so an
|
||||
# unresolvable title isn't re-fetched every run; left NULL only on a transient fetch error). ---
|
||||
tmdb_id: Mapped[int | None] = mapped_column(Integer)
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
tmdb_rating: Mapped[float | None] = mapped_column(Float) # TMDb vote_average (0–10)
|
||||
tmdb_vote_count: Mapped[int | None] = mapped_column(Integer)
|
||||
budget: Mapped[int | None] = mapped_column(BigInteger) # USD (movies; TV has none)
|
||||
revenue: Mapped[int | None] = mapped_column(BigInteger)
|
||||
keywords: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
crew: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{name, job, tmdb_id, thumb}]
|
||||
studios: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB production companies [str]
|
||||
production_countries: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
# [{tmdb_id, title, poster, media_type}] — resolved to library rows (by guid) at serve time.
|
||||
recommendations: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
backdrops: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # TMDB backdrop file_paths [str] (crossfade pool)
|
||||
# Hungarian-metadata twin fields, written by the overlay sync. NULL = no HU twin matched
|
||||
# (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
|
||||
title_hu: Mapped[str | None] = mapped_column(String(512))
|
||||
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline
|
||||
summary_hu: Mapped[str | None] = mapped_column(Text)
|
||||
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
|
||||
search_vector: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
search_vector_hu: Mapped[object | None] = mapped_column(
|
||||
TSVECTOR,
|
||||
Computed(
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
|
||||
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')",
|
||||
persisted=True,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A small cast/crew headshot cache, keyed by name, so person filter-chips (the cards for the
|
||||
actors currently in the filter) can show a photo + count without a live per-card metadata fetch.
|
||||
Populated opportunistically when an item's RICH metadata is fetched — the info-page view and the
|
||||
periodic rich re-sync — since the cheap section listing that drives the main sync only carries the
|
||||
top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
|
||||
removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
|
||||
metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
|
||||
row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
|
||||
(the headshot FILE lives in the image cache, filled by the people_enrich job)."""
|
||||
|
||||
__tablename__ = "plex_people"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String(255), unique=True, index=True)
|
||||
photo_thumb: Mapped[str | None] = mapped_column(Text) # proxied /person-image URL, or None
|
||||
# TMDB person id, linked by name from a title's TMDB credits (the /person
|
||||
# biography fetch builds on it). NULL until a TMDB-enriched title names this person.
|
||||
tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
|
||||
# --- TMDB person enrichment: fetched from /person/{tmdb_person_id} by the people_enrich job,
|
||||
# or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
|
||||
# no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
|
||||
# media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
|
||||
biography: Mapped[str | None] = mapped_column(Text)
|
||||
# TMDB's Hungarian biography, from the same request's `translations`; a Hungarian reader is
|
||||
# served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
|
||||
# keywords have no Hungarian on TMDB, so they stay in the instance language.
|
||||
biography_hu: Mapped[str | None] = mapped_column(Text)
|
||||
birthday: Mapped[str | None] = mapped_column(String(10)) # "YYYY-MM-DD", served as-is
|
||||
deathday: Mapped[str | None] = mapped_column(String(10))
|
||||
place_of_birth: Mapped[str | None] = mapped_column(Text)
|
||||
known_for_department: Mapped[str | None] = mapped_column(String(64))
|
||||
also_known_as: Mapped[list | None] = mapped_column(JSONB(none_as_null=True)) # [str]
|
||||
known_for: Mapped[list | None] = mapped_column(
|
||||
JSONB(none_as_null=True)
|
||||
) # [{tmdb_id, media_type, title}]
|
||||
tmdb_enriched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexCollection(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A mirrored Plex collection (a grouping of movies or shows — franchises like Avatar, curated
|
||||
sets like MCU, or auto/community lists like IMDb Top 250). Mirrored in full by the background
|
||||
sync so reads never hit Plex's slow collection queries. Membership is stored as `collection_keys`
|
||||
on the member movies (plex_items) / shows (plex_shows). `smart` = Plex's query-based flag;
|
||||
`editable` = we may write changes back to Plex (default False = read-only; set for curated ones)."""
|
||||
|
||||
__tablename__ = "plex_collections"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
rating_key: Mapped[str] = mapped_column(String(32), unique=True, index=True)
|
||||
library_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_libraries.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
summary: Mapped[str | None] = mapped_column(Text)
|
||||
thumb_key: Mapped[str | None] = mapped_column(String(512))
|
||||
child_count: Mapped[int | None] = mapped_column(Integer)
|
||||
smart: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
editable: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a Plex item — mirrors VideoState. Lives in Siftlode's own DB so
|
||||
it works for users WITHOUT a Plex account (the whole point of the access-layer design).
|
||||
`synced_to_plex` marks whether this state was pushed back to a linked Plex account."""
|
||||
|
||||
__tablename__ = "plex_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
synced_to_plex: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class PlexLink(Base, UpdatedAtMixin):
|
||||
"""Links a Siftlode user to a Plex account for two-way watch-state sync (one row per user).
|
||||
|
||||
For the owner (the MVP scope) the row uses the server's admin token — `uses_admin=True`,
|
||||
`plex_token_enc=None` — so no separate Plex login is needed; the admin token already reads/writes
|
||||
the owner account's watch state. `plex_account_id` is that Plex account's numeric id, used to
|
||||
filter the shared watch-history feed to this user (Phase C). Reserved for later: a friend can link
|
||||
their OWN Plex account via PIN OAuth, storing a personal `plex_token_enc` (uses_admin=False) — the
|
||||
sync loop then iterates link rows uniformly.
|
||||
|
||||
`initial_import_done` guards the one-time "Plex is master" import; `last_watch_sync_at` is the
|
||||
high-water mark for the incremental Plex→Siftlode pass."""
|
||||
|
||||
__tablename__ = "plex_link"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), unique=True, index=True
|
||||
)
|
||||
uses_admin: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
|
||||
plex_token_enc: Mapped[str | None] = mapped_column(Text) # P5b friend tokens (encrypted)
|
||||
plex_account_id: Mapped[int | None] = mapped_column(Integer)
|
||||
plex_username: Mapped[str | None] = mapped_column(String(255))
|
||||
sync_enabled: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
initial_import_done: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default="false"
|
||||
)
|
||||
last_watch_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
linked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class PlexPlaylist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user ORDERED watch-list of Plex items — Siftlode-native (lives in our own DB, works for
|
||||
users WITHOUT a Plex account, like plex_states). Unlike collections (shared, library-level), a
|
||||
playlist is owned by one user and its items are ordered. Optional Plex-direction sync is a later
|
||||
phase (`plex_rating_key` set once pushed back to the owner's Plex account)."""
|
||||
|
||||
__tablename__ = "plex_playlists"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
title: Mapped[str] = mapped_column(String(512))
|
||||
plex_rating_key: Mapped[str | None] = mapped_column(String(32))
|
||||
|
||||
|
||||
class PlexPlaylistItem(Base):
|
||||
"""One ordered entry in a playlist. `position` (0-based) orders playback."""
|
||||
|
||||
__tablename__ = "plex_playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("plex_playlists.id", ondelete="CASCADE"))
|
||||
item_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
@@ -13,7 +13,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexShow
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex.client import PlexClient
|
||||
from app.plex.genres import hu_genres
|
||||
|
||||
@@ -15,7 +15,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexPerson
|
||||
from app.plex.models import PlexPerson
|
||||
from app.plex import imgcache
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
|
||||
|
||||
@@ -0,0 +1,507 @@
|
||||
"""Plex's reads for the HTTP layer: the mirrored rows a page shows, and a user's watch state on them.
|
||||
|
||||
A user's own rows (`PlexState`) are read through `userscope`, so "someone else's row" is the same
|
||||
answer as a missing one. A rating key with a NUL in it names no row (`lookups.by_rating_key`).
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated
|
||||
from app.models import User
|
||||
from app.plex import rollup
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexLink,
|
||||
PlexPerson,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
)
|
||||
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
|
||||
from app.userscope import owned_by, owned_row
|
||||
from app.utils import like_escape
|
||||
|
||||
# --- one row ---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def item(db: Session, rating_key: str) -> PlexItem | None:
|
||||
return by_rating_key(db, PlexItem, rating_key)
|
||||
|
||||
|
||||
def movie(db: Session, rating_key: str) -> PlexItem | None:
|
||||
"""The movie with this key — an episode's key is not one."""
|
||||
it = item(db, rating_key)
|
||||
return it if it is not None and it.kind == "movie" else None
|
||||
|
||||
|
||||
def show(db: Session, rating_key: str) -> PlexShow | None:
|
||||
return by_rating_key(db, PlexShow, rating_key)
|
||||
|
||||
|
||||
def collection(db: Session, rating_key: str) -> PlexCollection | None:
|
||||
return by_rating_key(db, PlexCollection, rating_key)
|
||||
|
||||
|
||||
def season(db: Session, rating_key: str) -> PlexSeason | None:
|
||||
return by_rating_key(db, PlexSeason, rating_key)
|
||||
|
||||
|
||||
def season_episodes(db: Session, se: PlexSeason) -> list[PlexItem]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode")
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def show_by_id(db: Session, show_id: int) -> PlexShow | None:
|
||||
return db.get(PlexShow, show_id)
|
||||
|
||||
|
||||
def season_by_id(db: Session, season_id: int) -> PlexSeason | None:
|
||||
return db.get(PlexSeason, season_id)
|
||||
|
||||
|
||||
def library(db: Session, library_id: int) -> PlexLibrary | None:
|
||||
return db.get(PlexLibrary, library_id)
|
||||
|
||||
|
||||
def library_by_key(db: Session, plex_key: str) -> PlexLibrary | None:
|
||||
return (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(plex_key)))
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def library_key(db: Session, library_id: int | None) -> str | None:
|
||||
"""The Plex section key for a library id — the info/show page passes it to the admin collection
|
||||
editor (in the unified cross-library view there's no single library prop otherwise)."""
|
||||
if library_id is None:
|
||||
return None
|
||||
lib = db.get(PlexLibrary, library_id)
|
||||
return lib.plex_key if lib else None
|
||||
|
||||
|
||||
def person(db: Session, name: str) -> PlexPerson | None:
|
||||
return db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
|
||||
|
||||
|
||||
# --- a user's watch state --------------------------------------------------------------------
|
||||
|
||||
|
||||
def state(db: Session, user: User | int, item_id: int) -> PlexState | None:
|
||||
return (
|
||||
db.execute(
|
||||
select(PlexState).where(owned_by(PlexState.user_id, user), PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
# --- a show ----------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def seasons(db: Session, sh: PlexShow) -> list[PlexSeason]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexSeason)
|
||||
.where(PlexSeason.show_id == sh.id)
|
||||
.order_by(PlexSeason.season_number.asc().nullsfirst())
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def episodes(db: Session, sh: PlexShow) -> list[PlexItem]:
|
||||
"""A show's episodes in season/episode order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def episode_keys(db: Session, show_id: int) -> list[str]:
|
||||
"""The rating keys of a show's episodes, in the order `episodes` gives."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem.rating_key)
|
||||
.where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def shows_by_keys(db: Session, rating_keys: list[str], exclude: int) -> dict[str, PlexShow]:
|
||||
"""The mirrored shows among these keys, but `exclude`, by key."""
|
||||
return {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rating_keys), PlexShow.id != exclude)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
|
||||
# --- libraries -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def browsable_libraries(db: Session, lang: str) -> list[tuple[PlexLibrary, int]]:
|
||||
"""The mirrored libraries a user can browse, each with its title count (movies, or shows). An
|
||||
HU overlay library is left out: it is never browsed, it only feeds its EN twin's Hungarian
|
||||
metadata."""
|
||||
libs = db.execute(
|
||||
select(PlexLibrary)
|
||||
.where(PlexLibrary.overlay_of.is_(None))
|
||||
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang))
|
||||
).scalars()
|
||||
out = []
|
||||
for lib in libs:
|
||||
if lib.kind == "movie":
|
||||
count = db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.library_id == lib.id, PlexItem.kind == "movie"
|
||||
)
|
||||
)
|
||||
else:
|
||||
count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
|
||||
out.append((lib, count or 0))
|
||||
return out
|
||||
|
||||
|
||||
# --- people ----------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def person_photos(db: Session, names: list[str]) -> dict[str, str | None]:
|
||||
return {
|
||||
p.name: p.photo_thumb
|
||||
for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(names))).scalars()
|
||||
}
|
||||
|
||||
|
||||
def cast_count(db: Session, name: str, movie_lib_ids: list[int], show_lib_ids: list[int]) -> int:
|
||||
"""How many movies (in `movie_lib_ids`) and shows (in `show_lib_ids`) name this person in their
|
||||
cast. An empty id list leaves its side out."""
|
||||
count = 0
|
||||
if movie_lib_ids:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
PlexItem.cast_names.contains([name]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
if show_lib_ids:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexShow.id)).where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
PlexShow.cast_names.contains([name]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return count
|
||||
|
||||
|
||||
def filmography(
|
||||
db: Session,
|
||||
kind: Literal["movie", "show"],
|
||||
name: str,
|
||||
lib_ids: list[int],
|
||||
lang: str,
|
||||
limit: int,
|
||||
) -> list:
|
||||
"""The movies or shows in these libraries whose cast or directors name this person, newest
|
||||
first."""
|
||||
model: type[PlexItem] | type[PlexShow] = PlexItem if kind == "movie" else PlexShow
|
||||
query = select(model).where(
|
||||
model.library_id.in_(lib_ids),
|
||||
or_(model.cast_names.contains([name]), model.directors.contains([name])),
|
||||
)
|
||||
if model is PlexItem:
|
||||
query = query.where(PlexItem.kind == "movie")
|
||||
return list(
|
||||
db.execute(
|
||||
query.order_by(
|
||||
model.year.desc().nullslast(), collated(title_col(model, lang), lang)
|
||||
).limit(limit)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
# --- collections -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def collections_page(
|
||||
db: Session, library: str | None, q: str | None, keys: list[str] | None, lang: str
|
||||
) -> list[PlexCollection]:
|
||||
"""Collections, most-populated first: one library's (by its plex key), else every enabled
|
||||
library's; name-filtered by `q`; narrowed to `keys` when given (an empty list asks for none)."""
|
||||
if library is not None:
|
||||
lib = library_by_key(db, library)
|
||||
if lib is None:
|
||||
return []
|
||||
query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
|
||||
else:
|
||||
enabled_ids = list(
|
||||
db.execute(
|
||||
select(PlexLibrary.id).where(
|
||||
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
if not enabled_ids:
|
||||
return []
|
||||
query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
|
||||
term = (q or "").strip()
|
||||
if term:
|
||||
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
|
||||
if keys is not None:
|
||||
# An explicit but empty `keys=` asks for nothing, which is not the same as no `keys` (every
|
||||
# collection): answered here without a query. The empty IN () would also match nothing.
|
||||
if not keys:
|
||||
return []
|
||||
query = query.where(PlexCollection.rating_key.in_(keys))
|
||||
return list(
|
||||
db.execute(
|
||||
query.order_by(
|
||||
PlexCollection.child_count.desc().nullslast(), collated(PlexCollection.title, lang)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def library_collections(db: Session, library_id: int, lang: str) -> list[PlexCollection]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexCollection)
|
||||
.where(PlexCollection.library_id == library_id)
|
||||
.order_by(collated(PlexCollection.title, lang))
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def collections_by_keys(db: Session, keys: list[str]) -> list[PlexCollection]:
|
||||
return list(
|
||||
db.execute(select(PlexCollection).where(PlexCollection.rating_key.in_(keys))).scalars()
|
||||
)
|
||||
|
||||
|
||||
def collection_named(db: Session, library_id: int, title: str) -> PlexCollection | None:
|
||||
"""A collection of this library with this name, case aside."""
|
||||
return (
|
||||
db.execute(
|
||||
select(PlexCollection).where(
|
||||
PlexCollection.library_id == library_id,
|
||||
func.lower(PlexCollection.title) == title.lower(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
|
||||
|
||||
def collection_members(
|
||||
db: Session, rating_key: str, lang: str
|
||||
) -> tuple[list[PlexItem], list[PlexShow]]:
|
||||
"""A collection's movies and shows, each by year then title."""
|
||||
movies = db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
).scalars()
|
||||
shows = db.execute(
|
||||
select(PlexShow)
|
||||
.where(PlexShow.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
|
||||
).scalars()
|
||||
return list(movies), list(shows)
|
||||
|
||||
|
||||
def library_movies_in(
|
||||
db: Session, library_id: int, rating_key: str, lang: str, limit: int
|
||||
) -> list[PlexItem]:
|
||||
"""The movies of one library in this collection, by year then title."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.library_id == library_id,
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.collection_keys.contains([rating_key]),
|
||||
)
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
.limit(limit)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def show_rollups(db: Session, user_id: int, show_ids: list[int]) -> list:
|
||||
"""The user's episode rollup (total, watched, in progress, last resumed) of each of these shows
|
||||
that has episodes."""
|
||||
if not show_ids:
|
||||
return []
|
||||
return list(
|
||||
db.execute(
|
||||
rollup.agg_subquery(db, user_id, include_resumed=True).where(
|
||||
PlexItem.show_id.in_(show_ids)
|
||||
)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def collection_tiles(
|
||||
db: Session, keys: list[str]
|
||||
) -> tuple[dict[str, list[str]], dict[str, list[str]], dict[str, int]]:
|
||||
"""What each collection's tile shows: the top two movie members' keys, the top two show members'
|
||||
keys (a collection lives in one library, so exactly one side has any), and its movies' total
|
||||
runtime (a show has no single duration)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
|
||||
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
|
||||
runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
|
||||
return art_m, art_s, runtime
|
||||
|
||||
|
||||
# --- the admin page --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def sample_file_path(db: Session) -> str | None:
|
||||
"""Any mirrored item's Plex-side file path — a sample to test the mount and path map against."""
|
||||
return db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
|
||||
|
||||
|
||||
def watch_link(db: Session, user: User | int) -> PlexLink | None:
|
||||
"""The user's Plex watch-sync link, if they have one."""
|
||||
return db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user))).scalars().first()
|
||||
|
||||
|
||||
# --- a user's playlists ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_playlist(
|
||||
db: Session, user: User, playlist_id: int, *, lock: bool = False
|
||||
) -> PlexPlaylist | None:
|
||||
"""The user's playlist with this id; someone else's is the same answer as a missing one. `lock`
|
||||
takes it FOR UPDATE: every add, remove and reorder does, so two of them never read the same max
|
||||
position, and the list row is always locked before its items."""
|
||||
return owned_row(db, PlexPlaylist, playlist_id, user, lock=lock)
|
||||
|
||||
|
||||
def playlists_of(db: Session, user: User) -> list[PlexPlaylist]:
|
||||
"""The user's playlists, newest-touched first."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlexPlaylist)
|
||||
.where(owned_by(PlexPlaylist.user_id, user))
|
||||
.order_by(PlexPlaylist.updated_at.desc())
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def playlist_count(db: Session, playlist_id: int) -> int:
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def _playlist_rows(playlist_id: int):
|
||||
return (
|
||||
select(PlexPlaylistItem, PlexItem)
|
||||
.join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
|
||||
.where(PlexPlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
)
|
||||
|
||||
|
||||
def playlist_cover(db: Session, playlist_id: int) -> str | None:
|
||||
"""The rating key of a playlist's first item — its cover."""
|
||||
first = db.execute(_playlist_rows(playlist_id)).first()
|
||||
return first[1].rating_key if first else None
|
||||
|
||||
|
||||
def playlist_items(db: Session, playlist_id: int) -> list[PlexItem]:
|
||||
"""A playlist's items in play order."""
|
||||
return [it for _row, it in db.execute(_playlist_rows(playlist_id)).all()]
|
||||
|
||||
|
||||
def playlist_holds(db: Session, playlist_id: int, item_ids: set[int]) -> int:
|
||||
"""How many of these items the playlist holds."""
|
||||
if not item_ids:
|
||||
return 0
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id,
|
||||
PlexPlaylistItem.item_id.in_(item_ids),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
|
||||
def item_id(db: Session, rating_key: str) -> int | None:
|
||||
"""The id of the mirrored item with this key. Unlike `item`, a NUL in the key reaches the query,
|
||||
which the app answers as a 422 (`main._nul_in_text`), as for every other compared string."""
|
||||
return db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(rating_key)))
|
||||
|
||||
|
||||
def item_ids(db: Session, rating_keys: list[str]) -> set[int]:
|
||||
"""The ids of the mirrored items with these keys."""
|
||||
if not rating_keys:
|
||||
return set()
|
||||
return set(
|
||||
db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rating_keys))).scalars()
|
||||
)
|
||||
|
||||
|
||||
def items_by_keys(db: Session, rating_keys: list) -> list[PlexItem]:
|
||||
"""The mirrored items with these keys, in the given order; an unknown key and a repeat drop out."""
|
||||
order = [str(k) for k in rating_keys if str(k)]
|
||||
if not order:
|
||||
return []
|
||||
by_rk = {
|
||||
it.rating_key: it
|
||||
for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
|
||||
}
|
||||
seen: set[str] = set()
|
||||
out: list[PlexItem] = []
|
||||
for rk in order:
|
||||
it = by_rk.get(rk)
|
||||
if it is not None and rk not in seen:
|
||||
seen.add(rk)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
|
||||
def shows_by_ids(db: Session, show_ids: set[int]) -> dict[int, PlexShow]:
|
||||
if not show_ids:
|
||||
return {}
|
||||
return {
|
||||
sh.id: sh for sh in db.execute(select(PlexShow).where(PlexShow.id.in_(show_ids))).scalars()
|
||||
}
|
||||
@@ -10,7 +10,7 @@ from sqlalchemy import and_, case, func, or_, select, type_coerce
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import PlexItem, PlexLibrary
|
||||
from app.plex.models import PlexItem, PlexLibrary
|
||||
from app.plex.genres import hu_genre
|
||||
|
||||
TS_CONFIG = "public.unaccent_simple"
|
||||
@@ -236,7 +236,7 @@ def apply_meta_filters(q, model, p: dict, lang: str = "en"):
|
||||
# The 4 classes the feed filter + facet expose, derived from the stored `playable` + `transcode_hw`.
|
||||
# ONE definition (this SQL CASE) so the filter WHERE and the facet counts can't drift: `playable_conds`
|
||||
# filters on it, `facets` groups by it. transcode_hw IS NOT True groups NULL (unprobed) with software.
|
||||
# Public — it is the shared source of truth imported by routes.plex.browse (facet order + zero-fill).
|
||||
# Public — it is the shared source of truth imported by plex.library (facet order + zero-fill).
|
||||
PLAYBACK_CLASSES = ("direct", "remux", "transcode_hw", "transcode_sw")
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,9 @@ from datetime import datetime, timezone
|
||||
from sqlalchemy import and_, case, func, or_, select
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
|
||||
from app.models import PlexItem, PlexState
|
||||
from app.models import User
|
||||
from app.userscope import owned_by
|
||||
from app.plex.models import PlexItem, PlexState
|
||||
from app.plex.cards import ShowState
|
||||
|
||||
# Sort sentinel for a rail row whose timestamp is missing, so `key=lambda r: r[0] or NO_TS` reads the
|
||||
@@ -92,7 +94,7 @@ def agg_subquery(db: Session, user_id: int, include_resumed: bool = False):
|
||||
)
|
||||
return (
|
||||
select(*cols)
|
||||
.outerjoin(ps, and_(ps.item_id == PlexItem.id, ps.user_id == user_id))
|
||||
.outerjoin(ps, and_(ps.item_id == PlexItem.id, owned_by(ps.user_id, user_id)))
|
||||
.where(PlexItem.kind == "episode", PlexItem.show_id.isnot(None))
|
||||
.group_by(PlexItem.show_id)
|
||||
)
|
||||
@@ -105,7 +107,7 @@ def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, ShowS
|
||||
return {r.show_id: show_state(r.total, r.watched, r.inprog) for r in rows}
|
||||
|
||||
|
||||
def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int, PlexState]:
|
||||
def item_state_map(db: Session, user_id: User | int, item_ids: list[int]) -> dict[int, PlexState]:
|
||||
"""A user's PlexState rows for a set of leaf items (movies or episodes), keyed by item_id — the
|
||||
leaf analogue of `state_map` (which rolls a show's episodes up to a `ShowState`). One batched
|
||||
fetch, so the caller serializes cards without a per-card query. Empty in → empty dict (no `IN ()`)."""
|
||||
@@ -114,6 +116,8 @@ def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int,
|
||||
return {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id.in_(item_ids))
|
||||
select(PlexState).where(
|
||||
owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(item_ids)
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,413 @@
|
||||
"""Plex's writes for the HTTP layer, one function per use case. None of them commits: the handler
|
||||
does, so a page that fails to save an enrichment still renders, and a Plex write and its local
|
||||
mirror land together.
|
||||
|
||||
The rest of the mirror's writes stay with the sync (`plex.sync`): a collection's targeted re-sync
|
||||
after a write-back, its local cleanup after a delete.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import CursorResult, delete, func, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.db import SessionLocal
|
||||
from app.plex import detail as plex_detail
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup
|
||||
from app.plex.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexLink,
|
||||
PlexPerson,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
)
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def enrich_show(db: Session, sh: PlexShow, meta: dict) -> None:
|
||||
"""Fill a not-yet-enriched show from its live metadata: the full cast (filterable), IMDb and
|
||||
tagline, stamped so later views serve from the row."""
|
||||
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
|
||||
plex_detail.apply_rich_detail(sh, meta)
|
||||
sh.people_enriched_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def enrich_movie(db: Session, it: PlexItem, meta: dict) -> None:
|
||||
"""The same for a movie. The listing sync stored only its top ~3 cast; the live metadata
|
||||
carries all of them."""
|
||||
cast = plex_people.cast_from_meta(meta)
|
||||
if cast:
|
||||
plex_people.enrich_row_people(db, it, cast)
|
||||
plex_detail.apply_rich_detail(it, meta)
|
||||
it.people_enriched_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def set_collection_editable(col: PlexCollection, editable: bool) -> None:
|
||||
col.editable = editable
|
||||
|
||||
|
||||
def keep_created_collection(
|
||||
db: Session, lib: PlexLibrary, rating_key: str, title: str, seed: str
|
||||
) -> None:
|
||||
"""A collection Plex made, whose follow-up read failed: its row is kept, editable. The full sync
|
||||
preserves `editable` but creates a missing row read-only, so the collection would drop out of
|
||||
every "add to" list; and the duplicate-name check reads these rows, so a retry of the same name
|
||||
would make a second one on Plex. Plex made it with exactly the seed in it, so the seed's row says
|
||||
so too — otherwise the title showed unticked in the collection it was just added to, and the
|
||||
collection empty, until that sync."""
|
||||
db.add(
|
||||
PlexCollection(
|
||||
rating_key=rating_key, library_id=lib.id, title=title, editable=True, child_count=1
|
||||
)
|
||||
)
|
||||
Model = PlexItem if lib.kind == "movie" else PlexShow
|
||||
if row := by_rating_key(db, Model, seed):
|
||||
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), rating_key]))
|
||||
db.flush()
|
||||
|
||||
|
||||
def enrich_person_bg(person_id: int) -> None:
|
||||
"""Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
|
||||
AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
|
||||
"no live TMDB call at serve time" invariant). The request's session is already closed by then, so
|
||||
open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
|
||||
next open, and the bio lands from the row by then."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
person = db.get(PlexPerson, person_id)
|
||||
if person is not None:
|
||||
plex_tmdb.enrich_person(db, person)
|
||||
except Exception: # noqa: BLE001 — background best-effort; never surfaces to a user
|
||||
db.rollback()
|
||||
log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def set_watch_link(db: Session, user: User, enabled: bool) -> PlexLink:
|
||||
"""Turn the user's two-way watch sync on or off, creating their link on first use. The link uses
|
||||
the server admin token (`uses_admin`), so it needs no Plex login of its own."""
|
||||
link = queries.watch_link(db, user)
|
||||
if link is None:
|
||||
link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
|
||||
db.add(link)
|
||||
link.sync_enabled = enabled
|
||||
db.flush()
|
||||
return link
|
||||
|
||||
|
||||
# --- a user's playlists ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_playlist(db: Session, user: User, title: str, seed: PlexItem | None) -> PlexPlaylist:
|
||||
"""A new playlist of the user's, holding `seed` when one is given."""
|
||||
pl = PlexPlaylist(user_id=user.id, title=title)
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
if seed is not None:
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=seed.id, position=0))
|
||||
db.flush()
|
||||
return pl
|
||||
|
||||
|
||||
def rename_playlist(pl: PlexPlaylist, title: str) -> None:
|
||||
pl.title = title
|
||||
|
||||
|
||||
def delete_playlist(db: Session, pl: PlexPlaylist) -> None:
|
||||
db.delete(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def _touch(pl: PlexPlaylist) -> None:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _last_position(db: Session, playlist_id: int) -> int:
|
||||
return db.execute(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
).scalar_one()
|
||||
|
||||
|
||||
def add_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
|
||||
"""Append `it` to the (locked) playlist; an item already in it stays where it is."""
|
||||
exists = db.execute(
|
||||
select(PlexPlaylistItem.id).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
).first()
|
||||
if exists is None:
|
||||
pos = _last_position(db, pl.id) + 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def remove_item(db: Session, pl: PlexPlaylist, it: PlexItem) -> None:
|
||||
db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def add_items(db: Session, pl: PlexPlaylist, items: list[PlexItem]) -> int:
|
||||
"""Append many items in their given order (a whole season or show); those already in the list
|
||||
are skipped. Says how many were added."""
|
||||
existing = set(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
)
|
||||
pos = _last_position(db, pl.id)
|
||||
added = 0
|
||||
for it in items:
|
||||
if it.id in existing:
|
||||
continue
|
||||
pos += 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=pos))
|
||||
existing.add(it.id)
|
||||
added += 1
|
||||
if added:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return added
|
||||
|
||||
|
||||
def insert_items_at(db: Session, pl: PlexPlaylist, placed: list[tuple[int, PlexItem]]) -> int:
|
||||
"""Insert new rows at their indices (ascending, so each index is where the row stood when the
|
||||
rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
|
||||
already in the list does not move: the add stays idempotent."""
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem)
|
||||
.where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
present = {r.item_id for r in rows}
|
||||
added = 0
|
||||
for at, it in sorted(placed, key=lambda p: p[0]):
|
||||
if it.id in present:
|
||||
continue
|
||||
row = PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0)
|
||||
db.add(row)
|
||||
rows.insert(min(at, len(rows)), row)
|
||||
present.add(it.id)
|
||||
added += 1
|
||||
for i, r in enumerate(rows):
|
||||
r.position = i
|
||||
if added:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return added
|
||||
|
||||
|
||||
def remove_items(db: Session, pl: PlexPlaylist, item_ids: list[int]) -> int:
|
||||
"""Remove these items from the playlist; says how many were there."""
|
||||
if not item_ids:
|
||||
return 0
|
||||
result = db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id, PlexPlaylistItem.item_id.in_(item_ids)
|
||||
),
|
||||
execution_options={"synchronize_session": False},
|
||||
)
|
||||
removed = result.rowcount if isinstance(result, CursorResult) else 0
|
||||
if removed:
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
return removed
|
||||
|
||||
|
||||
def reorder_playlist(db: Session, pl: PlexPlaylist, rating_keys: list[str]) -> None:
|
||||
"""Set the play order from an explicit list of item keys (0-based positions)."""
|
||||
id_by_rk = {
|
||||
it.rating_key: it.id
|
||||
for it in db.execute(
|
||||
select(PlexItem).where(PlexItem.rating_key.in_(rating_keys or [""]))
|
||||
).scalars()
|
||||
}
|
||||
rows = {
|
||||
r.item_id: r
|
||||
for r in db.execute(
|
||||
select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[int] = set()
|
||||
for rk in rating_keys:
|
||||
row = rows.get(id_by_rk.get(rk, -1))
|
||||
if row is not None and row.item_id not in seen:
|
||||
row.position = pos
|
||||
pos += 1
|
||||
seen.add(row.item_id)
|
||||
# Rows the payload omitted (added in another tab meanwhile) keep their relative order after
|
||||
# the given ones, so no two rows share a position (the YouTube reorder does the same).
|
||||
for row in sorted(
|
||||
(r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
|
||||
):
|
||||
row.position = pos
|
||||
pos += 1
|
||||
_touch(pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- a user's watch state --------------------------------------------------------------------
|
||||
|
||||
|
||||
def mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
|
||||
"""Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
|
||||
checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
|
||||
read-then-write let both see "not watched" and both scrobble. The transition is decided by the
|
||||
write itself: an insert that did not conflict, or an update conditional on the old status —
|
||||
under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
|
||||
values = {
|
||||
"status": "watched",
|
||||
"watched_at": now,
|
||||
"position_seconds": 0,
|
||||
"progress_updated_at": now,
|
||||
"synced_to_plex": False,
|
||||
}
|
||||
inserted = db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(user_id=user_id, item_id=item_id, **values)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
.returning(PlexState.id)
|
||||
).first()
|
||||
if inserted is not None:
|
||||
return True
|
||||
mine = owned_by(PlexState.user_id, user_id) & (PlexState.item_id == item_id)
|
||||
result = db.execute(update(PlexState).where(mine, PlexState.status != "watched").values(values))
|
||||
if isinstance(result, CursorResult) and result.rowcount:
|
||||
return True
|
||||
# Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
|
||||
db.execute(update(PlexState).where(mine).values(values))
|
||||
return False
|
||||
|
||||
|
||||
def state_row(
|
||||
db: Session,
|
||||
user: User | int,
|
||||
item_id: int,
|
||||
st: PlexState | None = None,
|
||||
*,
|
||||
synced: bool = False,
|
||||
) -> PlexState:
|
||||
"""The user's state row on this item, made if the read before found none. Made by the insert
|
||||
itself (ON CONFLICT DO NOTHING) and then read: the first checkpoint of an item and the `final`
|
||||
flush beside it routinely both read "none", and a plain add let the second one fail on
|
||||
`uq_plex_user_item` — a 500, and the push it was carrying lost. Every first row goes through
|
||||
here: a checkpoint, a status, a season marked at once, a pull from Plex. `synced` is what a row
|
||||
made now says about Plex; a row another writer made first keeps its own."""
|
||||
if st is not None:
|
||||
return st
|
||||
db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(
|
||||
user_id=user if isinstance(user, int) else user.id,
|
||||
item_id=item_id,
|
||||
synced_to_plex=synced,
|
||||
)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
)
|
||||
made = queries.state(db, user, item_id)
|
||||
assert made is not None # inserted now, or committed by the request that won
|
||||
return made
|
||||
|
||||
|
||||
def clear_progress(db: Session, st: PlexState | None, now: datetime) -> None:
|
||||
"""A checkpoint too early to keep: a state that held nothing else goes, any other forgets its
|
||||
position."""
|
||||
if st is not None and st.status == "new":
|
||||
db.delete(st)
|
||||
elif st is not None:
|
||||
st.position_seconds = 0
|
||||
st.progress_updated_at = now
|
||||
db.flush()
|
||||
|
||||
|
||||
def checkpoint(
|
||||
db: Session,
|
||||
user: User,
|
||||
item_id: int,
|
||||
st: PlexState | None,
|
||||
position: int,
|
||||
now: datetime,
|
||||
*,
|
||||
dirty: bool,
|
||||
) -> PlexState:
|
||||
"""Record a resume position. `dirty` marks it not yet mirrored to Plex — set before a push, so
|
||||
the reconcile re-pushes it if the push fails and the push can settle it once Plex confirms."""
|
||||
st = state_row(db, user, item_id, st)
|
||||
st.position_seconds = position
|
||||
st.progress_updated_at = now
|
||||
if dirty:
|
||||
st.synced_to_plex = False
|
||||
db.flush()
|
||||
return st
|
||||
|
||||
|
||||
def forget_state(db: Session, st: PlexState) -> None:
|
||||
db.delete(st)
|
||||
db.flush()
|
||||
|
||||
|
||||
def set_status(
|
||||
db: Session, user: User, item_id: int, st: PlexState | None, status: str
|
||||
) -> PlexState:
|
||||
"""Set a watched or hidden status; watched also clears the resume point."""
|
||||
st = state_row(db, user, item_id, st)
|
||||
st.status = status
|
||||
if status == "watched":
|
||||
st.watched_at = datetime.now(timezone.utc)
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
db.flush()
|
||||
return st
|
||||
|
||||
|
||||
def bulk_state(
|
||||
db: Session, user_id: int, eps: list[PlexItem], watched: bool
|
||||
) -> list[tuple[int, str, str]]:
|
||||
"""Mark every episode in `eps` watched (or unwatched) for a user and say what changed, as the
|
||||
(item id, rating key, action) pushes a linked Plex account needs. Unwatching keeps the user's
|
||||
private "hidden" marks."""
|
||||
now = datetime.now(timezone.utc)
|
||||
existing = rollup.item_state_map(db, user_id, [e.id for e in eps])
|
||||
to_push: list[tuple[int, str, str]] = []
|
||||
for e in eps:
|
||||
st = existing.get(e.id)
|
||||
prev = st.status if st is not None else "new"
|
||||
if watched:
|
||||
st = state_row(db, user_id, e.id, st)
|
||||
prev = st.status # a first checkpoint beside this may have made it meanwhile
|
||||
st.status = "watched"
|
||||
st.watched_at = now
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
if prev != "watched":
|
||||
to_push.append((e.id, e.rating_key, "watched"))
|
||||
else: # → new: drop any state (but keep the user's private "hidden" marks)
|
||||
if st is not None and prev != "hidden":
|
||||
db.delete(st)
|
||||
if prev != "new":
|
||||
to_push.append((e.id, e.rating_key, "unwatched"))
|
||||
db.flush()
|
||||
return to_push
|
||||
@@ -33,7 +33,7 @@ from sqlalchemy.orm import Session as DbSession
|
||||
from app import logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.downloads.edit import encode_thread_cap
|
||||
from app.models import PlexItem
|
||||
from app.plex.models import PlexItem
|
||||
from app.plex import paths
|
||||
from app.plex.sync import HDR_TRANSFERS, HW_DECODABLE_CODECS
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.db import SessionLocal
|
||||
from app.models import PlexItem
|
||||
from app.plex.models import PlexItem
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex.cache import CACHE_DIR
|
||||
from app.plex.client import PlexClient, PlexError
|
||||
|
||||
@@ -21,7 +21,7 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
|
||||
from app.plex.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
|
||||
from app.plex import detail, overlay, people
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.genres import hu_genres
|
||||
|
||||
@@ -22,7 +22,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.config import settings
|
||||
from app.models import PlexItem, PlexPerson, PlexShow
|
||||
from app.plex.models import PlexItem, PlexPerson, PlexShow
|
||||
from app.plex.cards import display_title
|
||||
from app.plex.people import people_by_credits, warm_portraits
|
||||
|
||||
@@ -619,9 +619,9 @@ def enrich_person(db: Session, person: PlexPerson) -> bool:
|
||||
if not person.tmdb_person_id or not sysconfig.get_str(db, "tmdb_api_key"):
|
||||
return False
|
||||
try:
|
||||
# Called from a background task (off the request path — see people._enrich_person_bg), so it no
|
||||
# longer blocks a serve; still cap the outbound TMDB call with a short timeout so a hanging TMDB
|
||||
# doesn't tie up a threadpool worker for the default 30s.
|
||||
# Called from a background task (off the request path — see `plex.service.enrich_person_bg`),
|
||||
# so it no longer blocks a serve; still cap the outbound TMDB call with a short timeout so a
|
||||
# hanging TMDB doesn't tie up a threadpool worker for the default 30s.
|
||||
with TmdbClient(db, timeout=8.0) as tmdb:
|
||||
_fetch_person_into(tmdb, person)
|
||||
except TmdbError as exc:
|
||||
|
||||
@@ -29,8 +29,11 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import PlexItem, PlexLibrary, PlexLink, PlexState
|
||||
from app.userscope import owned_by
|
||||
from app.plex.models import PlexItem, PlexLibrary, PlexLink, PlexState
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex import queries
|
||||
from app.plex.service import state_row
|
||||
from app.plex.sync import (
|
||||
_TYPE_EPISODE,
|
||||
_TYPE_MOVIE,
|
||||
@@ -196,7 +199,7 @@ def link_for_push(db: Session, user_id: int) -> PlexLink | None:
|
||||
"""The active owner watch-push link for this user, or None when push is off / not the owner.
|
||||
Push is gated on `initial_import_done` so we never push a half-known local state over Plex's
|
||||
authoritative one before the one-time "Plex is master" import has reconciled them."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user_id)).scalars().first()
|
||||
link = db.execute(select(PlexLink).where(owned_by(PlexLink.user_id, user_id))).scalars().first()
|
||||
if link and link.uses_admin and link.sync_enabled and link.initial_import_done:
|
||||
return link
|
||||
return None
|
||||
@@ -251,13 +254,7 @@ def push_state_to_plex(
|
||||
# Flag the row synced — but an "unwatched" via status=new deletes the row, so there may be
|
||||
# nothing to flag; that's fine (unscrobble still happened). Skip the flag if the row was
|
||||
# re-edited since we captured `expect_ts` (leave it dirty for its own push / the reconcile).
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user_id, item_id)
|
||||
if st is not None and _row_matches(st, expect_ts):
|
||||
st.synced_to_plex = True
|
||||
db.commit()
|
||||
@@ -318,7 +315,7 @@ def push_bulk_state_to_plex(user_id: int, changes: list[tuple[int, str, str]]) -
|
||||
if synced_ids:
|
||||
db.execute(
|
||||
update(PlexState)
|
||||
.where(PlexState.user_id == user_id, PlexState.item_id.in_(synced_ids))
|
||||
.where(owned_by(PlexState.user_id, user_id), PlexState.item_id.in_(synced_ids))
|
||||
.values(synced_to_plex=True)
|
||||
)
|
||||
db.commit()
|
||||
@@ -391,13 +388,7 @@ def _pull_apply(
|
||||
"""Bring one Plex-side positive signal into Siftlode under last-write-wins. Returns
|
||||
"watched"/"in_progress" if it changed the row, else None. Never clears a row (un-watch is the
|
||||
full reconcile's job) — this only pulls in watched/resume."""
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user_id, item_id)
|
||||
if _same_state(st, status, pos):
|
||||
if st is not None and not st.synced_to_plex:
|
||||
st.synced_to_plex = True # already matches Plex → just settle the flag, no churn
|
||||
@@ -418,8 +409,9 @@ def _pull_apply(
|
||||
# "almost always already pushed" assumption above excludes.
|
||||
return None
|
||||
if st is None:
|
||||
st = PlexState(user_id=user_id, item_id=item_id)
|
||||
db.add(st)
|
||||
st = state_row(db, user_id, item_id, synced=True)
|
||||
if not st.synced_to_plex: # a local change made it first, beside this pull: the guard above
|
||||
return None
|
||||
if status == "watched":
|
||||
st.status, st.position_seconds, st.watched_at, st.progress_updated_at = (
|
||||
"watched",
|
||||
@@ -449,7 +441,7 @@ def _repush_dirty(db: Session, plex: PlexClient, link: PlexLink) -> int:
|
||||
dirty = (
|
||||
db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == link.user_id,
|
||||
owned_by(PlexState.user_id, link.user_id),
|
||||
PlexState.synced_to_plex.is_(False),
|
||||
PlexState.status != "hidden",
|
||||
)
|
||||
@@ -505,7 +497,7 @@ def _reconcile_inprogress_movies(
|
||||
and aren't affected). Note: like the rest of this job, `metadata()` reads the TOKEN account (the
|
||||
owner) — a HTPC playing under a managed Plex user (accountID>1) is out of scope here as elsewhere."""
|
||||
conds = [
|
||||
PlexState.user_id == link.user_id,
|
||||
owned_by(PlexState.user_id, link.user_id),
|
||||
PlexState.status.notin_(("watched", "hidden")),
|
||||
PlexState.position_seconds > _PROGRESS_MIN_S,
|
||||
PlexItem.kind == "movie",
|
||||
@@ -627,7 +619,7 @@ def _clear_unwatched(db: Session, user_id: int, plex_states: dict, settled: set[
|
||||
select(PlexState)
|
||||
.join(PlexItem, PlexItem.id == PlexState.item_id)
|
||||
.join(PlexLibrary, PlexLibrary.id == PlexItem.library_id)
|
||||
.where(PlexState.user_id == user_id, PlexLibrary.plex_key.in_(settled))
|
||||
.where(owned_by(PlexState.user_id, user_id), PlexLibrary.plex_key.in_(settled))
|
||||
)
|
||||
for st in rows.scalars().all():
|
||||
if st.item_id in plex_states or st.status == "hidden" or not st.synced_to_plex:
|
||||
|
||||
@@ -16,6 +16,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import ApiQuotaUsage, QuotaEvent
|
||||
@@ -173,7 +174,7 @@ def actions_today(db: Session, user_id: int, action: str) -> int:
|
||||
select(func.count())
|
||||
.select_from(QuotaEvent)
|
||||
.where(
|
||||
QuotaEvent.user_id == user_id,
|
||||
owned_by(QuotaEvent.user_id, user_id),
|
||||
QuotaEvent.action == action,
|
||||
QuotaEvent.created_at >= pacific_day_start_utc(),
|
||||
)
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
"""Reader's tables: what a user does with the shared catalog — where they are in a video
|
||||
(`VideoState`), their playlists, their saved views, their labels on channels (`Tag`, `ChannelTag`)
|
||||
and the comfort shelf on Home. Only this unit writes them; another unit reads them along a declared
|
||||
edge, or asks this unit's service."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
JSON,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
func,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.db import Base
|
||||
from app.models import TimestampMixin, UpdatedAtMixin
|
||||
|
||||
|
||||
class Tag(Base, TimestampMixin):
|
||||
"""A label. System tags (user_id NULL) are computed automatically and shared by all
|
||||
users; user tags (user_id set) are personal. category groups them in the UI."""
|
||||
|
||||
__tablename__ = "tags"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(64))
|
||||
color: Mapped[str | None] = mapped_column(String(16))
|
||||
# language | topic | other
|
||||
category: Mapped[str] = mapped_column(String(16), default="other", server_default="other")
|
||||
|
||||
|
||||
class ChannelTag(Base):
|
||||
"""Links a tag to a channel. source='auto' rows are managed by the auto-tagger and
|
||||
regenerated freely; source='user' rows are never touched by the system."""
|
||||
|
||||
__tablename__ = "channel_tags"
|
||||
__table_args__ = (UniqueConstraint("channel_id", "tag_id", "user_id", name="uq_channel_tag"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
channel_id: Mapped[str] = mapped_column(
|
||||
ForeignKey("channels.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
tag_id: Mapped[int] = mapped_column(ForeignKey("tags.id", ondelete="CASCADE"), index=True)
|
||||
# NULL = system/auto link visible to everyone; set = a specific user's link.
|
||||
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
source: Mapped[str] = mapped_column(String(8), default="auto", server_default="auto")
|
||||
confidence: Mapped[float | None] = mapped_column(Float)
|
||||
|
||||
|
||||
class VideoState(Base, UpdatedAtMixin):
|
||||
"""Per-user watch state for a video. Rows exist only for non-default states; the
|
||||
absence of a row means 'new/unseen'."""
|
||||
|
||||
__tablename__ = "video_states"
|
||||
__table_args__ = (UniqueConstraint("user_id", "video_id", name="uq_user_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
# new | watched | hidden ("saved" is now Watch-later playlist membership, not a status)
|
||||
status: Mapped[str] = mapped_column(String(12), default="new", server_default="new")
|
||||
watched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
# Resume position (seconds into the video) for the in-app player. Orthogonal to
|
||||
# status: a partially-watched video keeps status "new" but a non-zero position, so
|
||||
# it can render a progress bar and match the "in progress" feed filter. 0 = not
|
||||
# started / finished (trivially-early and near-finished positions are not stored).
|
||||
# int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
|
||||
position_seconds: Mapped[int] = mapped_column(
|
||||
Integer, default=0, server_default="0", nullable=False
|
||||
)
|
||||
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
|
||||
class Playlist(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user named, ordered collection of videos from the shared catalog.
|
||||
|
||||
`kind`: 'user' (normal) or 'watch_later' (built-in, undeletable, one per user).
|
||||
`source`: 'local' (created here) or 'youtube' (mirrored from the user's YouTube
|
||||
playlists). `yt_playlist_id` links a mirrored/exported playlist to its YouTube id.
|
||||
`dirty` marks a YouTube-linked playlist with local edits not yet pushed, so the read
|
||||
sync won't clobber them. (source/yt_playlist_id/dirty are forward-looking for the YT
|
||||
sync phases; the foundation phase only uses local playlists.)"""
|
||||
|
||||
__tablename__ = "playlists"
|
||||
__table_args__ = (
|
||||
# One Watch later per user, held by the database: two first saves at once each made one.
|
||||
Index(
|
||||
"uq_playlist_watch_later",
|
||||
"user_id",
|
||||
unique=True,
|
||||
postgresql_where=text("kind = 'watch_later'"),
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
name: Mapped[str] = mapped_column(String(255))
|
||||
kind: Mapped[str] = mapped_column(String(16), default="user", server_default="user")
|
||||
source: Mapped[str] = mapped_column(String(16), default="local", server_default="local")
|
||||
yt_playlist_id: Mapped[str | None] = mapped_column(String(64))
|
||||
# True when the current items/name differ from `synced_fingerprint` (unpushed edits).
|
||||
dirty: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
# SHA-256 of name + ordered video ids as last synced from / pushed to YouTube; the
|
||||
# baseline `dirty` is derived against. NULL = no baseline yet (treated as dirty).
|
||||
synced_fingerprint: Mapped[str | None] = mapped_column(String)
|
||||
# JSON list of the ordered video ids as of the last sync — the 3-way-merge baseline the
|
||||
# two-way YouTube sync classifies each item against (local-add vs YT-add vs real delete).
|
||||
# NULL = no baseline yet; the first merge then does a lossless union (never deletes).
|
||||
synced_video_ids: Mapped[str | None] = mapped_column(Text)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
|
||||
items: Mapped[list["PlaylistItem"]] = relationship(
|
||||
back_populates="playlist",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="PlaylistItem.position",
|
||||
)
|
||||
|
||||
|
||||
class PlaylistItem(Base):
|
||||
"""A video's membership in a playlist, with its order within that playlist."""
|
||||
|
||||
__tablename__ = "playlist_items"
|
||||
__table_args__ = (UniqueConstraint("playlist_id", "video_id", name="uq_playlist_video"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
playlist_id: Mapped[int] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
|
||||
video_id: Mapped[str] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
added_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
|
||||
|
||||
playlist: Mapped["Playlist"] = relationship(back_populates="items")
|
||||
|
||||
|
||||
class SavedView(Base, TimestampMixin, UpdatedAtMixin):
|
||||
"""A per-user named snapshot of a module's filter/sort state (a "smart view").
|
||||
|
||||
`source` names the module the view belongs to ("feed" | "plex"); `filters` is that module's
|
||||
serialized filter JSON the frontend applies with one click. `position` orders them in the sidebar
|
||||
(drag-reorder, per source); `is_default` marks the one view applied automatically on load — at most
|
||||
one per (user, source), enforced by the API on set. The name is unique per (user, source)."""
|
||||
|
||||
__tablename__ = "saved_views"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("user_id", "source", "name", name="uq_user_source_saved_view"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
|
||||
source: Mapped[str] = mapped_column(String(16), default="feed", server_default="feed")
|
||||
name: Mapped[str] = mapped_column(String(80))
|
||||
filters: Mapped[dict] = mapped_column(JSON)
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
|
||||
is_default: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
|
||||
|
||||
|
||||
class ComfortPin(Base, TimestampMixin):
|
||||
"""One entry on a user's comfort shelf — the anti-paralysis fallback: "just put on X".
|
||||
|
||||
Deliberately NOT a boolean on `VideoState`/`PlexState`, and NOT a polymorphic `(kind, ref)` string
|
||||
pair. A flag cannot ORDER a shelf and cannot hold a channel or a show — and comfort media is
|
||||
usually "put on X", a series or a channel, not one particular video. A string ref would order and
|
||||
hold anything, but it would also lose the foreign key: a Plex item dropped from the library, or an
|
||||
unsubscribed channel, would leave a dead tile on the one surface whose whole job is to be
|
||||
reliable. So instead: one nullable CASCADE-ing FK per pinnable kind, and a check constraint that
|
||||
exactly one of them is set.
|
||||
|
||||
That constraint is the house rule applied to a five-way union — the invariant lives in the schema,
|
||||
not in a comment every insert site has to remember. `kind`/`ref` below derive FROM the columns, so
|
||||
there is no second vocabulary to keep in sync.
|
||||
|
||||
NULLs are DISTINCT in a Postgres unique index, so the five per-kind unique constraints each stop a
|
||||
double-pin of the same thing without colliding across kinds."""
|
||||
|
||||
__tablename__ = "comfort_pins"
|
||||
__table_args__ = (
|
||||
CheckConstraint(
|
||||
"num_nonnulls(video_id, channel_id, plex_item_id, plex_show_id, playlist_id) = 1",
|
||||
name="ck_comfort_pin_one_ref",
|
||||
),
|
||||
UniqueConstraint("user_id", "video_id", name="uq_comfort_pin_video"),
|
||||
UniqueConstraint("user_id", "channel_id", name="uq_comfort_pin_channel"),
|
||||
UniqueConstraint("user_id", "plex_item_id", name="uq_comfort_pin_plex_item"),
|
||||
UniqueConstraint("user_id", "plex_show_id", name="uq_comfort_pin_plex_show"),
|
||||
UniqueConstraint("user_id", "playlist_id", name="uq_comfort_pin_playlist"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
|
||||
|
||||
video_id: Mapped[str | None] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
|
||||
channel_id: Mapped[str | None] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
|
||||
plex_item_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_items.id", ondelete="CASCADE")
|
||||
)
|
||||
plex_show_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("plex_shows.id", ondelete="CASCADE")
|
||||
)
|
||||
playlist_id: Mapped[int | None] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
|
||||
|
||||
# 0-based shelf order. The shelf is short and hand-curated, so a dense integer the reorder
|
||||
# endpoint rewrites wholesale beats a fractional index nobody would ever exercise here.
|
||||
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
|
||||
@@ -0,0 +1,361 @@
|
||||
"""Reader's reads: a user's labels, saved views, watch state and playlists, for the HTTP layer and
|
||||
for the other units that ask about them.
|
||||
|
||||
Every read of a user's row goes through `userscope`, so "someone else's row" is the same answer as
|
||||
a missing one. Reader reads Catalog's videos along its declared edge (a playlist's cover and
|
||||
length, whether a video exists); another unit asks the questions at the end of this module rather
|
||||
than joining Reader's tables itself.
|
||||
"""
|
||||
|
||||
from sqlalchemy import ColumnElement, and_, func, or_, select
|
||||
from sqlalchemy.orm import InstrumentedAttribute, Session, aliased
|
||||
|
||||
from app.collation import collated
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription, Video
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
|
||||
from app.userscope import owned_by, owned_row
|
||||
|
||||
# --- catalog, read along the edge ------------------------------------------------------------
|
||||
|
||||
|
||||
def video(db: Session, video_id: str) -> Video | None:
|
||||
return db.get(Video, video_id)
|
||||
|
||||
|
||||
# --- tags ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def tags_with_counts(db: Session, user: User) -> list[tuple[Tag, int]]:
|
||||
"""The tags this user sees (the shared system ones and their own), each with how many channel
|
||||
links it has, by category then name."""
|
||||
return [
|
||||
(tag, count)
|
||||
for tag, count in db.execute(
|
||||
select(Tag, func.count(ChannelTag.id))
|
||||
.outerjoin(ChannelTag, ChannelTag.tag_id == Tag.id)
|
||||
.where(or_(Tag.user_id.is_(None), owned_by(Tag.user_id, user)))
|
||||
.group_by(Tag.id)
|
||||
.order_by(Tag.category, Tag.name)
|
||||
).tuples()
|
||||
]
|
||||
|
||||
|
||||
def own_tag(db: Session, user: User, tag_id: int) -> Tag | None:
|
||||
"""One of the user's own tags. A system tag (no owner) is nobody's to edit, so it reads as
|
||||
missing too."""
|
||||
return owned_row(db, Tag, tag_id, user)
|
||||
|
||||
|
||||
# --- saved views -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def views(db: Session, user: User, source: str) -> list[SavedView]:
|
||||
return list(
|
||||
db.execute(
|
||||
select(SavedView)
|
||||
.where(owned_by(SavedView.user_id, user), SavedView.source == source)
|
||||
.order_by(SavedView.position, SavedView.id)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def all_views(db: Session, user: User) -> list[SavedView]:
|
||||
return list(db.execute(select(SavedView).where(owned_by(SavedView.user_id, user))).scalars())
|
||||
|
||||
|
||||
def own_view(db: Session, user: User, view_id: int) -> SavedView | None:
|
||||
return owned_row(db, SavedView, view_id, user)
|
||||
|
||||
|
||||
# --- watch state -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def video_state(db: Session, user: User, video_id: str) -> VideoState | None:
|
||||
return db.execute(
|
||||
select(VideoState).where(
|
||||
owned_by(VideoState.user_id, user), VideoState.video_id == video_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
# --- playlists -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def own_playlist(
|
||||
db: Session, user: User, playlist_id: int, *, lock: bool = False
|
||||
) -> Playlist | None:
|
||||
return owned_row(db, Playlist, playlist_id, user, lock=lock)
|
||||
|
||||
|
||||
def playlists_for(db: Session, user: User) -> list[Playlist]:
|
||||
"""The user's playlists in their rail order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Playlist)
|
||||
.where(owned_by(Playlist.user_id, user))
|
||||
.order_by(Playlist.position, Playlist.created_at)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def watch_later(db: Session, user: User) -> Playlist | None:
|
||||
return db.scalar(
|
||||
select(Playlist).where(owned_by(Playlist.user_id, user), Playlist.kind == "watch_later")
|
||||
)
|
||||
|
||||
|
||||
def item_counts(db: Session, ids: list[int]) -> dict[int, int]:
|
||||
return dict(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id, func.count())
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
)
|
||||
.tuples()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def durations(db: Session, ids: list[int]) -> dict[int, int]:
|
||||
"""Each playlist's running time: the sum of its videos' lengths."""
|
||||
return {
|
||||
pid: int(total or 0)
|
||||
for pid, total in db.execute(
|
||||
select(PlaylistItem.playlist_id, func.coalesce(func.sum(Video.duration_seconds), 0))
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
).tuples()
|
||||
}
|
||||
|
||||
|
||||
def containing(db: Session, ids: list[int], video_id: str) -> set[int]:
|
||||
"""Which of these playlists already hold the video (the add-to-playlist popover's ticks)."""
|
||||
return set(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id).where(
|
||||
PlaylistItem.playlist_id.in_(ids), PlaylistItem.video_id == video_id
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def covers(db: Session, ids: list[int]) -> dict[int, str | None]:
|
||||
"""Each playlist's cover: its first item's thumbnail, for every list in ONE window query rather
|
||||
than a lookup per playlist."""
|
||||
rn = (
|
||||
func.row_number()
|
||||
.over(partition_by=PlaylistItem.playlist_id, order_by=PlaylistItem.position)
|
||||
.label("rn")
|
||||
)
|
||||
first_items = (
|
||||
select(PlaylistItem.playlist_id.label("pid"), Video.thumbnail_url.label("thumb"), rn)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.subquery()
|
||||
)
|
||||
return dict(
|
||||
db.execute(select(first_items.c.pid, first_items.c.thumb).where(first_items.c.rn == 1))
|
||||
.tuples()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def playlist_totals(db: Session, playlist_id: int) -> tuple[int, int, str | None]:
|
||||
"""One playlist's item count, running time and cover. An outer join, so an empty list still
|
||||
answers (0, 0, None)."""
|
||||
count, total = db.execute(
|
||||
select(func.count(PlaylistItem.id), func.coalesce(func.sum(Video.duration_seconds), 0))
|
||||
.select_from(PlaylistItem)
|
||||
.outerjoin(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
).one()
|
||||
cover = db.scalar(
|
||||
select(Video.thumbnail_url)
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlaylistItem.position)
|
||||
.limit(1)
|
||||
)
|
||||
return count or 0, total or 0, cover
|
||||
|
||||
|
||||
def playlist_rows(db: Session, user: User, playlist_id: int) -> list:
|
||||
"""A playlist's videos as feed rows (the same columns the feed serializes), in list order."""
|
||||
state = aliased(VideoState)
|
||||
return list(
|
||||
db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
# `id` breaks a position tie the same way on every read (the renumbering add does too).
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def items(db: Session, playlist_id: int) -> list[PlaylistItem]:
|
||||
"""A playlist's item rows, in list order."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(PlaylistItem)
|
||||
.where(PlaylistItem.playlist_id == playlist_id)
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
|
||||
|
||||
def item(db: Session, playlist_id: int, video_id: str) -> PlaylistItem | None:
|
||||
return db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == playlist_id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# --- for the other units ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def kept_by_a_reader(video_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
|
||||
"""A video someone holds: watched, in progress or hidden (a `VideoState`), or in one of their
|
||||
playlists. The catalog's cleanup jobs keep such a video; as a clause, so their DELETE stays
|
||||
one statement."""
|
||||
return or_(
|
||||
select(VideoState.id).where(VideoState.video_id == video_id).exists(),
|
||||
select(PlaylistItem.id).where(PlaylistItem.video_id == video_id).exists(),
|
||||
)
|
||||
|
||||
|
||||
def playlisted_by(db: Session, video_ids: list[str]) -> list[tuple[int, str]]:
|
||||
"""(user id, video id) for every user who has one of these videos in a playlist."""
|
||||
return list(
|
||||
db.execute(
|
||||
select(Playlist.user_id, PlaylistItem.video_id)
|
||||
.join(PlaylistItem, PlaylistItem.playlist_id == Playlist.id)
|
||||
.where(PlaylistItem.video_id.in_(video_ids))
|
||||
).tuples()
|
||||
)
|
||||
|
||||
|
||||
def has_auto_tags(channel_id: InstrumentedAttribute[str]) -> ColumnElement[bool]:
|
||||
"""A channel the auto-tagger has already labelled, as a clause for its "only the missing" scan."""
|
||||
return (
|
||||
select(ChannelTag.id)
|
||||
.where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.source == "auto",
|
||||
ChannelTag.user_id.is_(None),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
|
||||
|
||||
def view_tag_ids(filters: object) -> list[int]:
|
||||
"""The tag ids a stored view's filters name; anything else in that slot names none."""
|
||||
tags = filters.get("tags") if isinstance(filters, dict) else None
|
||||
if not isinstance(tags, list):
|
||||
return []
|
||||
return [t for t in tags if type(t) is int] # `type`, not isinstance: True is not a tag id
|
||||
|
||||
|
||||
def tags_in_feed_views(db: Session) -> set[int]:
|
||||
"""Every tag id any user's saved feed view filters on. Read defensively (`view_tag_ids`): a view
|
||||
stored before its route checked the shape must not raise for everyone's job."""
|
||||
return {
|
||||
t
|
||||
for (filters,) in db.execute(select(SavedView.filters).where(SavedView.source == "feed"))
|
||||
for t in view_tag_ids(filters)
|
||||
}
|
||||
|
||||
|
||||
def held_by(db: Session, user: User, video_ids: list[str]) -> set[str]:
|
||||
"""Which of these videos the user already holds: watched, in progress or hidden, or in one of
|
||||
their playlists (a live search sinks them below the new finds)."""
|
||||
if not video_ids:
|
||||
return set()
|
||||
has_state = (
|
||||
select(VideoState.id)
|
||||
.where(VideoState.video_id == Video.id, owned_by(VideoState.user_id, user))
|
||||
.exists()
|
||||
)
|
||||
in_playlist = (
|
||||
select(PlaylistItem.id)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(PlaylistItem.video_id == Video.id, owned_by(Playlist.user_id, user))
|
||||
.exists()
|
||||
)
|
||||
return set(
|
||||
db.scalars(select(Video.id).where(Video.id.in_(video_ids), or_(has_state, in_playlist)))
|
||||
)
|
||||
|
||||
|
||||
def feed_rows_by_ids(db: Session, user: User, video_ids: list[str]) -> list[dict]:
|
||||
"""These videos serialized with the feed's projection (their channel, this user's watch state,
|
||||
saved), in the order given."""
|
||||
if not video_ids:
|
||||
return []
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, owned_by(state.user_id, user)))
|
||||
.where(Video.id.in_(video_ids))
|
||||
).all()
|
||||
by_id = {r.id: serialize(r) for r in rows}
|
||||
return [by_id[i] for i in video_ids if i in by_id]
|
||||
|
||||
|
||||
def playlist_channels(db: Session, user: User, exclude: str | None, lang: str) -> list:
|
||||
"""The channels of the videos in the user's playlists that they don't subscribe to (`exclude`:
|
||||
their own channel), each with how many of those videos it has and across how many playlists;
|
||||
most present first, then by title in the reader's alphabet."""
|
||||
subscribed = select(Subscription.channel_id).where(owned_by(Subscription.user_id, user))
|
||||
q = (
|
||||
select(
|
||||
Channel,
|
||||
func.count(func.distinct(PlaylistItem.video_id)),
|
||||
func.count(func.distinct(Playlist.id)),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.where(owned_by(Playlist.user_id, user), Channel.id.not_in(subscribed))
|
||||
)
|
||||
if exclude:
|
||||
q = q.where(Channel.id != exclude)
|
||||
return list(
|
||||
db.execute(
|
||||
q.group_by(Channel.id).order_by(
|
||||
func.count(func.distinct(PlaylistItem.video_id)).desc(),
|
||||
collated(Channel.title, lang),
|
||||
)
|
||||
).all()
|
||||
)
|
||||
|
||||
|
||||
def user_tag_links(db: Session, user: User, channel_id: str | None = None) -> dict[str, list[int]]:
|
||||
"""The user's own tag links by channel (one channel's, when given)."""
|
||||
stmt = select(ChannelTag.channel_id, ChannelTag.tag_id).where(
|
||||
owned_by(ChannelTag.user_id, user)
|
||||
)
|
||||
if channel_id is not None:
|
||||
stmt = stmt.where(ChannelTag.channel_id == channel_id)
|
||||
out: dict[str, list[int]] = {}
|
||||
for cid, tag_id in db.execute(stmt).tuples():
|
||||
out.setdefault(cid, []).append(tag_id)
|
||||
return out
|
||||
|
||||
|
||||
def user_tag_link(db: Session, user: User, channel_id: str, tag_id: int) -> ChannelTag | None:
|
||||
return db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == tag_id,
|
||||
owned_by(ChannelTag.user_id, user),
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
@@ -0,0 +1,497 @@
|
||||
"""Reader's writes: a user's labels and saved views, where they are in a video, their playlists'
|
||||
items and order — and the system tags the catalog's auto-tagger hands over.
|
||||
|
||||
Every function here flushes and leaves the commit to its caller (the handler, or the job that owns
|
||||
its session). A write that must not double up keeps its guard where it is written: an upsert on the
|
||||
unique key, a row lock taken before a presence check. The comfort shelf's writes are Home's
|
||||
(`discover.pins`); a playlist's YouTube sync is `sync.playlists`.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import exists, func, select, text, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import User
|
||||
from app.reader import queries
|
||||
from app.reader.models import ChannelTag, Playlist, PlaylistItem, SavedView, Tag, VideoState
|
||||
from app.sync.playlists import lock_playlists, recompute_dirty
|
||||
from app.userscope import owned_by
|
||||
|
||||
TAG_CATEGORIES = ("language", "topic", "other")
|
||||
VIEW_NAME_MAX = 80
|
||||
|
||||
# --- tags ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_tag(db: Session, user: User, name: str, category: str, color: str | None) -> Tag:
|
||||
"""A tag of the user's own. An unknown category falls back to "other" rather than refusing. A
|
||||
name the user already has raises `IntegrityError` at the flush."""
|
||||
tag = Tag(
|
||||
user_id=user.id,
|
||||
name=name,
|
||||
color=color,
|
||||
category=category if category in TAG_CATEGORIES else "other",
|
||||
)
|
||||
db.add(tag)
|
||||
db.flush()
|
||||
return tag
|
||||
|
||||
|
||||
def update_tag(tag: Tag, *, sent: set[str], name: str | None, color: str | None) -> None:
|
||||
"""Change the fields the caller sent: `color` sent as None clears it."""
|
||||
if "name" in sent and name is not None:
|
||||
tag.name = name
|
||||
if "color" in sent:
|
||||
tag.color = color
|
||||
|
||||
|
||||
def delete_tag(db: Session, tag: Tag) -> None:
|
||||
db.delete(tag) # its channel links cascade
|
||||
db.flush()
|
||||
|
||||
|
||||
def ensure_system_tag(db: Session, name: str, category: str, color: str | None) -> Tag:
|
||||
"""The shared system tag of this name, made the first time a channel earns it."""
|
||||
tag = db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), Tag.name == name)
|
||||
).scalar_one_or_none()
|
||||
if tag is None:
|
||||
tag = Tag(user_id=None, name=name, category=category, color=color)
|
||||
db.add(tag)
|
||||
db.flush()
|
||||
return tag
|
||||
|
||||
|
||||
def set_auto_tags(db: Session, channel_id: str, desired: dict[int, float | None]) -> None:
|
||||
"""Make a channel's automatic tag links exactly `desired` (tag id → confidence). Only the
|
||||
auto-tagger's own links move; a user's links on the channel are never touched."""
|
||||
existing = {
|
||||
ct.tag_id: ct
|
||||
for ct in db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.user_id.is_(None),
|
||||
ChannelTag.source == "auto",
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
for tag_id, ct in existing.items():
|
||||
if tag_id not in desired:
|
||||
db.delete(ct)
|
||||
for tag_id, conf in desired.items():
|
||||
ct = existing.get(tag_id)
|
||||
if ct is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel_id,
|
||||
tag_id=tag_id,
|
||||
user_id=None,
|
||||
source="auto",
|
||||
confidence=conf,
|
||||
)
|
||||
)
|
||||
else:
|
||||
ct.confidence = conf
|
||||
db.flush()
|
||||
|
||||
|
||||
def prune_orphan_system_tags(db: Session) -> int:
|
||||
"""Delete the system tags no channel links to any more (after a mapping change) — but not one a
|
||||
saved feed view filters on. Views hold tag IDS, and a deleted tag comes back under a new id when
|
||||
a channel earns it again, so the view's filter would point at nothing; the feed drops an unknown
|
||||
id, and the view showed the whole feed under its old name. The kept ids are filtered here rather
|
||||
than in SQL: a `NOT IN` over every view's ids could pass the driver's bind-parameter limit."""
|
||||
in_views = queries.tags_in_feed_views(db)
|
||||
orphans = [
|
||||
tag
|
||||
for tag in db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), ~exists().where(ChannelTag.tag_id == Tag.id))
|
||||
).scalars()
|
||||
if tag.id not in in_views
|
||||
]
|
||||
for tag in orphans:
|
||||
db.delete(tag)
|
||||
db.flush()
|
||||
return len(orphans)
|
||||
|
||||
|
||||
# --- saved views -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_view(db: Session, user: User, source: str, name: str, filters: dict) -> SavedView:
|
||||
"""A view appended after the user's others of the same module (each module numbers its own
|
||||
0..N). A name the user already has there raises `IntegrityError` at the flush."""
|
||||
maxpos = db.execute(
|
||||
select(func.coalesce(func.max(SavedView.position), -1)).where(
|
||||
owned_by(SavedView.user_id, user), SavedView.source == source
|
||||
)
|
||||
).scalar_one()
|
||||
view = SavedView(
|
||||
user_id=user.id,
|
||||
source=source,
|
||||
name=name[:VIEW_NAME_MAX],
|
||||
filters=filters,
|
||||
position=maxpos + 1,
|
||||
)
|
||||
db.add(view)
|
||||
db.flush()
|
||||
return view
|
||||
|
||||
|
||||
def update_view(
|
||||
db: Session,
|
||||
view: SavedView,
|
||||
*,
|
||||
sent: set[str],
|
||||
name: str | None = None,
|
||||
filters: dict | None = None,
|
||||
is_default: bool | None = None,
|
||||
) -> None:
|
||||
"""Change the fields the caller sent. Making a view the default clears the flag on the user's
|
||||
other views of the same module first: at most one default per (user, source)."""
|
||||
if "name" in sent and name is not None:
|
||||
view.name = name[:VIEW_NAME_MAX]
|
||||
if "filters" in sent and filters is not None:
|
||||
view.filters = filters
|
||||
if "is_default" in sent:
|
||||
make_default = bool(is_default)
|
||||
if make_default:
|
||||
db.execute(
|
||||
update(SavedView)
|
||||
.where(
|
||||
owned_by(SavedView.user_id, view.user_id),
|
||||
SavedView.source == view.source,
|
||||
SavedView.id != view.id,
|
||||
)
|
||||
.values(is_default=False)
|
||||
)
|
||||
view.is_default = make_default
|
||||
db.flush()
|
||||
|
||||
|
||||
def delete_view(db: Session, view: SavedView) -> None:
|
||||
db.delete(view)
|
||||
db.flush()
|
||||
|
||||
|
||||
def reorder_views(db: Session, user: User, ids: list[int]) -> None:
|
||||
"""Number the listed views 0..n in the listed order; ids that are not the user's are skipped."""
|
||||
by_id = {v.id: v for v in queries.all_views(db, user)}
|
||||
pos = 0
|
||||
for vid in ids:
|
||||
v = by_id.get(vid)
|
||||
if v is not None:
|
||||
v.position = pos
|
||||
pos += 1
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- watch state -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def unmark(db: Session, user: User, video_id: str) -> None:
|
||||
"""Back to "new". A row that still holds a resume point keeps it (un-marking "watched" restores
|
||||
the in-progress state, not where the user left off); otherwise the row goes. The player fires
|
||||
this beside a checkpoint that may delete the same row: the caller's commit then raises
|
||||
`StaleDataError`, which is that lost race and nothing worse."""
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is None:
|
||||
return
|
||||
if row.position_seconds:
|
||||
row.status = "new"
|
||||
row.watched_at = None
|
||||
else:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
|
||||
|
||||
def mark(db: Session, user: User, video_id: str, status: str) -> None:
|
||||
""" "watched" or "hidden", as one atomic upsert on (user, video). The player marks "watched" at
|
||||
the end of playback at the instant a checkpoint may delete the in-progress row; a SELECT then
|
||||
UPDATE raised `StaleDataError` on that race, and an INSERT … ON CONFLICT is immune to it.
|
||||
"hidden" keeps any earlier `watched_at`."""
|
||||
now = datetime.now(timezone.utc)
|
||||
set_: dict = {"status": status, "updated_at": now}
|
||||
if status == "watched":
|
||||
set_["watched_at"] = now
|
||||
db.execute(
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
status=status,
|
||||
watched_at=now if status == "watched" else None,
|
||||
)
|
||||
.on_conflict_do_update(constraint="uq_user_video", set_=set_)
|
||||
)
|
||||
|
||||
|
||||
def clear_state(db: Session, user: User, video_id: str) -> None:
|
||||
"""The video back to pristine for this user: status, resume point and `watched_at` all go.
|
||||
Playlist membership lives elsewhere and stays."""
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
|
||||
|
||||
def set_progress(db: Session, user: User, video_id: str, position: int) -> None:
|
||||
"""The player's resume point. 0 clears it: a row holding nothing else goes, one with a status
|
||||
(watched, hidden) keeps it with the position zeroed. The status itself is never changed here.
|
||||
A position is one upsert on (user, video): the first checkpoint and the end-of-playback mark
|
||||
beside it both read "no row", and a plain add let the second one fail on `uq_user_video`."""
|
||||
now = datetime.now(timezone.utc)
|
||||
if position:
|
||||
db.execute(
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
position_seconds=position,
|
||||
progress_updated_at=now,
|
||||
)
|
||||
.on_conflict_do_update(
|
||||
constraint="uq_user_video",
|
||||
set_={"position_seconds": position, "progress_updated_at": now, "updated_at": now},
|
||||
)
|
||||
)
|
||||
return
|
||||
row = queries.video_state(db, user, video_id)
|
||||
if row is None:
|
||||
return
|
||||
if row.status == "new":
|
||||
db.delete(row)
|
||||
else:
|
||||
row.position_seconds = 0
|
||||
row.progress_updated_at = now
|
||||
db.flush()
|
||||
|
||||
|
||||
# --- playlists -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MoveRefused(Exception):
|
||||
"""A move between playlists that cannot happen: `reason` is the code the UI translates."""
|
||||
|
||||
def __init__(self, reason: str):
|
||||
super().__init__(reason)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def touch(pl: Playlist) -> None:
|
||||
"""Bump the playlist's `updated_at` for an edit that only writes `playlist_items`.
|
||||
|
||||
`UpdatedAtMixin`'s `onupdate` fires on an UPDATE of the PLAYLIST row, and add / remove /
|
||||
reorder / move write the item rows instead — so without this the rail's "recently changed"
|
||||
sort would only ever see renames. The assignment is what makes the column mean "last time
|
||||
this list changed", which is what the rail promises the reader.
|
||||
|
||||
The DATABASE's clock, like the mixin's own `onupdate`: one sort key written from two clocks
|
||||
(this process's and Postgres's) could order a rename and an item edit the wrong way round."""
|
||||
pl.updated_at = func.now()
|
||||
|
||||
|
||||
def _next_playlist_position(db: Session, user_id: int) -> int:
|
||||
"""Append position for a new playlist in this user's rail."""
|
||||
return (
|
||||
db.execute(
|
||||
select(func.coalesce(func.max(Playlist.position), -1)).where(
|
||||
owned_by(Playlist.user_id, user_id)
|
||||
)
|
||||
).scalar_one()
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def next_item_position(db: Session, playlist_id: int) -> int:
|
||||
"""Append position for a new item at the end of a playlist. Locks the playlist row first: two
|
||||
adds (two tabs, the rail and the page) otherwise both read the same max and land on one
|
||||
position, and the list's order between them is then arbitrary."""
|
||||
lock_playlists(db, playlist_id)
|
||||
return (
|
||||
db.execute(
|
||||
select(func.coalesce(func.max(PlaylistItem.position), -1)).where(
|
||||
PlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
).scalar_one()
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def create_playlist(db: Session, user: User, name: str) -> Playlist:
|
||||
pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id))
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
return pl
|
||||
|
||||
|
||||
def watch_later(db: Session, user: User) -> Playlist:
|
||||
"""The user's built-in Watch later playlist, created on first use. Created by the insert itself
|
||||
(ON CONFLICT DO NOTHING on the one-per-user index) and then read: the first two saves at once
|
||||
both read "none", and each used to make a list of its own."""
|
||||
pl = queries.watch_later(db, user)
|
||||
if pl is None:
|
||||
db.execute(
|
||||
pg_insert(Playlist)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
name="Watch later",
|
||||
kind="watch_later",
|
||||
position=_next_playlist_position(db, user.id),
|
||||
)
|
||||
.on_conflict_do_nothing(
|
||||
index_elements=["user_id"], index_where=text("kind = 'watch_later'")
|
||||
)
|
||||
)
|
||||
pl = queries.watch_later(db, user)
|
||||
assert pl is not None # inserted now, or committed by the request that won
|
||||
return pl
|
||||
|
||||
|
||||
def add_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Append a video to a playlist if not already present (idempotent). Returns whether it
|
||||
was added. `mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later).
|
||||
Locked BEFORE the presence check: two adds of one video both saw "absent" and the second one's
|
||||
insert hit uq_playlist_video as a 500."""
|
||||
lock_playlists(db, pl.id)
|
||||
if queries.item(db, pl.id, video_id) is not None:
|
||||
return False
|
||||
db.add(
|
||||
PlaylistItem(playlist_id=pl.id, video_id=video_id, position=next_item_position(db, pl.id))
|
||||
)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
|
||||
def remove_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Remove a video from a playlist if present (idempotent). Returns whether it was removed.
|
||||
`mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later)."""
|
||||
lock_playlists(db, pl.id)
|
||||
found = queries.item(db, pl.id, video_id)
|
||||
if found is None:
|
||||
return False
|
||||
db.delete(found)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return True
|
||||
|
||||
|
||||
def place_item(db: Session, pl: Playlist, video_id: str, position: int) -> None:
|
||||
"""Put a video already in the list at `position` (clamped to the end), renumbering the whole
|
||||
list: positions can have gaps after removals, so "index N" is only meaningful over the
|
||||
ordered rows."""
|
||||
rows = queries.items(db, pl.id)
|
||||
new = next(r for r in rows if r.video_id == video_id)
|
||||
rows.remove(new)
|
||||
rows.insert(min(position, len(rows)), new)
|
||||
for pos, row in enumerate(rows):
|
||||
row.position = pos
|
||||
db.flush()
|
||||
|
||||
|
||||
def add_to_playlist(db: Session, pl: Playlist, video_id: str, position: int | None) -> None:
|
||||
"""Add a video to one of the user's lists, at `position` when given (the list page's "Undo" of
|
||||
a removal puts the row back where it was in ONE request), else at the end. A video already in
|
||||
the list stays where it is."""
|
||||
added = add_item(db, pl, video_id, mark_dirty=False)
|
||||
if added and position is not None:
|
||||
place_item(db, pl, video_id, position)
|
||||
recompute_dirty(db, pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def rename_playlist(db: Session, pl: Playlist, name: str) -> None:
|
||||
"""A new name moves `dirty`; the caller holds the playlist's lock (the scheduled mirror refresh
|
||||
reads `dirty` under it)."""
|
||||
if name != pl.name:
|
||||
pl.name = name
|
||||
recompute_dirty(db, pl)
|
||||
db.flush()
|
||||
|
||||
|
||||
def delete_playlist(db: Session, pl: Playlist) -> None:
|
||||
"""The playlist first, as every item write: the cascade deletes the items after the list row
|
||||
is locked."""
|
||||
lock_playlists(db, pl.id)
|
||||
db.delete(pl) # items cascade
|
||||
db.flush()
|
||||
|
||||
|
||||
def move_item(db: Session, src: Playlist, dst: Playlist, video_id: str) -> None:
|
||||
"""Move one video from `src` into `dst`, in the caller's one transaction.
|
||||
|
||||
The client-side shape — add to the target, then remove from the source — leaves the video in
|
||||
BOTH lists whenever the second call fails. Idempotent: a video already in the target is simply
|
||||
dropped from the source (moving into a list that has it is a removal, not a duplicate). Raises
|
||||
`MoveRefused` for the same list twice, or a video a concurrent edit already took out."""
|
||||
if src.id == dst.id:
|
||||
raise MoveRefused("same_playlist")
|
||||
# Both lists, in id order: a move A→B beside a move B→A would otherwise lock crosswise.
|
||||
lock_playlists(db, src.id, dst.id)
|
||||
row = queries.item(db, src.id, video_id)
|
||||
if row is None:
|
||||
raise MoveRefused("not_in_playlist")
|
||||
dst_changed = False
|
||||
if queries.item(db, dst.id, video_id) is None:
|
||||
# Re-parent the existing row rather than delete+insert: `added_at` ("when I saved this")
|
||||
# survives the move, and there is no window where the video belongs to neither list. The
|
||||
# check above holds: both lists are locked, and an insert into the target (another tab's
|
||||
# add, a YouTube merge) waits for the lock — its foreign key needs the target row.
|
||||
row.playlist_id = dst.id
|
||||
row.position = next_item_position(db, dst.id)
|
||||
dst_changed = True
|
||||
if not dst_changed:
|
||||
db.delete(row)
|
||||
# The source always changed; the target only when the row landed in it. Touching a target
|
||||
# that already had the video promoted an untouched list in the rail's "recently changed" sort.
|
||||
recompute_dirty(db, src)
|
||||
touch(src)
|
||||
if dst_changed:
|
||||
recompute_dirty(db, dst)
|
||||
touch(dst)
|
||||
db.flush()
|
||||
|
||||
|
||||
def reorder_items(db: Session, pl: Playlist, order: list[str]) -> int:
|
||||
"""Number the listed videos 0..n in the listed order. Items the list left out (added in another
|
||||
tab meanwhile) keep their relative order after them, so no two items share a position. The
|
||||
caller holds the playlist's lock. Returns how many items the list has."""
|
||||
by_video = {it.video_id: it for it in queries.items(db, pl.id)}
|
||||
pos = 0
|
||||
seen: set[str] = set()
|
||||
for vid in order:
|
||||
it = by_video.get(vid)
|
||||
if it is not None:
|
||||
it.position = pos
|
||||
pos += 1
|
||||
seen.add(vid)
|
||||
for it in sorted(
|
||||
(it for vid, it in by_video.items() if vid not in seen), key=lambda i: i.position
|
||||
):
|
||||
it.position = pos
|
||||
pos += 1
|
||||
recompute_dirty(db, pl)
|
||||
touch(pl)
|
||||
db.flush()
|
||||
return pos
|
||||
|
||||
|
||||
def attach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
|
||||
"""The user's own label on a channel (idempotent). The caller has checked the tag is theirs."""
|
||||
if queries.user_tag_link(db, user, channel_id, tag_id) is None:
|
||||
db.add(ChannelTag(channel_id=channel_id, tag_id=tag_id, user_id=user.id, source="user"))
|
||||
db.flush()
|
||||
|
||||
|
||||
def detach_user_tag(db: Session, user: User, channel_id: str, tag_id: int) -> None:
|
||||
row = queries.user_tag_link(db, user, channel_id, tag_id)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.flush()
|
||||
@@ -16,6 +16,7 @@ from app.db import Base
|
||||
from app.models import POSITION_MAX_S, User
|
||||
from app.issues import gitea
|
||||
from app.refusal import refused
|
||||
from app.userscope import owned_row
|
||||
from app.youtube.client import YouTubeError, refusal_reason
|
||||
|
||||
M = TypeVar("M", bound=Base)
|
||||
@@ -88,10 +89,11 @@ def owned_or_404(
|
||||
) -> M:
|
||||
"""Fetch a row by primary key that `user` owns, or raise 404 (`lock`: see `get_or_404`). A row owned by someone
|
||||
else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger.
|
||||
It replaced the near-identical per-route ownership helpers, and is called directly — a thin
|
||||
`_own_x` wrapper around it is the ad-hoc shape CLAUDE.md's patterns table rules out."""
|
||||
obj = get_or_404(db, model, id_, what=what, lock=lock)
|
||||
if getattr(obj, owner_attr) != user.id:
|
||||
A core route calls it directly. A unit's route cannot (it may not name a unit's model as a value):
|
||||
it asks its unit's `queries.own_x`, which goes through the same `owned_row`, and answers
|
||||
`not_found` itself."""
|
||||
obj = owned_row(db, model, id_, user, owner_attr=owner_attr, lock=lock)
|
||||
if obj is None:
|
||||
raise not_found(what)
|
||||
return obj
|
||||
|
||||
|
||||
+107
-108
@@ -5,9 +5,10 @@ helpers) stays in app/auth.py, imported below."""
|
||||
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import NamedTuple
|
||||
from typing import Literal, NamedTuple
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from authlib.common.security import generate_token
|
||||
from authlib.integrations.starlette_client import OAuth, OAuthError
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
|
||||
from fastapi.responses import JSONResponse, RedirectResponse
|
||||
@@ -16,6 +17,7 @@ from sqlalchemy import select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -24,10 +26,11 @@ from app import sysconfig
|
||||
from app.auth import (
|
||||
_YOUTUBE_SCOPES,
|
||||
BASE_SCOPES,
|
||||
MAX_SESSION_ACCOUNTS,
|
||||
EXPECTED_ACCOUNT_HEADER,
|
||||
PASSWORD_MIN_LEN,
|
||||
READ_SCOPES,
|
||||
WRITE_SCOPES,
|
||||
built_for_another_account,
|
||||
current_user,
|
||||
env_admin_canonicals,
|
||||
env_allowed_canonicals,
|
||||
@@ -36,7 +39,9 @@ from app.auth import (
|
||||
is_allowed,
|
||||
log,
|
||||
operator_contact,
|
||||
resolved_user_id,
|
||||
optional_current_user,
|
||||
session_user_id,
|
||||
stamp_session,
|
||||
)
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal, get_db
|
||||
@@ -319,6 +324,25 @@ class SetPasswordIn(BaseModel):
|
||||
current_password: str | None = None
|
||||
|
||||
|
||||
# A connect in flight, in the session: `{state, uid, explicit}`. It names the OAuth state of the flow
|
||||
# that set it, because the session is the browser's, not the flow's: a callback coming back with
|
||||
# another state (an older flow, which authlib then refuses) must not spend the marker of the connect
|
||||
# still open, or that connect would return as a plain sign-in — signing someone else in, or storing
|
||||
# a YouTube grant on whichever account the Google identity already belongs to.
|
||||
_LINK_MARKER = "oauth_link"
|
||||
|
||||
# Where a navigation from a tab showing another account than the one signed in lands: the app, which
|
||||
# boots as the account signed in now and says why (the frontend's `ACCOUNT_CHANGED_FLAG`).
|
||||
ACCOUNT_CHANGED_LANDING = "/?account=changed"
|
||||
|
||||
|
||||
def _take_link_marker(request: Request) -> dict | None:
|
||||
marker = request.session.get(_LINK_MARKER)
|
||||
if not isinstance(marker, dict) or marker.get("state") != request.query_params.get("state"):
|
||||
return None
|
||||
return request.session.pop(_LINK_MARKER)
|
||||
|
||||
|
||||
@router.get("/login")
|
||||
async def login(request: Request, db: Session = Depends(get_db)):
|
||||
# access_type=offline ensures a refresh_token on first authorization. We avoid
|
||||
@@ -329,6 +353,10 @@ async def login(request: Request, db: Session = Depends(get_db)):
|
||||
# Google sign-in isn't configured on this instance — land back on the login page (the UI
|
||||
# hides the button, so this is just a safety net for a stale/direct hit).
|
||||
return RedirectResponse(url="/")
|
||||
# A sign-in, never a link. Starting it retires every earlier flow's state (authlib keeps only the
|
||||
# newest), so a connect's marker left from before — abandoned at Google, or still open in
|
||||
# another tab — names a flow that can no longer finish.
|
||||
request.session.pop(_LINK_MARKER, None)
|
||||
return await client.authorize_redirect(
|
||||
request,
|
||||
_oauth_redirect_uri(request),
|
||||
@@ -349,24 +377,23 @@ async def callback(
|
||||
if client is None:
|
||||
return RedirectResponse(url="/")
|
||||
# Linking mode: an already-authenticated account is attaching this Google identity (a
|
||||
# password account enabling YouTube or connecting SSO). The markers are set by /auth/link and
|
||||
# /auth/upgrade; when present we attach to the current user instead of identifying by sub.
|
||||
# Pop them BEFORE the token exchange: if the user aborts Google's consent screen the exchange
|
||||
# throws below, and a marker left behind would silently turn the NEXT clean sign-in into a link
|
||||
# attempt. Popping up front means an aborted link flow leaves no residue in the session.
|
||||
link_uid = request.session.pop("oauth_link_uid", None)
|
||||
link_explicit = request.session.pop("oauth_link_explicit", False)
|
||||
# password account enabling YouTube or connecting SSO). The marker is set by
|
||||
# /auth/google/connect; when it is this flow's we attach to the current user instead of
|
||||
# identifying by sub. Taken BEFORE the token exchange: if the user aborts Google's consent screen
|
||||
# the exchange throws below, and a marker left behind would silently turn the NEXT clean sign-in
|
||||
# into a link attempt. A connect that never comes back here at all is cleared by /auth/login.
|
||||
link = _take_link_marker(request)
|
||||
try:
|
||||
token = await client.authorize_access_token(request)
|
||||
except OAuthError as exc:
|
||||
# The user cancelled or denied consent at Google — land back with a friendly notice instead
|
||||
# of a raw 400 JSON error page. An already-signed-in user (adding another account via
|
||||
# /auth/login, or linking/upgrading via /auth/link|upgrade) never sees the pre-auth login
|
||||
# banner, so surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on
|
||||
# the login page with the oauth_cancelled banner instead. Keyed on the live session, not on
|
||||
# link_uid, so the add-account path (no marker) is covered too.
|
||||
# of a raw 400 JSON error page. An already-signed-in user (signing in again via /auth/login,
|
||||
# or connecting via /auth/google/connect) never sees the pre-auth login banner, so
|
||||
# surface a neutral toast via ?oauth=cancelled; a fresh sign-in cancel lands on the login page
|
||||
# with the oauth_cancelled banner instead. Keyed on the live session, not on link_uid, so a
|
||||
# sign-in started while signed in (no marker) is covered too.
|
||||
log.info("Google OAuth cancelled/failed: %s", exc.error)
|
||||
if request.session.get("user_id") is not None:
|
||||
if session_user_id(request) is not None:
|
||||
return RedirectResponse(url="/?oauth=cancelled")
|
||||
return RedirectResponse(url="/?login=oauth_cancelled")
|
||||
|
||||
@@ -377,8 +404,8 @@ async def callback(
|
||||
sub = userinfo["sub"]
|
||||
email = (userinfo.get("email") or "").lower()
|
||||
|
||||
if link_uid is not None:
|
||||
return _complete_link(request, db, link_uid, link_explicit, sub, userinfo, token)
|
||||
if link is not None:
|
||||
return _complete_link(request, db, link["uid"], link["explicit"], sub, userinfo, token)
|
||||
|
||||
if not is_allowed(db, email):
|
||||
# No uid to name — having no account here is the branch. And the address is left out on
|
||||
@@ -579,19 +606,16 @@ def _complete_link(
|
||||
token: dict,
|
||||
):
|
||||
"""Attach the just-authorized Google identity to the already-signed-in account that started
|
||||
the link/upgrade. No is_allowed gate — they're already an active user. Refuses to hijack a
|
||||
the connect. No is_allowed gate — they're already an active user. Refuses to hijack a
|
||||
Google identity owned by another account, or to silently swap a different linked identity.
|
||||
`explicit` (the popped oauth_link_explicit marker) distinguishes a "Connect Google" click from
|
||||
a YouTube upgrade — the caller pops it up front so an aborted flow leaves nothing behind."""
|
||||
# SEC-11: the marker must name an account signed into THIS browser — the session default OR any
|
||||
# account in the wallet — not only the default. A tab acting as a secondary account links Google to
|
||||
# ITSELF (the marker was set from current_user, which resolves the ?account= it navigated with), so
|
||||
# accepting wallet membership stops the identity/token landing on the default account A instead of B.
|
||||
accounts = request.session.get("account_ids") or []
|
||||
if link_uid != request.session.get("user_id") and link_uid not in accounts:
|
||||
return RedirectResponse(url="/?link=error")
|
||||
user = db.get(User, link_uid)
|
||||
if user is None:
|
||||
`explicit` (from the link marker) distinguishes a "Connect Google" click from a YouTube grant —
|
||||
the caller takes the marker up front so an aborted flow leaves nothing behind."""
|
||||
# The marker must name the account still signed into THIS browser, and that session must still
|
||||
# hold: a sign-in as someone else between the start and the callback replaced it (the identity
|
||||
# must not land on the newcomer), and a session revoked meanwhile ("log out everywhere", a
|
||||
# password change, a suspension) must not finish what it started.
|
||||
user = optional_current_user(request, db)
|
||||
if user is None or user.id != link_uid:
|
||||
return RedirectResponse(url="/?link=error")
|
||||
other = db.execute(
|
||||
select(User).where(User.google_sub == sub, User.id != user.id)
|
||||
@@ -611,45 +635,31 @@ def _complete_link(
|
||||
_store_token(db, user, token)
|
||||
db.commit()
|
||||
log.info("Google linked: uid=%s scopes=%s", user.id, token.get("scope"))
|
||||
# An explicit "Connect Google" lands on Settings with a confirmation; a YouTube upgrade
|
||||
# An explicit "Connect Google" lands on Settings with a confirmation; a YouTube grant
|
||||
# (wizard / Settings access rows) returns to the app so the onboarding flow resumes as before.
|
||||
return RedirectResponse(url="/?link=ok" if explicit else "/")
|
||||
|
||||
|
||||
@router.post("/logout")
|
||||
async def logout(request: Request):
|
||||
"""Sign the requesting tab's active account out of this browser (removing it from the wallet).
|
||||
Per-tab aware: the account is taken from the X-Siftlode-Account header when present, so logging
|
||||
out in one tab doesn't disturb another tab running a different account. If the account being
|
||||
removed was the session default, promote the most recent remaining account (or clear the
|
||||
session when none remain)."""
|
||||
# A tab still pinned to an account already signed out in another tab resolves to nobody, so it
|
||||
# signs nobody out — it used to fall back to the default and sign THAT account out in its place.
|
||||
active, is_default = resolved_user_id(request)
|
||||
remaining = [a for a in (request.session.get("account_ids") or []) if a != active]
|
||||
request.session["account_ids"] = remaining
|
||||
if is_default:
|
||||
if remaining:
|
||||
request.session["user_id"] = remaining[-1]
|
||||
return JSONResponse({"ok": True, "switched": True})
|
||||
request.session.clear()
|
||||
return JSONResponse({"ok": True, "switched": False})
|
||||
# A non-default (per-tab) account signed out: it's gone from the wallet and the session default
|
||||
# is untouched. The tab drops its own override and falls back to the default on reload.
|
||||
return JSONResponse({"ok": True, "switched": False})
|
||||
"""Sign this browser out: the session holds one account, and it ends — unless the tab asking was
|
||||
built for another account than the one signed in now (a sign-in as someone else in another tab):
|
||||
its Sign out would end THAT person's session (see `EXPECTED_ACCOUNT_HEADER`)."""
|
||||
signed_in = session_user_id(request)
|
||||
if signed_in is not None and built_for_another_account(
|
||||
signed_in, request.headers.get(EXPECTED_ACCOUNT_HEADER)
|
||||
):
|
||||
raise refused(401, "account_changed")
|
||||
request.session.clear()
|
||||
return JSONResponse({"ok": True})
|
||||
|
||||
|
||||
def _establish_session(request: Request, user: User) -> None:
|
||||
"""Sign `user` in for this browser session — the ONE sign-in path, shared by the Google callback
|
||||
and password login, so both join the same account switcher and both stamp `last_login_at` (the
|
||||
caller commits). Multi-session: every account that authenticated in this browser is remembered so
|
||||
the user can switch between them; only accounts that actually proved themselves HERE land in that
|
||||
wallet (it is the proof they may be switched to)."""
|
||||
accounts = [a for a in (request.session.get("account_ids") or []) if a != user.id]
|
||||
accounts.append(user.id)
|
||||
request.session["account_ids"] = accounts[-MAX_SESSION_ACCOUNTS:]
|
||||
and password login, so both stamp `last_login_at` (the caller commits). One account per session:
|
||||
the sign-in replaces whoever was signed in here before."""
|
||||
request.session["user_id"] = user.id
|
||||
_remember_epoch(request, user)
|
||||
stamp_session(request, user)
|
||||
user.last_login_at = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
@@ -1124,7 +1134,7 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
|
||||
# Burn any other outstanding reset tokens for this user.
|
||||
for row in db.execute(
|
||||
select(AuthToken).where(
|
||||
AuthToken.user_id == user.id,
|
||||
owned_by(AuthToken.user_id, user),
|
||||
AuthToken.kind.in_(_PASSWORD_TOKEN_KINDS),
|
||||
AuthToken.used_at.is_(None),
|
||||
)
|
||||
@@ -1134,15 +1144,6 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
def _remember_epoch(request: Request, user: User) -> None:
|
||||
"""Record this account's current `session_epoch` in the signed cookie (SA4), so `current_user`
|
||||
can later reject the cookie if the account's epoch was bumped (password reset/change, or
|
||||
"log out everywhere"). Per-account so one account's revocation doesn't evict the others."""
|
||||
epochs = dict(request.session.get("epochs") or {})
|
||||
epochs[str(user.id)] = user.session_epoch
|
||||
request.session["epochs"] = epochs
|
||||
|
||||
|
||||
def bump_session_epoch(db: Session, user: User) -> None:
|
||||
"""Invalidate every EXISTING signed cookie for this account by advancing its epoch (SA4). Cookies
|
||||
minted before the bump carry a now-stale epoch and are rejected by `current_user`. The account's
|
||||
@@ -1219,35 +1220,11 @@ def logout_others(
|
||||
# session at a newer epoch than the DB (see set_password).
|
||||
bump_session_epoch(db, user)
|
||||
db.commit()
|
||||
_remember_epoch(request, user)
|
||||
stamp_session(request, user)
|
||||
log.info("Logout-others: uid=%s (epoch=%s)", user.id, user.session_epoch)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.get("/link")
|
||||
async def link_google(
|
||||
request: Request,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Start linking a Google account to the signed-in (e.g. password) account. Requests only
|
||||
the identity scopes — YouTube access is granted separately via /auth/upgrade. The callback
|
||||
sees `oauth_link_uid` and attaches the identity instead of creating a new account."""
|
||||
client = google_oauth(db)
|
||||
if client is None:
|
||||
return RedirectResponse(url="/?link=error")
|
||||
request.session["oauth_link_uid"] = user.id
|
||||
request.session["oauth_link_explicit"] = True
|
||||
return await client.authorize_redirect(
|
||||
request,
|
||||
_oauth_redirect_uri(request),
|
||||
access_type="offline",
|
||||
prompt="select_account",
|
||||
include_granted_scopes="true",
|
||||
scope=BASE_SCOPES,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/set-password")
|
||||
def set_password(
|
||||
body: SetPasswordIn,
|
||||
@@ -1271,35 +1248,57 @@ def set_password(
|
||||
# can't leave the cookie at a newer epoch than the DB (which would wrongly 401 THIS session).
|
||||
bump_session_epoch(db, user)
|
||||
db.commit()
|
||||
_remember_epoch(request, user)
|
||||
stamp_session(request, user)
|
||||
log.info("Password set/changed: uid=%s", user.id)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.get("/upgrade")
|
||||
async def upgrade(
|
||||
# What each Google connect asks Google for. `identity` attaches a Google sign-in to the account
|
||||
# (Settings → Connect Google) and lets the person pick which Google account; `read` and `write` grant
|
||||
# YouTube access (the onboarding wizard, the Settings access rows) and always show the consent screen,
|
||||
# so a scope granted earlier and since revoked at Google is asked for again.
|
||||
_CONNECT: dict[str, tuple[str, str]] = {
|
||||
"identity": (BASE_SCOPES, "select_account"),
|
||||
"read": (READ_SCOPES, "consent"),
|
||||
"write": (WRITE_SCOPES, "consent"),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/google/connect")
|
||||
async def google_connect(
|
||||
request: Request,
|
||||
access: str = "read",
|
||||
access: Literal["identity", "read", "write"],
|
||||
expect: str | None = None,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Incremental consent for the onboarding wizard. `access=read` grants YouTube
|
||||
read-only (enough to build the feed); `access=write` additionally grants management
|
||||
(unsubscribe). The shared callback stores whatever Google returns, so `can_read` /
|
||||
`can_write` flip on once granted. Anything other than an explicit `write` defaults to
|
||||
the least-privileged read scope."""
|
||||
"""Attach Google to the signed-in account: its identity, or YouTube read/write access on top.
|
||||
The shared callback sees the link marker and attaches what Google returns to THIS account —
|
||||
without it a password account (no google_sub) would get a brand-new, separate Google user
|
||||
created instead. It stores whatever scopes Google grants, so `can_read` / `can_write` flip on
|
||||
once granted.
|
||||
|
||||
`expect` is the account the tab was built for. A navigation carries no headers, so the check
|
||||
every request gets (`EXPECTED_ACCOUNT_HEADER`) rides the query here: a tab still showing the
|
||||
account signed in before would otherwise attach a Google identity to the one signed in now."""
|
||||
if built_for_another_account(user.id, expect):
|
||||
return RedirectResponse(url=ACCOUNT_CHANGED_LANDING)
|
||||
client = google_oauth(db)
|
||||
if client is None:
|
||||
return RedirectResponse(url="/")
|
||||
# Attach the grant to THIS account in the callback. Without it a password account (no
|
||||
# google_sub) would get a brand-new, separate Google user created instead of being linked.
|
||||
request.session["oauth_link_uid"] = user.id
|
||||
scope = WRITE_SCOPES if access == "write" else READ_SCOPES
|
||||
return RedirectResponse(url="/?link=error")
|
||||
scope, prompt = _CONNECT[access]
|
||||
state = generate_token(48)
|
||||
request.session[_LINK_MARKER] = {
|
||||
"state": state,
|
||||
"uid": user.id,
|
||||
"explicit": access == "identity",
|
||||
}
|
||||
return await client.authorize_redirect(
|
||||
request,
|
||||
_oauth_redirect_uri(request),
|
||||
state=state,
|
||||
access_type="offline",
|
||||
prompt="consent",
|
||||
prompt=prompt,
|
||||
include_granted_scopes="true",
|
||||
scope=scope,
|
||||
)
|
||||
|
||||
+70
-261
@@ -4,31 +4,22 @@ hidden, tags) and per-channel sync state. Local reads/writes plus YouTube-side w
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from functools import partial
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import Select, func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app.mailtext import lang_for_user
|
||||
from app import logs, quota, scheduler
|
||||
from app.auth import admin_user, current_user, require_write_scope
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
ChannelTag,
|
||||
ExploredChannel,
|
||||
Playlist,
|
||||
PlaylistItem,
|
||||
Subscription,
|
||||
Tag,
|
||||
User,
|
||||
Video,
|
||||
)
|
||||
from app.routes._common import get_or_404, not_found, owned_or_404, youtube_refusal
|
||||
from app.models import User
|
||||
from app.catalog import queries, service
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
from app.routes._common import not_found, youtube_refusal
|
||||
from app.sync.explore import explore_ingest_page
|
||||
from app.sync.runner import backfill_channel_now, run_recent_backfill
|
||||
from app.sync.subscriptions import apply_channel_details, details_stale
|
||||
@@ -36,35 +27,17 @@ from app.utils import iso
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.catalog.models import Channel, Subscription
|
||||
|
||||
router = APIRouter(prefix="/api/channels", tags=["channels"])
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def channel_aggregates(channel_ids: list[str]) -> Select:
|
||||
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
|
||||
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
|
||||
alone; a test holds that (test_channel_list_index_only)."""
|
||||
return (
|
||||
select(
|
||||
Video.channel_id,
|
||||
func.count(Video.id),
|
||||
func.max(Video.published_at),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.where(Video.channel_id.in_(channel_ids))
|
||||
.group_by(Video.channel_id)
|
||||
)
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_channels(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
rows = db.execute(
|
||||
select(Subscription, Channel)
|
||||
.join(Channel, Channel.id == Subscription.channel_id)
|
||||
.where(Subscription.user_id == user.id)
|
||||
.order_by(Subscription.priority.desc(), collated(Channel.title, lang_for_user(user)))
|
||||
).all()
|
||||
rows = queries.subscriptions(db, user, lang_for_user(user))
|
||||
|
||||
# Per-channel aggregates over the shared catalog (channel_aggregates), computed on read rather than
|
||||
# denormalised: an Index Only Scan over a ~20 MB index, where it used to read every video's heap page
|
||||
@@ -73,36 +46,21 @@ def list_channels(user: User = Depends(current_user), db: Session = Depends(get_
|
||||
# computed here — the manager table dropped its N/S/L column.)
|
||||
channel_ids = [c.id for _, c in rows]
|
||||
agg: dict[str, dict] = {}
|
||||
if channel_ids:
|
||||
for cid, n, last_up, total_dur in db.execute(channel_aggregates(channel_ids)).all():
|
||||
agg[cid] = {
|
||||
"stored": n,
|
||||
"last_video_at": iso(last_up),
|
||||
"total_duration_seconds": int(total_dur or 0),
|
||||
}
|
||||
for cid, n, last_up, total_dur in queries.aggregates(db, channel_ids):
|
||||
agg[cid] = {
|
||||
"stored": n,
|
||||
"last_video_at": iso(last_up),
|
||||
"total_duration_seconds": int(total_dur or 0),
|
||||
}
|
||||
|
||||
# Channels already in the *global* deep-backfill queue — i.e. at least one user (any
|
||||
# user) has requested full history and it isn't done yet. Their whole back-catalogue is
|
||||
# coming for everyone (video data is shared), so the UI shouldn't offer "get full
|
||||
# history" to a user who simply hasn't personally opted in.
|
||||
deep_queued: set[str] = set()
|
||||
if channel_ids:
|
||||
for (cid,) in db.execute(
|
||||
select(Subscription.channel_id)
|
||||
.where(
|
||||
Subscription.channel_id.in_(channel_ids),
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.distinct()
|
||||
).all():
|
||||
deep_queued.add(cid)
|
||||
deep_queued = queries.deep_queued(db, channel_ids)
|
||||
|
||||
# The user's personal tag links, grouped by channel.
|
||||
tags_by_channel: dict[str, list[int]] = {}
|
||||
for cid, tag_id in db.execute(
|
||||
select(ChannelTag.channel_id, ChannelTag.tag_id).where(ChannelTag.user_id == user.id)
|
||||
).all():
|
||||
tags_by_channel.setdefault(cid, []).append(tag_id)
|
||||
tags_by_channel = reader_queries.user_tag_links(db, user)
|
||||
|
||||
return [
|
||||
{
|
||||
@@ -131,27 +89,7 @@ DISCOVERY_ENRICH_CAP = 200
|
||||
def _discovery_rows(db: Session, user: User) -> list:
|
||||
"""The user's playlist channels they don't subscribe to (and that aren't their own),
|
||||
each with how many playlist videos come from it and across how many playlists."""
|
||||
subscribed = select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
q = (
|
||||
select(
|
||||
Channel,
|
||||
func.count(func.distinct(PlaylistItem.video_id)),
|
||||
func.count(func.distinct(Playlist.id)),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.where(Playlist.user_id == user.id, Channel.id.not_in(subscribed))
|
||||
)
|
||||
if user.yt_channel_id:
|
||||
q = q.where(Channel.id != user.yt_channel_id)
|
||||
return db.execute(
|
||||
q.group_by(Channel.id).order_by(
|
||||
func.count(func.distinct(PlaylistItem.video_id)).desc(),
|
||||
collated(Channel.title, lang_for_user(user)),
|
||||
)
|
||||
).all()
|
||||
return reader_queries.playlist_channels(db, user, user.yt_channel_id, lang_for_user(user))
|
||||
|
||||
|
||||
@router.get("/discovery")
|
||||
@@ -173,7 +111,7 @@ def discover_channels(
|
||||
):
|
||||
own = yt.get_my_channel_id()
|
||||
if own:
|
||||
user.yt_channel_id = own
|
||||
service.remember_own_channel(user, own)
|
||||
db.commit()
|
||||
except YouTubeError as exc:
|
||||
# No traceback anywhere in this file's YouTube handlers: for an upstream answer
|
||||
@@ -229,7 +167,7 @@ def discover_channels(
|
||||
]
|
||||
|
||||
|
||||
def _channel_summary(ch: Channel) -> dict:
|
||||
def _channel_summary(ch: "Channel") -> dict:
|
||||
"""The channel fields common to the list and discovery projections."""
|
||||
return {
|
||||
"id": ch.id,
|
||||
@@ -243,7 +181,7 @@ def _channel_summary(ch: Channel) -> dict:
|
||||
|
||||
|
||||
def _channel_detail_dict(
|
||||
channel: Channel,
|
||||
channel: "Channel",
|
||||
*,
|
||||
subscribed: bool,
|
||||
explored: bool,
|
||||
@@ -312,7 +250,7 @@ def channel_detail(
|
||||
needed (a stub) and refreshes it once it is stale (`DETAILS_MAX_AGE`), which is how a channel nobody subscribes to (an explored one, outside the
|
||||
subscription import) gets a banner that still resolves. The enrich uses the API key (1 unit,
|
||||
cached after)."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
# No `published_at is None` sentinel: a terminated channel never gets one, so it re-fetched on
|
||||
# every view; rows enriched before that field existed are long past DETAILS_MAX_AGE anyway.
|
||||
stale = details_stale(channel, datetime.now(timezone.utc))
|
||||
@@ -324,7 +262,7 @@ def channel_detail(
|
||||
):
|
||||
apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
|
||||
db.commit()
|
||||
channel = db.get(Channel, channel_id)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
except YouTubeError as exc:
|
||||
log.warning(
|
||||
"channel detail enrich failed: %s",
|
||||
@@ -334,41 +272,15 @@ def channel_detail(
|
||||
),
|
||||
)
|
||||
db.rollback()
|
||||
sub = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
sub = queries.subscription(db, user, channel_id)
|
||||
subscribed = sub is not None
|
||||
explored = (
|
||||
db.execute(
|
||||
select(ExploredChannel.id).where(
|
||||
ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
blocked = _is_blocked(db, user, channel_id)
|
||||
explored = queries.exploration(db, user, channel_id) is not None
|
||||
blocked = queries.is_blocked(db, user, channel_id)
|
||||
# Stored total + the Normal/Shorts/Live split in one grouped pass (same filters as the manager
|
||||
# list), so the channel page can show the per-type breakdown that moved off the table.
|
||||
n_all, n_short, n_live = db.execute(
|
||||
select(
|
||||
func.count(Video.id),
|
||||
func.count(Video.id).filter(Video.is_short.is_(True)),
|
||||
func.count(Video.id).filter(Video.live_status.in_(LIVE_OR_UPCOMING)),
|
||||
).where(Video.channel_id == channel_id)
|
||||
).one()
|
||||
stored = int(n_all or 0)
|
||||
count_short = int(n_short or 0)
|
||||
count_live = int(n_live or 0)
|
||||
stored, count_short, count_live = queries.video_counts(db, channel_id)
|
||||
count_normal = max(0, stored - count_short - count_live)
|
||||
tag_ids = list(
|
||||
db.scalars(
|
||||
select(ChannelTag.tag_id).where(
|
||||
ChannelTag.user_id == user.id, ChannelTag.channel_id == channel_id
|
||||
)
|
||||
).all()
|
||||
)
|
||||
tag_ids = reader_queries.user_tag_links(db, user, channel_id).get(channel_id, [])
|
||||
return _channel_detail_dict(
|
||||
channel,
|
||||
subscribed=subscribed,
|
||||
@@ -398,28 +310,15 @@ def explore_channel(
|
||||
it, then ingests one page of its uploads (newest-first) marked via_explore + enriched.
|
||||
Returns the next page token so the page can "load more". Guarded against the shared
|
||||
backfill quota reserve."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
if _is_blocked(db, user, channel_id):
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
if queries.is_blocked(db, user, channel_id):
|
||||
raise refused(403, "channel_blocked")
|
||||
if quota.backfill_reserve_reached():
|
||||
raise refused(429, "quota_reserve")
|
||||
|
||||
# Record/refresh the exploration (DB only) before ingesting, so the explorer can see the
|
||||
# videos as soon as they land. Flag the channel ephemeral unless it's organically followed.
|
||||
existing = db.execute(
|
||||
select(ExploredChannel).where(
|
||||
ExploredChannel.user_id == user.id, ExploredChannel.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is None:
|
||||
db.add(ExploredChannel(user_id=user.id, channel_id=channel_id))
|
||||
else:
|
||||
existing.created_at = datetime.now(timezone.utc) # refresh the grace clock
|
||||
followed = db.execute(
|
||||
select(Subscription.id).where(Subscription.channel_id == channel_id).limit(1)
|
||||
).first()
|
||||
if followed is None and not channel.from_explore:
|
||||
channel.from_explore = True
|
||||
service.explore(db, user, channel)
|
||||
db.commit()
|
||||
|
||||
try:
|
||||
@@ -433,7 +332,7 @@ def explore_channel(
|
||||
if details_stale(channel, datetime.now(timezone.utc)):
|
||||
apply_channel_details(db, yt.get_channels([channel_id]), [channel_id])
|
||||
db.commit()
|
||||
channel = db.get(Channel, channel_id)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
result = explore_ingest_page(db, yt, channel, page_token)
|
||||
except YouTubeError as exc:
|
||||
raise youtube_refusal(exc) from exc
|
||||
@@ -445,30 +344,20 @@ def explore_channel(
|
||||
}
|
||||
|
||||
|
||||
def _user_subscription(db: Session, user: User, channel_id: str) -> Subscription:
|
||||
sub = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
def _channel_or_404(db: Session, channel_id: str) -> "Channel":
|
||||
channel = queries.channel(db, channel_id)
|
||||
if channel is None:
|
||||
raise not_found("channel")
|
||||
return channel
|
||||
|
||||
|
||||
def _user_subscription(db: Session, user: User, channel_id: str) -> "Subscription":
|
||||
sub = queries.subscription(db, user, channel_id)
|
||||
if sub is None:
|
||||
raise not_found("subscription")
|
||||
return sub
|
||||
|
||||
|
||||
def _is_blocked(db: Session, user: User, channel_id: str) -> bool:
|
||||
"""Whether this user has blocked this channel."""
|
||||
return (
|
||||
db.execute(
|
||||
select(BlockedChannel.id).where(
|
||||
BlockedChannel.user_id == user.id,
|
||||
BlockedChannel.channel_id == channel_id,
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
class ChannelUpdateIn(BaseModel):
|
||||
# PATCH semantics: a field left unset (None) is untouched. Every field is optional so the
|
||||
# UI can send just the one control it changed. int/bool coercion + rejection of garbage
|
||||
@@ -486,19 +375,14 @@ def update_channel(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
sub = _user_subscription(db, user, channel_id)
|
||||
channel = db.get(Channel, channel_id)
|
||||
deep_turned_on = False
|
||||
if body.priority is not None:
|
||||
sub.priority = body.priority
|
||||
if body.hidden is not None:
|
||||
sub.hidden = body.hidden
|
||||
if body.deep_requested is not None:
|
||||
# Opt this channel into (or out of) full-history backfill. The deep scheduler
|
||||
# picks the flag up on its next run; turning it off won't undo already-fetched
|
||||
# videos, it just stops further deep paging if nobody else still wants it.
|
||||
want = body.deep_requested
|
||||
deep_turned_on = want and not sub.deep_requested
|
||||
sub.deep_requested = want
|
||||
channel = queries.channel(db, channel_id)
|
||||
deep_turned_on = service.update_subscription(
|
||||
db,
|
||||
sub,
|
||||
priority=body.priority,
|
||||
hidden=body.hidden,
|
||||
deep_requested=body.deep_requested,
|
||||
)
|
||||
db.commit()
|
||||
|
||||
# Opting a channel into full history should give an immediate feed: if we haven't even
|
||||
@@ -542,19 +426,8 @@ def reset_backfill(
|
||||
state (a "reset" trigger). Clears the channel's backfill markers, opts it back into deep
|
||||
backfill, and re-runs its recent pull immediately; the deep scheduler re-pages the full
|
||||
back-catalog on its next run. Idempotent — videos upsert by id, so nothing duplicates."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
channel.backfill_done = False
|
||||
channel.backfill_cursor = None
|
||||
channel.recent_synced_at = None
|
||||
# Re-arm the demand-driven deep scheduler for this channel. run_deep_backfill gates on an
|
||||
# EXISTS over subscriptions with deep_requested=True, so flip it on EVERY subscriber rather
|
||||
# than requiring the admin to be personally subscribed (they reset from the manager, not
|
||||
# their own feed). No-op row count if the channel has no subscribers.
|
||||
db.execute(
|
||||
update(Subscription)
|
||||
.where(Subscription.channel_id == channel_id)
|
||||
.values(deep_requested=True)
|
||||
)
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
service.reset_backfill(db, channel)
|
||||
db.commit()
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_BACKFILL_RECENT):
|
||||
run_recent_backfill(db, [channel], max_channels=1)
|
||||
@@ -572,13 +445,8 @@ def subscribe(
|
||||
local subscription with YouTube's returned resource id and enriches the channel's metadata
|
||||
if it's still a stub, so the new row renders properly in the manager. It does NOT pull the
|
||||
channel's videos — the scheduler picks those up on its next run, like any subscribed channel."""
|
||||
channel = get_or_404(db, Channel, channel_id, what="channel")
|
||||
existing = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == channel_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
channel = _channel_or_404(db, channel_id)
|
||||
if queries.subscription(db, user, channel_id) is not None:
|
||||
raise refused(409, "already_subscribed")
|
||||
with (
|
||||
quota.attribute(user.id, quota.QuotaAction.CHANNELS_SUBSCRIBE),
|
||||
@@ -618,24 +486,7 @@ def subscribe(
|
||||
reason=exc.reason,
|
||||
),
|
||||
)
|
||||
db.add(
|
||||
Subscription(
|
||||
user_id=user.id,
|
||||
channel_id=channel_id,
|
||||
yt_subscription_id=yt_sub_id or None,
|
||||
subscribed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
# Subscribing is the "keep" signal: a previously-explored channel becomes a normal organic
|
||||
# one. Drop its ephemeral flags so the cleanup job leaves it alone and its videos stop being
|
||||
# per-user private (they join the shared Library like any subscribed channel's uploads).
|
||||
if channel.from_explore:
|
||||
channel.from_explore = False
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.channel_id == channel_id, Video.via_explore.is_(True))
|
||||
.values(via_explore=False)
|
||||
)
|
||||
service.record_subscription(db, user, channel, yt_sub_id)
|
||||
db.commit()
|
||||
return {"ok": True, "subscribed": channel_id}
|
||||
|
||||
@@ -662,7 +513,7 @@ def unsubscribe(
|
||||
# 422 (a real, explainable error → speaking modal), not 502 which the client treats as
|
||||
# a transient "connection lost" gateway blip.
|
||||
raise youtube_refusal(exc) from exc
|
||||
db.delete(sub)
|
||||
service.drop_subscription(db, sub)
|
||||
db.commit()
|
||||
return {"ok": True, "unsubscribed": channel_id}
|
||||
|
||||
@@ -670,16 +521,7 @@ def unsubscribe(
|
||||
@router.get("/blocked/list")
|
||||
def list_blocked(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
"""The user's blocked channels (videos from these are excluded from their search/feed/explore)."""
|
||||
rows = (
|
||||
db.execute(
|
||||
select(Channel)
|
||||
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
|
||||
.where(BlockedChannel.user_id == user.id)
|
||||
.order_by(collated(Channel.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = queries.blocked_channels(db, user, lang_for_user(user))
|
||||
return [
|
||||
{"id": c.id, "title": c.title, "handle": c.handle, "thumbnail_url": c.thumbnail_url}
|
||||
for c in rows
|
||||
@@ -695,16 +537,9 @@ def block_channel(
|
||||
"""Block a channel for this user: its videos won't appear in their live search (and won't be
|
||||
ingested for them) and are hidden from their feed / explore. Idempotent. The now-hidden
|
||||
un-kept search/explore videos are reclaimed by the discovery-cleanup job in due course."""
|
||||
get_or_404(db, Channel, channel_id, what="channel")
|
||||
if not _is_blocked(db, user, channel_id):
|
||||
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
|
||||
# Stop any active exploration of it so it can be cleaned up.
|
||||
db.execute(
|
||||
ExploredChannel.__table__.delete().where(
|
||||
(ExploredChannel.user_id == user.id) & (ExploredChannel.channel_id == channel_id)
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
_channel_or_404(db, channel_id)
|
||||
service.block(db, user, channel_id)
|
||||
db.commit()
|
||||
return {"ok": True, "blocked": channel_id}
|
||||
|
||||
|
||||
@@ -714,11 +549,7 @@ def unblock_channel(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
db.execute(
|
||||
BlockedChannel.__table__.delete().where(
|
||||
(BlockedChannel.user_id == user.id) & (BlockedChannel.channel_id == channel_id)
|
||||
)
|
||||
)
|
||||
service.unblock(db, user, channel_id)
|
||||
db.commit()
|
||||
return {"ok": True, "unblocked": channel_id}
|
||||
|
||||
@@ -735,26 +566,12 @@ def attach_tag(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
_user_subscription(db, user, channel_id)
|
||||
# Only the user's own tags may be attached as personal links (owned_or_404 → 404 if the tag
|
||||
# is missing OR belongs to someone else, without leaking which).
|
||||
owned_or_404(db, Tag, body.tag_id, user, what="tag")
|
||||
exists = db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == body.tag_id,
|
||||
ChannelTag.user_id == user.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if exists is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel_id,
|
||||
tag_id=body.tag_id,
|
||||
user_id=user.id,
|
||||
source="user",
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
# Only the user's own tags may be attached as personal links (a 404 if the tag is missing OR
|
||||
# belongs to someone else, without leaking which).
|
||||
if reader_queries.own_tag(db, user, body.tag_id) is None:
|
||||
raise not_found("tag")
|
||||
reader_service.attach_user_tag(db, user, channel_id, body.tag_id)
|
||||
db.commit()
|
||||
return {"ok": True, "channel_id": channel_id, "tag_id": body.tag_id}
|
||||
|
||||
|
||||
@@ -765,14 +582,6 @@ def detach_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
row = db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel_id,
|
||||
ChannelTag.tag_id == tag_id,
|
||||
ChannelTag.user_id == user.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.commit()
|
||||
reader_service.detach_user_tag(db, user, channel_id, tag_id)
|
||||
db.commit()
|
||||
return {"ok": True, "channel_id": channel_id, "tag_id": tag_id}
|
||||
|
||||
@@ -18,6 +18,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs, logsink
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -95,7 +96,7 @@ def stop_session(
|
||||
def list_sessions(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sessions = db.scalars(
|
||||
select(DebugSession)
|
||||
.where(DebugSession.user_id == user.id)
|
||||
.where(owned_by(DebugSession.user_id, user))
|
||||
.order_by(DebugSession.started_at.desc())
|
||||
.limit(LIST_MAX)
|
||||
).all()
|
||||
@@ -219,6 +220,7 @@ def send_issue(sid: str, user: User = Depends(current_user), db: Session = Depen
|
||||
raise refused(409, "issue_sending")
|
||||
new = session.issue_url is None
|
||||
if not new and not debugsession.unfinished(session):
|
||||
db.commit() # nothing to send: the lock ends here, not when the request's session closes
|
||||
return {"ok": True, "session": _view(db, session)}
|
||||
key = str(user.id)
|
||||
if new:
|
||||
|
||||
@@ -13,7 +13,6 @@ from typing import Literal
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -22,9 +21,9 @@ from app.db import get_db
|
||||
from app.discover import hero as hero_mod
|
||||
from app.discover import pins, rails
|
||||
from app.discover.budget import Budget, DEFAULT_BUDGET
|
||||
from app.models import ComfortPin, User
|
||||
from app.models import User
|
||||
from app.plex.cards import norm_lang
|
||||
from app.routes._common import owned_or_404, not_found
|
||||
from app.routes._common import not_found
|
||||
|
||||
router = APIRouter(prefix="/api/discover", tags=["discover"])
|
||||
|
||||
@@ -140,30 +139,19 @@ def add_pin(
|
||||
raise not_found("item")
|
||||
column, value = resolved
|
||||
|
||||
existing = db.execute(
|
||||
select(ComfortPin).where(
|
||||
ComfortPin.user_id == user.id,
|
||||
getattr(ComfortPin, column) == value,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
existing = pins.pin_of(db, user, column, value)
|
||||
if existing is not None:
|
||||
return {"ok": True, "id": existing.id, "created": False}
|
||||
|
||||
pin = ComfortPin(user_id=user.id, position=pins.next_position(db, user), **{column: value})
|
||||
db.add(pin)
|
||||
try:
|
||||
pin = pins.add(db, user, column, value)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
# A concurrent pin of the same thing from a second tab. The unique constraint did its job;
|
||||
# the user's intent ("be on my shelf") is satisfied either way, so report the winner rather
|
||||
# than a 500 the user can do nothing with.
|
||||
db.rollback()
|
||||
winner = db.execute(
|
||||
select(ComfortPin).where(
|
||||
ComfortPin.user_id == user.id,
|
||||
getattr(ComfortPin, column) == value,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
winner = pins.pin_of(db, user, column, value)
|
||||
if winner is None:
|
||||
# Not the unique constraint, then: the only other way in is the target row being pruned
|
||||
# between `resolve_ref` confirming it and this INSERT (a Plex sync dropping a removed
|
||||
@@ -180,8 +168,10 @@ def remove_pin(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pin = owned_or_404(db, ComfortPin, pin_id, user, what="pin")
|
||||
db.delete(pin)
|
||||
pin = pins.own_pin(db, user, pin_id)
|
||||
if pin is None:
|
||||
raise not_found("pin")
|
||||
pins.remove(db, pin)
|
||||
db.commit()
|
||||
return {"deleted": pin_id}
|
||||
|
||||
|
||||
+106
-337
@@ -3,23 +3,22 @@
|
||||
All routes are behind `current_user`; admin routes add `admin_user`. Live job state is polled by the
|
||||
frontend (the worker is a separate process); this module never downloads — it enqueues, manages job
|
||||
state, and shares finished files. The library itself — listing, renaming, moving, deleting and
|
||||
serving a done file, and uploading one — is `routes.files`.
|
||||
serving a done file, and uploading one — is `routes.files`. Reads come from `downloads.queries`,
|
||||
writes go through Capture's service (`service`, `links`, `quota`); this module validates, answers in
|
||||
HTTP and commits.
|
||||
"""
|
||||
|
||||
import re
|
||||
from typing import Literal
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated
|
||||
from app import audit, sysconfig
|
||||
from app.auth import admin_user, current_user
|
||||
from app.config import settings
|
||||
@@ -29,45 +28,32 @@ from app.downloads import credentials as dlcreds
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import folders
|
||||
from app.downloads import links as linksmod
|
||||
from app.downloads import quota, service, storage
|
||||
from app.downloads import queries, quota, service, storage
|
||||
from app.downloads import urlguard
|
||||
from app.downloads.folders import FolderKind
|
||||
from app.downloads.status import FINISHED, NOT_REFETCHABLE, JobStatus
|
||||
from app.models import (
|
||||
DownloadJob,
|
||||
DownloadLink,
|
||||
FileFolder,
|
||||
DownloadProfile,
|
||||
DownloadQuota,
|
||||
DownloadShare,
|
||||
LibraryState,
|
||||
MediaAsset,
|
||||
User,
|
||||
display_text,
|
||||
)
|
||||
from app.downloads.status import NOT_REFETCHABLE, JobStatus
|
||||
from app.models import User, display_text
|
||||
from app.routes._common import (
|
||||
ProgressIn,
|
||||
get_or_404,
|
||||
kept_position,
|
||||
not_found,
|
||||
owned_or_404,
|
||||
)
|
||||
from app.routes.files import (
|
||||
MAX_LIST,
|
||||
accessible_job,
|
||||
assets_for,
|
||||
own_folder,
|
||||
own_job,
|
||||
own_job_for_update,
|
||||
positions_for,
|
||||
quota_detail,
|
||||
serialize,
|
||||
serialize_job,
|
||||
source_keys,
|
||||
)
|
||||
from app.security import hash_password
|
||||
from app.userscope import is_messageable_user, messageable_clauses
|
||||
from app.utils import iso
|
||||
from app.userscope import emails_of, is_messageable_user, messageable_others, user_by_email
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.downloads.models import DownloadJob, DownloadLink, DownloadProfile, FileFolder
|
||||
|
||||
router = APIRouter(prefix="/api/downloads", tags=["downloads"])
|
||||
admin_router = APIRouter(prefix="/api/admin/downloads", tags=["admin-downloads"])
|
||||
|
||||
@@ -119,7 +105,7 @@ def resolve_source(raw: str) -> tuple[str, str]:
|
||||
# --- profiles ------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _serialize_profile(p: DownloadProfile) -> dict:
|
||||
def _serialize_profile(p: "DownloadProfile") -> dict:
|
||||
return {
|
||||
"id": p.id,
|
||||
"name": p.name,
|
||||
@@ -131,18 +117,7 @@ def _serialize_profile(p: DownloadProfile) -> dict:
|
||||
|
||||
@router.get("/profiles")
|
||||
def list_profiles(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
rows = (
|
||||
db.execute(
|
||||
select(DownloadProfile)
|
||||
.where((DownloadProfile.is_builtin.is_(True)) | (DownloadProfile.user_id == user.id))
|
||||
.order_by(
|
||||
DownloadProfile.is_builtin.desc(), DownloadProfile.sort_order, DownloadProfile.id
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return [_serialize_profile(p) for p in rows]
|
||||
return [_serialize_profile(p) for p in queries.profiles_for(db, user)]
|
||||
|
||||
|
||||
class CreateProfileIn(BaseModel):
|
||||
@@ -158,15 +133,7 @@ def create_profile(
|
||||
spec = body.spec
|
||||
if not name:
|
||||
raise refused(400, "profile_name_required")
|
||||
maxpos = db.execute(
|
||||
select(func.coalesce(func.max(DownloadProfile.sort_order), 100)).where(
|
||||
DownloadProfile.user_id == user.id
|
||||
)
|
||||
).scalar_one()
|
||||
p = DownloadProfile(
|
||||
user_id=user.id, name=name[:80], spec=spec, is_builtin=False, sort_order=maxpos + 1
|
||||
)
|
||||
db.add(p)
|
||||
p = service.create_profile(db, user, name, spec)
|
||||
db.commit()
|
||||
return _serialize_profile(p)
|
||||
|
||||
@@ -183,31 +150,32 @@ def update_profile(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
p = db.get(DownloadProfile, profile_id)
|
||||
if p is None or p.is_builtin or p.user_id != user.id:
|
||||
raise not_found("profile")
|
||||
p = _own_profile(db, user, profile_id)
|
||||
sent = body.model_fields_set
|
||||
name = None
|
||||
if "name" in sent:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise refused(400, "profile_name_required")
|
||||
p.name = name[:80]
|
||||
if "spec" in sent:
|
||||
if body.spec is None:
|
||||
raise refused(400, "spec_not_object")
|
||||
p.spec = body.spec
|
||||
if "spec" in sent and body.spec is None:
|
||||
raise refused(400, "spec_not_object")
|
||||
service.update_profile(p, name=name, spec=body.spec if "spec" in sent else None)
|
||||
db.commit()
|
||||
return _serialize_profile(p)
|
||||
|
||||
|
||||
def _own_profile(db: Session, user: User, profile_id: int) -> "DownloadProfile":
|
||||
p = queries.own_profile(db, user, profile_id)
|
||||
if p is None:
|
||||
raise not_found("profile")
|
||||
return p
|
||||
|
||||
|
||||
@router.delete("/profiles/{profile_id}")
|
||||
def delete_profile(
|
||||
profile_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
p = db.get(DownloadProfile, profile_id)
|
||||
if p is None or p.is_builtin or p.user_id != user.id:
|
||||
raise not_found("profile")
|
||||
db.delete(p)
|
||||
service.delete_profile(db, _own_profile(db, user, profile_id))
|
||||
db.commit()
|
||||
return {"deleted": profile_id}
|
||||
|
||||
@@ -277,18 +245,13 @@ def _resolve_spec(
|
||||
"""Pick the format spec for an enqueue: an explicit profile_id (builtin or the user's own),
|
||||
an inline spec, or the first builtin as a fallback."""
|
||||
if profile_id is not None:
|
||||
p = db.get(DownloadProfile, profile_id)
|
||||
if p is None or (not p.is_builtin and p.user_id != user.id):
|
||||
p = queries.usable_profile(db, user, profile_id)
|
||||
if p is None:
|
||||
raise not_found("profile")
|
||||
return p.spec, p.id
|
||||
if spec is not None:
|
||||
return spec, None
|
||||
fallback = db.execute(
|
||||
select(DownloadProfile)
|
||||
.where(DownloadProfile.is_builtin.is_(True))
|
||||
.order_by(DownloadProfile.sort_order)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
fallback = queries.default_profile(db)
|
||||
if fallback is None:
|
||||
raise refused(400, "no_profile")
|
||||
return fallback.spec, fallback.id
|
||||
@@ -326,6 +289,7 @@ def enqueue_download(
|
||||
job = service.enqueue(db, user.id, source_kind, source_ref, spec, profile_id, display_name)
|
||||
except quota.QuotaExceeded as e:
|
||||
raise HTTPException(status_code=422, detail=quota_detail(e)) from e
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
@@ -355,6 +319,7 @@ def enqueue_edit(
|
||||
raise HTTPException(status_code=422, detail=quota_detail(e)) from e
|
||||
except service.EditError as e:
|
||||
raise HTTPException(status_code=400, detail={"code": "edit", "reason": e.reason}) from e
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
@@ -398,38 +363,22 @@ def enqueue_clip(
|
||||
raise HTTPException(status_code=status, detail=detail) from e
|
||||
except quota.QuotaExceeded as e:
|
||||
raise HTTPException(status_code=422, detail=quota_detail(e)) from e
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
# How long a finished job stays in the queue: the tray polls it from every page and must see a job
|
||||
# turn done (its "Done · Open" row and the done notice) — the finished file itself is the library's.
|
||||
# A done job is timed from when it became done, never from its last write: a rename, a move or the
|
||||
# GC's expiry warning touches an old file's row, and the tray would announce it as just finished.
|
||||
QUEUE_RECENT = timedelta(minutes=10)
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_downloads(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
"""The queue: what is waiting, running, paused or failed, and what finished a moment ago."""
|
||||
recent = datetime.now(timezone.utc) - QUEUE_RECENT
|
||||
jobs = (
|
||||
db.execute(
|
||||
select(DownloadJob)
|
||||
.where(
|
||||
DownloadJob.user_id == user.id,
|
||||
DownloadJob.status.not_in(FINISHED)
|
||||
| ((DownloadJob.status == JobStatus.DONE) & (DownloadJob.ready_at >= recent))
|
||||
| ((DownloadJob.status == JobStatus.CANCELED) & (DownloadJob.updated_at >= recent)),
|
||||
)
|
||||
.order_by(DownloadJob.created_at.desc(), DownloadJob.id.desc())
|
||||
.limit(MAX_LIST)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
assets = assets_for(db, jobs)
|
||||
positions = positions_for(db, user, jobs)
|
||||
sources = source_keys(db, jobs)
|
||||
jobs = queries.queue(db, user, datetime.now(timezone.utc) - QUEUE_RECENT)
|
||||
assets = queries.assets_for(db, jobs)
|
||||
positions = queries.positions_for(db, user, jobs)
|
||||
sources = queries.source_keys(db, jobs)
|
||||
return [
|
||||
{
|
||||
**serialize(j, assets.get(j.asset_id) if j.asset_id is not None else None),
|
||||
@@ -453,14 +402,8 @@ _INDEX_RANK: dict[str, int] = {
|
||||
def my_download_index(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""source_ref -> best active status, for the feed's per-card "downloaded / queued" badge.
|
||||
Small + cheap so the feed can poll it without loading the full job list."""
|
||||
rows = db.execute(
|
||||
select(DownloadJob.source_ref, DownloadJob.status).where(
|
||||
DownloadJob.user_id == user.id,
|
||||
DownloadJob.status.in_(tuple(_INDEX_RANK)),
|
||||
)
|
||||
).all()
|
||||
out: dict[str, str] = {}
|
||||
for ref, status in rows:
|
||||
for ref, status in queries.statuses_by_source(db, user, tuple(_INDEX_RANK)):
|
||||
if _INDEX_RANK.get(status, 0) > _INDEX_RANK.get(out.get(ref, ""), 0):
|
||||
out[ref] = status
|
||||
return out
|
||||
@@ -472,6 +415,7 @@ def pause_download(
|
||||
) -> dict:
|
||||
job = own_job_for_update(db, user, job_id)
|
||||
service.pause(db, job)
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
@@ -481,6 +425,7 @@ def resume_download(
|
||||
) -> dict:
|
||||
job = own_job_for_update(db, user, job_id)
|
||||
service.resume(db, job)
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
@@ -508,6 +453,7 @@ def redownload_download(
|
||||
if job.status in (JobStatus.QUEUED, JobStatus.RUNNING, JobStatus.PAUSED):
|
||||
raise refused(409, "still_downloading")
|
||||
service.redownload(db, job)
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
@@ -517,6 +463,7 @@ def cancel_download(
|
||||
) -> dict:
|
||||
job = own_job_for_update(db, user, job_id)
|
||||
service.cancel(db, job)
|
||||
db.commit()
|
||||
return serialize_job(db, job)
|
||||
|
||||
|
||||
@@ -534,24 +481,7 @@ def set_library_progress(
|
||||
one shared with them). A position too early or too close to the end clears it instead."""
|
||||
accessible_job(db, user, job_id)
|
||||
position = kept_position(body)
|
||||
mine = (LibraryState.user_id == user.id) & (LibraryState.job_id == job_id)
|
||||
if not position:
|
||||
# A statement, not a loaded row's delete: the periodic checkpoint and the pagehide beacon of
|
||||
# one stop may both clear it, and the second then deletes nothing rather than failing.
|
||||
db.execute(delete(LibraryState).where(mine))
|
||||
db.commit()
|
||||
return {"ok": True, "job_id": job_id, "position_seconds": 0}
|
||||
now = datetime.now(timezone.utc)
|
||||
# One row per (user, job) even when two checkpoints race (the same pair): an upsert, not a
|
||||
# read-then-insert that the second would fail on the unique key.
|
||||
db.execute(
|
||||
pg_insert(LibraryState)
|
||||
.values(user_id=user.id, job_id=job_id, position_seconds=position, progress_updated_at=now)
|
||||
.on_conflict_do_update(
|
||||
constraint="uq_library_user_job",
|
||||
set_={"position_seconds": position, "progress_updated_at": now},
|
||||
)
|
||||
)
|
||||
service.set_position(db, user, job_id, position)
|
||||
db.commit()
|
||||
return {"ok": True, "job_id": job_id, "position_seconds": position}
|
||||
|
||||
@@ -564,8 +494,7 @@ def storyboard_meta(
|
||||
|
||||
Best-effort: for a very long/high-res source the sprite may not generate in time — then
|
||||
`available` is False and the editor shows a plain timeline."""
|
||||
job = accessible_job(db, user, job_id)
|
||||
asset = db.get(MediaAsset, job.asset_id) if job.asset_id else None
|
||||
asset = queries.asset_of(db, accessible_job(db, user, job_id))
|
||||
if asset is None or asset.status != "ready" or not asset.rel_path:
|
||||
raise refused(409, "download_not_ready")
|
||||
meta = editmod.ensure_storyboard(asset, settings.files_root)
|
||||
@@ -585,8 +514,7 @@ def storyboard_meta(
|
||||
@router.get("/{job_id}/poster.jpg")
|
||||
def poster_image(job_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)):
|
||||
"""The locally-generated poster frame for a download whose source had no thumbnail."""
|
||||
job = accessible_job(db, user, job_id)
|
||||
asset = db.get(MediaAsset, job.asset_id) if job.asset_id else None
|
||||
asset = queries.asset_of(db, accessible_job(db, user, job_id))
|
||||
if asset is None or not asset.poster_path:
|
||||
raise not_found("poster")
|
||||
path = storage.safe_abs_path(settings.files_root, asset.poster_path)
|
||||
@@ -599,8 +527,7 @@ def poster_image(job_id: int, user: User = Depends(current_user), db: Session =
|
||||
def storyboard_image(
|
||||
job_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
):
|
||||
job = accessible_job(db, user, job_id)
|
||||
asset = db.get(MediaAsset, job.asset_id) if job.asset_id else None
|
||||
asset = queries.asset_of(db, accessible_job(db, user, job_id))
|
||||
if asset is None or not asset.storyboard_path:
|
||||
raise not_found("filmstrip")
|
||||
path = storage.safe_abs_path(settings.files_root, asset.storyboard_path)
|
||||
@@ -623,45 +550,20 @@ def share_download(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
job = owned_or_404(db, DownloadJob, job_id, user, what="download")
|
||||
job = own_job(db, user, job_id)
|
||||
if job.status != JobStatus.DONE or job.asset_id is None:
|
||||
raise refused(409, "share_unfinished")
|
||||
target = body.email.strip().lower()
|
||||
if not target:
|
||||
raise refused(400, "recipient_required")
|
||||
recipient = db.execute(
|
||||
select(User).where(func.lower(User.email) == target)
|
||||
).scalar_one_or_none()
|
||||
recipient = user_by_email(db, target)
|
||||
# Uniform "no user" for a missing OR non-messageable (suspended/deactivated) target, so
|
||||
# sharing can't be used to probe which addresses exist or to reach a suspended account.
|
||||
if not is_messageable_user(recipient):
|
||||
if recipient is None or not is_messageable_user(recipient):
|
||||
raise not_found("recipient")
|
||||
if recipient.id == user.id:
|
||||
raise refused(400, "share_own")
|
||||
exists = db.execute(
|
||||
select(DownloadShare.id).where(
|
||||
DownloadShare.job_id == job_id, DownloadShare.to_user_id == recipient.id
|
||||
)
|
||||
).first()
|
||||
if exists is None:
|
||||
db.add(DownloadShare(job_id=job_id, from_user_id=user.id, to_user_id=recipient.id))
|
||||
from app.notifications import create_notification
|
||||
|
||||
create_notification(
|
||||
db,
|
||||
user_id=recipient.id,
|
||||
type="download_shared",
|
||||
title=(job.display_name or "A download") + " was shared with you",
|
||||
# The client names the notice in the reader's language from `data`; the title is the
|
||||
# English fallback only.
|
||||
data={
|
||||
"kind": "download_shared",
|
||||
"from": user.email,
|
||||
"job_id": job_id,
|
||||
"title": job.display_name,
|
||||
},
|
||||
commit=False,
|
||||
)
|
||||
if service.share(db, job, user, recipient):
|
||||
db.commit()
|
||||
return {"ok": True, "shared_with": recipient.email}
|
||||
|
||||
@@ -673,22 +575,10 @@ def unshare_download(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
owned_or_404(
|
||||
db, DownloadJob, job_id, user, what="download"
|
||||
) # ownership guard (404s if not the caller's job)
|
||||
recipient = db.execute(
|
||||
select(User).where(func.lower(User.email) == email.strip().lower())
|
||||
).scalar_one_or_none()
|
||||
if recipient is not None:
|
||||
row = db.execute(
|
||||
select(DownloadShare).where(
|
||||
DownloadShare.job_id == job_id,
|
||||
DownloadShare.to_user_id == recipient.id,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.commit()
|
||||
own_job(db, user, job_id) # ownership guard (404s if not the caller's job)
|
||||
recipient = user_by_email(db, email)
|
||||
if recipient is not None and service.unshare(db, job_id, recipient):
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -701,54 +591,26 @@ def share_recipients(
|
||||
) -> list[dict]:
|
||||
"""Registered users this user can share a download with (excludes self + any non-messageable
|
||||
account — suspended or deactivated — so the picker never leaks those addresses)."""
|
||||
rows = (
|
||||
db.execute(
|
||||
select(User)
|
||||
.where(User.id != user.id, *messageable_clauses())
|
||||
.order_by(collated(User.email, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = messageable_others(db, user, lang_for_user(user))
|
||||
return [{"id": u.id, "email": u.email, "display_name": u.display_name} for u in rows]
|
||||
|
||||
|
||||
# --- public watch links (share by link) ----------------------------------------------------
|
||||
|
||||
|
||||
def _own_link(db: Session, user: User, link_id: int) -> DownloadLink:
|
||||
link = db.get(DownloadLink, link_id)
|
||||
def _own_link(db: Session, user: User, link_id: int) -> "DownloadLink":
|
||||
link = queries.own_link(db, user, link_id)
|
||||
if link is None:
|
||||
raise not_found("link")
|
||||
target = (
|
||||
db.get(DownloadJob, link.job_id)
|
||||
if link.job_id is not None
|
||||
else db.get(FileFolder, link.folder_id)
|
||||
)
|
||||
if target is None or target.user_id != user.id:
|
||||
raise not_found("link")
|
||||
return link
|
||||
|
||||
|
||||
def _link_expiry(days: int | None) -> datetime | None:
|
||||
# None/0/negative → no expiry; otherwise N days out, capped at ~10 years. The Pydantic model
|
||||
# already coerced/validated the type, so a non-number is a 422 before we get here.
|
||||
if not days or days <= 0:
|
||||
return None
|
||||
return datetime.now(timezone.utc) + timedelta(days=min(days, 3650))
|
||||
|
||||
|
||||
def _links_of(db: Session, *where) -> list[dict]:
|
||||
rows = db.execute(select(DownloadLink).where(*where).order_by(DownloadLink.id.desc()))
|
||||
return [linksmod.owner_view(row) for row in rows.scalars()]
|
||||
|
||||
|
||||
@router.get("/{job_id}/links")
|
||||
def list_links(
|
||||
job_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> list[dict]:
|
||||
owned_or_404(db, DownloadJob, job_id, user, what="download")
|
||||
return _links_of(db, DownloadLink.job_id == job_id)
|
||||
own_job(db, user, job_id)
|
||||
return [linksmod.owner_view(link) for link in queries.links_to(db, job_id=job_id)]
|
||||
|
||||
|
||||
class CreateLinkIn(BaseModel):
|
||||
@@ -764,30 +626,18 @@ class CreateLinkIn(BaseModel):
|
||||
|
||||
def _new_link(
|
||||
db: Session, user: User, body: CreateLinkIn, **target: int
|
||||
) -> tuple[DownloadLink, str]:
|
||||
"""Make a link to `target` (`job_id=` or `folder_id=`, already checked to be the user's), and
|
||||
the token it is opened by — which is in hand only now."""
|
||||
pw = body.password.strip()
|
||||
token = linksmod.new_token()
|
||||
link = DownloadLink(
|
||||
**target,
|
||||
created_by=user.id,
|
||||
allow_download=body.allow_download,
|
||||
password_hash=hash_password(pw) if pw else None,
|
||||
expires_at=_link_expiry(body.expires_days),
|
||||
)
|
||||
linksmod.seal(link, token)
|
||||
db.add(link)
|
||||
db.flush()
|
||||
audit.record(
|
||||
) -> tuple["DownloadLink", str]:
|
||||
"""A link to `target` (`job_id=` or `folder_id=`, already checked to be the user's) and its
|
||||
token (`links.create`). The Pydantic model already coerced the expiry, so a non-number is a 422
|
||||
before this runs."""
|
||||
return linksmod.create(
|
||||
db,
|
||||
user.id,
|
||||
audit.AuditAction.DOWNLOAD_LINK_CREATE,
|
||||
target_type="download_link",
|
||||
target_id=link.id,
|
||||
after={**target, **_link_audit(link)},
|
||||
user,
|
||||
password=body.password.strip(),
|
||||
allow_download=body.allow_download,
|
||||
expires_days=body.expires_days,
|
||||
**target,
|
||||
)
|
||||
return link, token
|
||||
|
||||
|
||||
@router.post("/{job_id}/links")
|
||||
@@ -805,28 +655,20 @@ def create_link(
|
||||
return linksmod.owner_view(link, token=token)
|
||||
|
||||
|
||||
def _shareable_job(db: Session, user: User, job_id: int) -> DownloadJob:
|
||||
"""One of the user's files, held (FOR SHARE) for the rest of the transaction: a delete of it at
|
||||
the same moment (the user's, or the retention sweep's) waits and takes the new link with it,
|
||||
rather than failing the link's foreign key — or, done first, leaves a 404."""
|
||||
owned_or_404(db, DownloadJob, job_id, user, what="download")
|
||||
job = db.execute(
|
||||
select(DownloadJob)
|
||||
.where(DownloadJob.id == job_id)
|
||||
.with_for_update(read=True)
|
||||
.execution_options(populate_existing=True)
|
||||
).scalar_one_or_none()
|
||||
def _shareable_job(db: Session, user: User, job_id: int) -> "DownloadJob":
|
||||
"""One of the user's files, held for the link's transaction (`service.hold_for_share`)."""
|
||||
job = service.hold_for_share(db, user, job_id)
|
||||
if job is None:
|
||||
raise not_found("download")
|
||||
return job
|
||||
|
||||
|
||||
def _shareable_folder(db: Session, user: User, folder_id: int) -> FileFolder:
|
||||
def _shareable_folder(db: Session, user: User, folder_id: int) -> "FileFolder":
|
||||
"""One of the user's own folders. A source folder lets its files go on its own clock, and the Hub
|
||||
is the user's channel to a machine of theirs: neither is a shelf to hand out. Held (FOR SHARE)
|
||||
for the rest of the transaction, so a delete of it at the same moment waits and takes the new
|
||||
link with it, rather than failing the link's foreign key."""
|
||||
owned_or_404(db, FileFolder, folder_id, user, what="folder")
|
||||
own_folder(db, user, folder_id)
|
||||
try:
|
||||
folder = folders.own_folder(db, user.id, folder_id)
|
||||
except folders.FolderError as e:
|
||||
@@ -840,8 +682,8 @@ def _shareable_folder(db: Session, user: User, folder_id: int) -> FileFolder:
|
||||
def list_folder_links(
|
||||
folder_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> list[dict]:
|
||||
owned_or_404(db, FileFolder, folder_id, user, what="folder")
|
||||
return _links_of(db, DownloadLink.folder_id == folder_id)
|
||||
own_folder(db, user, folder_id)
|
||||
return [linksmod.owner_view(link) for link in queries.links_to(db, folder_id=folder_id)]
|
||||
|
||||
|
||||
@router.post("/folders/{folder_id}/links")
|
||||
@@ -857,15 +699,6 @@ def create_folder_link(
|
||||
return linksmod.owner_view(link, token=token)
|
||||
|
||||
|
||||
def _link_audit(link: DownloadLink) -> dict:
|
||||
"""A link's settings as the audit records them: whether it has a password, never which."""
|
||||
return {
|
||||
"allow_download": link.allow_download,
|
||||
"has_password": link.password_hash is not None,
|
||||
"expires_at": iso(link.expires_at),
|
||||
}
|
||||
|
||||
|
||||
class UpdateLinkIn(BaseModel):
|
||||
allow_download: bool | None = None
|
||||
expires_days: int | None = None
|
||||
@@ -880,29 +713,15 @@ def update_link(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
link = _own_link(db, user, link_id)
|
||||
before = _link_audit(link)
|
||||
sent = body.model_fields_set
|
||||
if "allow_download" in sent:
|
||||
link.allow_download = bool(body.allow_download)
|
||||
if "expires_days" in sent:
|
||||
link.expires_at = _link_expiry(body.expires_days)
|
||||
if "password" in sent:
|
||||
pw = (body.password or "").strip()
|
||||
link.password_hash = hash_password(pw) if pw else None
|
||||
# Bump the version so any grant issued under the previous password stops working now, rather
|
||||
# than living out its 6h TTL (setting/clearing/changing the password all count as a rotation).
|
||||
link.password_version += 1
|
||||
after = _link_audit(link)
|
||||
if after != before or "password" in sent:
|
||||
audit.record(
|
||||
db,
|
||||
user.id,
|
||||
audit.AuditAction.DOWNLOAD_LINK_UPDATE,
|
||||
target_type="download_link",
|
||||
target_id=link_id,
|
||||
before=before,
|
||||
after=after,
|
||||
)
|
||||
linksmod.update(
|
||||
db,
|
||||
user,
|
||||
link,
|
||||
allow_download=body.allow_download,
|
||||
expires_days=body.expires_days,
|
||||
password=body.password,
|
||||
sent=frozenset(body.model_fields_set),
|
||||
)
|
||||
db.commit()
|
||||
return linksmod.owner_view(link)
|
||||
|
||||
@@ -911,16 +730,7 @@ def update_link(
|
||||
def revoke_link(
|
||||
link_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
link = _own_link(db, user, link_id)
|
||||
db.delete(link)
|
||||
audit.record(
|
||||
db,
|
||||
user.id,
|
||||
audit.AuditAction.DOWNLOAD_LINK_REVOKE,
|
||||
target_type="download_link",
|
||||
target_id=link_id,
|
||||
after={"job_id": link.job_id} if link.job_id is not None else {"folder_id": link.folder_id},
|
||||
)
|
||||
linksmod.revoke(db, user, _own_link(db, user, link_id))
|
||||
db.commit()
|
||||
return {"ok": True, "deleted": link_id}
|
||||
|
||||
@@ -932,20 +742,9 @@ def revoke_link(
|
||||
def admin_list_downloads(
|
||||
admin: User = Depends(admin_user), db: Session = Depends(get_db)
|
||||
) -> list[dict]:
|
||||
jobs = (
|
||||
db.execute(
|
||||
select(DownloadJob)
|
||||
.order_by(DownloadJob.created_at.desc(), DownloadJob.id.desc())
|
||||
.limit(MAX_LIST)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
assets = assets_for(db, jobs)
|
||||
emails = {
|
||||
u.id: u.email
|
||||
for u in db.execute(select(User).where(User.id.in_({j.user_id for j in jobs}))).scalars()
|
||||
}
|
||||
jobs = queries.recent_jobs(db)
|
||||
assets = queries.assets_for(db, jobs)
|
||||
emails = emails_of(db, {j.user_id for j in jobs})
|
||||
out = []
|
||||
for j in jobs:
|
||||
data = serialize(j, assets.get(j.asset_id) if j.asset_id is not None else None)
|
||||
@@ -957,27 +756,11 @@ def admin_list_downloads(
|
||||
|
||||
@admin_router.get("/storage")
|
||||
def admin_storage(admin: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
|
||||
# Every file on disk (`quota.ON_DISK`): ready ones, and uploads waiting for or failed in their
|
||||
# conversion — the same disk the footprint and the GC cap count.
|
||||
on_disk = db.execute(
|
||||
select(func.count(), func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(quota.ON_DISK)
|
||||
).one()
|
||||
total_assets = db.execute(select(func.count()).select_from(MediaAsset)).scalar()
|
||||
files_on_disk, total_bytes, total_assets = queries.storage_totals(db)
|
||||
# EVERY user, not only those holding a file: a quota is set before anyone downloads.
|
||||
emails: dict[int, str] = {
|
||||
uid: email
|
||||
for uid, email in db.execute(select(User.id, User.email).order_by(User.email)).tuples()
|
||||
}
|
||||
emails = emails_of(db)
|
||||
uids = list(emails)
|
||||
# What each one's Files lists: their finished files.
|
||||
files: dict[int, int] = {
|
||||
uid: n
|
||||
for uid, n in db.execute(
|
||||
select(DownloadJob.user_id, func.count())
|
||||
.where(DownloadJob.status == JobStatus.DONE)
|
||||
.group_by(DownloadJob.user_id)
|
||||
).tuples()
|
||||
}
|
||||
files = queries.done_files_per_user(db)
|
||||
|
||||
def row(uid: int) -> dict:
|
||||
lim = quota.resolve(db, uid)
|
||||
@@ -991,8 +774,8 @@ def admin_storage(admin: User = Depends(admin_user), db: Session = Depends(get_d
|
||||
}
|
||||
|
||||
return {
|
||||
"files_on_disk": on_disk[0],
|
||||
"total_bytes": int(on_disk[1]),
|
||||
"files_on_disk": files_on_disk,
|
||||
"total_bytes": total_bytes,
|
||||
"total_assets": total_assets,
|
||||
"total_cap_bytes": sysconfig.get_int(db, "download_total_max_bytes"),
|
||||
"per_user": [row(uid) for uid in uids],
|
||||
@@ -1004,10 +787,9 @@ def admin_get_quota(
|
||||
user_id: int, admin: User = Depends(admin_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
lim = quota.resolve(db, user_id)
|
||||
row = db.get(DownloadQuota, user_id)
|
||||
return {
|
||||
"user_id": user_id,
|
||||
"custom": row is not None,
|
||||
"custom": queries.custom_quota(db, user_id) is not None,
|
||||
"max_bytes": lim.max_bytes,
|
||||
"max_concurrent": lim.max_concurrent,
|
||||
"max_jobs": lim.max_jobs,
|
||||
@@ -1032,25 +814,14 @@ def admin_set_quota(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
get_or_404(db, User, user_id, what="user")
|
||||
cur = quota.resolve(db, user_id)
|
||||
row = db.get(DownloadQuota, user_id)
|
||||
if row is None:
|
||||
row = DownloadQuota(
|
||||
user_id=user_id,
|
||||
max_bytes=cur.max_bytes,
|
||||
max_concurrent=cur.max_concurrent,
|
||||
max_jobs=cur.max_jobs,
|
||||
unlimited=cur.unlimited,
|
||||
)
|
||||
db.add(row)
|
||||
if body.max_bytes is not None:
|
||||
row.max_bytes = max(0, body.max_bytes)
|
||||
if body.max_concurrent is not None:
|
||||
row.max_concurrent = max(1, body.max_concurrent)
|
||||
if body.max_jobs is not None:
|
||||
row.max_jobs = max(1, body.max_jobs)
|
||||
if body.unlimited is not None:
|
||||
row.unlimited = body.unlimited
|
||||
quota.set_custom(
|
||||
db,
|
||||
user_id,
|
||||
max_bytes=body.max_bytes,
|
||||
max_concurrent=body.max_concurrent,
|
||||
max_jobs=body.max_jobs,
|
||||
unlimited=body.unlimited,
|
||||
)
|
||||
db.commit()
|
||||
return admin_get_quota(user_id, admin, db)
|
||||
|
||||
@@ -1059,8 +830,6 @@ def admin_set_quota(
|
||||
def admin_reset_quota(
|
||||
user_id: int, admin: User = Depends(admin_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
row = db.get(DownloadQuota, user_id)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
if quota.reset_custom(db, user_id):
|
||||
db.commit()
|
||||
return admin_get_quota(user_id, admin, db)
|
||||
|
||||
+51
-183
@@ -1,9 +1,7 @@
|
||||
from datetime import date, datetime, timezone
|
||||
from datetime import date
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm.exc import StaleDataError
|
||||
|
||||
@@ -11,34 +9,10 @@ from app import quota
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.mailtext import lang_for_user
|
||||
from app.feedquery import (
|
||||
decode_cursor,
|
||||
encode_cursor,
|
||||
feed_summary,
|
||||
feed_years,
|
||||
filtered_query,
|
||||
keyset_predicate,
|
||||
order_by,
|
||||
rank_window_start,
|
||||
serialize,
|
||||
sort_keys,
|
||||
windowed_rank,
|
||||
windowed_relevance_rows,
|
||||
)
|
||||
from app.models import (
|
||||
ChannelTag,
|
||||
Tag,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.routes._common import (
|
||||
ProgressIn,
|
||||
get_or_404,
|
||||
kept_position,
|
||||
not_found,
|
||||
youtube_refusal,
|
||||
)
|
||||
from app.feedquery import BadCursor, facet_counts, feed_count, feed_page, feed_summary, feed_years
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.routes._common import ProgressIn, kept_position, not_found, youtube_refusal
|
||||
from app.sync.videos import parse_iso8601_duration
|
||||
from app.utils import iso
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
@@ -108,49 +82,20 @@ def get_feed(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
query, rank_expr = filtered_query(db, user, **params)
|
||||
lang = lang_for_user(user)
|
||||
keys = sort_keys(sort, seed, rank_expr, lang)
|
||||
|
||||
vals, cursor_id, window = None, None, None
|
||||
if cursor:
|
||||
# The read-model raises a plain ValueError on a malformed cursor; translate it to the
|
||||
# HTTP 400 here so feedquery stays web-framework-free.
|
||||
try:
|
||||
vals, cursor_id, window = decode_cursor(keys, cursor)
|
||||
except ValueError:
|
||||
# `from None`: the ValueError is an expected bad-input signal, not an error to chain.
|
||||
raise refused(400, "invalid_cursor") from None
|
||||
ranked = sort == "relevance" and rank_expr is not None
|
||||
if ranked and not cursor:
|
||||
window = rank_window_start(db, query)
|
||||
if ranked and window is not None:
|
||||
keys = sort_keys(sort, seed, windowed_rank(rank_expr, window), lang)
|
||||
|
||||
# Expose each key column on the row so we can build the next cursor from the last item.
|
||||
query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
|
||||
if vals is not None:
|
||||
for k, v in zip(keys, vals, strict=True):
|
||||
k["value"] = v
|
||||
if ranked and window is not None:
|
||||
rows = windowed_relevance_rows(db, query, keys, window, cursor_id, limit + 1)
|
||||
else:
|
||||
query = query.order_by(*order_by(keys))
|
||||
if cursor_id is not None:
|
||||
query = query.where(keyset_predicate(keys, cursor_id))
|
||||
rows = db.execute(query.limit(limit + 1)).all()
|
||||
has_more = len(rows) > limit
|
||||
page = rows[:limit]
|
||||
return {
|
||||
"items": [serialize(r) for r in page],
|
||||
"next_cursor": (
|
||||
encode_cursor(keys, page[-1], window if ranked else None)
|
||||
if (has_more and page)
|
||||
else None
|
||||
),
|
||||
"has_more": has_more,
|
||||
"limit": limit,
|
||||
}
|
||||
try:
|
||||
return feed_page(
|
||||
db,
|
||||
user,
|
||||
params,
|
||||
sort=sort,
|
||||
seed=seed,
|
||||
limit=limit,
|
||||
cursor=cursor,
|
||||
lang=lang_for_user(user),
|
||||
)
|
||||
except BadCursor:
|
||||
# `from None`: a malformed cursor is an expected bad-input signal, not an error to chain.
|
||||
raise refused(400, "invalid_cursor") from None
|
||||
|
||||
|
||||
@router.get("/feed/count")
|
||||
@@ -162,9 +107,7 @@ def get_feed_count(
|
||||
# SUPERSEDED by /feed/summary (PERF-13) — the app no longer calls this. Kept as the reference
|
||||
# oracle that test_feed_summary_matches_count_plus_facets asserts /feed/summary's `count` against;
|
||||
# remove both together with that test if this endpoint is ever retired.
|
||||
query, _rank = filtered_query(db, user, **params)
|
||||
total = db.scalar(select(func.count()).select_from(query.subquery()))
|
||||
return {"count": total or 0}
|
||||
return {"count": feed_count(db, user, **params)}
|
||||
|
||||
|
||||
@router.get("/facets")
|
||||
@@ -186,31 +129,7 @@ def get_facets(
|
||||
SUPERSEDED by /feed/summary (PERF-13) — the app no longer calls this. Kept as the reference
|
||||
oracle for test_feed_summary_matches_count_plus_facets (it asserts /feed/summary's `facets`
|
||||
against these `counts`); retire the two together."""
|
||||
visible = or_(ChannelTag.user_id.is_(None), ChannelTag.user_id == user.id)
|
||||
counts: dict[int, int] = {}
|
||||
for category in ("language", "topic", "other"):
|
||||
# Disjunctive (OR) facets drop the category's own selections so its chips keep
|
||||
# independent counts (you can OR more of them in). Conjunctive (AND, topics only)
|
||||
# keeps them applied, so each remaining chip narrows to channels that ALSO have all
|
||||
# already-selected topics — and tags that can't co-occur drop to zero (hidden).
|
||||
conjunctive = params.get("tag_modes", {}).get(category) == "and"
|
||||
base, _rank = filtered_query(
|
||||
db,
|
||||
user,
|
||||
**{**params, "exclude_tag_category": None if conjunctive else category},
|
||||
)
|
||||
channels = base.with_only_columns(Video.channel_id).distinct().subquery()
|
||||
rows = db.execute(
|
||||
select(ChannelTag.tag_id, func.count(func.distinct(ChannelTag.channel_id)))
|
||||
.select_from(channels)
|
||||
.join(ChannelTag, ChannelTag.channel_id == channels.c.channel_id)
|
||||
.join(Tag, and_(Tag.id == ChannelTag.tag_id, Tag.category == category))
|
||||
.where(visible)
|
||||
.group_by(ChannelTag.tag_id)
|
||||
).all()
|
||||
for tag_id, count in rows:
|
||||
counts[tag_id] = count
|
||||
return {"counts": counts}
|
||||
return {"counts": facet_counts(db, user, **params)}
|
||||
|
||||
|
||||
@router.get("/feed/summary")
|
||||
@@ -239,6 +158,11 @@ def get_feed_years(
|
||||
return {"years": feed_years(db, user, **params)}
|
||||
|
||||
|
||||
def _video_or_404(db: Session, video_id: str) -> None:
|
||||
if queries.video(db, video_id) is None:
|
||||
raise not_found("video")
|
||||
|
||||
|
||||
class VideoStateIn(BaseModel):
|
||||
status: str
|
||||
|
||||
@@ -253,50 +177,17 @@ def set_video_state(
|
||||
status = body.status
|
||||
if status not in VALID_STATES:
|
||||
raise refused(400, "invalid_status")
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
_video_or_404(db, video_id)
|
||||
if status == "new":
|
||||
# Un-marking: keep the row if it still holds a resume position (un-marking
|
||||
# "watched" should restore the in-progress state, not wipe where the user left
|
||||
# off); otherwise drop it. The in-app player fires this alongside a progress
|
||||
# checkpoint that may DELETE the same row, so tolerate it vanishing under us.
|
||||
row = db.execute(
|
||||
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
if row.position_seconds:
|
||||
row.status = "new"
|
||||
row.watched_at = None
|
||||
else:
|
||||
db.delete(row)
|
||||
try:
|
||||
db.commit()
|
||||
# silent-ok: the documented lost race — a concurrent player checkpoint already removed
|
||||
# or moved this row. The state the caller asked for is the state that holds either way.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
try:
|
||||
service.unmark(db, user, video_id)
|
||||
db.commit()
|
||||
# silent-ok: the documented lost race — a concurrent player checkpoint already removed or
|
||||
# moved this row (`service.unmark`). The state the caller asked for holds either way.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
return {"ok": True, "video_id": video_id, "status": "new"}
|
||||
|
||||
# watched / hidden: atomic upsert. The player marks "watched" at end-of-playback at
|
||||
# the same instant a progress checkpoint may DELETE the in-progress row; a plain
|
||||
# SELECT-then-UPDATE then raises StaleDataError ("0 rows matched") on that race. An
|
||||
# INSERT … ON CONFLICT DO UPDATE keyed on uq_user_video is immune to it.
|
||||
set_: dict = {"status": status, "updated_at": now}
|
||||
if status == "watched":
|
||||
set_["watched_at"] = now # hidden keeps any existing watched_at
|
||||
stmt = (
|
||||
pg_insert(VideoState)
|
||||
.values(
|
||||
user_id=user.id,
|
||||
video_id=video_id,
|
||||
status=status,
|
||||
watched_at=now if status == "watched" else None,
|
||||
)
|
||||
.on_conflict_do_update(constraint="uq_user_video", set_=set_)
|
||||
)
|
||||
db.execute(stmt)
|
||||
service.mark(db, user, video_id, status)
|
||||
db.commit()
|
||||
return {"ok": True, "video_id": video_id, "status": status}
|
||||
|
||||
@@ -310,16 +201,12 @@ def clear_video_state(
|
||||
"""Reset a video to pristine for this user — drop the whole VideoState row (status,
|
||||
watch position and watched_at), as if it had never been opened. Saved/playlist membership
|
||||
lives elsewhere and is untouched."""
|
||||
row = db.execute(
|
||||
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
try:
|
||||
db.commit()
|
||||
# silent-ok: the row went under the delete, which is the outcome the delete wanted.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
try:
|
||||
service.clear_state(db, user, video_id)
|
||||
db.commit()
|
||||
# silent-ok: the row went under the delete, which is the outcome the delete wanted.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
return {"ok": True, "video_id": video_id, "status": "new", "position_seconds": 0}
|
||||
|
||||
|
||||
@@ -335,36 +222,17 @@ def set_video_progress(
|
||||
partially-watched video can render a progress bar and match the "in progress" filter.
|
||||
Trivially-early and near-finished positions clear the position rather than store it."""
|
||||
position = kept_position(body)
|
||||
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
|
||||
row = db.execute(
|
||||
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
_video_or_404(db, video_id)
|
||||
if not position:
|
||||
# Nothing meaningful to store: drop a status-less row entirely, otherwise just
|
||||
# zero the position (a saved/hidden video keeps its status).
|
||||
if row is not None:
|
||||
if row.status == "new":
|
||||
db.delete(row)
|
||||
else:
|
||||
row.position_seconds = 0
|
||||
row.progress_updated_at = datetime.now(timezone.utc)
|
||||
# A concurrent state change (e.g. the end-of-playback "watched" mark) may
|
||||
# have removed/updated this row already — tolerate the lost race.
|
||||
try:
|
||||
db.commit()
|
||||
# silent-ok: exactly the race the comment above describes; the response stays true.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
try:
|
||||
service.set_progress(db, user, video_id, 0)
|
||||
db.commit()
|
||||
# silent-ok: a concurrent state change (the end-of-playback "watched" mark) removed or
|
||||
# updated this row already — the lost race; the response stays true.
|
||||
except StaleDataError:
|
||||
db.rollback()
|
||||
return {"ok": True, "video_id": video_id, "position_seconds": 0}
|
||||
|
||||
if row is None:
|
||||
row = VideoState(user_id=user.id, video_id=video_id)
|
||||
db.add(row)
|
||||
row.position_seconds = position
|
||||
row.progress_updated_at = datetime.now(timezone.utc)
|
||||
service.set_progress(db, user, video_id, position)
|
||||
db.commit()
|
||||
return {"ok": True, "video_id": video_id, "position_seconds": position}
|
||||
|
||||
@@ -382,7 +250,7 @@ def get_video_detail(
|
||||
our DB (e.g. a YouTube link inside another video's description that the in-app
|
||||
player navigated to) is resolved via the YouTube API (videos.list, 1 unit,
|
||||
attributed to the requesting user)."""
|
||||
v = db.get(Video, video_id)
|
||||
v = queries.video(db, video_id)
|
||||
if v is not None:
|
||||
return {
|
||||
"id": v.id,
|
||||
|
||||
+93
-243
@@ -5,8 +5,10 @@ the per-user layer (name, folder, kept, expiry, shares and links all hang off it
|
||||
bytes. Folders are rows (`downloads.folders`), so a rename or a move never touches the disk.
|
||||
|
||||
In-flight jobs are not here: they are the download tray's (`routes.downloads`). The helpers above
|
||||
the routes are shared with that module — serialization, the ownership locks, the upload spool —
|
||||
because a download and a library file are one row at two moments.
|
||||
the routes are shared with that module — serialization, the ownership checks, the upload spool —
|
||||
because a download and a library file are one row at two moments. What is read comes from
|
||||
`downloads.queries`, what is written goes through Capture's service (`service`, `folders`); this
|
||||
module validates, answers in HTTP and commits.
|
||||
|
||||
A machine with an API token (`app.apitokens`) may use the routes that take `files_caller`: list the
|
||||
folders and files, fetch a file and its hash, upload, delete — each only in a folder its token was
|
||||
@@ -19,51 +21,37 @@ import hashlib
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
from collections.abc import AsyncIterator, Sequence
|
||||
from collections.abc import AsyncIterator
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import BinaryIO, Literal
|
||||
from typing import TYPE_CHECKING, BinaryIO, Literal
|
||||
from urllib.parse import unquote, urlparse
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from fastapi.responses import FileResponse, Response
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, or_, select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, sysconfig
|
||||
from app.apitokens import FilesCaller, Right, files_caller
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
from app.collation import collated
|
||||
from app.config import settings
|
||||
from app.db import get_db, release_read_connection
|
||||
from app.downloads import folders, preview, quota, service, storage
|
||||
from app.downloads import folders, preview, queries, quota, service, storage
|
||||
from app.downloads import upload as uploadmod
|
||||
from app.downloads.status import JobStatus
|
||||
from app.mailtext import lang_for_user
|
||||
from app.models import (
|
||||
DISPLAY_TEXT_MAX,
|
||||
DownloadJob,
|
||||
DownloadLink,
|
||||
DownloadShare,
|
||||
FileFolder,
|
||||
LibraryState,
|
||||
MediaAsset,
|
||||
User,
|
||||
display_text,
|
||||
)
|
||||
from app.models import DISPLAY_TEXT_MAX, User, display_text
|
||||
from app.refusal import refused
|
||||
from app.routes._common import capped_stream_to, not_found, owned_or_404
|
||||
from app.utils import iso, like_escape
|
||||
from app.routes._common import capped_stream_to, not_found
|
||||
from app.utils import iso
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.downloads.models import DownloadJob, FileFolder, MediaAsset
|
||||
|
||||
router = APIRouter(prefix="/api/files", tags=["files"])
|
||||
|
||||
# Cap the listing so a heavy library can't return an unbounded result set (the old Download Center's
|
||||
# own cap; far beyond any real library).
|
||||
MAX_LIST = 500
|
||||
# The longest a folder may keep a file, in days — ten years is "forever" in all but name.
|
||||
_MAX_RETENTION_DAYS = 3650
|
||||
_HASH_CHUNK = 1 << 20
|
||||
@@ -75,7 +63,7 @@ _HASHING = threading.BoundedSemaphore(_HASHERS)
|
||||
# --- shared with routes.downloads -------------------------------------------------------
|
||||
|
||||
|
||||
def serialize(job: DownloadJob, asset: MediaAsset | None) -> dict:
|
||||
def serialize(job: "DownloadJob", asset: "MediaAsset | None") -> dict:
|
||||
ready = asset is not None and asset.status == "ready" and bool(asset.rel_path)
|
||||
return {
|
||||
"id": job.id,
|
||||
@@ -135,58 +123,18 @@ def serialize(job: DownloadJob, asset: MediaAsset | None) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def assets_for(db: Session, jobs: Sequence[DownloadJob]) -> dict[int, MediaAsset]:
|
||||
ids = {j.asset_id for j in jobs if j.asset_id}
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(select(MediaAsset).where(MediaAsset.id.in_(ids))).scalars()
|
||||
return {a.id: a for a in rows}
|
||||
|
||||
|
||||
def positions_for(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, int]:
|
||||
"""`user`'s resume point in each of `jobs` that has one (the library player's `library_states`)."""
|
||||
ids = [j.id for j in jobs]
|
||||
if not ids:
|
||||
return {}
|
||||
rows = db.execute(
|
||||
select(LibraryState.job_id, LibraryState.position_seconds).where(
|
||||
LibraryState.user_id == user.id, LibraryState.job_id.in_(ids)
|
||||
)
|
||||
).all()
|
||||
return {job_id: pos for job_id, pos in rows}
|
||||
|
||||
|
||||
def serialize_job(db: Session, job: DownloadJob) -> dict:
|
||||
def serialize_job(db: Session, job: "DownloadJob") -> dict:
|
||||
"""Resolve a job's cache asset (if any) and serialize the pair — the resolve-then-serialize step
|
||||
every single-job handler shares."""
|
||||
asset = db.get(MediaAsset, job.asset_id) if job.asset_id else None
|
||||
return serialize(job, asset)
|
||||
return serialize(job, queries.asset_of(db, job))
|
||||
|
||||
|
||||
def own_job_for_update(db: Session, user: User, job_id: int) -> DownloadJob:
|
||||
"""The ownership check (`owned_or_404`) for a transition that moves its asset's ref count (resume, cancel, delete, redownload)
|
||||
or writes the status those decide on (pause). The job is re-read under a row lock, so two
|
||||
overlapping requests for one job run one after the other and the second decides on the first's
|
||||
committed result — never both releasing the same hold, and never a pause turning a job a cancel
|
||||
just released back into a holder; either can free a file another job still holds.
|
||||
|
||||
The asset is locked BEFORE the job: the order a finishing worker child takes them in
|
||||
(`service.finalize_if_running`), so the two can never deadlock."""
|
||||
while True:
|
||||
asset_id = owned_or_404(db, DownloadJob, job_id, user, what="download").asset_id
|
||||
if asset_id is not None:
|
||||
db.execute(select(MediaAsset.id).where(MediaAsset.id == asset_id).with_for_update())
|
||||
job = db.execute(
|
||||
select(DownloadJob)
|
||||
.where(DownloadJob.id == job_id)
|
||||
.with_for_update()
|
||||
.execution_options(populate_existing=True)
|
||||
).scalar_one_or_none()
|
||||
if job is None:
|
||||
raise not_found("download")
|
||||
if job.asset_id == asset_id:
|
||||
return job
|
||||
db.rollback() # another request moved the job to another asset meanwhile: lock that one
|
||||
def own_job_for_update(db: Session, user: User, job_id: int) -> "DownloadJob":
|
||||
"""One of the user's jobs under its row lock (`service.lock_own_job`), or a 404."""
|
||||
job = service.lock_own_job(db, user, job_id)
|
||||
if job is None:
|
||||
raise not_found("download")
|
||||
return job
|
||||
|
||||
|
||||
def quota_detail(e: quota.QuotaExceeded) -> dict:
|
||||
@@ -322,30 +270,32 @@ async def spool_upload(
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def accessible_job(db: Session, user: User, job_id: int) -> DownloadJob:
|
||||
"""A file the caller may read: their own, or one shared with them."""
|
||||
job = db.get(DownloadJob, job_id)
|
||||
def accessible_job(db: Session, user: User, job_id: int) -> "DownloadJob":
|
||||
"""A file the caller may read (`queries.accessible_job`: their own, or shared with them)."""
|
||||
job = queries.accessible_job(db, user, job_id)
|
||||
if job is None:
|
||||
raise not_found("download")
|
||||
if job.user_id == user.id:
|
||||
return job
|
||||
shared = db.execute(
|
||||
select(DownloadShare.id).where(
|
||||
DownloadShare.job_id == job_id, DownloadShare.to_user_id == user.id
|
||||
)
|
||||
).first()
|
||||
if shared is None:
|
||||
raise not_found("download")
|
||||
return job
|
||||
|
||||
|
||||
def readable_job(db: Session, caller: FilesCaller, job_id: int) -> DownloadJob:
|
||||
"""A file `caller` may read: as `accessible_job` for the user; for a token, one of the user's
|
||||
own in a folder it may read."""
|
||||
if caller.grants is None:
|
||||
return accessible_job(db, caller.user, job_id)
|
||||
job = db.get(DownloadJob, job_id)
|
||||
if job is None or job.user_id != caller.user.id or not caller.may(job.folder_id, Right.READ):
|
||||
def own_job(db: Session, user: User, job_id: int) -> "DownloadJob":
|
||||
job = queries.own_job(db, user, job_id)
|
||||
if job is None:
|
||||
raise not_found("download")
|
||||
return job
|
||||
|
||||
|
||||
def own_folder(db: Session, user: User, folder_id: int) -> "FileFolder":
|
||||
folder = queries.own_folder(db, user, folder_id)
|
||||
if folder is None:
|
||||
raise not_found("folder")
|
||||
return folder
|
||||
|
||||
|
||||
def readable_job(db: Session, caller: FilesCaller, job_id: int) -> "DownloadJob":
|
||||
"""A file `caller` may read (`queries.readable_job`), or a 404."""
|
||||
job = queries.readable_job(db, caller, job_id)
|
||||
if job is None:
|
||||
raise not_found("download")
|
||||
return job
|
||||
|
||||
@@ -353,7 +303,7 @@ def readable_job(db: Session, caller: FilesCaller, job_id: int) -> DownloadJob:
|
||||
# --- folders -------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _serialize_folder(folder: FileFolder, count: int, size: int, linked: bool = False) -> dict:
|
||||
def _serialize_folder(folder: "FileFolder", count: int, size: int, linked: bool = False) -> dict:
|
||||
return {
|
||||
"id": folder.id,
|
||||
"kind": folder.kind,
|
||||
@@ -369,18 +319,6 @@ def _serialize_folder(folder: FileFolder, count: int, size: int, linked: bool =
|
||||
}
|
||||
|
||||
|
||||
def _linked(db: Session, folder_ids: list[int]) -> set[int]:
|
||||
"""Which of these folders a live public link opens (one not past its expiry)."""
|
||||
now = datetime.now(timezone.utc)
|
||||
rows = db.execute(
|
||||
select(DownloadLink.folder_id).where(
|
||||
DownloadLink.folder_id.in_(folder_ids),
|
||||
or_(DownloadLink.expires_at.is_(None), DownloadLink.expires_at > now),
|
||||
)
|
||||
).scalars()
|
||||
return {folder_id for folder_id in rows if folder_id is not None}
|
||||
|
||||
|
||||
@router.get("/folders")
|
||||
def list_folders(
|
||||
caller: FilesCaller = Depends(files_caller), db: Session = Depends(get_db)
|
||||
@@ -388,23 +326,9 @@ def list_folders(
|
||||
"""The caller's folders with how many files each holds and their size (a token: the ones it may
|
||||
list). A source folder its files have all left stays (count 0): the client decides whether to
|
||||
show it."""
|
||||
done = (DownloadJob.folder_id == FileFolder.id) & (DownloadJob.status == JobStatus.DONE)
|
||||
stmt = select(
|
||||
FileFolder,
|
||||
func.count(DownloadJob.id),
|
||||
func.coalesce(func.sum(MediaAsset.size_bytes), 0),
|
||||
).where(FileFolder.user_id == caller.user.id)
|
||||
listable = caller.folders_with(Right.LIST)
|
||||
if listable is not None:
|
||||
stmt = stmt.where(FileFolder.id.in_(listable))
|
||||
rows = db.execute(
|
||||
stmt.outerjoin(DownloadJob, done)
|
||||
.outerjoin(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
|
||||
.group_by(FileFolder.id)
|
||||
.order_by(FileFolder.kind.desc(), FileFolder.source_key, FileFolder.name, FileFolder.id)
|
||||
).all()
|
||||
linked = _linked(db, [f.id for f, _, _ in rows])
|
||||
return [_serialize_folder(f, count, int(size), f.id in linked) for f, count, size in rows]
|
||||
rows = queries.folders_with_totals(db, caller.user, caller.folders_with(Right.LIST))
|
||||
linked = queries.linked_folders(db, [f.id for f, _, _ in rows])
|
||||
return [_serialize_folder(f, count, size, f.id in linked) for f, count, size in rows]
|
||||
|
||||
|
||||
class FolderIn(BaseModel):
|
||||
@@ -422,13 +346,10 @@ def _folder_name(raw: str) -> str:
|
||||
def create_folder(
|
||||
body: FolderIn, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
folder = FileFolder(user_id=user.id, kind=folders.FolderKind.OWN, name=_folder_name(body.name))
|
||||
try:
|
||||
with db.begin_nested():
|
||||
db.add(folder)
|
||||
db.flush()
|
||||
except IntegrityError as e: # the unique (user, lower(name)): one folder per name
|
||||
raise refused(409, "file_folder_name_taken") from e
|
||||
folder = folders.create_own(db, user.id, _folder_name(body.name))
|
||||
except folders.FolderError as e:
|
||||
raise refused(409, e.reason) from e
|
||||
audit.record(
|
||||
db, user.id, AuditAction.FILE_FOLDER_CREATE, target_type="file_folder", target_id=folder.id
|
||||
)
|
||||
@@ -451,17 +372,16 @@ def update_folder(
|
||||
) -> dict:
|
||||
"""Rename one of the caller's own folders (a source folder's name is its origin's), and/or set
|
||||
how long any of their folders keeps a file (`folders.set_retention`)."""
|
||||
folder = owned_or_404(db, FileFolder, folder_id, user, what="folder")
|
||||
folder = own_folder(db, user, folder_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
if folder.kind != folders.FolderKind.OWN:
|
||||
# Which folder may be renamed is answered before what the name is, as it always was.
|
||||
if not folders.renamable(folder):
|
||||
raise refused(409, "file_folder_not_own")
|
||||
folder.name = _folder_name(body.name or "")
|
||||
try:
|
||||
with db.begin_nested():
|
||||
db.flush()
|
||||
except IntegrityError as e:
|
||||
raise refused(409, "file_folder_name_taken") from e
|
||||
folders.rename(db, folder, _folder_name(body.name or ""))
|
||||
except folders.FolderError as e:
|
||||
raise refused(409, e.reason) from e
|
||||
audit.record(
|
||||
db,
|
||||
user.id,
|
||||
@@ -486,18 +406,9 @@ def update_folder(
|
||||
after={"retention_days": body.retention_days},
|
||||
)
|
||||
db.commit()
|
||||
count, size = _folder_totals(db, folder.id)
|
||||
return _serialize_folder(folder, count, size, folder.id in _linked(db, [folder.id]))
|
||||
|
||||
|
||||
def _folder_totals(db: Session, folder_id: int) -> tuple[int, int]:
|
||||
count, size = db.execute(
|
||||
select(func.count(DownloadJob.id), func.coalesce(func.sum(MediaAsset.size_bytes), 0))
|
||||
.select_from(DownloadJob)
|
||||
.outerjoin(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
|
||||
.where(DownloadJob.folder_id == folder_id, DownloadJob.status == JobStatus.DONE)
|
||||
).one()
|
||||
return count, int(size)
|
||||
count, size = queries.folder_totals(db, folder.id)
|
||||
linked = folder.id in queries.linked_folders(db, [folder.id])
|
||||
return _serialize_folder(folder, count, size, linked)
|
||||
|
||||
|
||||
@router.delete("/folders/{folder_id}")
|
||||
@@ -505,7 +416,7 @@ def delete_folder(
|
||||
folder_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
"""Delete one of the caller's own folders; its files go back to their source folders."""
|
||||
folder = owned_or_404(db, FileFolder, folder_id, user, what="folder")
|
||||
folder = own_folder(db, user, folder_id)
|
||||
try:
|
||||
moved = folders.delete_own(db, folder)
|
||||
except folders.FolderError as e:
|
||||
@@ -547,13 +458,7 @@ def remove_shared_with_me(
|
||||
"""Remove a file that was shared WITH this user from their 'Shared with me' list. Deletes
|
||||
only the recipient's share grant — the owner's job and the physical file are untouched (this is
|
||||
a per-user dismissal, not a delete)."""
|
||||
row = db.execute(
|
||||
select(DownloadShare).where(
|
||||
DownloadShare.job_id == job_id, DownloadShare.to_user_id == user.id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
if service.unshare(db, job_id, user):
|
||||
db.commit()
|
||||
return {"ok": True, "removed": job_id}
|
||||
|
||||
@@ -582,46 +487,22 @@ def list_files(
|
||||
raise refused(403, "api_token_scope")
|
||||
if folder_id is not None and folder_id not in listable:
|
||||
raise not_found("folder")
|
||||
stmt = (
|
||||
select(DownloadJob)
|
||||
.join(MediaAsset, MediaAsset.id == DownloadJob.asset_id)
|
||||
.where(DownloadJob.status == JobStatus.DONE)
|
||||
)
|
||||
shared = scope == "shared"
|
||||
if shared:
|
||||
stmt = stmt.join(DownloadShare, DownloadShare.job_id == DownloadJob.id).where(
|
||||
DownloadShare.to_user_id == user.id
|
||||
)
|
||||
else:
|
||||
stmt = stmt.where(DownloadJob.user_id == user.id)
|
||||
if folder_id is not None:
|
||||
stmt = stmt.where(DownloadJob.folder_id == folder_id)
|
||||
elif listable is not None:
|
||||
stmt = stmt.where(DownloadJob.folder_id.in_(listable))
|
||||
name = func.coalesce(DownloadJob.display_name, MediaAsset.title, "")
|
||||
term = q.strip()
|
||||
if term:
|
||||
like = f"%{like_escape(term)}%"
|
||||
stmt = stmt.where(
|
||||
or_(
|
||||
name.ilike(like),
|
||||
func.coalesce(DownloadJob.display_uploader, MediaAsset.uploader, "").ilike(like),
|
||||
)
|
||||
)
|
||||
# A file is added to its owner's list when it is done, not when it was asked for (a download that
|
||||
# failed for days and then went through is new); one shared with the caller, when it was shared.
|
||||
key = {
|
||||
"added": DownloadShare.created_at if shared else DownloadJob.ready_at,
|
||||
"name": collated(name, lang_for_user(user)),
|
||||
"size": MediaAsset.size_bytes,
|
||||
}[sort]
|
||||
first = key.asc() if order == "asc" else key.desc()
|
||||
tie = DownloadJob.id.asc() if order == "asc" else DownloadJob.id.desc()
|
||||
jobs = db.execute(stmt.order_by(first.nulls_last(), tie).limit(MAX_LIST)).scalars().all()
|
||||
assets = assets_for(db, jobs)
|
||||
positions = positions_for(db, user, jobs)
|
||||
shared_at = _shared_at(db, user, jobs) if shared else {}
|
||||
sources = source_keys(db, jobs)
|
||||
jobs = queries.library(
|
||||
db,
|
||||
user,
|
||||
folder_id=folder_id,
|
||||
listable=listable,
|
||||
shared=shared,
|
||||
term=q.strip(),
|
||||
sort=sort,
|
||||
order=order,
|
||||
lang=lang_for_user(user),
|
||||
)
|
||||
assets = queries.assets_for(db, jobs)
|
||||
positions = queries.positions_for(db, user, jobs)
|
||||
shared_at = queries.shared_at(db, user, jobs) if shared else {}
|
||||
sources = queries.source_keys(db, jobs)
|
||||
return [
|
||||
{
|
||||
**serialize(j, assets.get(j.asset_id) if j.asset_id is not None else None),
|
||||
@@ -634,36 +515,6 @@ def list_files(
|
||||
]
|
||||
|
||||
|
||||
def source_keys(db: Session, jobs: Sequence[DownloadJob]) -> dict[int, str]:
|
||||
"""Where each file came from (its source folder's key), for the list's Source column, the
|
||||
"back to its source folder" move and the download tray's site: a file in a source folder has its folder's key; one moved into
|
||||
an own folder (or someone else's, shared) is traced back by `folders.source_key_for`."""
|
||||
ids = {j.folder_id for j in jobs if j.folder_id is not None}
|
||||
by_id = (
|
||||
{f.id: f for f in db.execute(select(FileFolder).where(FileFolder.id.in_(ids))).scalars()}
|
||||
if ids
|
||||
else {}
|
||||
)
|
||||
out: dict[int, str] = {}
|
||||
for j in jobs:
|
||||
folder = by_id.get(j.folder_id) if j.folder_id is not None else None
|
||||
if folder is not None and folder.kind == folders.FolderKind.SOURCE and folder.source_key:
|
||||
out[j.id] = folder.source_key
|
||||
else:
|
||||
out[j.id] = folders.source_key_for(db, j)
|
||||
return out
|
||||
|
||||
|
||||
def _shared_at(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, datetime]:
|
||||
rows = db.execute(
|
||||
select(DownloadShare.job_id, DownloadShare.created_at).where(
|
||||
DownloadShare.to_user_id == user.id,
|
||||
DownloadShare.job_id.in_([j.id for j in jobs]),
|
||||
)
|
||||
).all()
|
||||
return {job_id: at for job_id, at in rows}
|
||||
|
||||
|
||||
_MAX_EXTRA_LINKS = 8
|
||||
|
||||
|
||||
@@ -711,18 +562,20 @@ def update_file(
|
||||
"""Rename a file and its channel, set its reference links (all display-only: the file on disk
|
||||
keeps its own name), move it to another folder (`folders.move`), or keep it until deleted /
|
||||
hand it back to its folder's retention."""
|
||||
job = owned_or_404(db, DownloadJob, job_id, user, what="download")
|
||||
job = own_job(db, user, job_id)
|
||||
sent = body.model_fields_set
|
||||
display: dict[str, object] = {}
|
||||
if "name" in sent:
|
||||
job.display_name = display_text(body.name)
|
||||
display["display_name"] = display_text(body.name)
|
||||
if "uploader" in sent:
|
||||
job.display_uploader = display_text(body.uploader)
|
||||
display["display_uploader"] = display_text(body.uploader)
|
||||
if "uploader_url" in sent:
|
||||
job.display_uploader_url = _clean_url(body.uploader_url)
|
||||
display["display_uploader_url"] = _clean_url(body.uploader_url)
|
||||
if "extra_links" in sent:
|
||||
raw = body.extra_links or []
|
||||
cleaned = [u for u in (_clean_url(x) for x in raw) if u][:_MAX_EXTRA_LINKS]
|
||||
job.extra_links = cleaned or None
|
||||
display["extra_links"] = cleaned or None
|
||||
service.set_display(job, **display)
|
||||
if "folder_id" in sent:
|
||||
if body.folder_id is None:
|
||||
raise refused(400, "file_folder_gone")
|
||||
@@ -768,7 +621,8 @@ def delete_file(
|
||||
target_id=job_id,
|
||||
summary=caller.via,
|
||||
)
|
||||
service.delete(db, job) # commits, the audit row with it
|
||||
service.delete(db, job)
|
||||
db.commit() # the audit row with it
|
||||
return {"ok": True, "deleted": job_id}
|
||||
|
||||
|
||||
@@ -787,7 +641,7 @@ def upload_file(
|
||||
return serialize_job(db, ingest_spooled(db, upload))
|
||||
|
||||
|
||||
def ingest_spooled(db: Session, upload: SpooledUpload) -> DownloadJob:
|
||||
def ingest_spooled(db: Session, upload: SpooledUpload) -> "DownloadJob":
|
||||
"""The spooled upload into the library (`upload.ingest_upload`, which commits its own rows),
|
||||
its refusals answered as HTTP."""
|
||||
if upload.size == 0:
|
||||
@@ -811,8 +665,8 @@ def ingest_spooled(db: Session, upload: SpooledUpload) -> DownloadJob:
|
||||
return job
|
||||
|
||||
|
||||
def _ready_file(db: Session, job: DownloadJob) -> tuple[MediaAsset, Path]:
|
||||
asset = db.get(MediaAsset, job.asset_id) if job.asset_id else None
|
||||
def _ready_file(db: Session, job: "DownloadJob") -> tuple["MediaAsset", Path]:
|
||||
asset = queries.asset_of(db, job)
|
||||
if asset is None or asset.status != "ready" or not asset.rel_path:
|
||||
raise refused(409, "download_not_ready")
|
||||
path = storage.safe_abs_path(settings.files_root, asset.rel_path)
|
||||
@@ -821,7 +675,7 @@ def _ready_file(db: Session, job: DownloadJob) -> tuple[MediaAsset, Path]:
|
||||
return asset, path
|
||||
|
||||
|
||||
def serve_file(db: Session, job: DownloadJob) -> FileResponse:
|
||||
def serve_file(db: Session, job: "DownloadJob") -> FileResponse:
|
||||
"""The file of `job` (the caller may read it), range-aware, named by the user's display name.
|
||||
|
||||
The served Content-Type comes from the ffprobe-verified media_kind + a container allow-list
|
||||
@@ -839,7 +693,7 @@ def serve_file(db: Session, job: DownloadJob) -> FileResponse:
|
||||
if asset.sha256:
|
||||
digest = base64.b64encode(bytes.fromhex(asset.sha256)).decode()
|
||||
headers["Repr-Digest"] = f"sha-256=:{digest}:"
|
||||
asset.last_access_at = datetime.now(timezone.utc) # the handler commits it
|
||||
service.touch(asset) # the handler commits it
|
||||
media_type = storage.served_media_type(asset.media_kind, asset.container)
|
||||
if media_type is None:
|
||||
return FileResponse(
|
||||
@@ -894,13 +748,13 @@ def preview_file(
|
||||
text = preview.text_utf8(head)
|
||||
if text is None:
|
||||
raise refused(415, "file_no_preview")
|
||||
asset.last_access_at = datetime.now(timezone.utc)
|
||||
service.touch(asset)
|
||||
db.commit()
|
||||
return Response(text, media_type=preview.media_type(kind, asset.container), headers=headers)
|
||||
filename = storage.download_filename(
|
||||
job.display_name or asset.title or asset.source_ref, asset.container, path
|
||||
)
|
||||
asset.last_access_at = datetime.now(timezone.utc)
|
||||
service.touch(asset)
|
||||
db.commit()
|
||||
return FileResponse(
|
||||
path,
|
||||
@@ -936,10 +790,6 @@ def get_sha256(
|
||||
finally:
|
||||
_HASHING.release()
|
||||
digest = h.hexdigest()
|
||||
db.execute(
|
||||
update(MediaAsset)
|
||||
.where(MediaAsset.id == asset_id, MediaAsset.sha256.is_(None))
|
||||
.values(sha256=digest)
|
||||
)
|
||||
service.store_sha256(db, asset_id, digest)
|
||||
db.commit()
|
||||
return {"sha256": digest}
|
||||
|
||||
+10
-81
@@ -9,11 +9,9 @@ from app.auth import (
|
||||
count_admins,
|
||||
count_pending_activations,
|
||||
current_user,
|
||||
drop_from_wallet,
|
||||
google_enabled,
|
||||
has_read_scope,
|
||||
has_write_scope,
|
||||
is_allowed,
|
||||
optional_current_user,
|
||||
purge_user,
|
||||
)
|
||||
@@ -26,79 +24,10 @@ from app.db import get_db
|
||||
from app.models import Invite, User
|
||||
from app.utils import iso
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
router = APIRouter(prefix="/api/me", tags=["me"])
|
||||
|
||||
|
||||
@router.get("/accounts")
|
||||
def my_accounts(
|
||||
request: Request,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> list[dict]:
|
||||
"""The accounts that have authenticated in this browser session (switchable without a
|
||||
new Google sign-in). The active one is flagged."""
|
||||
ids = request.session.get("account_ids") or [user.id]
|
||||
rows = {u.id: u for u in db.execute(select(User).where(User.id.in_(ids))).scalars()}
|
||||
out = []
|
||||
for uid in ids: # preserve recency order from the session
|
||||
u = rows.get(uid)
|
||||
if u is not None:
|
||||
out.append(
|
||||
{
|
||||
"id": u.id,
|
||||
"email": u.email,
|
||||
"display_name": u.display_name,
|
||||
"avatar_url": u.avatar_url,
|
||||
"active": u.id == user.id,
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
class SwitchAccountIn(BaseModel):
|
||||
user_id: int
|
||||
|
||||
|
||||
@router.post("/switch")
|
||||
def switch_account(
|
||||
body: SwitchAccountIn,
|
||||
request: Request,
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Switch the active account to another one already authenticated in this browser. No
|
||||
Google round-trip — but only accounts present in the session list (proof they signed in
|
||||
here) are allowed, and access is re-checked in case the invite was revoked since."""
|
||||
target = body.user_id
|
||||
if target not in (request.session.get("account_ids") or []):
|
||||
raise refused(403, "not_your_account")
|
||||
# An account `_session_user` would refuse — gone, deactivated, suspended, or its session revoked — is
|
||||
# refused here too, and dropped from the switch list the way `_session_user` drops it. Made the session
|
||||
# default instead, the next request's current_user would refuse it as the DEFAULT and clear the whole
|
||||
# wallet session, logging out every account here. (B-core-9)
|
||||
u = db.get(User, target)
|
||||
if u is None:
|
||||
drop_from_wallet(request.session, target)
|
||||
raise not_found("account")
|
||||
if not is_allowed(db, u.email):
|
||||
raise refused(403, "account_no_access")
|
||||
if not u.is_active:
|
||||
drop_from_wallet(request.session, target)
|
||||
raise refused(403, "account_no_access")
|
||||
if u.is_suspended:
|
||||
drop_from_wallet(request.session, target)
|
||||
raise refused(403, "account_suspended")
|
||||
# Revoked: a password reset/change or "log out everywhere" bumped the account's epoch.
|
||||
epochs = request.session.get("epochs") or {}
|
||||
if int(epochs.get(str(target), 0)) != (u.session_epoch or 0):
|
||||
drop_from_wallet(request.session, target)
|
||||
raise refused(403, "account_signed_out")
|
||||
request.session["user_id"] = target
|
||||
return {"ok": True, "user_id": target}
|
||||
|
||||
|
||||
@router.get("")
|
||||
def get_me(
|
||||
user: User | None = Depends(optional_current_user), db: Session = Depends(get_db)
|
||||
@@ -187,17 +116,11 @@ def delete_my_account(
|
||||
surfaces)."""
|
||||
if user.role == "admin" and count_admins(db) <= 1:
|
||||
raise refused(400, "only_admin_delete")
|
||||
user_id = user.id
|
||||
# Full erasure (cascades) + access-request cleanup + Google-grant revocation; shared with the
|
||||
# admin delete path. Capture the id first — `user` is detached once purge_user deletes the row.
|
||||
# admin delete path.
|
||||
purge_user(db, user, background)
|
||||
# Drop this account from the browser session; switch to another signed-in account if any.
|
||||
remaining = [a for a in (request.session.get("account_ids") or []) if a != user_id]
|
||||
if remaining:
|
||||
request.session["account_ids"] = remaining
|
||||
request.session["user_id"] = remaining[-1]
|
||||
else:
|
||||
request.session.clear()
|
||||
# The session's one account is gone: so is the session.
|
||||
request.session.clear()
|
||||
return {"deleted": True}
|
||||
|
||||
|
||||
@@ -213,7 +136,13 @@ def update_preferences(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Merge so partial updates don't wipe other keys.
|
||||
# Merge so partial updates don't wipe other keys — over the stored row read again under a row
|
||||
# lock: a client saves keys one at a time (several at once on a boot that moves this browser's
|
||||
# copies to the account), and two merges over the same earlier read would keep only the last.
|
||||
# FOR NO KEY UPDATE, as bump_session_epoch: it still orders two merges, and a transaction that
|
||||
# only added a row referring to the account (a playlist pull, then minutes of YouTube) does not
|
||||
# hold the save up.
|
||||
db.refresh(user, with_for_update={"key_share": True})
|
||||
merged = dict(user.preferences or {})
|
||||
merged.update(preferences.root)
|
||||
user.preferences = merged
|
||||
|
||||
@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import Notification, User
|
||||
@@ -36,7 +37,7 @@ def list_notifications(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
where = [Notification.user_id == user.id]
|
||||
where = [owned_by(Notification.user_id, user)]
|
||||
if not include_dismissed:
|
||||
where.append(Notification.dismissed.is_(False))
|
||||
total = db.scalar(select(func.count()).select_from(Notification).where(*where)) or 0
|
||||
@@ -61,7 +62,7 @@ def unread_count(user: User = Depends(current_user), db: Session = Depends(get_d
|
||||
select(func.count())
|
||||
.select_from(Notification)
|
||||
.where(
|
||||
Notification.user_id == user.id,
|
||||
owned_by(Notification.user_id, user),
|
||||
Notification.read.is_(False),
|
||||
Notification.dismissed.is_(False),
|
||||
)
|
||||
@@ -77,8 +78,8 @@ def mark_read(
|
||||
) -> dict:
|
||||
n = owned_or_404(db, Notification, notification_id, user, what="notification")
|
||||
n.read = True
|
||||
db.commit()
|
||||
trim_read(db, user.id)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -86,11 +87,11 @@ def mark_read(
|
||||
def mark_all_read(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.read.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.read.is_(False))
|
||||
.values(read=True)
|
||||
)
|
||||
db.commit()
|
||||
trim_read(db, user.id)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -113,7 +114,7 @@ def clear_all(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"""Dismiss every (still-visible) notification for this user."""
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.dismissed.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.dismissed.is_(False))
|
||||
.values(dismissed=True, read=True)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
+61
-304
@@ -3,31 +3,34 @@ CRUD (create / rename / delete, add / remove / reorder items) plus YouTube two-w
|
||||
mirror the user's own YouTube playlists in (`/sync-youtube`) and push/revert local edits back
|
||||
out (`/{id}/push-plan`, `/{id}/revert-youtube`), behind the optional write scope."""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, select, and_
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import quota
|
||||
from app.auth import current_user, has_read_scope, has_write_scope, require_write_scope
|
||||
from app.db import get_db
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.models import Channel, Playlist, PlaylistItem, User, Video, VideoState
|
||||
from app.feedquery import serialize
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.sync.playlists import (
|
||||
PlaylistGone,
|
||||
lock_playlists,
|
||||
merge_playlist,
|
||||
plan_merge,
|
||||
recompute_dirty,
|
||||
pull_user_playlists,
|
||||
repull_playlist,
|
||||
sync_claim,
|
||||
user_sync_claim,
|
||||
)
|
||||
from app.ratelimit import RateLimiter
|
||||
from app.routes._common import get_or_404, not_found, owned_or_404, youtube_refusal
|
||||
from app.routes._common import not_found, youtube_refusal
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.reader.models import Playlist
|
||||
|
||||
router = APIRouter(prefix="/api/playlists", tags=["playlists"])
|
||||
|
||||
# SEC-14, same reasoning as the sync module's limiter: a playlist pull lists every playlist the
|
||||
@@ -56,90 +59,20 @@ def _refused(status: int, reason: str) -> HTTPException:
|
||||
return HTTPException(status_code=status, detail={"code": "playlist", "reason": reason})
|
||||
|
||||
|
||||
def _touch(pl: Playlist) -> None:
|
||||
"""Bump the playlist's `updated_at` for an edit that only writes `playlist_items`.
|
||||
|
||||
`UpdatedAtMixin`'s `onupdate` fires on an UPDATE of the PLAYLIST row, and add / remove /
|
||||
reorder / move write the item rows instead — so without this the rail's "recently changed"
|
||||
sort would only ever see renames. The assignment is what makes the column mean "last time
|
||||
this list changed", which is what the rail promises the reader.
|
||||
|
||||
The DATABASE's clock, like the mixin's own `onupdate`: one sort key written from two clocks
|
||||
(this process's and Postgres's) could order a rename and an item edit the wrong way round."""
|
||||
pl.updated_at = func.now()
|
||||
def _own_playlist(db: Session, user: User, playlist_id: int, *, lock: bool = False) -> "Playlist":
|
||||
pl = queries.own_playlist(db, user, playlist_id, lock=lock)
|
||||
if pl is None:
|
||||
raise not_found("playlist")
|
||||
return pl
|
||||
|
||||
|
||||
def _next_playlist_position(db: Session, user_id: int) -> int:
|
||||
"""Append position for a new playlist in this user's rail."""
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(Playlist.position), -1)).where(
|
||||
Playlist.user_id == user_id
|
||||
)
|
||||
)
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def _next_item_position(db: Session, playlist_id: int) -> int:
|
||||
"""Append position for a new item at the end of a playlist. Locks the playlist row first: two
|
||||
adds (two tabs, the rail and the page) otherwise both read the same max and land on one
|
||||
position, and the list's order between them is then arbitrary."""
|
||||
lock_playlists(db, playlist_id)
|
||||
return (
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(PlaylistItem.position), -1)).where(
|
||||
PlaylistItem.playlist_id == playlist_id
|
||||
)
|
||||
)
|
||||
+ 1
|
||||
)
|
||||
|
||||
|
||||
def _add_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Append a video to a playlist if not already present (idempotent). Returns whether it
|
||||
was added. `mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later).
|
||||
Locked BEFORE the presence check: two adds of one video both saw "absent" and the second one's
|
||||
insert hit uq_playlist_video as a 500."""
|
||||
lock_playlists(db, pl.id)
|
||||
exists = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == pl.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
if exists is not None:
|
||||
return False
|
||||
db.add(
|
||||
PlaylistItem(playlist_id=pl.id, video_id=video_id, position=_next_item_position(db, pl.id))
|
||||
)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
_touch(pl)
|
||||
db.flush() # the handler commits (CLAUDE.md: helpers flush)
|
||||
return True
|
||||
|
||||
|
||||
def _remove_item(db: Session, pl: Playlist, video_id: str, *, mark_dirty: bool) -> bool:
|
||||
"""Remove a video from a playlist if present (idempotent). Returns whether it was removed.
|
||||
`mark_dirty` recomputes the YouTube-sync dirty flag (skip for Watch later)."""
|
||||
lock_playlists(db, pl.id)
|
||||
item = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == pl.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
if item is None:
|
||||
return False
|
||||
db.delete(item)
|
||||
if mark_dirty:
|
||||
recompute_dirty(db, pl)
|
||||
_touch(pl)
|
||||
db.flush() # the handler commits (CLAUDE.md: helpers flush)
|
||||
return True
|
||||
def _video_or_404(db: Session, video_id: str) -> None:
|
||||
if queries.video(db, video_id) is None:
|
||||
raise not_found("video")
|
||||
|
||||
|
||||
def _summary(
|
||||
pl: Playlist,
|
||||
pl: "Playlist",
|
||||
count: int,
|
||||
total_duration: int = 0,
|
||||
has_video: bool | None = None,
|
||||
@@ -156,7 +89,7 @@ def _summary(
|
||||
"total_duration_seconds": total_duration,
|
||||
"cover_thumbnail": cover,
|
||||
# When this list last CHANGED (rename, add/remove/move/reorder) — the rail's
|
||||
# "recently changed first" sort reads it. See _touch for why it isn't automatic.
|
||||
# "recently changed first" sort reads it. See `service.touch` for why it isn't automatic.
|
||||
"updated_at": pl.updated_at,
|
||||
}
|
||||
if has_video is not None:
|
||||
@@ -172,73 +105,14 @@ def list_playlists(
|
||||
) -> list[dict]:
|
||||
"""The user's playlists (newest custom order). With `contains=<video_id>`, each entry
|
||||
also reports whether that video is already in it — used by the add-to-playlist popover."""
|
||||
pls = (
|
||||
db.execute(
|
||||
select(Playlist)
|
||||
.where(Playlist.user_id == user.id)
|
||||
.order_by(Playlist.position, Playlist.created_at)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
pls = queries.playlists_for(db, user)
|
||||
if not pls:
|
||||
return []
|
||||
ids = [p.id for p in pls]
|
||||
counts = dict(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id, func.count())
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
).all()
|
||||
)
|
||||
durations = dict(
|
||||
db.execute(
|
||||
select(
|
||||
PlaylistItem.playlist_id,
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.group_by(PlaylistItem.playlist_id)
|
||||
).all()
|
||||
)
|
||||
member: set[int] = set()
|
||||
if contains:
|
||||
member = set(
|
||||
db.execute(
|
||||
select(PlaylistItem.playlist_id).where(
|
||||
PlaylistItem.playlist_id.in_(ids),
|
||||
PlaylistItem.video_id == contains,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
# Batch the cover thumbnails (first item per playlist by position) in ONE window query
|
||||
# instead of a per-playlist SELECT inside _summary (the old N+1).
|
||||
rn = (
|
||||
func.row_number()
|
||||
.over(
|
||||
partition_by=PlaylistItem.playlist_id,
|
||||
order_by=PlaylistItem.position,
|
||||
)
|
||||
.label("rn")
|
||||
)
|
||||
first_items = (
|
||||
select(
|
||||
PlaylistItem.playlist_id.label("pid"),
|
||||
Video.thumbnail_url.label("thumb"),
|
||||
rn,
|
||||
)
|
||||
.join(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id.in_(ids))
|
||||
.subquery()
|
||||
)
|
||||
covers = dict(
|
||||
db.execute(
|
||||
select(first_items.c.pid, first_items.c.thumb).where(first_items.c.rn == 1)
|
||||
).all()
|
||||
)
|
||||
counts = queries.item_counts(db, ids)
|
||||
durations = queries.durations(db, ids)
|
||||
member = queries.containing(db, ids, contains) if contains else set()
|
||||
covers = queries.covers(db, ids)
|
||||
return [
|
||||
_summary(
|
||||
p,
|
||||
@@ -269,29 +143,11 @@ def create_playlist(
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise _refused(400, "name_required")
|
||||
pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id))
|
||||
db.add(pl)
|
||||
pl = service.create_playlist(db, user, name)
|
||||
db.commit()
|
||||
return _summary(pl, 0)
|
||||
|
||||
|
||||
def _watch_later(db: Session, user: User) -> Playlist:
|
||||
"""The user's built-in Watch later playlist, created on first use."""
|
||||
pl = db.scalar(
|
||||
select(Playlist).where(Playlist.user_id == user.id, Playlist.kind == "watch_later")
|
||||
)
|
||||
if pl is None:
|
||||
pl = Playlist(
|
||||
user_id=user.id,
|
||||
name="Watch later",
|
||||
kind="watch_later",
|
||||
position=_next_playlist_position(db, user.id),
|
||||
)
|
||||
db.add(pl)
|
||||
db.flush() # the id, for the item the caller adds next; the handler commits
|
||||
return pl
|
||||
|
||||
|
||||
@router.post("/watch-later")
|
||||
def add_watch_later(
|
||||
body: VideoRefIn,
|
||||
@@ -300,9 +156,9 @@ def add_watch_later(
|
||||
) -> dict:
|
||||
"""Bookmark shortcut: add a video to the built-in Watch later playlist."""
|
||||
video_id = body.video_id
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
pl = _watch_later(db, user)
|
||||
_add_item(db, pl, video_id, mark_dirty=False)
|
||||
_video_or_404(db, video_id)
|
||||
pl = service.watch_later(db, user)
|
||||
service.add_item(db, pl, video_id, mark_dirty=False)
|
||||
db.commit()
|
||||
return {"ok": True, "saved": True, "playlist_id": pl.id}
|
||||
|
||||
@@ -313,11 +169,9 @@ def remove_watch_later(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = db.scalar(
|
||||
select(Playlist).where(Playlist.user_id == user.id, Playlist.kind == "watch_later")
|
||||
)
|
||||
pl = queries.watch_later(db, user)
|
||||
if pl is not None:
|
||||
_remove_item(db, pl, video_id, mark_dirty=False)
|
||||
service.remove_item(db, pl, video_id, mark_dirty=False)
|
||||
db.commit()
|
||||
return {"ok": True, "saved": False}
|
||||
|
||||
@@ -354,7 +208,7 @@ def revert_youtube(
|
||||
"""Reset one YouTube-linked playlist to its current YouTube state, discarding local
|
||||
edits (and clearing the dirty flag). The user-facing 'undo all my changes' for a list
|
||||
once the in-session undo history is gone."""
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
if not pl.yt_playlist_id:
|
||||
raise _refused(400, "not_on_youtube")
|
||||
if not has_read_scope(user):
|
||||
@@ -375,10 +229,10 @@ def revert_youtube(
|
||||
raise youtube_refusal(exc, playlist=True) from exc
|
||||
|
||||
|
||||
def _pushable(db: Session, user: User, playlist_id: int) -> Playlist:
|
||||
def _pushable(db: Session, user: User, playlist_id: int) -> "Playlist":
|
||||
"""A local playlist the user may push to YouTube (create or update). The built-in
|
||||
Watch later and read-only YouTube mirrors are never pushable."""
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
if pl.kind == "watch_later":
|
||||
raise _refused(400, "watch_later_not_pushable")
|
||||
return pl
|
||||
@@ -448,17 +302,8 @@ def get_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
# `id` breaks a position tie the same way on every read (the renumbering add does too).
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).all()
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
rows = queries.playlist_rows(db, user, pl.id)
|
||||
return {
|
||||
"id": pl.id,
|
||||
"name": pl.name,
|
||||
@@ -482,34 +327,15 @@ def rename_playlist(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked: the new name moves `dirty`, which the scheduled mirror refresh reads under this lock.
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist", lock=True)
|
||||
pl = _own_playlist(db, user, playlist_id, lock=True)
|
||||
if "name" in body.model_fields_set:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise _refused(400, "name_required")
|
||||
if name != pl.name:
|
||||
pl.name = name
|
||||
recompute_dirty(db, pl)
|
||||
service.rename_playlist(db, pl, name)
|
||||
db.commit()
|
||||
# One aggregate for count + duration (outer join so an empty playlist still returns 0),
|
||||
# plus a single cover lookup — was 3 separate round-trips.
|
||||
count, total = db.execute(
|
||||
select(
|
||||
func.count(PlaylistItem.id),
|
||||
func.coalesce(func.sum(Video.duration_seconds), 0),
|
||||
)
|
||||
.select_from(PlaylistItem)
|
||||
.outerjoin(Video, Video.id == PlaylistItem.video_id)
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
).one()
|
||||
cover = db.scalar(
|
||||
select(Video.thumbnail_url)
|
||||
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlaylistItem.position)
|
||||
.limit(1)
|
||||
)
|
||||
return _summary(pl, count or 0, total or 0, cover=cover)
|
||||
count, total, cover = queries.playlist_totals(db, pl.id)
|
||||
return _summary(pl, count, total, cover=cover)
|
||||
|
||||
|
||||
@router.delete("/{playlist_id}")
|
||||
@@ -519,7 +345,7 @@ def delete_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
if pl.kind == "watch_later":
|
||||
raise _refused(400, "watch_later_undeletable")
|
||||
# Not in the middle of a sync of it (`sync_claim`): a push that read the list before the delete
|
||||
@@ -540,9 +366,7 @@ def delete_playlist(
|
||||
yt.delete_playlist(pl.yt_playlist_id)
|
||||
except YouTubeError as exc:
|
||||
raise youtube_refusal(exc, playlist=True) from exc
|
||||
# The playlist first, as every item write: the cascade deletes the items before the list row.
|
||||
lock_playlists(db, pl.id)
|
||||
db.delete(pl) # items cascade
|
||||
service.delete_playlist(db, pl)
|
||||
db.commit()
|
||||
return {"deleted": playlist_id}
|
||||
|
||||
@@ -561,26 +385,10 @@ def add_item(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
video_id = body.video_id
|
||||
get_or_404(db, Video, video_id, what="video")
|
||||
added = _add_item(db, pl, video_id, mark_dirty=False)
|
||||
if added and body.position is not None:
|
||||
# Renumber the whole list with the new row at `position` (clamped to the end): positions
|
||||
# can have gaps after removals, so "index N" is only meaningful over the ordered rows.
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlaylistItem)
|
||||
.where(PlaylistItem.playlist_id == pl.id)
|
||||
.order_by(PlaylistItem.position, PlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
new = next(r for r in rows if r.video_id == video_id)
|
||||
rows.remove(new)
|
||||
rows.insert(min(body.position, len(rows)), new)
|
||||
for pos, row in enumerate(rows):
|
||||
row.position = pos
|
||||
recompute_dirty(db, pl)
|
||||
_video_or_404(db, video_id)
|
||||
service.add_to_playlist(db, pl, video_id, body.position)
|
||||
db.commit()
|
||||
return {"ok": True, "playlist_id": pl.id, "video_id": video_id}
|
||||
|
||||
@@ -592,8 +400,8 @@ def remove_item(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
_remove_item(db, pl, video_id, mark_dirty=True)
|
||||
pl = _own_playlist(db, user, playlist_id)
|
||||
service.remove_item(db, pl, video_id, mark_dirty=True)
|
||||
db.commit()
|
||||
return {"ok": True, "playlist_id": pl.id, "video_id": video_id}
|
||||
|
||||
@@ -617,49 +425,21 @@ def move_item(
|
||||
exactly the kind of half-applied state this codebase treats as a bug rather than an edge
|
||||
case. Idempotent: if the video is already in the target, the source row is simply dropped
|
||||
(moving into a list that already has it is a removal, not a duplicate)."""
|
||||
src = owned_or_404(db, Playlist, playlist_id, user, what="playlist")
|
||||
src = _own_playlist(db, user, playlist_id)
|
||||
# Its own reason: "this playlist doesn't exist any more" about a TARGET that is gone was read as
|
||||
# the list on screen — which is still there.
|
||||
dst = owned_or_404(db, Playlist, body.target_playlist_id, user, what="move_target")
|
||||
if src.id == dst.id:
|
||||
dst = queries.own_playlist(db, user, body.target_playlist_id)
|
||||
if dst is None:
|
||||
raise not_found("move_target")
|
||||
try:
|
||||
service.move_item(db, src, dst, video_id)
|
||||
except service.MoveRefused as e:
|
||||
# A same-list move is a bad request; a video a concurrent edit (another tab, a YouTube
|
||||
# re-pull) already took out is gone.
|
||||
status = 400 if e.reason == "same_playlist" else 404
|
||||
raise HTTPException(
|
||||
status_code=400, detail={"code": "playlist_move", "reason": "same_playlist"}
|
||||
)
|
||||
# Both lists, in id order: a move A→B beside a move B→A would otherwise lock crosswise.
|
||||
lock_playlists(db, src.id, dst.id)
|
||||
item = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == src.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
if item is None:
|
||||
# A concurrent edit (another tab, or a YouTube re-pull) already took it out.
|
||||
raise HTTPException(
|
||||
status_code=404, detail={"code": "playlist_move", "reason": "not_in_playlist"}
|
||||
)
|
||||
already = db.scalar(
|
||||
select(PlaylistItem).where(
|
||||
PlaylistItem.playlist_id == dst.id, PlaylistItem.video_id == video_id
|
||||
)
|
||||
)
|
||||
dst_changed = False
|
||||
if already is None:
|
||||
# Re-parent the existing row rather than delete+insert: `added_at` ("when I saved this")
|
||||
# survives the move, and there is no window where the video belongs to neither list. The
|
||||
# check above holds: both lists are locked, and an insert into the target (another tab's
|
||||
# add, a YouTube merge) waits for the lock — its foreign key needs the target row.
|
||||
item.playlist_id = dst.id
|
||||
item.position = _next_item_position(db, dst.id)
|
||||
dst_changed = True
|
||||
if not dst_changed:
|
||||
db.delete(item)
|
||||
# The source always changed; the target only when the row landed in it. `_touch` on a target
|
||||
# that already had the video promoted an untouched list in the rail's "recently changed" sort.
|
||||
recompute_dirty(db, src)
|
||||
_touch(src)
|
||||
if dst_changed:
|
||||
recompute_dirty(db, dst)
|
||||
_touch(dst)
|
||||
status_code=status, detail={"code": "playlist_move", "reason": e.reason}
|
||||
) from None
|
||||
db.commit()
|
||||
return {"ok": True, "from_playlist_id": src.id, "to_playlist_id": dst.id, "video_id": video_id}
|
||||
|
||||
@@ -676,30 +456,7 @@ def reorder_items(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked: a concurrent add between reading the items and writing the order would collide.
|
||||
pl = owned_or_404(db, Playlist, playlist_id, user, what="playlist", lock=True)
|
||||
order = body.video_ids
|
||||
items = {
|
||||
it.video_id: it
|
||||
for it in db.execute(
|
||||
select(PlaylistItem).where(PlaylistItem.playlist_id == pl.id)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[str] = set()
|
||||
for vid in order:
|
||||
it = items.get(vid)
|
||||
if it is not None:
|
||||
it.position = pos
|
||||
pos += 1
|
||||
seen.add(vid)
|
||||
# Any items the payload omitted (e.g. added concurrently in another tab) keep their relative
|
||||
# order but get fresh trailing positions, so no two items collide on the same position.
|
||||
for it in sorted(
|
||||
(it for vid, it in items.items() if vid not in seen), key=lambda i: i.position
|
||||
):
|
||||
it.position = pos
|
||||
pos += 1
|
||||
recompute_dirty(db, pl)
|
||||
_touch(pl)
|
||||
pl = _own_playlist(db, user, playlist_id, lock=True)
|
||||
pos = service.reorder_items(db, pl, body.video_ids)
|
||||
db.commit()
|
||||
return {"ok": True, "playlist_id": pl.id, "count": pos}
|
||||
|
||||
@@ -1,18 +1,22 @@
|
||||
"""Shared helpers, serializers and constants for the plex route package."""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import PlexItem
|
||||
from app.plex import queries
|
||||
from app.plex.cards import norm_lang as _norm_lang # noqa: F401 — the package's historic name
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
def item_or_404(db: Session, rating_key: str) -> PlexItem:
|
||||
it = by_rating_key(db, PlexItem, rating_key)
|
||||
def item_or_404(db: Session, rating_key: str) -> "PlexItem":
|
||||
it = queries.item(db, rating_key)
|
||||
if it is None:
|
||||
raise not_found("plex_item")
|
||||
return it
|
||||
|
||||
@@ -1,27 +1,26 @@
|
||||
"""Plex admin: connection test, catalog sync, and owner watch-link config."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexLink,
|
||||
User,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex import queries, service
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex import watch_sync as plex_watch
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.utils import iso
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexLink
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
@@ -69,7 +68,7 @@ def _sample_plex_path(db: Session, plex: PlexClient, sections: list[dict]) -> st
|
||||
"""A real Plex-side media file path to validate the mount + path-map against. Prefers the
|
||||
already-mirrored catalog (cheap); before the first sync, pulls one movie leaf from an enabled
|
||||
section straight from Plex."""
|
||||
fp = db.scalar(select(PlexItem.file_path).where(PlexItem.file_path.isnot(None)))
|
||||
fp = queries.sample_file_path(db)
|
||||
if fp:
|
||||
return fp
|
||||
wanted = plex_sync._enabled_section_keys(db)
|
||||
@@ -106,7 +105,7 @@ def trigger_sync(_: User = Depends(admin_user), db: Session = Depends(get_db)) -
|
||||
return {"ok": True, **plex_sync.sync(db)}
|
||||
|
||||
|
||||
def _link_dict(link: PlexLink | None) -> dict:
|
||||
def _link_dict(link: "PlexLink | None") -> dict:
|
||||
return {
|
||||
"linked": link is not None,
|
||||
"sync_enabled": bool(link and link.sync_enabled),
|
||||
@@ -119,8 +118,7 @@ def _link_dict(link: PlexLink | None) -> dict:
|
||||
@router.get("/watch/link")
|
||||
def watch_link_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""This user's Plex watch-sync link status (drives the Settings toggle)."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
return _link_dict(link)
|
||||
return _link_dict(queries.watch_link(db, user))
|
||||
|
||||
|
||||
class WatchLinkIn(BaseModel):
|
||||
@@ -139,11 +137,7 @@ def watch_link_set(
|
||||
if not sysconfig.get_bool(db, "plex_enabled"):
|
||||
raise refused(409, "plex_disabled")
|
||||
enabled = body.enabled
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
if link is None:
|
||||
link = PlexLink(user_id=user.id, uses_admin=True, linked_at=datetime.now(timezone.utc))
|
||||
db.add(link)
|
||||
link.sync_enabled = enabled
|
||||
link = service.set_watch_link(db, user, enabled)
|
||||
db.commit()
|
||||
|
||||
result: dict = {}
|
||||
@@ -156,7 +150,7 @@ def watch_link_set(
|
||||
def watch_import_now(user: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""Admin: (re-)run the "Plex is master" import for this account — Plex re-wins on the
|
||||
intersection; Siftlode-only states are preserved. Idempotent."""
|
||||
link = db.execute(select(PlexLink).where(PlexLink.user_id == user.id)).scalars().first()
|
||||
link = queries.watch_link(db, user)
|
||||
if link is None or not link.sync_enabled:
|
||||
raise refused(409, "watch_sync_off")
|
||||
result = plex_watch.import_owner_watch_state(db, link)
|
||||
|
||||
@@ -1,110 +1,30 @@
|
||||
"""Plex browse: the library list, the unified cross-library browse feed, and its faceted filters."""
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
Integer,
|
||||
String,
|
||||
and_,
|
||||
cast,
|
||||
func,
|
||||
literal,
|
||||
null,
|
||||
or_,
|
||||
select,
|
||||
)
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated, sorted_words
|
||||
from app import sysconfig
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.mailtext import lang_for_user
|
||||
from app.models import User
|
||||
from app.plex import library as plex_library
|
||||
from app.plex import queries
|
||||
from app.plex.cards import display_title, episode_card
|
||||
from app.plex.query import (
|
||||
FIELD_COLUMNS,
|
||||
PLAYBACK_CLASSES,
|
||||
TS_CONFIG,
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_movie_state_filter,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
field_recheck,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
movie_status_case,
|
||||
playable_conds,
|
||||
playback_class_expr,
|
||||
show_status_case,
|
||||
split_csv,
|
||||
summary_col,
|
||||
title_col,
|
||||
vector_col,
|
||||
vector_match,
|
||||
)
|
||||
from app.plex.query import LibraryFilters
|
||||
from app.routes.plex._shared import _norm_lang
|
||||
from app.utils import to_tsquery_str
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def sort_column(u, sort: str, lang: str = "en"):
|
||||
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
|
||||
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
|
||||
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
|
||||
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
|
||||
in the reader's alphabet (`collation`)."""
|
||||
return {
|
||||
"added": u.c.added_at,
|
||||
"title": collated(u.c.title, lang),
|
||||
"year": u.c.year,
|
||||
"rating": u.c.rating,
|
||||
"duration": u.c.duration_s,
|
||||
"release": u.c.rel,
|
||||
"lastwatched": u.c.last_watched,
|
||||
}.get(sort, u.c.added_at)
|
||||
|
||||
|
||||
@router.get("/libraries")
|
||||
def list_libraries(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""The mirrored libraries the user can browse (drives the Plex scope selector). HU overlay
|
||||
libraries are excluded — they're not browsable, they only feed the EN twin's Hungarian metadata."""
|
||||
libs = (
|
||||
db.execute(
|
||||
select(PlexLibrary)
|
||||
.where(PlexLibrary.overlay_of.is_(None))
|
||||
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
out = []
|
||||
for lib in libs:
|
||||
if lib.kind == "movie":
|
||||
count = db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.library_id == lib.id, PlexItem.kind == "movie"
|
||||
)
|
||||
)
|
||||
else:
|
||||
count = db.scalar(select(func.count(PlexShow.id)).where(PlexShow.library_id == lib.id))
|
||||
out.append(
|
||||
{
|
||||
"key": lib.plex_key,
|
||||
"title": lib.title,
|
||||
"kind": lib.kind,
|
||||
"count": count or 0,
|
||||
}
|
||||
)
|
||||
out = [
|
||||
{"key": lib.plex_key, "title": lib.title, "kind": lib.kind, "count": count}
|
||||
for lib, count in queries.browsable_libraries(db, lang_for_user(user))
|
||||
]
|
||||
return {"enabled": sysconfig.get_bool(db, "plex_enabled"), "libraries": out}
|
||||
|
||||
|
||||
@@ -129,193 +49,60 @@ def unified_library(
|
||||
episodes come back in a separate `episodes` list (grouped result — the 'Episodes' section)."""
|
||||
offset = max(0, offset)
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
|
||||
ts = to_tsquery_str(q) if q else None
|
||||
tsq = func.to_tsquery(TS_CONFIG, ts) if ts else None
|
||||
if field not in FIELD_COLUMNS and field != "all":
|
||||
field = "all"
|
||||
# `episode` scope targets episode rows only — the movie/show grid stays empty; the recheck for
|
||||
# the specific title/people/summary fields is a strict narrowing on top of the GIN prefilter.
|
||||
grid_field = field != "episode"
|
||||
recheck = field_recheck if tsq is not None and field not in ("all", "episode") else None
|
||||
|
||||
selects = []
|
||||
if want_movies and grid_field:
|
||||
st = aliased(PlexState)
|
||||
m_status = movie_status_case(st)
|
||||
mq = (
|
||||
select(
|
||||
PlexItem.rating_key.label("rk"),
|
||||
literal("movie").label("kind"),
|
||||
title_col(PlexItem, lang).label("title"),
|
||||
PlexItem.year.label("year"),
|
||||
PlexItem.rating.label("rating"),
|
||||
# Truncated so the card hover-overview costs a few hundred bytes, not the full synopsis.
|
||||
func.left(summary_col(PlexItem, lang), 300).label("summary"),
|
||||
PlexItem.added_at.label("added_at"),
|
||||
PlexItem.originally_available_at.label("rel"),
|
||||
PlexItem.duration_s.label("duration_s"),
|
||||
PlexItem.playable.label("playable"),
|
||||
PlexItem.transcode_hw.label("transcode_hw"),
|
||||
cast(null(), Integer).label("season_count"),
|
||||
func.coalesce(st.position_seconds, 0).label("position_seconds"),
|
||||
m_status.label("status"),
|
||||
literal(False).label("resumable"),
|
||||
# "last watched" sort key: when the movie was last resumed, else when it was marked
|
||||
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
|
||||
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexItem, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(st, and_(st.item_id == PlexItem.id, st.user_id == user.id))
|
||||
.where(PlexItem.kind == "movie", PlexItem.library_id.in_(movie_lib_ids))
|
||||
)
|
||||
mq = apply_movie_state_filter(mq, st, show)
|
||||
mq = apply_meta_filters(mq, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
mq = mq.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
mq = mq.where(PlexItem.duration_s <= f.duration_max)
|
||||
# Playback-class filter (movie-only — a show is an aggregate with no single class).
|
||||
mq = mq.where(*playable_conds(split_csv(f.playable)))
|
||||
if tsq is not None:
|
||||
mq = mq.where(vector_match(PlexItem, lang, tsq))
|
||||
if recheck is not None:
|
||||
mq = mq.where(recheck(PlexItem, field, tsq, lang))
|
||||
selects.append(mq)
|
||||
if want_shows and grid_field:
|
||||
agg = rollup.agg_subquery(db, user.id, include_resumed=True).subquery()
|
||||
tot = func.coalesce(agg.c.total, 0)
|
||||
wat = func.coalesce(agg.c.watched, 0)
|
||||
inp = func.coalesce(agg.c.inprog, 0)
|
||||
s_status = show_status_case(tot, wat, inp)
|
||||
sq = (
|
||||
select(
|
||||
PlexShow.rating_key.label("rk"),
|
||||
literal("show").label("kind"),
|
||||
title_col(PlexShow, lang).label("title"),
|
||||
PlexShow.year.label("year"),
|
||||
PlexShow.rating.label("rating"),
|
||||
func.left(summary_col(PlexShow, lang), 300).label("summary"),
|
||||
PlexShow.added_at.label("added_at"),
|
||||
PlexShow.originally_available_at.label("rel"),
|
||||
cast(null(), Integer).label("duration_s"),
|
||||
cast(null(), String).label("playable"),
|
||||
cast(null(), Boolean).label("transcode_hw"),
|
||||
PlexShow.child_count.label("season_count"),
|
||||
literal(0).label("position_seconds"),
|
||||
s_status.label("status"),
|
||||
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
|
||||
(inp > 0).label("resumable"),
|
||||
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
|
||||
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
|
||||
agg.c.resumed_at.label("last_watched"),
|
||||
(
|
||||
func.ts_rank(vector_col(PlexShow, lang), tsq)
|
||||
if tsq is not None
|
||||
else literal(0.0)
|
||||
).label("rank"),
|
||||
)
|
||||
.outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_lib_ids))
|
||||
)
|
||||
sq = apply_show_state_filter(sq, tot, wat, inp, show)
|
||||
sq = apply_meta_filters(sq, PlexShow, p, lang)
|
||||
if tsq is not None:
|
||||
sq = sq.where(vector_match(PlexShow, lang, tsq))
|
||||
if recheck is not None:
|
||||
sq = sq.where(recheck(PlexShow, field, tsq, lang))
|
||||
selects.append(sq)
|
||||
|
||||
total, rows, eps = plex_library.library_page(
|
||||
db,
|
||||
user,
|
||||
scope=scope,
|
||||
q=q,
|
||||
field=field,
|
||||
sort=sort,
|
||||
sort_dir=sort_dir,
|
||||
show=show,
|
||||
lang=lang,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
f=f,
|
||||
)
|
||||
items = []
|
||||
total = 0
|
||||
if selects:
|
||||
union = selects[0] if len(selects) == 1 else selects[0].union_all(*selects[1:])
|
||||
u = union.subquery()
|
||||
total = db.scalar(select(func.count()).select_from(u)) or 0
|
||||
|
||||
if tsq is not None:
|
||||
order = [u.c.rank.desc()]
|
||||
else:
|
||||
col = sort_column(u, sort, lang)
|
||||
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
|
||||
order.append(u.c.rk.desc())
|
||||
rows = db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all()
|
||||
|
||||
for r in rows:
|
||||
if r.kind == "movie":
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "movie",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"duration_seconds": r.duration_s,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"playable": r.playable,
|
||||
"transcode_hw": r.transcode_hw,
|
||||
"status": r.status,
|
||||
"position_seconds": r.position_seconds,
|
||||
}
|
||||
)
|
||||
else:
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "show",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"season_count": r.season_count,
|
||||
"status": r.status,
|
||||
"resumable": r.resumable,
|
||||
}
|
||||
)
|
||||
|
||||
# Episode "grouped result" section: shown for Everywhere (full-vector match, today's behaviour)
|
||||
# and for the Episode scope (title-only match, empty item grid). The title/people/summary scopes
|
||||
# target the movie/show grid, so they suppress episodes.
|
||||
episodes = []
|
||||
if tsq is not None and want_shows and field in ("all", "episode"):
|
||||
ep_st = aliased(PlexState)
|
||||
# In HU an episode is found by its Hungarian title too (the overlay fills it) — and
|
||||
# still by its English one, like `vector_match` does for the grid.
|
||||
if field == "episode":
|
||||
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
|
||||
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
|
||||
else:
|
||||
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
|
||||
hit = ep_vec.op("@@")(tsq)
|
||||
if lang == "hu":
|
||||
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
|
||||
ep_vec = ep_vec_hu
|
||||
eps = db.execute(
|
||||
select(PlexItem, ep_st, PlexShow)
|
||||
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, ep_st.user_id == user.id))
|
||||
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_lib_ids),
|
||||
hit,
|
||||
for r in rows:
|
||||
if r.kind == "movie":
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "movie",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"duration_seconds": r.duration_s,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"playable": r.playable,
|
||||
"transcode_hw": r.transcode_hw,
|
||||
"status": r.status,
|
||||
"position_seconds": r.position_seconds,
|
||||
}
|
||||
)
|
||||
.order_by(func.ts_rank(ep_vec, tsq).desc())
|
||||
.limit(24)
|
||||
).all()
|
||||
for e, s, sh in eps:
|
||||
card = episode_card(e, s, lang=lang)
|
||||
card["show_title"] = display_title(sh, lang) if sh is not None else None
|
||||
episodes.append(card)
|
||||
else:
|
||||
items.append(
|
||||
{
|
||||
"id": r.rk,
|
||||
"type": "show",
|
||||
"title": r.title,
|
||||
"year": r.year,
|
||||
"rating": r.rating,
|
||||
"summary": r.summary,
|
||||
"thumb": f"/api/plex/image/{r.rk}",
|
||||
"season_count": r.season_count,
|
||||
"status": r.status,
|
||||
"resumable": r.resumable,
|
||||
}
|
||||
)
|
||||
|
||||
episodes = []
|
||||
for e, s, sh in eps:
|
||||
card = episode_card(e, s, lang=lang)
|
||||
card["show_title"] = display_title(sh, lang) if sh is not None else None
|
||||
episodes.append(card)
|
||||
|
||||
return {
|
||||
"scope": scope,
|
||||
@@ -341,237 +128,4 @@ def facets(
|
||||
filters applied but NOT its own — so selecting a filter narrows what the other groups offer, while
|
||||
a multi-select group still lets you add more. Merged across the scope's libraries; duration is
|
||||
movie-only. (Search text is not factored in — facets track the sidebar filters.)"""
|
||||
empty = {
|
||||
"genres": [],
|
||||
"content_ratings": [],
|
||||
"studios": [],
|
||||
"playable": [],
|
||||
"year_min": None,
|
||||
"year_max": None,
|
||||
"rating_max": None,
|
||||
"duration_min": None,
|
||||
"duration_max": None,
|
||||
}
|
||||
all_movie_ids, all_show_ids = content_libs(db)
|
||||
movie_ids = all_movie_ids if scope in ("movie", "both") else []
|
||||
show_ids = all_show_ids if scope in ("show", "both") else []
|
||||
if not movie_ids and not show_ids:
|
||||
return empty
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
|
||||
uid = user.id
|
||||
|
||||
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
|
||||
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
|
||||
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call.
|
||||
# The same filter on the same rows, so the counts do not change
|
||||
# (test_plex_facets_grid_parity).
|
||||
shows_in_scope: list[int] | None = None
|
||||
if show_ids and show in ("watched", "in_progress", "unwatched"):
|
||||
agg = rollup.agg_subquery(db, uid).subquery()
|
||||
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
|
||||
in_scope = apply_show_state_filter(
|
||||
in_scope,
|
||||
func.coalesce(agg.c.total, 0),
|
||||
func.coalesce(agg.c.watched, 0),
|
||||
func.coalesce(agg.c.inprog, 0),
|
||||
show,
|
||||
)
|
||||
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
|
||||
|
||||
def wsq(query, model, ids, pp, kind_movie, with_dur):
|
||||
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
|
||||
facet aggregation query, so the facets reflect exactly the visible result set."""
|
||||
query = query.where(model.library_id.in_(ids), *meta_filter_conds(model, pp, lang))
|
||||
if kind_movie:
|
||||
query = query.where(model.kind == "movie")
|
||||
if with_dur and pp.get("duration_min") is not None:
|
||||
query = query.where(model.duration_s >= pp["duration_min"])
|
||||
if with_dur and pp.get("duration_max") is not None:
|
||||
query = query.where(model.duration_s <= pp["duration_max"])
|
||||
# Playback-class filter (movie-only) — applied for the OTHER groups so an active class
|
||||
# narrows them; the playable facet itself passes pp with playable=None (self-exclusion).
|
||||
query = query.where(*playable_conds(split_csv(pp.get("playable"))))
|
||||
# Join watch-state and reuse the grid's exact movie state filter (unified_library) —
|
||||
# hidden movies never appear in the grid, so they must not inflate facet counts/bounds either.
|
||||
st = aliased(PlexState)
|
||||
query = query.outerjoin(st, and_(st.item_id == model.id, st.user_id == uid))
|
||||
query = apply_movie_state_filter(query, st, show)
|
||||
elif shows_in_scope is not None:
|
||||
query = query.where(model.id.in_(shows_in_scope))
|
||||
return query
|
||||
|
||||
genre_counts: dict[str, int] = {}
|
||||
cr_counts: dict[str, int] = {}
|
||||
studio_counts: dict[str, int] = {}
|
||||
playable_counts: dict[str, int] = {}
|
||||
years: list[int] = []
|
||||
ratings: list[float] = []
|
||||
durs: list[int] = []
|
||||
|
||||
def per_table(model, ids, kind_movie: bool) -> None:
|
||||
if not ids:
|
||||
return
|
||||
# Genres — in "any"/OR mode, exclude the genre filter itself so each offered genre is a valid
|
||||
# OR-addition that only widens (keep adding). In "all"/AND mode, KEEP the filter applied so we
|
||||
# offer only genres that CO-OCCUR on the current result set — a non-co-occurring genre would AND
|
||||
# the result down to zero, i.e. a dead-end chip. (The already-selected genres still appear, since
|
||||
# every matching title carries them, so they stay togglable.)
|
||||
genre_pp = p if p.get("genre_mode") == "all" else {**p, "genres": None}
|
||||
# Only rows whose `genres` is a JSON array can be unnested. A genre-less item stores JSON
|
||||
# `null` (SQLAlchemy's JSONB persists a Python None as JSON null, not SQL NULL), and
|
||||
# jsonb_array_elements_text() raises on a scalar ("cannot extract elements from a scalar") —
|
||||
# so a single metadata-poor library (e.g. one bare Shows library with no genres) 500'd the
|
||||
# whole facets endpoint. Mirror the exact guard people.py already applies to the same columns.
|
||||
# In HU the vocabulary comes from the active-language genre column (coalesce(genres_hu, genres)).
|
||||
gcol = genres_col(model, lang)
|
||||
expanded = (
|
||||
wsq(
|
||||
select(func.jsonb_array_elements_text(gcol).label("g")),
|
||||
model,
|
||||
ids,
|
||||
genre_pp,
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(func.jsonb_typeof(gcol) == "array")
|
||||
.subquery()
|
||||
)
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
genre_counts[g] = genre_counts.get(g, 0) + c
|
||||
# Content ratings — exclude its own.
|
||||
crq = (
|
||||
wsq(
|
||||
select(model.content_rating, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "content_ratings": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.content_rating.isnot(None))
|
||||
.group_by(model.content_rating)
|
||||
)
|
||||
for cr, c in db.execute(crq).all():
|
||||
cr_counts[cr] = cr_counts.get(cr, 0) + c
|
||||
# Studios — exclude its own (each offered studio is a valid OR-addition that only widens).
|
||||
stq = (
|
||||
wsq(
|
||||
select(model.studio, func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "studios": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(model.studio.isnot(None))
|
||||
.group_by(model.studio)
|
||||
)
|
||||
for s, c in db.execute(stq).all():
|
||||
studio_counts[s] = studio_counts.get(s, 0) + c
|
||||
# Playback class (movie-only) — exclude its own so each class stays an addable OR option.
|
||||
if kind_movie:
|
||||
pcls = playback_class_expr()
|
||||
pcq = (
|
||||
wsq(
|
||||
select(pcls.label("cls"), func.count()),
|
||||
model,
|
||||
ids,
|
||||
{**p, "playable": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
.where(pcls.isnot(None))
|
||||
.group_by(pcls)
|
||||
)
|
||||
for cls, c in db.execute(pcq).all():
|
||||
playable_counts[cls] = playable_counts.get(cls, 0) + c
|
||||
# Year bounds — exclude the year filter.
|
||||
by = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.year), func.max(model.year)),
|
||||
model,
|
||||
ids,
|
||||
{**p, "year_min": None, "year_max": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
).first()
|
||||
if by[0] is not None:
|
||||
years.append(by[0])
|
||||
if by[1] is not None:
|
||||
years.append(by[1])
|
||||
# Rating max — exclude the rating filter. Coalesce(imdb, audience) to match the score the
|
||||
# rating filter and the UI badge use (IMDb preferred, audience fallback).
|
||||
br = db.scalar(
|
||||
wsq(
|
||||
select(func.max(func.coalesce(model.imdb_rating, model.rating))),
|
||||
model,
|
||||
ids,
|
||||
{**p, "rating_min": None},
|
||||
kind_movie,
|
||||
True,
|
||||
)
|
||||
)
|
||||
if br is not None:
|
||||
ratings.append(float(br))
|
||||
# Duration bounds (movies) — exclude the duration filter.
|
||||
if kind_movie:
|
||||
bd = db.execute(
|
||||
wsq(
|
||||
select(func.min(model.duration_s), func.max(model.duration_s)),
|
||||
model,
|
||||
ids,
|
||||
p,
|
||||
kind_movie,
|
||||
False,
|
||||
)
|
||||
).first()
|
||||
if bd[0] is not None:
|
||||
durs.append(bd[0])
|
||||
if bd[1] is not None:
|
||||
durs.append(bd[1])
|
||||
|
||||
per_table(PlexItem, movie_ids, True)
|
||||
per_table(PlexShow, show_ids, False)
|
||||
# Always surface the actively-selected genres (even a zero-match AND combo, where they'd otherwise
|
||||
# be absent): the sidebar renders genres solely from this list and hides the whole genre section —
|
||||
# chips AND the Any/All toggle — when it's empty, which would trap the user with no per-chip way to
|
||||
# undo a zero-result selection. Count 0 is fine (genre chips don't display counts).
|
||||
for g in split_csv(f.genres):
|
||||
genre_counts.setdefault(g, 0)
|
||||
# Same for the actively-selected studios, so a selected studio stays removable even at zero matches.
|
||||
for s in split_csv(f.studios):
|
||||
studio_counts.setdefault(s, 0)
|
||||
# Keep an actively-selected playback class visible even at zero matches (the sidebar hides the whole
|
||||
# section when its facet list is empty, which would trap a zero-result selection with no way to undo).
|
||||
for c in split_csv(f.playable):
|
||||
if c in PLAYBACK_CLASSES:
|
||||
playable_counts.setdefault(c, 0)
|
||||
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
|
||||
return {
|
||||
"genres": [
|
||||
{"value": g, "count": c}
|
||||
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
|
||||
],
|
||||
"content_ratings": [
|
||||
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
|
||||
],
|
||||
"studios": [
|
||||
{"value": s, "count": c}
|
||||
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
|
||||
],
|
||||
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
|
||||
"playable": [
|
||||
{"value": c, "count": playable_counts[c]}
|
||||
for c in PLAYBACK_CLASSES
|
||||
if c in playable_counts
|
||||
],
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"rating_max": max(ratings) if ratings else None,
|
||||
"duration_min": min(durs) if durs else None,
|
||||
"duration_max": max(durs) if durs else None,
|
||||
}
|
||||
return plex_library.facets(db, user, scope=scope, show=show, lang=lang, f=f)
|
||||
|
||||
@@ -1,96 +1,41 @@
|
||||
"""Plex collections: browse mirrored collections + the admin manual-collection editor."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime
|
||||
from typing import Any, NamedTuple
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.mailtext import lang_for_user
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app import logs
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex import sync as plex_sync
|
||||
from app.plex.cards import SHOW_NEW, display_title, movie_card, norm_lang, show_card
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
|
||||
from app.utils import like_escape
|
||||
from app.plex.collections import (
|
||||
DEFAULT_MEMBER_SORT,
|
||||
Member,
|
||||
collection_card,
|
||||
collection_cert,
|
||||
collection_editable,
|
||||
collection_source,
|
||||
sort_members,
|
||||
)
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexCollection, PlexState
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
|
||||
# members (Plex shows one cert on the collection; we don't store it, so derive it).
|
||||
_CERT_RANK = {
|
||||
"G": 1,
|
||||
"TV-Y": 1,
|
||||
"TV-Y7": 1,
|
||||
"TV-G": 1,
|
||||
"PG": 2,
|
||||
"TV-PG": 2,
|
||||
"PG-13": 3,
|
||||
"TV-14": 3,
|
||||
"R": 4,
|
||||
"TV-MA": 4,
|
||||
"NC-17": 5,
|
||||
}
|
||||
|
||||
|
||||
def _collection_cert(certs: list[str | None]) -> str | None:
|
||||
ranked = [(c, _CERT_RANK.get(c, 0)) for c in certs if c]
|
||||
return max(ranked, key=lambda x: x[1])[0] if ranked else None
|
||||
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
_STRIP_PROVIDERS = ("imdb", "tmdb", "tvdb", "trakt")
|
||||
|
||||
|
||||
def _collection_source(col: PlexCollection) -> str:
|
||||
first = (col.title or "").strip().split(" ", 1)[0].lower()
|
||||
if first in _STRIP_PROVIDERS:
|
||||
return first
|
||||
if col.smart:
|
||||
return "smart"
|
||||
return "collection"
|
||||
|
||||
|
||||
def _collection_editable(c: PlexCollection) -> bool:
|
||||
"""Effective editability: the admin marked it editable AND it's a plain manual collection — never
|
||||
smart, never an external auto-list (IMDb/TMDb/…), which are managed outside Plex."""
|
||||
return bool(c.editable) and not c.smart and _collection_source(c) == "collection"
|
||||
|
||||
|
||||
def _collection_card(c: PlexCollection) -> dict:
|
||||
return {
|
||||
"id": c.rating_key,
|
||||
"title": c.title,
|
||||
"summary": c.summary,
|
||||
"thumb": f"/api/plex/image/{c.rating_key}" if c.thumb_key else None,
|
||||
"child_count": c.child_count,
|
||||
"smart": c.smart,
|
||||
"source": _collection_source(c),
|
||||
"editable": c.editable,
|
||||
"can_edit": _collection_editable(
|
||||
c
|
||||
), # whether the edit controls should be offered (admin UI)
|
||||
}
|
||||
|
||||
|
||||
@router.get("/collections")
|
||||
def collections(
|
||||
@@ -110,64 +55,22 @@ def collections(
|
||||
sidebar token and the applied-filter chip had nothing to render but the raw id. Resolving them
|
||||
through this endpoint costs one small row each; pulling the whole list to find two titles is
|
||||
~200 KB."""
|
||||
if library is not None:
|
||||
lib = (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(library)))
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if lib is None:
|
||||
return {"collections": []}
|
||||
query = select(PlexCollection).where(PlexCollection.library_id == lib.id)
|
||||
else:
|
||||
enabled_ids = list(
|
||||
db.execute(
|
||||
select(PlexLibrary.id).where(
|
||||
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
if not enabled_ids:
|
||||
return {"collections": []}
|
||||
query = select(PlexCollection).where(PlexCollection.library_id.in_(enabled_ids))
|
||||
term = (q or "").strip()
|
||||
if term:
|
||||
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
|
||||
if keys is not None:
|
||||
wanted = [k for k in (keys or "").split(",") if k]
|
||||
# An explicit but empty `keys=` asks for nothing — answer nothing, rather than falling through
|
||||
# to "every collection" (which is what an unguarded empty IN () would have to become).
|
||||
if not wanted:
|
||||
return {"collections": []}
|
||||
query = query.where(PlexCollection.rating_key.in_(wanted))
|
||||
rows = (
|
||||
db.execute(
|
||||
query.order_by(
|
||||
PlexCollection.child_count.desc().nullslast(),
|
||||
collated(PlexCollection.title, lang_for_user(user)),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
wanted = [k for k in keys.split(",") if k] if keys is not None else None
|
||||
rows = queries.collections_page(db, library, q, wanted, lang_for_user(user))
|
||||
# A montage of the top titles per tile + the collection's total runtime. Art unions MOVIE and SHOW
|
||||
# members (a collection lives in one library, so exactly one side has art) — a show collection would
|
||||
# otherwise render blank; the collection's own poster is the next fallback, then the colour hash.
|
||||
# Runtime is movie-only (shows have no single duration).
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
keys = [c.rating_key for c in rows]
|
||||
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
|
||||
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
|
||||
runtime = runtime_by_key(db, PlexItem, movie_ids, PlexItem.collection_keys, keys)
|
||||
art_m, art_s, runtime = queries.collection_tiles(db, [c.rating_key for c in rows])
|
||||
|
||||
def _art(c: PlexCollection) -> list[str]:
|
||||
def _art(c: "PlexCollection") -> list[str]:
|
||||
member = art_m.get(c.rating_key) or art_s.get(c.rating_key)
|
||||
return member or ([c.rating_key] if c.thumb_key else [])
|
||||
|
||||
return {
|
||||
"collections": [
|
||||
{
|
||||
**_collection_card(c),
|
||||
**collection_card(c),
|
||||
"art": _art(c),
|
||||
"total_runtime_s": runtime.get(c.rating_key, 0),
|
||||
}
|
||||
@@ -185,89 +88,21 @@ def collections_for_item(
|
||||
"""What a movie or show card's "Add to collection" lists: the editable collections of the title's
|
||||
library (a collection lives in one library, and only a manual one can take a title — the same
|
||||
`can_edit` the editor offers), each saying whether the title is in it already, by name."""
|
||||
movie = by_rating_key(db, PlexItem, item_rating_key)
|
||||
title: PlexItem | PlexShow | None = (
|
||||
movie if movie is not None and movie.kind == "movie" else None
|
||||
) or by_rating_key(db, PlexShow, item_rating_key)
|
||||
movie = queries.movie(db, item_rating_key)
|
||||
title = movie or queries.show(db, item_rating_key)
|
||||
if title is None:
|
||||
raise not_found("item")
|
||||
member_of = set(title.collection_keys or [])
|
||||
rows = (
|
||||
db.execute(
|
||||
select(PlexCollection)
|
||||
.where(PlexCollection.library_id == title.library_id)
|
||||
.order_by(collated(PlexCollection.title, lang_for_user(user)))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
rows = queries.library_collections(db, title.library_id, lang_for_user(user))
|
||||
return {
|
||||
"collections": [
|
||||
{**_collection_card(c), "contains": c.rating_key in member_of}
|
||||
{**collection_card(c), "contains": c.rating_key in member_of}
|
||||
for c in rows
|
||||
if _collection_editable(c)
|
||||
if collection_editable(c)
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
class Member(NamedTuple):
|
||||
"""One collection member flattened to everything the Sort control can order by. The page's two
|
||||
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
|
||||
ordering is a pure list sort over these, which is also what makes it unit-testable."""
|
||||
|
||||
kind: str # "movie" | "show"
|
||||
obj: Any # PlexItem | PlexShow, carried through to the card serializer
|
||||
title: str # lower-cased; never NULL (the column is not nullable)
|
||||
year: int | None
|
||||
rating: float | None
|
||||
added: datetime | None
|
||||
release: date | None
|
||||
duration: int | None # movies only — a show has no single runtime
|
||||
last_watched: datetime | None
|
||||
|
||||
|
||||
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
|
||||
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
|
||||
# that silently does nothing here.
|
||||
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
|
||||
"added": lambda m: m.added,
|
||||
"lastwatched": lambda m: m.last_watched,
|
||||
"release": lambda m: m.release,
|
||||
"year": lambda m: m.year,
|
||||
"rating": lambda m: m.rating,
|
||||
"duration": lambda m: m.duration,
|
||||
"title": lambda m: m.title,
|
||||
}
|
||||
|
||||
# What the page ordered by before it had a control, and still its default: a film series wants to be
|
||||
# read in release order, not by whenever Plex happened to ingest it.
|
||||
DEFAULT_MEMBER_SORT = "year"
|
||||
|
||||
|
||||
def sort_members(
|
||||
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
|
||||
) -> list[Member]:
|
||||
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
|
||||
|
||||
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
|
||||
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
|
||||
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
|
||||
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
|
||||
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
|
||||
|
||||
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
|
||||
compare as Python strings, which puts every accented initial after "z"."""
|
||||
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
|
||||
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
|
||||
if key is MEMBER_SORT_KEYS["title"]:
|
||||
key = by_title
|
||||
ordered = sorted(members, key=by_title) # stable secondary, applied first
|
||||
have = [m for m in ordered if key(m) is not None]
|
||||
missing = [m for m in ordered if key(m) is None]
|
||||
have.sort(key=key, reverse=sort_dir != "asc")
|
||||
return have + missing
|
||||
|
||||
|
||||
@router.get("/collections/{rating_key}")
|
||||
def collection_detail(
|
||||
rating_key: str,
|
||||
@@ -283,45 +118,20 @@ def collection_detail(
|
||||
what this page did before it had a control. Read-only here; edits go through the admin mutation
|
||||
endpoints below."""
|
||||
lang = norm_lang(lang)
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow)
|
||||
.where(PlexShow.collection_keys.contains([rating_key]))
|
||||
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
movies, shows = queries.collection_members(db, rating_key, lang)
|
||||
mstate = rollup.item_state_map(db, user.id, [m.id for m in movies])
|
||||
# ONE rollup pass over the show members, for both the aggregate badge status and `resumed_at` —
|
||||
# the browse grid's "last watched" for a show (the newest resume/watched across its episodes). The
|
||||
# cheaper `rollup.state_map` gives only the status, which would leave `lastwatched` unorderable on
|
||||
# a show collection while the control still offered it.
|
||||
show_rows = (
|
||||
db.execute(
|
||||
rollup.agg_subquery(db, user.id, include_resumed=True).where(
|
||||
PlexItem.show_id.in_([s.id for s in shows])
|
||||
)
|
||||
).all()
|
||||
if shows
|
||||
else []
|
||||
)
|
||||
show_rows = queries.show_rollups(db, user.id, [s.id for s in shows])
|
||||
sstate = {r.show_id: rollup.show_state(r.total, r.watched, r.inprog) for r in show_rows}
|
||||
sresumed = {r.show_id: r.resumed_at for r in show_rows}
|
||||
|
||||
def _movie_last_watched(st: PlexState | None) -> datetime | None:
|
||||
def _movie_last_watched(st: "PlexState | None") -> datetime | None:
|
||||
# Same definition as the browse grid's: when it was last resumed, else when it was marked
|
||||
# watched (coalesce(progress_updated_at, watched_at)).
|
||||
return (st.progress_updated_at or st.watched_at) if st else None
|
||||
@@ -365,8 +175,8 @@ def collection_detail(
|
||||
years = [m.year for m in members if m.year is not None]
|
||||
certs = [m.content_rating for m in movies] + [s.content_rating for s in shows]
|
||||
return {
|
||||
**_collection_card(col),
|
||||
"content_rating": _collection_cert(certs),
|
||||
**collection_card(col),
|
||||
"content_rating": collection_cert(certs),
|
||||
"year_min": min(years) if years else None,
|
||||
"year_max": max(years) if years else None,
|
||||
"total_runtime_s": sum(m.duration_s or 0 for m in movies),
|
||||
@@ -374,11 +184,11 @@ def collection_detail(
|
||||
}
|
||||
|
||||
|
||||
def _editable_collection_or_403(db: Session, rating_key: str) -> PlexCollection:
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
def _editable_collection_or_403(db: Session, rating_key: str) -> "PlexCollection":
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
if not _collection_editable(col):
|
||||
if not collection_editable(col):
|
||||
raise refused(403, "collection_read_only")
|
||||
return col
|
||||
|
||||
@@ -389,16 +199,7 @@ def _reject_duplicate_name(
|
||||
"""No two collections in a section may share a name: Plex CONFLATES same-named collections'
|
||||
membership (both rating_keys resolve to one merged member list — a real footgun), so forbid it up
|
||||
front on create + rename rather than mirror the mess."""
|
||||
dup = (
|
||||
db.execute(
|
||||
select(PlexCollection).where(
|
||||
PlexCollection.library_id == library_id,
|
||||
func.lower(PlexCollection.title) == title.lower(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
dup = queries.collection_named(db, library_id, title)
|
||||
if dup is not None and dup.rating_key != exclude_rk:
|
||||
# Structured, so the bilingual UI says it in the reader's language (it was toasted in
|
||||
# Hungarian beside an English error dialog saying the same thing).
|
||||
@@ -441,15 +242,13 @@ def create_collection(
|
||||
seed = body.item_rating_key
|
||||
if not title or not seed:
|
||||
raise refused(400, "collection_fields_required")
|
||||
seed_show = by_rating_key(db, PlexShow, seed)
|
||||
seed_show = queries.show(db, seed)
|
||||
if library:
|
||||
lib = (
|
||||
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == library)).scalars().first()
|
||||
)
|
||||
lib = queries.library_by_key(db, library)
|
||||
else:
|
||||
# The "New collection" flow only knows the seed title → derive its library.
|
||||
seed_item = by_rating_key(db, PlexItem, seed) or seed_show
|
||||
lib = db.get(PlexLibrary, seed_item.library_id) if seed_item else None
|
||||
seed_item = queries.item(db, seed) or seed_show
|
||||
lib = queries.library(db, seed_item.library_id) if seed_item else None
|
||||
if lib is None:
|
||||
raise not_found("library")
|
||||
_reject_duplicate_name(db, lib.id, title)
|
||||
@@ -462,28 +261,16 @@ def create_collection(
|
||||
wrote = True
|
||||
new_rk = str(new.get("ratingKey") or "")
|
||||
col = plex_sync.resync_collection(db, plex, lib, new_rk)
|
||||
col.editable = True
|
||||
service.set_collection_editable(col, True)
|
||||
db.commit()
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
if wrote and new_rk:
|
||||
# It is on Plex now, and only the follow-up read failed. Its row is kept, editable: the
|
||||
# full sync preserves `editable` but creates a missing row read-only, so the collection
|
||||
# would drop out of every "add to" list; and the duplicate-name check reads these rows,
|
||||
# so a retry of the same name would make a second one on Plex. Plex made it with exactly
|
||||
# the seed in it, so the seed's row says so too — otherwise the title showed unticked in
|
||||
# the collection it was just added to, and the collection empty, until that sync.
|
||||
db.add(
|
||||
PlexCollection(
|
||||
rating_key=new_rk, library_id=lib.id, title=title, editable=True, child_count=1
|
||||
)
|
||||
)
|
||||
Model = PlexItem if lib.kind == "movie" else PlexShow
|
||||
if row := by_rating_key(db, Model, seed):
|
||||
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), new_rk]))
|
||||
# It is on Plex now, and only the follow-up read failed: keep its row (see the service).
|
||||
service.keep_created_collection(db, lib, new_rk, title, seed)
|
||||
db.commit()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.post("/collections/{rating_key}/items/{item_rating_key}")
|
||||
@@ -494,7 +281,7 @@ def collection_add_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -505,7 +292,7 @@ def collection_add_item(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.delete("/collections/{rating_key}/items/{item_rating_key}")
|
||||
@@ -516,7 +303,7 @@ def collection_remove_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -527,7 +314,7 @@ def collection_remove_item(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
class CollectionRenameIn(BaseModel):
|
||||
@@ -546,7 +333,7 @@ def collection_rename(
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
_reject_duplicate_name(db, col.library_id, title, exclude_rk=rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
wrote = False
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
@@ -557,7 +344,7 @@ def collection_rename(
|
||||
except (PlexError, PlexNotConfigured) as e:
|
||||
db.rollback()
|
||||
raise _write_or_pending_error(wrote, e) from e
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
|
||||
@router.delete("/collections/{rating_key}")
|
||||
@@ -565,7 +352,7 @@ def collection_delete(
|
||||
rating_key: str, _: User = Depends(admin_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
col = _editable_collection_or_403(db, rating_key)
|
||||
lib = db.get(PlexLibrary, col.library_id)
|
||||
lib = queries.library(db, col.library_id)
|
||||
try:
|
||||
with PlexClient(db) as plex:
|
||||
plex.delete_collection(rating_key)
|
||||
@@ -590,12 +377,12 @@ def collection_set_editable(
|
||||
) -> dict:
|
||||
"""Mark an existing Plex collection as 'mine to edit' (or unmark). Smart / auto-list collections
|
||||
(IMDb/TMDb/…) can never be made editable — they're regenerated outside Plex."""
|
||||
col = by_rating_key(db, PlexCollection, rating_key)
|
||||
col = queries.collection(db, rating_key)
|
||||
if col is None:
|
||||
raise not_found("collection")
|
||||
want = body.editable
|
||||
if want and (col.smart or _collection_source(col) != "collection"):
|
||||
if want and (col.smart or collection_source(col) != "collection"):
|
||||
raise refused(400, "collection_smart")
|
||||
col.editable = want
|
||||
service.set_collection_editable(col, want)
|
||||
db.commit()
|
||||
return _collection_card(col)
|
||||
return collection_card(col)
|
||||
|
||||
@@ -4,52 +4,34 @@ import re
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated, sorted_words
|
||||
from app.collation import sorted_words
|
||||
from app import logs
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.discover import queries
|
||||
from app.models import (
|
||||
PlexCollection,
|
||||
PlexItem,
|
||||
PlexLibrary,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.discover import queries as discover_queries
|
||||
from app.models import User
|
||||
from app.plex import detail as plex_detail
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import rollup, subtitles
|
||||
from app.plex.query import title_col
|
||||
from app.plex import queries, rollup, service, subtitles
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.collections import collection_source
|
||||
from app.plex.cards import SHOW_NEW, episode_card, movie_card, show_card
|
||||
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes.plex._shared import _norm_lang, item_or_404
|
||||
from app.routes.plex.collections import _collection_source
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem, PlexSeason
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def _lib_key(db: Session, library_id: int | None) -> str | None:
|
||||
"""The Plex section key for a library id — the info/show page passes it to the admin collection
|
||||
editor (in the unified cross-library view there's no single library prop otherwise)."""
|
||||
if library_id is None:
|
||||
return None
|
||||
lib = db.get(PlexLibrary, library_id)
|
||||
return lib.plex_key if lib else None
|
||||
|
||||
|
||||
_SHOW_ENRICH_TTL_S = 300
|
||||
|
||||
_show_enrich_cache: dict[str, tuple[float, dict, list]] = {}
|
||||
@@ -113,39 +95,12 @@ def show_detail(
|
||||
"""A show's seasons + episodes (the drill-down view) with per-user watch state."""
|
||||
lang = _norm_lang(lang)
|
||||
hu = lang == "hu"
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
seasons = (
|
||||
db.execute(
|
||||
select(PlexSeason)
|
||||
.where(PlexSeason.show_id == sh.id)
|
||||
.order_by(PlexSeason.season_number.asc().nullsfirst())
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(PlexItem.show_id == sh.id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.item_id.in_([e.id for e in eps] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
seasons = queries.seasons(db, sh)
|
||||
eps = queries.episodes(db, sh)
|
||||
states = rollup.item_state_map(db, user, [e.id for e in eps])
|
||||
by_season: dict[int | None, list] = {}
|
||||
all_cards: list[dict] = [] # every episode, in season/episode order → show-level rollup
|
||||
for e in eps:
|
||||
@@ -163,9 +118,7 @@ def show_detail(
|
||||
meta, related_raw = _show_enrichment(db, sh.rating_key)
|
||||
related = _relatedshow_cards(db, user.id, related_raw, exclude=sh.id, lang=lang)
|
||||
if sh.people_enriched_at is None and meta:
|
||||
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
|
||||
plex_detail.apply_rich_detail(sh, meta) # imdb/tagline/full_cast for future views
|
||||
sh.people_enriched_at = datetime.now(timezone.utc)
|
||||
service.enrich_show(db, sh, meta) # imdb/tagline/full_cast for future views
|
||||
try:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — a failed enrich must not fail the page
|
||||
@@ -198,7 +151,7 @@ def show_detail(
|
||||
"year": sh.year,
|
||||
"thumb": f"/api/plex/image/{sh.rating_key}",
|
||||
"art": f"/api/plex/image/{sh.rating_key}?variant=art",
|
||||
"library": _lib_key(db, sh.library_id),
|
||||
"library": queries.library_key(db, sh.library_id),
|
||||
"content_rating": sh.content_rating,
|
||||
"rating": sh.rating,
|
||||
"genres": (sh.genres_hu or sh.genres if hu else sh.genres) or [],
|
||||
@@ -234,13 +187,13 @@ def show_next(
|
||||
Asked one show at a time, when a card is hovered or played, rather than carried on every card of a
|
||||
grid: three probes a show would cost a 60-card page 180 queries for the few cards anyone plays.
|
||||
`next` is null for a show with no episodes."""
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
return {"next": queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
|
||||
return {"next": discover_queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
|
||||
|
||||
|
||||
def _season_block(se: PlexSeason, cards: list[dict]) -> dict:
|
||||
def _season_block(se: "PlexSeason", cards: list[dict]) -> dict:
|
||||
"""A season's card for the show page (with its aggregate rollup) plus its episodes (used by the
|
||||
season subpage, read from the same cached show-detail payload)."""
|
||||
roll = rollup.rollup(cards)
|
||||
@@ -265,12 +218,7 @@ def _relatedshow_cards(
|
||||
rks = [str(m.get("ratingKey")) for m in related if m.get("ratingKey")]
|
||||
if not rks:
|
||||
return []
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks[:60]), PlexShow.id != exclude)
|
||||
).scalars()
|
||||
}
|
||||
shows = queries.shows_by_keys(db, rks[:60], exclude)
|
||||
statuses = rollup.state_map(db, user_id, [s.id for s in shows.values()])
|
||||
out: list[dict] = []
|
||||
seen: set[str] = set()
|
||||
@@ -285,73 +233,36 @@ def _relatedshow_cards(
|
||||
return out
|
||||
|
||||
|
||||
def _collection_strips(db: Session, it: PlexItem, user_id: int, lang: str) -> list[dict]:
|
||||
def _collection_strips(db: Session, it: "PlexItem", user_id: int, lang: str) -> list[dict]:
|
||||
"""The collections this movie belongs to, each with its member titles (playable cards) — the
|
||||
'Avatar Collection' strip on the info page. Pure local lookup (no Plex call)."""
|
||||
keys = it.collection_keys or []
|
||||
if not keys or it.kind != "movie":
|
||||
return []
|
||||
cols = {
|
||||
c.rating_key: c
|
||||
for c in db.execute(
|
||||
select(PlexCollection).where(PlexCollection.rating_key.in_(keys))
|
||||
).scalars()
|
||||
}
|
||||
cols = {c.rating_key: c for c in queries.collections_by_keys(db, keys)}
|
||||
# Smallest (most specific — e.g. a franchise) first; the big auto-lists (IMDb/TMDb Popular) last.
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, [c.title for c in cols.values()], lang))}
|
||||
ordered = sorted(cols.values(), key=lambda c: (c.child_count or 0, alpha[c.title]))
|
||||
out = []
|
||||
for col in ordered:
|
||||
rk = col.rating_key
|
||||
members = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.library_id == it.library_id,
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.collection_keys.contains([rk]),
|
||||
)
|
||||
.order_by(
|
||||
PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang)
|
||||
)
|
||||
.limit(60)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
ids = [m.id for m in members]
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user_id, PlexState.item_id.in_(ids or [0])
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
members = queries.library_movies_in(db, it.library_id, rk, lang, limit=60)
|
||||
states = rollup.item_state_map(db, user_id, [m.id for m in members])
|
||||
out.append(
|
||||
{
|
||||
"id": rk,
|
||||
"title": col.title,
|
||||
"source": _collection_source(col),
|
||||
"source": collection_source(col),
|
||||
"items": [movie_card(m, states.get(m.id), lang=lang) for m in members],
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _episode_nav(db: Session, it: PlexItem) -> tuple[str | None, str | None]:
|
||||
def _episode_nav(db: Session, it: "PlexItem") -> tuple[str | None, str | None]:
|
||||
if it.kind != "episode" or it.show_id is None:
|
||||
return None, None
|
||||
keys = list(
|
||||
db.execute(
|
||||
select(PlexItem.rating_key)
|
||||
.where(PlexItem.show_id == it.show_id, PlexItem.kind == "episode")
|
||||
.order_by(
|
||||
PlexItem.season_number.asc().nullsfirst(),
|
||||
PlexItem.episode_number.asc().nullsfirst(),
|
||||
)
|
||||
).scalars()
|
||||
)
|
||||
keys = queries.episode_keys(db, it.show_id)
|
||||
try:
|
||||
i = keys.index(it.rating_key)
|
||||
except ValueError:
|
||||
@@ -398,13 +309,7 @@ def item_detail(
|
||||
prev/next, and the per-user resume position + watch status."""
|
||||
lang = _norm_lang(lang)
|
||||
it = item_or_404(db, rating_key)
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
|
||||
# Markers + audio/subtitle streams are file-specific, so they stay LIVE; the rich detail (cast/IMDb/
|
||||
# tagline/genres) is served from the DB row for movies (persisted below / by the backfill) and only
|
||||
@@ -485,11 +390,7 @@ def item_detail(
|
||||
# only stored the listing's top ~3), so fill a not-yet-enriched movie — its cast becomes filterable
|
||||
# and future views serve from the row. Episodes aren't enriched; a show's cast is filled from /show.
|
||||
if it.kind == "movie" and it.people_enriched_at is None and meta:
|
||||
cast = plex_people.cast_from_meta(meta)
|
||||
if cast:
|
||||
plex_people.enrich_row_people(db, it, cast)
|
||||
plex_detail.apply_rich_detail(it, meta)
|
||||
it.people_enriched_at = datetime.now(timezone.utc)
|
||||
service.enrich_movie(db, it, meta)
|
||||
try:
|
||||
db.commit()
|
||||
except Exception: # noqa: BLE001 — a failed enrich must not fail the page
|
||||
@@ -522,8 +423,8 @@ def item_detail(
|
||||
rich = plex_detail.rich_detail(meta)
|
||||
|
||||
prev_id, next_id = _episode_nav(db, it)
|
||||
show = db.get(PlexShow, it.show_id) if it.kind == "episode" and it.show_id else None
|
||||
season = db.get(PlexSeason, it.season_id) if show and it.season_id else None
|
||||
show = queries.show_by_id(db, it.show_id) if it.kind == "episode" and it.show_id else None
|
||||
season = queries.season_by_id(db, it.season_id) if show and it.season_id else None
|
||||
return {
|
||||
"id": it.rating_key,
|
||||
"kind": it.kind,
|
||||
@@ -538,7 +439,7 @@ def item_detail(
|
||||
"video_bit_depth": it.video_bit_depth,
|
||||
"thumb": f"/api/plex/image/{it.rating_key}",
|
||||
"art": f"/api/plex/image/{it.rating_key}?variant=art",
|
||||
"library": _lib_key(db, it.library_id),
|
||||
"library": queries.library_key(db, it.library_id),
|
||||
"cast": rich.get("cast", []),
|
||||
"imdb_rating": rich.get("imdb_rating"),
|
||||
"imdb_id": rich.get("imdb_id"),
|
||||
|
||||
@@ -1,391 +1,18 @@
|
||||
"""Plex Discover: the rail-surface read-model.
|
||||
|
||||
A consolidated payload for the dedicated Discover landing — a stack of horizontal rails:
|
||||
Continue watching · Recently added · Browse-by-genre tiles · Because you watched X — plus a
|
||||
separate, reshuffling Surprise-me sample. Serve-from-DB only: the recommendation rail reads the
|
||||
recs the TMDB engine already persisted (`PlexItem.recommendations`), never a live TMDB call, and
|
||||
every card comes from the same `movie_card`/`show_card` serializers the grid uses (so rails inherit
|
||||
their `art` + rating/summary for free). The rails carry EN metadata (like the other non-grid
|
||||
card paths); the HU overlay coalesces only in the main grid.
|
||||
|
||||
The sidebar metadata filters (genre / rating / year / collection / …) narrow EVERY rail too, so the
|
||||
Discover surface behaves like the Browse grid — the same `LibraryFilters` / `meta_filter_conds` the
|
||||
grid uses, applied to each rail's movie + show query (a rail may legitimately come back empty, which
|
||||
the client hides). Genre-language matches the grid; duration is movie-only.
|
||||
"""
|
||||
"""Plex Discover: the rail-surface landing and its Surprise-me draw (`plex.discover`)."""
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import sorted_words
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import PlexItem, PlexShow, PlexState, User
|
||||
from app.plex import rollup
|
||||
from app.plex.cards import (
|
||||
SHOW_NEW,
|
||||
display_title,
|
||||
leaf_card,
|
||||
movie_card,
|
||||
show_card,
|
||||
)
|
||||
from app.plex.query import (
|
||||
LibraryFilters,
|
||||
apply_meta_filters,
|
||||
apply_show_state_filter,
|
||||
content_libs,
|
||||
genres_col,
|
||||
meta_filter_conds,
|
||||
top_art_keys,
|
||||
)
|
||||
from app.discover.queries import RAIL_LIMIT
|
||||
from app.plex.tmdb import resolve_recs
|
||||
from app.models import User
|
||||
from app.plex import discover as plex_discover
|
||||
from app.plex.discover import SURPRISE_PER_KIND
|
||||
from app.plex.query import LibraryFilters
|
||||
from app.routes.plex._shared import _norm_lang
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
GENRE_TILES = 12
|
||||
SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows (6 total)
|
||||
|
||||
|
||||
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
|
||||
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
|
||||
movie_ids, show_ids = content_libs(db)
|
||||
return (
|
||||
movie_ids if scope in ("movie", "both") else [],
|
||||
show_ids if scope in ("show", "both") else [],
|
||||
)
|
||||
|
||||
|
||||
def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
|
||||
"""Apply the shared metadata filters + movie-only duration to a movie query (same as the grid)."""
|
||||
q = apply_meta_filters(q, PlexItem, p, lang)
|
||||
if f.duration_min is not None:
|
||||
q = q.where(PlexItem.duration_s >= f.duration_min)
|
||||
if f.duration_max is not None:
|
||||
q = q.where(PlexItem.duration_s <= f.duration_max)
|
||||
return q
|
||||
|
||||
|
||||
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
|
||||
sts = rollup.item_state_map(db, uid, [it.id for it in items])
|
||||
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
|
||||
|
||||
|
||||
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
|
||||
statuses = rollup.state_map(db, uid, [s.id for s in shows])
|
||||
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
|
||||
|
||||
|
||||
def _recently_added(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Newest movies + shows, interleaved by `added_at` desc into one rail."""
|
||||
movies = (
|
||||
db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie", PlexItem.library_id.in_(movie_ids)
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
.order_by(PlexItem.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids)), PlexShow, p, lang
|
||||
)
|
||||
.order_by(PlexShow.added_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
|
||||
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — a
|
||||
# review found this one still on the older `(is not None, value)` form, i.e. one rule in two
|
||||
# spellings, with the two comments explaining it differently.
|
||||
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
|
||||
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
|
||||
top = tagged[:RAIL_LIMIT]
|
||||
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
|
||||
# share one), so the merge doesn't depend on object identity staying alive.
|
||||
top_movies = [o for _, kind, o in top if kind == "movie"]
|
||||
top_shows = [o for _, kind, o in top if kind == "show"]
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang), strict=True)
|
||||
}
|
||||
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
|
||||
|
||||
|
||||
def _recently_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Recently watched rail: the titles the user last FULLY watched, newest first. A movie is its own
|
||||
card; a show contributes the specific EPISODE watched (carrying its show title). Movies honour the
|
||||
sidebar filters (like the other rails); episodes carry no filterable metadata, so they don't."""
|
||||
rows: list[tuple[object, dict]] = [] # (watched_at, card)
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
)
|
||||
for it, st in db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all():
|
||||
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
|
||||
if show_ids:
|
||||
eq = (
|
||||
select(PlexItem, PlexState, PlexShow)
|
||||
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
|
||||
.join(PlexShow, PlexShow.id == PlexItem.show_id)
|
||||
.where(
|
||||
PlexItem.kind == "episode",
|
||||
PlexItem.library_id.in_(show_ids),
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
)
|
||||
for ep, st, sh in db.execute(eq).all():
|
||||
card = leaf_card(
|
||||
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
|
||||
)
|
||||
rows.append((st.watched_at, card))
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _continue_watching(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> list[dict]:
|
||||
"""Resume rail: in-progress movies + partially-watched shows, most-recently-progressed first.
|
||||
A movie's resume time is its own `progress_updated_at`; a show's is the max across its episodes."""
|
||||
rows: list[tuple[object, dict]] = [] # (resumed_at, card)
|
||||
|
||||
if movie_ids:
|
||||
mq = (
|
||||
select(PlexItem, PlexState)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.position_seconds > 0,
|
||||
# A NULL status is still "started" (mirrors apply_movie_state_filter's in_progress) —
|
||||
# `notin_` alone would drop it, since `NULL NOT IN (...)` is NULL, not TRUE.
|
||||
or_(
|
||||
PlexState.status.is_(None),
|
||||
PlexState.status.notin_(["watched", "hidden"]),
|
||||
),
|
||||
)
|
||||
)
|
||||
mrows = db.execute(
|
||||
_movie_meta(mq, p, lang, f)
|
||||
.order_by(PlexState.progress_updated_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for it, st in mrows:
|
||||
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
|
||||
|
||||
if show_ids:
|
||||
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
|
||||
# its `resumed_at` = max(progress_updated_at); the in-progress predicate comes from the same shared
|
||||
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
|
||||
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
|
||||
sq = apply_show_state_filter(
|
||||
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
|
||||
prog, prog.c.show_id == PlexShow.id
|
||||
),
|
||||
prog.c.total,
|
||||
prog.c.watched,
|
||||
prog.c.inprog,
|
||||
"in_progress",
|
||||
).where(PlexShow.library_id.in_(show_ids))
|
||||
srows = db.execute(
|
||||
apply_meta_filters(sq, PlexShow, p, lang)
|
||||
.order_by(prog.c.resumed_at.desc().nullslast())
|
||||
.limit(RAIL_LIMIT)
|
||||
).all()
|
||||
for sh, resumed_at, total, watched, inprog in srows:
|
||||
state = rollup.show_state(total, watched, inprog)
|
||||
rows.append((resumed_at, show_card(sh, state, lang=lang)))
|
||||
|
||||
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
|
||||
return [card for _, card in rows[:RAIL_LIMIT]]
|
||||
|
||||
|
||||
def _genre_tiles(
|
||||
db: Session, movie_ids: list[int], show_ids: list[int], p: dict, lang: str
|
||||
) -> list[dict]:
|
||||
"""The most-common genres across the scope, as {value, count} tiles → tap to filter the grid.
|
||||
The genre vocabulary is language-specific (coalesce(genres_hu, genres) in HU), matching the grid.
|
||||
The active metadata filters narrow the counts too, so the tiles agree with the filtered rails."""
|
||||
counts: dict[str, int] = {}
|
||||
for model, ids in ((PlexItem, movie_ids), (PlexShow, show_ids)):
|
||||
if not ids:
|
||||
continue
|
||||
gcol = genres_col(model, lang)
|
||||
base = select(func.jsonb_array_elements_text(gcol).label("g")).where(
|
||||
model.library_id.in_(ids),
|
||||
func.jsonb_typeof(gcol) == "array",
|
||||
*meta_filter_conds(model, p, lang),
|
||||
)
|
||||
if model is PlexItem:
|
||||
base = base.where(model.kind == "movie")
|
||||
expanded = base.subquery()
|
||||
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
|
||||
counts[g] = counts.get(g, 0) + c
|
||||
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
|
||||
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
|
||||
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
|
||||
art = top_art_keys(
|
||||
db,
|
||||
PlexItem,
|
||||
movie_ids,
|
||||
genres_col(PlexItem, lang),
|
||||
[g for g, _ in top],
|
||||
extra_conds=(PlexItem.kind == "movie", *meta_filter_conds(PlexItem, p, lang)),
|
||||
)
|
||||
return [{"value": g, "count": c, "art": art.get(g, [])} for g, c in top]
|
||||
|
||||
|
||||
def _because_you_watched(
|
||||
db: Session,
|
||||
uid: int,
|
||||
movie_ids: list[int],
|
||||
show_ids: list[int],
|
||||
p: dict,
|
||||
lang: str,
|
||||
f: LibraryFilters,
|
||||
) -> dict | None:
|
||||
"""Recommendations anchored on the user's most-recently-watched, TMDB-enriched movie. Only the
|
||||
recs that resolve to OUR library (by guid) become cards — the rest aren't actionable here. Returns
|
||||
None (rail hidden) when there's no enriched watch history or none of its recs are in the library.
|
||||
The metadata filters narrow the resolved recs (not the anchor pick) so the rail matches the grid."""
|
||||
if not movie_ids:
|
||||
return None
|
||||
anchor = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.join(
|
||||
PlexState,
|
||||
and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid),
|
||||
)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexState.status == "watched",
|
||||
PlexItem.recommendations.isnot(None),
|
||||
)
|
||||
.order_by(PlexState.watched_at.desc().nullslast())
|
||||
.limit(1)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if anchor is None:
|
||||
return None
|
||||
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
|
||||
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
|
||||
rks = [
|
||||
r["rating_key"]
|
||||
for r in resolved
|
||||
if r.get("in_library") and r.get("rating_key") != anchor.rating_key
|
||||
]
|
||||
if not rks:
|
||||
return None
|
||||
movies = {
|
||||
m.rating_key: m
|
||||
for m in db.execute(
|
||||
_movie_meta(
|
||||
select(PlexItem).where(PlexItem.kind == "movie", PlexItem.rating_key.in_(rks)),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
shows = {
|
||||
s.rating_key: s
|
||||
for s in db.execute(
|
||||
apply_meta_filters(
|
||||
select(PlexShow).where(PlexShow.rating_key.in_(rks)), PlexShow, p, lang
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
ordered_movies = [movies[rk] for rk in rks if rk in movies]
|
||||
ordered_shows = [shows[rk] for rk in rks if rk in shows]
|
||||
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
|
||||
mcards = {
|
||||
m.rating_key: c
|
||||
for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang), strict=True)
|
||||
}
|
||||
scards = {
|
||||
s.rating_key: c
|
||||
for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang), strict=True)
|
||||
}
|
||||
items: list[dict] = []
|
||||
for rk in rks:
|
||||
if rk in mcards:
|
||||
items.append(mcards[rk])
|
||||
elif rk in scards:
|
||||
items.append(scards[rk])
|
||||
if not items:
|
||||
return None
|
||||
return {
|
||||
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/discover")
|
||||
def discover(
|
||||
@@ -398,72 +25,7 @@ def discover(
|
||||
"""The Discover landing payload — every rail in one pass (serve-from-DB), narrowed by the active
|
||||
sidebar filters. Empty rails come back as empty lists (the client hides them); `because_you_watched`
|
||||
is null when there's no anchor."""
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
return {
|
||||
"scope": scope,
|
||||
"continue_watching": _continue_watching(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_watched": _recently_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"recently_added": _recently_added(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
"genres": _genre_tiles(db, movie_ids, show_ids, p, lang),
|
||||
"because_you_watched": _because_you_watched(db, user.id, movie_ids, show_ids, p, lang, f),
|
||||
}
|
||||
|
||||
|
||||
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
|
||||
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
|
||||
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
|
||||
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
|
||||
fresh = (
|
||||
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
|
||||
.scalars()
|
||||
.all()
|
||||
if exclude
|
||||
else []
|
||||
)
|
||||
if len(fresh) == per:
|
||||
return list(fresh)
|
||||
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
|
||||
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
|
||||
return [*fresh, *rest.all()]
|
||||
|
||||
|
||||
def _kept(
|
||||
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
|
||||
) -> list[dict]:
|
||||
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
|
||||
stays in it, marked watched, and one gone from the scope's libraries drops out."""
|
||||
movies = (
|
||||
db.execute(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
PlexItem.rating_key.in_(keep),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if movie_ids
|
||||
else []
|
||||
)
|
||||
shows = (
|
||||
db.execute(
|
||||
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
if show_ids
|
||||
else []
|
||||
)
|
||||
cards = {
|
||||
c["id"]: c
|
||||
for c in [
|
||||
*_movie_cards(db, uid, list(movies), lang),
|
||||
*_show_cards(db, uid, list(shows), lang),
|
||||
]
|
||||
}
|
||||
return [cards[k] for k in keep if k in cards]
|
||||
return plex_discover.discover_page(db, user, scope, _norm_lang(lang), f)
|
||||
|
||||
|
||||
@router.get("/surprise")
|
||||
@@ -485,69 +47,4 @@ def surprise(
|
||||
excluded (the point is to surface something to watch). Capped at SURPRISE_PER_KIND movies +
|
||||
SURPRISE_PER_KIND shows, drawn separately then interleaved so a lopsided library doesn't crowd out
|
||||
one type."""
|
||||
lang = _norm_lang(lang)
|
||||
p = f.as_dict()
|
||||
movie_ids, show_ids = _scope_libs(db, scope)
|
||||
if keep:
|
||||
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
|
||||
per = min(limit, SURPRISE_PER_KIND)
|
||||
movies = []
|
||||
if movie_ids:
|
||||
# Exclude movies this user has watched.
|
||||
watched_movie = (
|
||||
select(PlexState.id)
|
||||
.where(
|
||||
PlexState.item_id == PlexItem.id,
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.status == "watched",
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
movies = _draw(
|
||||
db,
|
||||
_movie_meta(
|
||||
select(PlexItem).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_ids),
|
||||
~watched_movie,
|
||||
),
|
||||
p,
|
||||
lang,
|
||||
f,
|
||||
),
|
||||
PlexItem.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
shows = []
|
||||
if show_ids:
|
||||
# Exclude fully-watched shows (LEFT JOIN so a never-started show — no rollup row — still counts
|
||||
# as unseen). `total IS NULL / == 0 / watched < total` == "not fully watched".
|
||||
prog = rollup.agg_subquery(db, user.id).subquery()
|
||||
not_fully_watched = or_(
|
||||
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
|
||||
)
|
||||
shows = _draw(
|
||||
db,
|
||||
apply_meta_filters(
|
||||
select(PlexShow)
|
||||
.outerjoin(prog, prog.c.show_id == PlexShow.id)
|
||||
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
|
||||
PlexShow,
|
||||
p,
|
||||
lang,
|
||||
),
|
||||
PlexShow.rating_key,
|
||||
per,
|
||||
exclude,
|
||||
)
|
||||
mcards = _movie_cards(db, user.id, movies, lang)
|
||||
scards = _show_cards(db, user.id, shows, lang)
|
||||
# Interleave movie/show cards, then trim to `limit`.
|
||||
items: list[dict] = []
|
||||
for i in range(max(len(mcards), len(scards))):
|
||||
if i < len(mcards):
|
||||
items.append(mcards[i])
|
||||
if i < len(scards):
|
||||
items.append(scards[i])
|
||||
return {"scope": scope, "items": items[:limit]}
|
||||
return plex_discover.surprise(db, user, scope, _norm_lang(lang), limit, exclude, keep, f)
|
||||
|
||||
@@ -4,26 +4,18 @@ import threading
|
||||
import time
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.collation import collated
|
||||
from app import logs, sysconfig
|
||||
from app import sysconfig
|
||||
from app.auth import current_user
|
||||
from app.db import SessionLocal, get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexPerson,
|
||||
PlexShow,
|
||||
User,
|
||||
)
|
||||
from app.plex import rollup
|
||||
from app.db import get_db
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex import tmdb as plex_tmdb
|
||||
from app.plex.cards import SHOW_NEW, movie_card, norm_lang, show_card
|
||||
from app.plex.query import content_libs, split_csv, title_col
|
||||
from app.plex.query import content_libs, split_csv
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
|
||||
# How many "Known for" titles the person page shows (the stored pool is deeper; see plex.tmdb).
|
||||
@@ -80,35 +72,13 @@ def people_cards(
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
|
||||
want_shows = scope in ("show", "both") and bool(show_lib_ids)
|
||||
photos = {
|
||||
p.name: p.photo_thumb
|
||||
for p in db.execute(select(PlexPerson).where(PlexPerson.name.in_(wanted))).scalars()
|
||||
}
|
||||
out = []
|
||||
for nm in wanted:
|
||||
count = 0
|
||||
if want_movies:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexItem.id)).where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
PlexItem.cast_names.contains([nm]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
if want_shows:
|
||||
count += (
|
||||
db.scalar(
|
||||
select(func.count(PlexShow.id)).where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
PlexShow.cast_names.contains([nm]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
out.append({"name": nm, "photo": photos.get(nm), "count": count})
|
||||
photos = queries.person_photos(db, wanted)
|
||||
movies = movie_lib_ids if want_movies else []
|
||||
shows = show_lib_ids if want_shows else []
|
||||
out = [
|
||||
{"name": nm, "photo": photos.get(nm), "count": queries.cast_count(db, nm, movies, shows)}
|
||||
for nm in wanted
|
||||
]
|
||||
return {"people": out}
|
||||
|
||||
|
||||
@@ -144,24 +114,6 @@ def _should_schedule_enrich(person_id: int) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _enrich_person_bg(person_id: int) -> None:
|
||||
"""Fetch + persist one person's TMDB bio OFF the request path — a FastAPI background task that runs
|
||||
AFTER the response is sent, so the outbound TMDB call never sits on the serve latency (the module's
|
||||
"no live TMDB call at serve time" invariant). The request's session is already closed by then, so
|
||||
open a fresh one. Best-effort: a TMDB failure just leaves `tmdb_enriched_at` NULL to retry on the
|
||||
next open, and the bio lands from the row by then."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
person = db.get(PlexPerson, person_id)
|
||||
if person is not None:
|
||||
plex_tmdb.enrich_person(db, person)
|
||||
except Exception: # noqa: BLE001 — background best-effort; never surfaces to a user
|
||||
db.rollback()
|
||||
log.warning("background person bio enrich failed for id=%s", person_id, exc_info=True)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@router.get("/person/{name}")
|
||||
def person_detail(
|
||||
name: str,
|
||||
@@ -183,7 +135,7 @@ def person_detail(
|
||||
fetches everyone ahead of time, so this lazy path is only for a person the job hasn't reached yet.
|
||||
"""
|
||||
lang = norm_lang(lang)
|
||||
person = db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
|
||||
person = queries.person(db, name)
|
||||
# Lazy one-time bio enrich, scheduled AFTER the response so the TMDB call is never on the serve path.
|
||||
# Gated by a per-person cooldown so concurrent/rapid opens (or repeated opens while TMDB is degraded)
|
||||
# can't pile background tasks onto the shared threadpool.
|
||||
@@ -197,53 +149,18 @@ def person_detail(
|
||||
and bool(sysconfig.get_str(db, "tmdb_api_key"))
|
||||
)
|
||||
if enriching and person is not None and _should_schedule_enrich(person.id):
|
||||
background_tasks.add_task(_enrich_person_bg, person.id)
|
||||
background_tasks.add_task(service.enrich_person_bg, person.id)
|
||||
movie_lib_ids, show_lib_ids = content_libs(db)
|
||||
|
||||
movies: list[dict] = []
|
||||
if movie_lib_ids:
|
||||
items = (
|
||||
db.execute(
|
||||
select(PlexItem)
|
||||
.where(
|
||||
PlexItem.kind == "movie",
|
||||
PlexItem.library_id.in_(movie_lib_ids),
|
||||
or_(
|
||||
PlexItem.cast_names.contains([name]),
|
||||
PlexItem.directors.contains([name]),
|
||||
),
|
||||
)
|
||||
.order_by(
|
||||
PlexItem.year.desc().nullslast(), collated(title_col(PlexItem, lang), lang)
|
||||
)
|
||||
.limit(_PERSON_SECTION_MAX)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
items = queries.filmography(db, "movie", name, movie_lib_ids, lang, _PERSON_SECTION_MAX)
|
||||
states = rollup.item_state_map(db, user.id, [it.id for it in items])
|
||||
movies = [movie_card(it, states.get(it.id), lang=lang) for it in items]
|
||||
|
||||
shows: list[dict] = []
|
||||
if show_lib_ids:
|
||||
show_rows = (
|
||||
db.execute(
|
||||
select(PlexShow)
|
||||
.where(
|
||||
PlexShow.library_id.in_(show_lib_ids),
|
||||
or_(
|
||||
PlexShow.cast_names.contains([name]),
|
||||
PlexShow.directors.contains([name]),
|
||||
),
|
||||
)
|
||||
.order_by(
|
||||
PlexShow.year.desc().nullslast(), collated(title_col(PlexShow, lang), lang)
|
||||
)
|
||||
.limit(_PERSON_SECTION_MAX)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
show_rows = queries.filmography(db, "show", name, show_lib_ids, lang, _PERSON_SECTION_MAX)
|
||||
statuses = rollup.state_map(db, user.id, [s.id for s in show_rows])
|
||||
shows = [show_card(sh, statuses.get(sh.id, SHOW_NEW), lang=lang) for sh in show_rows]
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Plex playback: image/subtitle proxies, HLS streaming, and watch-state writes."""
|
||||
|
||||
import threading
|
||||
from typing import TYPE_CHECKING
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import httpx
|
||||
@@ -16,22 +17,13 @@ from fastapi import (
|
||||
)
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user, resolved_user_id
|
||||
from app.auth import current_user, session_user_id
|
||||
from app.db import SessionLocal, get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexSeason,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.plex import imgcache, subtitles
|
||||
from app.models import User
|
||||
from app.plex import imgcache, queries, service, subtitles
|
||||
from app.plex import paths as plex_paths
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.plex import people as plex_people
|
||||
from app.plex import stream as plex_stream
|
||||
from app.plex import watch_sync as plex_watch
|
||||
@@ -40,6 +32,9 @@ from app.routes.plex._shared import item_or_404
|
||||
from app.refusal import refused
|
||||
from app.routes._common import ProgressIn, not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexItem
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
@@ -47,7 +42,7 @@ def _require_session(request: Request) -> int:
|
||||
"""Lightweight auth for the high-fan-out image proxies: validate the signed session cookie WITHOUT
|
||||
a DB user-load, so these endpoints never hold a pool connection for the auth. Poster images are
|
||||
low-sensitivity; a valid session is enough."""
|
||||
uid, _ = resolved_user_id(request)
|
||||
uid = session_user_id(request)
|
||||
if not uid:
|
||||
raise refused(401, "not_authenticated")
|
||||
return uid
|
||||
@@ -280,14 +275,7 @@ def _push_watch(
|
||||
return
|
||||
# Capture the row's freshest timestamp NOW so the background push can tell if a newer local edit
|
||||
# lands before it settles synced_to_plex (which would otherwise strand that newer value).
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user_id, PlexState.item_id == item_id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
marker = plex_watch._state_marker(st)
|
||||
marker = plex_watch._state_marker(queries.state(db, user_id, item_id))
|
||||
if action == "resume" and intent is not None:
|
||||
background.add_task(
|
||||
_push_resume_guarded, user_id, item_id, rating_key, time_ms, duration_ms, marker, intent
|
||||
@@ -305,37 +293,6 @@ def _push_watch(
|
||||
)
|
||||
|
||||
|
||||
def _mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bool:
|
||||
"""Mark the item watched (position cleared) and say whether THIS call made it so. Two near-end
|
||||
checkpoints arrive together routinely (the 10 s tick and the debounced `seeked` flush), and a
|
||||
read-then-write let both see "not watched" and both scrobble. The transition is decided by the
|
||||
write itself: an insert that did not conflict, or an update conditional on the old status —
|
||||
under READ COMMITTED the second writer re-reads the committed row and matches nothing."""
|
||||
values = {
|
||||
"status": "watched",
|
||||
"watched_at": now,
|
||||
"position_seconds": 0,
|
||||
"progress_updated_at": now,
|
||||
"synced_to_plex": False,
|
||||
}
|
||||
inserted = db.execute(
|
||||
pg_insert(PlexState)
|
||||
.values(user_id=user_id, item_id=item_id, **values)
|
||||
.on_conflict_do_nothing(constraint="uq_plex_user_item")
|
||||
.returning(PlexState.id)
|
||||
).first()
|
||||
if inserted is not None:
|
||||
return True
|
||||
mine = (PlexState.user_id == user_id) & (PlexState.item_id == item_id)
|
||||
if db.execute(
|
||||
update(PlexState).where(mine, PlexState.status != "watched").values(values)
|
||||
).rowcount:
|
||||
return True
|
||||
# Already watched: a re-finish still restamps the row, as it always did — just no scrobble.
|
||||
db.execute(update(PlexState).where(mine).values(values))
|
||||
return False
|
||||
|
||||
|
||||
class ItemProgressIn(ProgressIn):
|
||||
"""The shared checkpoint (its bounds included), plus whether the player is leaving the item."""
|
||||
|
||||
@@ -371,7 +328,7 @@ def item_progress(
|
||||
now = datetime.now(timezone.utc)
|
||||
near_end = duration > 0 and position > duration - _FINISH_MARGIN_S
|
||||
if near_end:
|
||||
flipped = _mark_finished(db, user.id, it.id, now)
|
||||
flipped = service.mark_finished(db, user.id, it.id, now)
|
||||
# The finish supersedes every resume push still in flight for this item: stamping it as the
|
||||
# latest push makes `_resume_push_superseded` drop them. Popping the entry instead (to keep
|
||||
# the map small) removed the very guard, and an older resume push could then land at Plex
|
||||
@@ -385,33 +342,17 @@ def item_progress(
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "watched")
|
||||
return {"ok": True, "status": "watched", "position_seconds": 0}
|
||||
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
|
||||
if position < _PROGRESS_MIN_S:
|
||||
if st is not None and st.status == "new":
|
||||
db.delete(st)
|
||||
elif st is not None:
|
||||
st.position_seconds = 0
|
||||
st.progress_updated_at = now
|
||||
service.clear_progress(db, st, now)
|
||||
db.commit()
|
||||
return {"ok": True, "position_seconds": 0}
|
||||
|
||||
if st is None:
|
||||
st = PlexState(user_id=user.id, item_id=it.id)
|
||||
db.add(st)
|
||||
st.position_seconds = position
|
||||
st.progress_updated_at = now
|
||||
push_now = _should_push_resume(user.id, it.id, now, final)
|
||||
if push_now:
|
||||
# Mark dirty before the (background) push so the reconcile knows to re-push if it fails, and so
|
||||
# `push_state_to_plex` can settle synced_to_plex once Plex confirms.
|
||||
st.synced_to_plex = False
|
||||
# Marked dirty before the (background) push so the reconcile knows to re-push if it fails, and so
|
||||
# `push_state_to_plex` can settle synced_to_plex once Plex confirms.
|
||||
service.checkpoint(db, user, it.id, st, position, now, dirty=push_now)
|
||||
db.commit()
|
||||
if push_now:
|
||||
_push_watch(
|
||||
@@ -447,13 +388,7 @@ def item_state(
|
||||
status = body.status
|
||||
if status not in ("new", "watched", "hidden"):
|
||||
raise refused(400, "invalid_status")
|
||||
st = (
|
||||
db.execute(
|
||||
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == it.id)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
st = queries.state(db, user, it.id)
|
||||
prev_status = st.status if st is not None else "new"
|
||||
if status == "new":
|
||||
# Does Plex hold anything for this item that the un-mark has to clear? Two things qualify: a
|
||||
@@ -467,19 +402,12 @@ def item_state(
|
||||
st.status == "watched" or (st.position_seconds or 0) > 0
|
||||
)
|
||||
if st is not None:
|
||||
db.delete(st)
|
||||
service.forget_state(db, st)
|
||||
db.commit()
|
||||
if mirrored_on_plex:
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "unwatched")
|
||||
return {"ok": True, "status": "new"}
|
||||
if st is None:
|
||||
st = PlexState(user_id=user.id, item_id=it.id)
|
||||
db.add(st)
|
||||
st.status = status
|
||||
if status == "watched":
|
||||
st.watched_at = datetime.now(timezone.utc)
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
service.set_status(db, user, it.id, st, status)
|
||||
db.commit()
|
||||
if status == "watched" and prev_status != "watched":
|
||||
_push_watch(background, db, user.id, it.id, rating_key, "watched")
|
||||
@@ -490,44 +418,15 @@ def _bulk_state(
|
||||
background: BackgroundTasks,
|
||||
db: Session,
|
||||
user_id: int,
|
||||
eps: list[PlexItem],
|
||||
eps: "list[PlexItem]",
|
||||
watched: bool,
|
||||
) -> int:
|
||||
"""Mark every episode in `eps` watched (or unwatched) for a user, mirroring each change to a
|
||||
linked Plex account in the background (best-effort). Returns how many rows actually changed."""
|
||||
now = datetime.now(timezone.utc)
|
||||
existing = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user_id,
|
||||
PlexState.item_id.in_([e.id for e in eps] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
linked Plex account in the background (best-effort). Returns how many rows actually changed.
|
||||
The handler commits before the response — which is when the background push runs, so the push
|
||||
always reads the committed marks."""
|
||||
pushable = plex_watch.link_for_push(db, user_id) is not None
|
||||
to_push: list[tuple[int, str, str]] = []
|
||||
for e in eps:
|
||||
st = existing.get(e.id)
|
||||
prev = st.status if st is not None else "new"
|
||||
if watched:
|
||||
if st is None:
|
||||
st = PlexState(user_id=user_id, item_id=e.id)
|
||||
db.add(st)
|
||||
st.status = "watched"
|
||||
st.watched_at = now
|
||||
st.position_seconds = 0
|
||||
st.synced_to_plex = False
|
||||
if prev != "watched":
|
||||
to_push.append((e.id, e.rating_key, "watched"))
|
||||
else: # → new: drop any state (but keep the user's private "hidden" marks)
|
||||
if st is not None and prev != "hidden":
|
||||
db.delete(st)
|
||||
if prev != "new":
|
||||
to_push.append((e.id, e.rating_key, "unwatched"))
|
||||
# Flush, not commit: the handler commits, and it does so before the response — which is when
|
||||
# the background push below runs, so the push always reads the committed marks.
|
||||
db.flush()
|
||||
to_push = service.bulk_state(db, user_id, eps, watched)
|
||||
if pushable and to_push:
|
||||
# One coalesced background task for the whole batch (reuses a single session + Plex client),
|
||||
# not one task per episode — a 200-ep show mark is 1 task, not 200.
|
||||
@@ -548,15 +447,11 @@ def show_state(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Mark a whole show watched/unwatched (all its episodes) for the current user + push to Plex."""
|
||||
sh = by_rating_key(db, PlexShow, rating_key)
|
||||
sh = queries.show(db, rating_key)
|
||||
if sh is None:
|
||||
raise not_found("show")
|
||||
watched = body.watched
|
||||
eps = (
|
||||
db.execute(select(PlexItem).where(PlexItem.show_id == sh.id, PlexItem.kind == "episode"))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = queries.episodes(db, sh)
|
||||
changed = _bulk_state(background, db, user.id, eps, watched)
|
||||
db.commit()
|
||||
return {"ok": True, "changed": changed, "watched": watched}
|
||||
@@ -571,15 +466,11 @@ def season_state(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Mark a whole season watched/unwatched (all its episodes) for the current user + push to Plex."""
|
||||
se = by_rating_key(db, PlexSeason, rating_key)
|
||||
se = queries.season(db, rating_key)
|
||||
if se is None:
|
||||
raise not_found("season")
|
||||
watched = body.watched
|
||||
eps = (
|
||||
db.execute(select(PlexItem).where(PlexItem.season_id == se.id, PlexItem.kind == "episode"))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
eps = queries.season_episodes(db, se)
|
||||
changed = _bulk_state(background, db, user.id, eps, watched)
|
||||
db.commit()
|
||||
return {"ok": True, "changed": changed, "watched": watched}
|
||||
|
||||
@@ -1,33 +1,26 @@
|
||||
"""Plex playlists: the current user's personal, orderable playlists of playable items."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING, Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
PlexItem,
|
||||
PlexPlaylist,
|
||||
PlexPlaylistItem,
|
||||
PlexShow,
|
||||
PlexState,
|
||||
User,
|
||||
)
|
||||
from app.models import User
|
||||
from app.plex import queries, rollup, service
|
||||
from app.plex.cards import display_title, leaf_card, norm_lang
|
||||
from app.plex.lookups import by_rating_key
|
||||
from app.routes._common import owned_or_404, not_found
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.plex.models import PlexPlaylist
|
||||
|
||||
router = APIRouter(prefix="/api/plex", tags=["plex"])
|
||||
|
||||
|
||||
def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
|
||||
def _playlist_card(pl: "PlexPlaylist", count: int, thumb_rk: str | None) -> dict:
|
||||
return {
|
||||
"id": pl.id,
|
||||
"title": pl.title,
|
||||
@@ -36,13 +29,15 @@ def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _playlist_item_query(pid: int):
|
||||
return (
|
||||
select(PlexPlaylistItem, PlexItem)
|
||||
.join(PlexItem, PlexItem.id == PlexPlaylistItem.item_id)
|
||||
.where(PlexPlaylistItem.playlist_id == pid)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
)
|
||||
def _card_of(db: Session, pl: "PlexPlaylist") -> dict:
|
||||
return _playlist_card(pl, queries.playlist_count(db, pl.id), queries.playlist_cover(db, pl.id))
|
||||
|
||||
|
||||
def _own_playlist(db: Session, user: User, pid: int, *, lock: bool = False) -> "PlexPlaylist":
|
||||
pl = queries.own_playlist(db, user, pid, lock=lock)
|
||||
if pl is None:
|
||||
raise not_found("playlist")
|
||||
return pl
|
||||
|
||||
|
||||
@router.get("/playlists")
|
||||
@@ -56,49 +51,17 @@ def playlists(
|
||||
`contains`=an item rating_key adds `has_item` per playlist (for the single 'add to playlist' dialog).
|
||||
`contains_group`=a comma-separated list of rating_keys adds `group_in` per playlist (how many of the
|
||||
group it already holds) + a top-level `group_size`, for the 'add a whole season/show' dialog."""
|
||||
contains_id = None
|
||||
if contains:
|
||||
contains_id = db.scalar(select(PlexItem.id).where(PlexItem.rating_key == str(contains)))
|
||||
contains_id = queries.item_id(db, contains) if contains else None
|
||||
group_ids: set[int] | None = None
|
||||
if contains_group is not None:
|
||||
rks = [s for s in (contains_group or "").split(",") if s]
|
||||
group_ids = set(
|
||||
db.execute(select(PlexItem.id).where(PlexItem.rating_key.in_(rks or [""]))).scalars()
|
||||
)
|
||||
group_ids = queries.item_ids(db, [s for s in (contains_group or "").split(",") if s])
|
||||
out = []
|
||||
for pl in db.execute(
|
||||
select(PlexPlaylist)
|
||||
.where(PlexPlaylist.user_id == user.id)
|
||||
.order_by(PlexPlaylist.updated_at.desc())
|
||||
).scalars():
|
||||
count = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
first = db.execute(_playlist_item_query(pl.id)).first()
|
||||
card = _playlist_card(pl, count, first[1].rating_key if first else None)
|
||||
for pl in queries.playlists_of(db, user):
|
||||
card = _card_of(db, pl)
|
||||
if contains_id is not None:
|
||||
card["has_item"] = (
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.id).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id,
|
||||
PlexPlaylistItem.item_id == contains_id,
|
||||
)
|
||||
).first()
|
||||
is not None
|
||||
)
|
||||
card["has_item"] = queries.playlist_holds(db, pl.id, {contains_id}) > 0
|
||||
if group_ids is not None:
|
||||
card["group_in"] = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(
|
||||
PlexPlaylistItem.playlist_id == pl.id,
|
||||
PlexPlaylistItem.item_id.in_(group_ids or [0]),
|
||||
)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
card["group_in"] = queries.playlist_holds(db, pl.id, group_ids)
|
||||
out.append(card)
|
||||
resp: dict = {"playlists": out}
|
||||
if group_ids is not None:
|
||||
@@ -120,19 +83,10 @@ def create_playlist(
|
||||
title = body.title.strip()
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
pl = PlexPlaylist(user_id=user.id, title=title)
|
||||
db.add(pl)
|
||||
db.flush()
|
||||
seed = body.item_rating_key
|
||||
count = 0
|
||||
thumb_rk = None
|
||||
if seed:
|
||||
it = by_rating_key(db, PlexItem, seed)
|
||||
if it is not None:
|
||||
db.add(PlexPlaylistItem(playlist_id=pl.id, item_id=it.id, position=0))
|
||||
count, thumb_rk = 1, it.rating_key
|
||||
seed = queries.item(db, body.item_rating_key) if body.item_rating_key else None
|
||||
pl = service.create_playlist(db, user, title, seed)
|
||||
db.commit()
|
||||
return _playlist_card(pl, count, thumb_rk)
|
||||
return _playlist_card(pl, 1 if seed else 0, seed.rating_key if seed else None)
|
||||
|
||||
|
||||
@router.get("/playlists/{pid}")
|
||||
@@ -143,26 +97,10 @@ def playlist_detail(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
lang = norm_lang(lang)
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
rows = db.execute(_playlist_item_query(pid)).all()
|
||||
items = [it for (_pi, it) in rows]
|
||||
states = {
|
||||
s.item_id: s
|
||||
for s in db.execute(
|
||||
select(PlexState).where(
|
||||
PlexState.user_id == user.id,
|
||||
PlexState.item_id.in_([it.id for it in items] or [0]),
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
shows = {
|
||||
sh.id: sh
|
||||
for sh in db.execute(
|
||||
select(PlexShow).where(
|
||||
PlexShow.id.in_([it.show_id for it in items if it.show_id] or [0])
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
pl = _own_playlist(db, user, pid)
|
||||
items = queries.playlist_items(db, pl.id)
|
||||
states = rollup.item_state_map(db, user, [it.id for it in items])
|
||||
shows = queries.shows_by_ids(db, {it.show_id for it in items if it.show_id})
|
||||
cards = [
|
||||
leaf_card(
|
||||
it,
|
||||
@@ -187,28 +125,21 @@ def rename_playlist(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
pl = _own_playlist(db, user, pid)
|
||||
title = body.title.strip()
|
||||
if not title:
|
||||
raise refused(400, "title_required")
|
||||
pl.title = title
|
||||
service.rename_playlist(pl, title)
|
||||
db.commit()
|
||||
count = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pl.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
first = db.execute(_playlist_item_query(pl.id)).first()
|
||||
return _playlist_card(pl, count, first[1].rating_key if first else None)
|
||||
return _playlist_card(pl, queries.playlist_count(db, pl.id), queries.playlist_cover(db, pl.id))
|
||||
|
||||
|
||||
@router.delete("/playlists/{pid}")
|
||||
def delete_playlist(
|
||||
pid: int, user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
) -> dict:
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
|
||||
db.delete(pl)
|
||||
pl = _own_playlist(db, user, pid)
|
||||
service.delete_playlist(db, pl)
|
||||
db.commit()
|
||||
return {"deleted": pid}
|
||||
|
||||
@@ -225,27 +156,11 @@ def playlist_add_item(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# Locked (like every add / reorder here): two adds otherwise read the same max position.
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
it = by_rating_key(db, PlexItem, body.item_rating_key)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
it = queries.item(db, body.item_rating_key)
|
||||
if it is None:
|
||||
raise not_found("item")
|
||||
exists = (
|
||||
db.execute(
|
||||
select(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.first()
|
||||
)
|
||||
if exists is None:
|
||||
maxpos = db.scalar(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == pid
|
||||
)
|
||||
)
|
||||
db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=int(maxpos) + 1))
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
service.add_item(db, pl, it)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -259,15 +174,10 @@ def playlist_remove_item(
|
||||
) -> dict:
|
||||
# Locked like the adds and the reorder: the list row first, then its items — a remove that
|
||||
# locked an item row before a reorder locked the list deadlocked against it.
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
it = by_rating_key(db, PlexItem, item_rating_key)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
it = queries.item(db, item_rating_key)
|
||||
if it is not None:
|
||||
db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id == it.id
|
||||
)
|
||||
)
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
service.remove_item(db, pl, it)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -276,25 +186,6 @@ class PlexItemKeysIn(BaseModel):
|
||||
item_rating_keys: list[str] = []
|
||||
|
||||
|
||||
def _items_by_rks(db: Session, rks: list) -> list[PlexItem]:
|
||||
"""Resolve rating_keys to PlexItems, preserving the input order (drops unknown keys)."""
|
||||
order = [str(k) for k in rks if str(k)]
|
||||
if not order:
|
||||
return []
|
||||
by_rk = {
|
||||
it.rating_key: it
|
||||
for it in db.execute(select(PlexItem).where(PlexItem.rating_key.in_(order))).scalars()
|
||||
}
|
||||
seen: set[str] = set()
|
||||
out: list[PlexItem] = []
|
||||
for rk in order:
|
||||
it = by_rk.get(rk)
|
||||
if it is not None and rk not in seen:
|
||||
seen.add(rk)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
|
||||
class PlexItemAddBulkIn(PlexItemKeysIn):
|
||||
# Where each key goes back (the list page's "Undo" of a removal): an index over the
|
||||
# ORDERED rows, one per key. The two-call shape (append, then send the whole order) leaves the rows
|
||||
@@ -309,32 +200,6 @@ class PlexItemAddBulkIn(PlexItemKeysIn):
|
||||
return self
|
||||
|
||||
|
||||
def _insert_at(db: Session, pid: int, placed: list[tuple[int, PlexItem]]) -> int:
|
||||
"""Insert new rows at their indices (ascending, so each index is where the row stood when the
|
||||
rows before it were back), renumber the whole list 0..n-1, and say how many were added. An item
|
||||
already in the list does not move: the add stays idempotent."""
|
||||
rows = list(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem)
|
||||
.where(PlexPlaylistItem.playlist_id == pid)
|
||||
.order_by(PlexPlaylistItem.position, PlexPlaylistItem.id)
|
||||
).scalars()
|
||||
)
|
||||
present = {r.item_id for r in rows}
|
||||
added = 0
|
||||
for at, it in sorted(placed, key=lambda p: p[0]):
|
||||
if it.id in present:
|
||||
continue
|
||||
row = PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=0)
|
||||
db.add(row)
|
||||
rows.insert(min(at, len(rows)), row)
|
||||
present.add(it.id)
|
||||
added += 1
|
||||
for i, r in enumerate(rows):
|
||||
r.position = i
|
||||
return added
|
||||
|
||||
|
||||
@router.post("/playlists/{pid}/items/bulk")
|
||||
def playlist_add_bulk(
|
||||
pid: int,
|
||||
@@ -345,56 +210,20 @@ def playlist_add_bulk(
|
||||
"""Append many items at once (a whole season/show). Already-present items are skipped; the input
|
||||
order is preserved for the newly-added ones. With `positions`, each key is put back at its index
|
||||
instead of appended. Returns how many were added + the new total."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
items = queries.items_by_keys(db, body.item_rating_keys)
|
||||
if body.positions is not None:
|
||||
by_rk = {it.rating_key: it for it in _items_by_rks(db, body.item_rating_keys)}
|
||||
by_rk = {it.rating_key: it for it in items}
|
||||
placed = [
|
||||
(at, by_rk[str(rk)])
|
||||
for rk, at in zip(body.item_rating_keys, body.positions, strict=True)
|
||||
if str(rk) in by_rk
|
||||
]
|
||||
added = _insert_at(db, pid, placed)
|
||||
if added:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "added": added, "item_count": total}
|
||||
items = _items_by_rks(db, body.item_rating_keys)
|
||||
existing = set(
|
||||
db.execute(
|
||||
select(PlexPlaylistItem.item_id).where(PlexPlaylistItem.playlist_id == pid)
|
||||
).scalars()
|
||||
)
|
||||
pos = int(
|
||||
db.scalar(
|
||||
select(func.coalesce(func.max(PlexPlaylistItem.position), -1)).where(
|
||||
PlexPlaylistItem.playlist_id == pid
|
||||
)
|
||||
)
|
||||
)
|
||||
added = 0
|
||||
for it in items:
|
||||
if it.id in existing:
|
||||
continue
|
||||
pos += 1
|
||||
db.add(PlexPlaylistItem(playlist_id=pid, item_id=it.id, position=pos))
|
||||
existing.add(it.id)
|
||||
added += 1
|
||||
if added:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
added = service.insert_items_at(db, pl, placed)
|
||||
else:
|
||||
added = service.add_items(db, pl, items)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "added": added, "item_count": total}
|
||||
return {"ok": True, "added": added, "item_count": queries.playlist_count(db, pid)}
|
||||
|
||||
|
||||
@router.post("/playlists/{pid}/items/remove-bulk")
|
||||
@@ -406,26 +235,11 @@ def playlist_remove_bulk(
|
||||
) -> dict:
|
||||
"""Remove many items at once (a whole season/show, or a multi-select). Returns how many were
|
||||
removed + the new total."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
ids = [it.id for it in _items_by_rks(db, body.item_rating_keys)]
|
||||
removed = 0
|
||||
if ids:
|
||||
removed = db.execute(
|
||||
delete(PlexPlaylistItem).where(
|
||||
PlexPlaylistItem.playlist_id == pid, PlexPlaylistItem.item_id.in_(ids)
|
||||
),
|
||||
execution_options={"synchronize_session": False},
|
||||
).rowcount
|
||||
if removed:
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
ids = [it.id for it in queries.items_by_keys(db, body.item_rating_keys)]
|
||||
removed = service.remove_items(db, pl, ids)
|
||||
db.commit()
|
||||
total = (
|
||||
db.scalar(
|
||||
select(func.count(PlexPlaylistItem.id)).where(PlexPlaylistItem.playlist_id == pid)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
return {"ok": True, "removed": removed, "item_count": total}
|
||||
return {"ok": True, "removed": removed, "item_count": queries.playlist_count(db, pid)}
|
||||
|
||||
|
||||
@router.put("/playlists/{pid}/order")
|
||||
@@ -436,35 +250,7 @@ def reorder_playlist(
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
"""Set the playlist order from an explicit list of item rating_keys (0-based positions)."""
|
||||
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist", lock=True)
|
||||
order = body.item_rating_keys
|
||||
id_by_rk = {
|
||||
it.rating_key: it.id
|
||||
for it in db.execute(
|
||||
select(PlexItem).where(PlexItem.rating_key.in_(order or [""]))
|
||||
).scalars()
|
||||
}
|
||||
rows = {
|
||||
r.item_id: r
|
||||
for r in db.execute(
|
||||
select(PlexPlaylistItem).where(PlexPlaylistItem.playlist_id == pid)
|
||||
).scalars()
|
||||
}
|
||||
pos = 0
|
||||
seen: set[int] = set()
|
||||
for rk in order:
|
||||
row = rows.get(id_by_rk.get(rk, -1))
|
||||
if row is not None and row.item_id not in seen:
|
||||
row.position = pos
|
||||
pos += 1
|
||||
seen.add(row.item_id)
|
||||
# Rows the payload omitted (added in another tab meanwhile) keep their relative order after
|
||||
# the given ones, so no two rows share a position (the YouTube reorder does the same).
|
||||
for row in sorted(
|
||||
(r for iid, r in rows.items() if iid not in seen), key=lambda r: (r.position, r.id)
|
||||
):
|
||||
row.position = pos
|
||||
pos += 1
|
||||
pl.updated_at = datetime.now(timezone.utc)
|
||||
pl = _own_playlist(db, user, pid, lock=True)
|
||||
service.reorder_playlist(db, pl, body.item_rating_keys)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -6,23 +6,26 @@ the app or any account data. Password-protected links exchange the password once
|
||||
a short-lived signed grant that the media URL carries (a `<video src>` can't send a header).
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import settings
|
||||
from app.db import get_db
|
||||
from app.downloads import links as linksmod
|
||||
from app.downloads import storage
|
||||
from app.downloads import queries, storage
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DownloadJob, DownloadLink, FileFolder, MediaAsset, User
|
||||
from app.ratelimit import RateLimiter, client_key
|
||||
from app.security import verify_password
|
||||
from app.refusal import refused
|
||||
from app.routes._common import not_found
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.downloads.models import DownloadJob, DownloadLink, FileFolder, MediaAsset
|
||||
|
||||
router = APIRouter(prefix="/api/public", tags=["public"])
|
||||
|
||||
# Slow password guessing on a link (the token itself is 192-bit, so it needs no throttle). Counted per
|
||||
@@ -37,26 +40,25 @@ _unlock_limit = RateLimiter(max_events=10, window_seconds=300)
|
||||
_unlock_failures = RateLimiter(max_events=50, window_seconds=300)
|
||||
|
||||
|
||||
def _link_or_404(db: Session, token: str) -> DownloadLink:
|
||||
def _link_or_404(db: Session, token: str) -> "DownloadLink":
|
||||
link = linksmod.find(db, token)
|
||||
if link is None or linksmod.is_expired(link):
|
||||
raise refused(404, "link_gone")
|
||||
return link
|
||||
|
||||
|
||||
def _being_made(db: Session, job: DownloadJob) -> bool:
|
||||
def _being_made(db: Session, job: "DownloadJob") -> bool:
|
||||
"""The job is running, or queued for a worker that will take it: the worker leaves a suspended
|
||||
owner's queued jobs where they are (`worker._claim_job`), as long as the suspension lasts."""
|
||||
if job.status == JobStatus.RUNNING:
|
||||
return True
|
||||
if job.status != JobStatus.QUEUED:
|
||||
return False
|
||||
owner = db.get(User, job.user_id)
|
||||
return owner is not None and not owner.is_suspended
|
||||
return not queries.owner_suspended(db, job)
|
||||
|
||||
|
||||
def _ready_job(db: Session, job: DownloadJob | None) -> tuple[DownloadJob, MediaAsset]:
|
||||
asset = db.get(MediaAsset, job.asset_id) if job and job.asset_id else None
|
||||
def _ready_job(db: Session, job: "DownloadJob | None") -> tuple["DownloadJob", "MediaAsset"]:
|
||||
asset = queries.asset_of(db, job)
|
||||
if (
|
||||
job is not None
|
||||
and asset is not None
|
||||
@@ -78,25 +80,24 @@ def _ready_job(db: Session, job: DownloadJob | None) -> tuple[DownloadJob, Media
|
||||
return job, asset
|
||||
|
||||
|
||||
def _ready_target(db: Session, link: DownloadLink) -> tuple[DownloadJob, MediaAsset]:
|
||||
def _ready_target(db: Session, link: "DownloadLink") -> tuple["DownloadJob", "MediaAsset"]:
|
||||
"""A file link's file. A folder link names no one file: its files are asked for by id."""
|
||||
if link.job_id is None:
|
||||
raise refused(404, "video_gone")
|
||||
return _ready_job(db, db.get(DownloadJob, link.job_id))
|
||||
return _ready_job(db, queries.job(db, link.job_id))
|
||||
|
||||
|
||||
def _link_folder(db: Session, link: DownloadLink) -> FileFolder:
|
||||
folder = db.get(FileFolder, link.folder_id) if link.folder_id is not None else None
|
||||
def _link_folder(db: Session, link: "DownloadLink") -> "FileFolder":
|
||||
folder = queries.link_folder(db, link)
|
||||
if folder is None:
|
||||
raise refused(404, "link_gone")
|
||||
return folder
|
||||
|
||||
|
||||
def _folder_job(db: Session, link: DownloadLink, job_id: int) -> tuple[DownloadJob, MediaAsset]:
|
||||
folder = _link_folder(db, link)
|
||||
job = db.execute(
|
||||
select(DownloadJob).where(DownloadJob.id == job_id, *linksmod.in_folder(folder))
|
||||
).scalar_one_or_none()
|
||||
def _folder_job(
|
||||
db: Session, link: "DownloadLink", job_id: int
|
||||
) -> tuple["DownloadJob", "MediaAsset"]:
|
||||
job = queries.folder_job(db, _link_folder(db, link), job_id)
|
||||
if job is None:
|
||||
raise refused(404, "video_gone")
|
||||
return _ready_job(db, job)
|
||||
@@ -113,7 +114,7 @@ def _file_url(token: str, grant: str | None = None, job_id: int | None = None) -
|
||||
|
||||
|
||||
def _poster_url(
|
||||
token: str, asset: MediaAsset, grant: str | None = None, job_id: int | None = None
|
||||
token: str, asset: "MediaAsset", grant: str | None = None, job_id: int | None = None
|
||||
) -> str | None:
|
||||
if not asset.poster_path:
|
||||
return None
|
||||
@@ -121,7 +122,7 @@ def _poster_url(
|
||||
return f"{url}?g={grant}" if grant else url
|
||||
|
||||
|
||||
def _folder_meta(db: Session, link: DownloadLink, token: str, grant: str | None) -> dict:
|
||||
def _folder_meta(db: Session, link: "DownloadLink", token: str, grant: str | None) -> dict:
|
||||
"""A folder link's page: the folder's name and each file's own watch metadata
|
||||
(`linksmod.folder_files`: ready, on disk, newest first)."""
|
||||
folder = _link_folder(db, link)
|
||||
@@ -147,7 +148,7 @@ def _folder_meta(db: Session, link: DownloadLink, token: str, grant: str | None)
|
||||
}
|
||||
|
||||
|
||||
def _file_meta(db: Session, link: DownloadLink, token: str, grant: str | None) -> dict:
|
||||
def _file_meta(db: Session, link: "DownloadLink", token: str, grant: str | None) -> dict:
|
||||
job, asset = _ready_target(db, link)
|
||||
return {
|
||||
"needs_password": False,
|
||||
@@ -158,20 +159,14 @@ def _file_meta(db: Session, link: DownloadLink, token: str, grant: str | None) -
|
||||
}
|
||||
|
||||
|
||||
def _opened(db: Session, link: DownloadLink, token: str, grant: str | None = None) -> dict:
|
||||
def _opened(db: Session, link: "DownloadLink", token: str, grant: str | None = None) -> dict:
|
||||
"""The page an open link shows, counted as one view (the handler commits the count)."""
|
||||
meta = (
|
||||
_folder_meta(db, link, token, grant)
|
||||
if link.folder_id is not None
|
||||
else _file_meta(db, link, token, grant)
|
||||
)
|
||||
# In SQL, not `+= 1` on the loaded row: two opens at once would both write the same count.
|
||||
db.execute(
|
||||
update(DownloadLink)
|
||||
.where(DownloadLink.id == link.id)
|
||||
.values(view_count=DownloadLink.view_count + 1)
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
linksmod.count_view(db, link)
|
||||
return meta
|
||||
|
||||
|
||||
@@ -217,13 +212,13 @@ def watch_unlock(
|
||||
return {"ok": True, **meta}
|
||||
|
||||
|
||||
def _granted(link: DownloadLink, token: str, g: str | None) -> None:
|
||||
def _granted(link: "DownloadLink", token: str, g: str | None) -> None:
|
||||
"""A password link's files want the grant its unlock handed out."""
|
||||
if link.password_hash and not linksmod.check_grant(token, g, link.password_version):
|
||||
raise refused(403, "share_password_required")
|
||||
|
||||
|
||||
def _serve(link: DownloadLink, asset: MediaAsset) -> FileResponse:
|
||||
def _serve(link: "DownloadLink", asset: "MediaAsset") -> FileResponse:
|
||||
"""Range-aware stream of a shared file. Inline by default (plays in the page); an
|
||||
`allow_download` link serves it as an attachment."""
|
||||
path = storage.safe_abs_path(settings.files_root, asset.rel_path)
|
||||
@@ -252,7 +247,7 @@ def _serve(link: DownloadLink, asset: MediaAsset) -> FileResponse:
|
||||
)
|
||||
|
||||
|
||||
def _poster(asset: MediaAsset) -> FileResponse:
|
||||
def _poster(asset: "MediaAsset") -> FileResponse:
|
||||
if not asset.poster_path:
|
||||
raise not_found("poster")
|
||||
path = storage.safe_abs_path(settings.files_root, asset.poster_path)
|
||||
|
||||
@@ -10,6 +10,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import quota
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
@@ -27,7 +28,7 @@ def _sum(db: Session, *conds) -> int:
|
||||
@router.get("/my-usage")
|
||||
def my_usage(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
now = datetime.now(timezone.utc)
|
||||
mine = QuotaEvent.user_id == user.id
|
||||
mine = owned_by(QuotaEvent.user_id, user)
|
||||
by_action = {
|
||||
action: int(units)
|
||||
for action, units in db.execute(
|
||||
|
||||
@@ -18,7 +18,8 @@ from sqlalchemy.orm import Session
|
||||
from app import scheduler
|
||||
from app.auth import admin_user
|
||||
from app.db import get_db
|
||||
from app.models import Channel, JobRun, LogEvent, User
|
||||
from app.models import JobRun, LogEvent, User
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.routes import _account as account
|
||||
from app.routes._keyset import PAGE_MAX, accounts, older_than, page, utc
|
||||
from app.runstatus import RunStatus
|
||||
@@ -116,10 +117,7 @@ def list_runs(
|
||||
lambda r: r.started_at,
|
||||
)
|
||||
channels = {r.target for r in runs if r.job in scheduler.AD_HOC_JOBS and r.target}
|
||||
titles: dict[str, str] = {}
|
||||
if channels:
|
||||
rows = db.execute(select(Channel.id, Channel.title).where(Channel.id.in_(channels)))
|
||||
titles = {cid: title for cid, title in rows.tuples() if title is not None}
|
||||
titles = catalog_queries.channel_titles(db, channels)
|
||||
lines: dict[int, int] = {}
|
||||
if runs:
|
||||
counted = db.execute(
|
||||
|
||||
@@ -1,15 +1,20 @@
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel, field_validator
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import SavedView, User
|
||||
from app.routes._common import owned_or_404
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.routes._common import not_found
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.reader.models import SavedView
|
||||
|
||||
# Saved "smart views": per-user named snapshots of a module's filters (feed or plex, keyed by
|
||||
# `source`).
|
||||
router = APIRouter(prefix="/api/saved-views", tags=["saved-views"])
|
||||
@@ -23,7 +28,7 @@ def _norm_source(source: str) -> str:
|
||||
return source if source in _SOURCES else "feed"
|
||||
|
||||
|
||||
def _serialize(v: SavedView) -> dict:
|
||||
def _serialize(v: "SavedView") -> dict:
|
||||
return {
|
||||
"id": v.id,
|
||||
"name": v.name,
|
||||
@@ -33,23 +38,20 @@ def _serialize(v: SavedView) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _own_view(db: Session, user: User, view_id: int) -> "SavedView":
|
||||
view = queries.own_view(db, user, view_id)
|
||||
if view is None:
|
||||
raise not_found("view")
|
||||
return view
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_views(
|
||||
source: str = "feed",
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> list[dict]:
|
||||
source = _norm_source(source)
|
||||
rows = (
|
||||
db.execute(
|
||||
select(SavedView)
|
||||
.where(SavedView.user_id == user.id, SavedView.source == source)
|
||||
.order_by(SavedView.position, SavedView.id)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return [_serialize(v) for v in rows]
|
||||
return [_serialize(v) for v in queries.views(db, user, _norm_source(source))]
|
||||
|
||||
|
||||
# A feed view's tag pick is read by more than the feed: the autotag job keeps every tag a view names
|
||||
@@ -86,22 +88,8 @@ def create_view(
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
source = _norm_source(body.source)
|
||||
# Position is per (user, source), so each module's views number 0..N independently.
|
||||
maxpos = db.execute(
|
||||
select(func.coalesce(func.max(SavedView.position), -1)).where(
|
||||
SavedView.user_id == user.id, SavedView.source == source
|
||||
)
|
||||
).scalar_one()
|
||||
view = SavedView(
|
||||
user_id=user.id,
|
||||
source=source,
|
||||
name=name[:80],
|
||||
filters=body.filters,
|
||||
position=maxpos + 1,
|
||||
)
|
||||
db.add(view)
|
||||
try:
|
||||
view = service.create_view(db, user, _norm_source(body.source), name, body.filters)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -126,32 +114,17 @@ def update_view(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
view = owned_or_404(db, SavedView, view_id, user, what="view")
|
||||
view = _own_view(db, user, view_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
view.name = name[:80]
|
||||
if "filters" in sent:
|
||||
if body.filters is None:
|
||||
raise refused(400, "filters_not_object")
|
||||
view.filters = body.filters
|
||||
if "is_default" in sent:
|
||||
make_default = bool(body.is_default)
|
||||
if make_default:
|
||||
# At most one default per (user, source): clear the flag on this module's others first.
|
||||
db.execute(
|
||||
update(SavedView)
|
||||
.where(
|
||||
SavedView.user_id == user.id,
|
||||
SavedView.source == view.source,
|
||||
SavedView.id != view.id,
|
||||
)
|
||||
.values(is_default=False)
|
||||
)
|
||||
view.is_default = make_default
|
||||
name = (body.name or "").strip()
|
||||
if "name" in sent and not name:
|
||||
raise refused(400, "name_required")
|
||||
if "filters" in sent and body.filters is None:
|
||||
raise refused(400, "filters_not_object")
|
||||
try:
|
||||
service.update_view(
|
||||
db, view, sent=sent, name=name, filters=body.filters, is_default=body.is_default
|
||||
)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -165,8 +138,7 @@ def delete_view(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
view = owned_or_404(db, SavedView, view_id, user, what="view")
|
||||
db.delete(view)
|
||||
service.delete_view(db, _own_view(db, user, view_id))
|
||||
db.commit()
|
||||
return {"deleted": view_id}
|
||||
|
||||
@@ -181,14 +153,6 @@ def reorder_views(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
ids = body.ids
|
||||
rows = db.execute(select(SavedView).where(SavedView.user_id == user.id)).scalars().all()
|
||||
by_id = {v.id: v for v in rows}
|
||||
pos = 0
|
||||
for vid in ids:
|
||||
v = by_id.get(vid)
|
||||
if v is not None:
|
||||
v.position = pos
|
||||
pos += 1
|
||||
service.reorder_views(db, user, body.ids)
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
@@ -4,7 +4,6 @@ and the shared quota picture."""
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, func, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, quota, state, sysconfig
|
||||
@@ -12,7 +11,8 @@ from app.audit import AuditAction
|
||||
from app.auth import admin_user
|
||||
from app.config import settings
|
||||
from app.db import get_db
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.scheduler import (
|
||||
JOB_INTERVALS,
|
||||
MAX_INTERVAL,
|
||||
@@ -23,7 +23,6 @@ from app.scheduler import (
|
||||
trigger_all,
|
||||
trigger_job,
|
||||
)
|
||||
from app.sync.explore import followed_channel
|
||||
from app.sync.runner import estimate_deep_backfill
|
||||
from app.routes._common import not_found
|
||||
from app.refusal import refused
|
||||
@@ -122,58 +121,14 @@ def get_scheduler(_: User = Depends(admin_user), db: Session = Depends(get_db))
|
||||
snap = scheduler_snapshot(db)
|
||||
|
||||
# Work still queued (shared catalog, so these are instance-wide, not per-user).
|
||||
deep_requested = (
|
||||
select(Subscription.id)
|
||||
.where(
|
||||
Subscription.channel_id == Channel.id,
|
||||
Subscription.deep_requested.is_(True),
|
||||
)
|
||||
.exists()
|
||||
)
|
||||
deep_channels = (
|
||||
db.execute(select(Channel).where(Channel.backfill_done.is_(False), deep_requested))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
deep_eta = estimate_deep_backfill(db, deep_channels)
|
||||
|
||||
queue = {
|
||||
"channels_recent_pending": db.scalar(
|
||||
# The same selection run_recent_backfill works through, or the card counts channels
|
||||
# the job will never touch and never reaches zero.
|
||||
select(func.count())
|
||||
.select_from(Channel)
|
||||
.where(Channel.recent_synced_at.is_(None), followed_channel())
|
||||
)
|
||||
or 0,
|
||||
# The whole opted-in BACKLOG, deliberately not `run_deep_backfill`'s next-pass selection
|
||||
# (which also wants `recent_synced_at`): the same figure /api/sync/my-status shows the user,
|
||||
# and a channel missing its recent pull is one the same job tick fetches first, not one the
|
||||
# job will never touch. Narrowing it here would under-report work that is genuinely queued.
|
||||
"channels_deep_pending": deep_eta["channels_pending"],
|
||||
"deep_eta_seconds": deep_eta["eta_seconds"],
|
||||
"videos_pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_pending_shorts": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(Video.shorts_probed.is_(False), Video.enriched_at.is_not(None))
|
||||
)
|
||||
or 0,
|
||||
"videos_live_refresh": db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Video)
|
||||
.where(
|
||||
or_(
|
||||
Video.live_status.in_(("live", "upcoming")),
|
||||
and_(Video.live_status == "was_live", Video.duration_seconds.is_(None)),
|
||||
)
|
||||
)
|
||||
)
|
||||
or 0,
|
||||
}
|
||||
deep_eta = estimate_deep_backfill(db, catalog_queries.deep_backlog(db))
|
||||
queue = catalog_queries.work_queue(db)
|
||||
# The whole opted-in BACKLOG, deliberately not `run_deep_backfill`'s next-pass selection
|
||||
# (which also wants `recent_synced_at`): the same figure /api/sync/my-status shows the user,
|
||||
# and a channel missing its recent pull is one the same job tick fetches first, not one the
|
||||
# job will never touch. Narrowing it here would under-report work that is genuinely queued.
|
||||
queue["channels_deep_pending"] = deep_eta["channels_pending"]
|
||||
queue["deep_eta_seconds"] = deep_eta["eta_seconds"]
|
||||
|
||||
used = quota.units_used_today(db)
|
||||
quota_info = {
|
||||
|
||||
+14
-245
@@ -11,39 +11,24 @@ regardless of source.
|
||||
Ingested results are flagged (videos.via_search / channels.from_search) so the Library can hide
|
||||
this search-discovered "noise" by default. Shorts and currently-live/upcoming videos are never
|
||||
ingested this way (Shorts are confirmed by the youtube.com/shorts probe; live/upcoming are
|
||||
dropped from the search response up front)."""
|
||||
dropped from the search response up front).
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
This is the HTTP side — the paging, the caps and the refusals; what a page of results leaves in the
|
||||
catalog is `catalog/search`."""
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import and_, delete, func, or_, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session, aliased
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, quota, sysconfig
|
||||
from app.auth import current_user
|
||||
from app.catalog import queries
|
||||
from app.catalog import search as catalog_search
|
||||
from app.db import get_db
|
||||
from app.models import (
|
||||
LIVE_OR_UPCOMING,
|
||||
BlockedChannel,
|
||||
Channel,
|
||||
Playlist,
|
||||
PlaylistItem,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
User,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.feedquery import feed_columns, serialize
|
||||
from app.utils import like_escape
|
||||
from app.sync.subscriptions import apply_channel_details
|
||||
from app.sync.videos import insert_stubs, apply_video_details, parse_dt
|
||||
from app.models import User
|
||||
from app.reader import queries as reader_queries
|
||||
from app.youtube import search_scrape
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
from app.refusal import refused
|
||||
from app.routes._common import youtube_refusal
|
||||
|
||||
@@ -54,198 +39,6 @@ router = APIRouter(prefix="/api/search", tags=["search"])
|
||||
SEARCH_COST = 100
|
||||
|
||||
|
||||
def _classify_shorts(db: Session, videos: list[Video]) -> None:
|
||||
"""Confirm/deny Shorts for the freshly-ingested videos so none slip into the feed.
|
||||
|
||||
Mirrors the scheduler's run_shorts_classification, but scoped to just these rows and run
|
||||
synchronously (the search is an explicit, awaited action). Non-candidates (too long, no
|
||||
duration, or live) are finalised without a probe; the rest hit youtube.com/shorts (no
|
||||
quota). Undetermined probes are left for the scheduler to retry."""
|
||||
probe_max = sysconfig.get_int(db, "shorts_probe_max_seconds")
|
||||
to_probe: list[Video] = []
|
||||
for v in videos:
|
||||
if v.shorts_probed:
|
||||
continue
|
||||
if v.enriched_at is None:
|
||||
# Enrichment failed (proxy blip) or YouTube omitted the row — duration is unknown,
|
||||
# so DON'T finalise shorts_probed off a null duration; leave it for the scheduler
|
||||
# (which only classifies enriched rows) to retry. Otherwise a real Short would leak.
|
||||
continue
|
||||
if v.live_status != "none" or v.duration_seconds is None or v.duration_seconds > probe_max:
|
||||
v.is_short = False
|
||||
v.shorts_probed = True
|
||||
else:
|
||||
to_probe.append(v)
|
||||
if to_probe:
|
||||
with make_client() as client:
|
||||
|
||||
def work(v: Video):
|
||||
return v, probe_is_short(client, v.id)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=16) as pool:
|
||||
for v, result in pool.map(work, to_probe):
|
||||
if result is None:
|
||||
continue
|
||||
v.is_short = result
|
||||
v.shorts_probed = True
|
||||
db.commit()
|
||||
|
||||
|
||||
def _serialize_ids(db: Session, user: User, ids: list[str]) -> list[dict]:
|
||||
"""Serialize the given video ids with the shared feed projection (joined channel + this
|
||||
user's watch state + saved), preserving the input (search relevance) order."""
|
||||
if not ids:
|
||||
return []
|
||||
state = aliased(VideoState)
|
||||
rows = db.execute(
|
||||
select(*feed_columns(user, state))
|
||||
.join(Channel, Channel.id == Video.channel_id)
|
||||
.outerjoin(state, and_(state.video_id == Video.id, state.user_id == user.id))
|
||||
.where(Video.id.in_(ids))
|
||||
).all()
|
||||
by_id = {r.id: serialize(r) for r in rows}
|
||||
return [by_id[i] for i in ids if i in by_id]
|
||||
|
||||
|
||||
def _ingest_candidates(
|
||||
db: Session, yt: YouTubeClient, user: User, candidates: list[dict]
|
||||
) -> list[str]:
|
||||
"""Materialise one page of search candidates into the catalog (channel stubs + enrich,
|
||||
video stubs flagged via_search, enrich, Shorts classification) and return the KEEPABLE
|
||||
video ids — non-Short, non-live — in the input relevance order."""
|
||||
ids = [it["id"] for it in candidates]
|
||||
if not ids:
|
||||
return []
|
||||
|
||||
# 1) Channels are the FK target — create stubs for unknown ones (flagged from_search),
|
||||
# then enrich the new ones (channels.list, 1 unit). Stubs survive an enrich failure.
|
||||
channel_ids = list({it["channel_id"] for it in candidates})
|
||||
existing = set(db.scalars(select(Channel.id).where(Channel.id.in_(channel_ids))).all())
|
||||
new_channel_ids = [c for c in channel_ids if c not in existing]
|
||||
if new_channel_ids:
|
||||
titles = {it["channel_id"]: it["channel_title"] for it in candidates}
|
||||
for cid in new_channel_ids:
|
||||
db.add(Channel(id=cid, title=titles.get(cid), from_search=True))
|
||||
db.commit()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.CHANNELS_DISCOVER):
|
||||
details = yt.get_channels(new_channel_ids)
|
||||
apply_channel_details(db, details, new_channel_ids)
|
||||
db.commit()
|
||||
except YouTubeError:
|
||||
# The stubs remain and the details can be backfilled later — but until they are, those
|
||||
# channels sit in the results with no thumbnail and no subscriber count, looking like
|
||||
# channels that have neither.
|
||||
log.warning(
|
||||
"the channels found by a search could not be enriched",
|
||||
extra=logs.event("search.channel_enrich_failed", channels=len(new_channel_ids)),
|
||||
)
|
||||
db.rollback() # details can be backfilled later; the stubs remain
|
||||
|
||||
# 2) Video stubs, flagged via_search. Existing rows keep their flags (ON CONFLICT DO
|
||||
# NOTHING), so an organically-synced video stays organic.
|
||||
# published_at is NOT NULL (migration 0068); the scrape source exposes no publish date
|
||||
# (only a relative "3 years ago"), so fall back to now() to satisfy the constraint — the
|
||||
# enrich in step 3 overwrites it with the video's real snippet date.
|
||||
now = datetime.now(timezone.utc)
|
||||
insert_stubs(
|
||||
db,
|
||||
[
|
||||
{
|
||||
"id": it["id"],
|
||||
"channel_id": it["channel_id"],
|
||||
"title": it["title"],
|
||||
"published_at": parse_dt(it["published_at"]) or now,
|
||||
"thumbnail_url": it["thumbnail_url"],
|
||||
"via_search": True,
|
||||
}
|
||||
for it in candidates
|
||||
],
|
||||
)
|
||||
|
||||
# 3) Enrich (videos.list, 1 unit/50) so cards have duration/stats and we can judge live.
|
||||
videos = db.scalars(select(Video).where(Video.id.in_(ids))).all()
|
||||
try:
|
||||
with quota.attribute(user.id, quota.QuotaAction.VIDEOS_ENRICH):
|
||||
detail_by_id = {it["id"]: it for it in yt.get_videos(ids)}
|
||||
except YouTubeError:
|
||||
# Every candidate then stays unenriched, and an unenriched video cannot be judged live or
|
||||
# Short — so it is dropped. A working search returns nothing and reads as "no results".
|
||||
log.warning(
|
||||
"the videos found by a search could not be enriched",
|
||||
extra=logs.event("search.video_enrich_failed", candidates=len(ids)),
|
||||
)
|
||||
detail_by_id = {}
|
||||
enriched_at = datetime.now(
|
||||
timezone.utc
|
||||
) # when THIS enrich ran (distinct from the stub placeholder)
|
||||
for v in videos:
|
||||
item = detail_by_id.get(v.id)
|
||||
if item is None:
|
||||
continue
|
||||
apply_video_details(v, item)
|
||||
v.enriched_at = enriched_at
|
||||
db.commit()
|
||||
|
||||
# 4) Confirm/deny Shorts (no quota) so none enter via search.
|
||||
_classify_shorts(db, videos)
|
||||
|
||||
keep = {
|
||||
v.id
|
||||
for v in videos
|
||||
if v.enriched_at is not None and not v.is_short and v.live_status not in LIVE_OR_UPCOMING
|
||||
}
|
||||
return [vid for vid in ids if vid in keep]
|
||||
|
||||
|
||||
def _new_first(db: Session, user: User, ids: list[str]) -> list[str]:
|
||||
"""Reorder results so genuinely-new discoveries come first and things the user already has
|
||||
(a subscribed channel, or a video they've watched / in-progress / saved / playlisted) sink
|
||||
to the back — keeping the relevance order within each group."""
|
||||
if not ids:
|
||||
return ids
|
||||
sub_ch = select(Subscription.channel_id).where(Subscription.user_id == user.id)
|
||||
has_state = (
|
||||
select(VideoState.id)
|
||||
.where(VideoState.video_id == Video.id, VideoState.user_id == user.id)
|
||||
.exists()
|
||||
)
|
||||
in_playlist = (
|
||||
select(PlaylistItem.id)
|
||||
.join(Playlist, Playlist.id == PlaylistItem.playlist_id)
|
||||
.where(PlaylistItem.video_id == Video.id, Playlist.user_id == user.id)
|
||||
.exists()
|
||||
)
|
||||
have = set(
|
||||
db.scalars(
|
||||
select(Video.id).where(
|
||||
Video.id.in_(ids),
|
||||
or_(Video.channel_id.in_(sub_ch), has_state, in_playlist),
|
||||
)
|
||||
).all()
|
||||
)
|
||||
return [i for i in ids if i not in have] + [i for i in ids if i in have]
|
||||
|
||||
|
||||
def _fold_term(db: Session, video_ids: list[str], term: str) -> None:
|
||||
"""Append `term` to each video's space-joined `search_terms`, unless it is already one of them.
|
||||
|
||||
"Already one of them" is a whole-term match, as a literal: a substring test skipped `cat` on a
|
||||
video already found for `category`, and an unescaped one read the `_` in `my_channel` as a
|
||||
wildcard — either way the term was never folded in, and local search lost that relevance."""
|
||||
padded = func.concat(" ", func.coalesce(Video.search_terms, ""), " ")
|
||||
db.execute(
|
||||
update(Video)
|
||||
.where(Video.id.in_(video_ids))
|
||||
.where(padded.notilike(f"% {like_escape(term)} %"))
|
||||
.values(
|
||||
search_terms=func.trim(
|
||||
func.coalesce(Video.search_terms, "").op("||")(" ").op("||")(term)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@router.get("/youtube")
|
||||
def search_youtube(
|
||||
q: str | None = None,
|
||||
@@ -287,9 +80,7 @@ def search_youtube(
|
||||
raise refused(429, "quota_spent")
|
||||
|
||||
# Channels this user has blocked are dropped before ingestion — never shown, never stored.
|
||||
blocked = set(
|
||||
db.scalars(select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)).all()
|
||||
)
|
||||
blocked = queries.blocked_ids(db, user)
|
||||
|
||||
collected: list[str] = []
|
||||
next_cursor = cursor
|
||||
@@ -330,7 +121,7 @@ def search_youtube(
|
||||
# write/shape failure degrades to a 422 with a message on this JSON endpoint instead of a
|
||||
# bare 500 hidden behind the FE's generic "search failed" (same rationale as B-core-1).
|
||||
try:
|
||||
collected.extend(_ingest_candidates(db, yt, user, candidates))
|
||||
collected.extend(catalog_search.ingest_candidates(db, yt, user, candidates))
|
||||
except Exception as exc: # any ingest failure, not just YouTubeError
|
||||
db.rollback()
|
||||
# The term is NOT in this line. A search string is exactly what the privacy
|
||||
@@ -378,17 +169,12 @@ def search_youtube(
|
||||
# Record these as THIS user's search finds (Mine feed / Source filter), and fold the
|
||||
# query into each result's search_terms so local full-text search inherits YouTube's
|
||||
# relevance. Idempotent; the generated search_vector updates automatically.
|
||||
db.execute(
|
||||
pg_insert(SearchFind)
|
||||
.values([{"user_id": user.id, "video_id": vid} for vid in ordered])
|
||||
.on_conflict_do_nothing(index_elements=["user_id", "video_id"])
|
||||
)
|
||||
_fold_term(db, ordered, term)
|
||||
catalog_search.record_finds(db, user, ordered, term)
|
||||
db.commit()
|
||||
|
||||
ordered = _new_first(db, user, ordered)
|
||||
ordered = catalog_search.new_first(db, user, ordered)
|
||||
return {
|
||||
"items": _serialize_ids(db, user, ordered),
|
||||
"items": reader_queries.feed_rows_by_ids(db, user, ordered),
|
||||
"next_cursor": next_cursor,
|
||||
"has_more": bool(next_cursor),
|
||||
"source": source,
|
||||
@@ -412,23 +198,6 @@ def clear_search(
|
||||
ids = body.video_ids
|
||||
if not ids:
|
||||
return {"ok": True, "removed": 0}
|
||||
db.execute(
|
||||
delete(SearchFind).where(SearchFind.user_id == user.id, SearchFind.video_id.in_(ids))
|
||||
)
|
||||
db.commit()
|
||||
has_state = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
in_playlist = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
subscribed = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
other_find = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
removed = db.execute(
|
||||
delete(Video).where(
|
||||
Video.id.in_(ids),
|
||||
Video.via_search.is_(True),
|
||||
~has_state,
|
||||
~in_playlist,
|
||||
~subscribed,
|
||||
~other_find,
|
||||
)
|
||||
).rowcount
|
||||
removed = catalog_search.clear(db, user, ids)
|
||||
db.commit()
|
||||
return {"ok": True, "removed": removed}
|
||||
|
||||
+11
-54
@@ -1,12 +1,12 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy import and_, case, func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit, quota, state
|
||||
from app.audit import AuditAction
|
||||
from app.auth import admin_user, current_user, has_read_scope
|
||||
from app.db import get_db
|
||||
from app.models import Channel, Subscription, User, Video
|
||||
from app.models import User
|
||||
from app.catalog import queries, service
|
||||
from app.routes._common import youtube_refusal
|
||||
from app.ratelimit import RateLimiter
|
||||
from app.sync.runner import estimate_deep_backfill
|
||||
@@ -47,19 +47,8 @@ def sync_subscriptions(user: User = Depends(current_user), db: Session = Depends
|
||||
|
||||
@router.get("/status")
|
||||
def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sub_count = db.scalar(
|
||||
select(func.count()).select_from(Subscription).where(Subscription.user_id == user.id)
|
||||
)
|
||||
return {
|
||||
"subscriptions": sub_count,
|
||||
"channels_total": db.scalar(select(func.count()).select_from(Channel)),
|
||||
"channels_backfilling": db.scalar(
|
||||
select(func.count()).select_from(Channel).where(Channel.backfill_done.is_(False))
|
||||
),
|
||||
"videos_total": db.scalar(select(func.count()).select_from(Video)),
|
||||
"pending_enrich": db.scalar(
|
||||
select(func.count()).select_from(Video).where(Video.enriched_at.is_(None))
|
||||
),
|
||||
**queries.catalog_totals(db, user),
|
||||
"quota_used_today": quota.units_used_today(db),
|
||||
"quota_remaining_today": quota.remaining_today(db),
|
||||
"paused": state.is_sync_paused(db),
|
||||
@@ -70,51 +59,21 @@ def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db
|
||||
@router.get("/my-status")
|
||||
def my_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
"""Per-user sync progress: how far the channels *this* user subscribes to have synced."""
|
||||
sub_join = and_(Subscription.channel_id == Channel.id, Subscription.user_id == user.id)
|
||||
total, details_synced, recent_synced, all_deep_done = db.execute(
|
||||
select(
|
||||
func.count(Channel.id),
|
||||
func.count(Channel.details_synced_at),
|
||||
func.count(Channel.recent_synced_at),
|
||||
func.sum(case((Channel.backfill_done.is_(True), 1), else_=0)),
|
||||
)
|
||||
.select_from(Channel)
|
||||
.join(Subscription, sub_join)
|
||||
).one()
|
||||
total = total or 0
|
||||
all_deep_done = int(all_deep_done or 0)
|
||||
total, details_synced, recent_synced, all_deep_done = queries.my_progress(db, user)
|
||||
|
||||
# Per-user full-history (deep) backfill: channels this user has opted in, and an ETA
|
||||
# for the still-pending ones (quota-bound estimate).
|
||||
deep_requested_channels = (
|
||||
db.execute(
|
||||
select(Channel).join(
|
||||
Subscription,
|
||||
and_(sub_join, Subscription.deep_requested.is_(True)),
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
deep_requested_channels = queries.my_deep_requested(db, user)
|
||||
deep_eta = estimate_deep_backfill(db, deep_requested_channels)
|
||||
# "done / requested" is shown as one ratio, so both halves count the SAME channels: the ones this
|
||||
# user asked full history for. Counting every subscribed channel whose back-catalog happens to be
|
||||
# complete (another user asked, or it was short) gave "342/329".
|
||||
deep_done = sum(1 for c in deep_requested_channels if c.backfill_done)
|
||||
|
||||
my_videos = db.scalar(
|
||||
select(func.count(Video.id)).join(
|
||||
Subscription,
|
||||
and_(
|
||||
Subscription.channel_id == Video.channel_id,
|
||||
Subscription.user_id == user.id,
|
||||
Subscription.hidden.is_(False),
|
||||
),
|
||||
)
|
||||
)
|
||||
my_videos = queries.my_video_count(db, user)
|
||||
# Whole-catalog total (every video any user's subscriptions have pulled in), so the
|
||||
# header can show "yours" against the shared library size.
|
||||
total_videos = db.scalar(select(func.count(Video.id)))
|
||||
total_videos = queries.video_total(db)
|
||||
return {
|
||||
"channels_total": total,
|
||||
"channels_details_synced": details_synced,
|
||||
@@ -125,8 +84,8 @@ def my_status(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"channels_deep_requested": len(deep_requested_channels),
|
||||
"deep_pending_count": deep_eta["channels_pending"],
|
||||
"deep_eta_seconds": deep_eta["eta_seconds"],
|
||||
"my_videos": my_videos or 0,
|
||||
"total_videos": total_videos or 0,
|
||||
"my_videos": my_videos,
|
||||
"total_videos": total_videos,
|
||||
"quota_used_today": quota.units_used_today(db),
|
||||
"quota_remaining_today": quota.remaining_today(db),
|
||||
"paused": state.is_sync_paused(db),
|
||||
@@ -143,11 +102,9 @@ def deep_all(
|
||||
on: bool = True,
|
||||
) -> dict:
|
||||
"""Opt every one of the user's channels into (or out of) full-history backfill."""
|
||||
result = db.execute(
|
||||
update(Subscription).where(Subscription.user_id == user.id).values(deep_requested=on)
|
||||
)
|
||||
updated = service.set_deep_all(db, user, on)
|
||||
db.commit()
|
||||
return {"ok": True, "updated": result.rowcount, "deep_requested": on}
|
||||
return {"ok": True, "updated": updated, "deep_requested": on}
|
||||
|
||||
|
||||
@router.post("/pause")
|
||||
|
||||
+24
-32
@@ -1,19 +1,32 @@
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import ChannelTag, Tag, User
|
||||
from app.routes._common import owned_or_404
|
||||
from app.models import User
|
||||
from app.reader import queries, service
|
||||
from app.routes._common import not_found
|
||||
from app.sync.tagsuggest import compute_tag_suggestions
|
||||
from app.refusal import refused
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.reader.models import Tag
|
||||
|
||||
router = APIRouter(prefix="/api/tags", tags=["tags"])
|
||||
|
||||
|
||||
def _own_tag(db: Session, user: User, tag_id: int) -> "Tag":
|
||||
"""One of the user's own tags, or a 404 — a system tag (no owner) is not theirs to edit."""
|
||||
tag = queries.own_tag(db, user, tag_id)
|
||||
if tag is None:
|
||||
raise not_found("tag")
|
||||
return tag
|
||||
|
||||
|
||||
@router.get("/suggestions")
|
||||
def tag_suggestions(
|
||||
user: User = Depends(current_user), db: Session = Depends(get_db)
|
||||
@@ -27,13 +40,6 @@ def tag_suggestions(
|
||||
|
||||
@router.get("")
|
||||
def list_tags(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
rows = db.execute(
|
||||
select(Tag, func.count(ChannelTag.id))
|
||||
.outerjoin(ChannelTag, ChannelTag.tag_id == Tag.id)
|
||||
.where(or_(Tag.user_id.is_(None), Tag.user_id == user.id))
|
||||
.group_by(Tag.id)
|
||||
.order_by(Tag.category, Tag.name)
|
||||
).all()
|
||||
return [
|
||||
{
|
||||
"id": tag.id,
|
||||
@@ -43,7 +49,7 @@ def list_tags(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"system": tag.user_id is None,
|
||||
"channel_count": count,
|
||||
}
|
||||
for tag, count in rows
|
||||
for tag, count in queries.tags_with_counts(db, user)
|
||||
]
|
||||
|
||||
|
||||
@@ -62,16 +68,8 @@ def create_tag(
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
# An unknown category silently falls back to "other" (unchanged behaviour) rather than 422.
|
||||
category = body.category if body.category in ("language", "topic", "other") else "other"
|
||||
tag = Tag(
|
||||
user_id=user.id,
|
||||
name=name,
|
||||
color=body.color,
|
||||
category=category,
|
||||
)
|
||||
db.add(tag)
|
||||
try:
|
||||
tag = service.create_tag(db, user, name, body.category, body.color)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -99,17 +97,13 @@ def update_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# System tags (user_id NULL) are not user-editable: owned_or_404 404s them too.
|
||||
tag = owned_or_404(db, Tag, tag_id, user, what="tag")
|
||||
tag = _own_tag(db, user, tag_id)
|
||||
sent = body.model_fields_set
|
||||
if "name" in sent:
|
||||
name = (body.name or "").strip()
|
||||
if not name:
|
||||
raise refused(400, "name_required")
|
||||
tag.name = name
|
||||
if "color" in sent:
|
||||
tag.color = body.color
|
||||
name = (body.name or "").strip()
|
||||
if "name" in sent and not name:
|
||||
raise refused(400, "name_required")
|
||||
try:
|
||||
service.update_tag(tag, sent=sent, name=name, color=body.color)
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
db.rollback()
|
||||
@@ -129,8 +123,6 @@ def delete_tag(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
# System tags (user_id NULL) are not user-editable: owned_or_404 404s them too.
|
||||
tag = owned_or_404(db, Tag, tag_id, user, what="tag")
|
||||
db.delete(tag) # ChannelTag rows cascade on tag delete.
|
||||
service.delete_tag(db, _own_tag(db, user, tag_id))
|
||||
db.commit()
|
||||
return {"deleted": tag_id}
|
||||
|
||||
@@ -16,13 +16,14 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens, audit
|
||||
from app.apitokens import GRANT_COLUMNS, Right
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.downloads import folders
|
||||
from app.models import ApiToken, ApiTokenGrant, FileFolder, User
|
||||
from app.downloads import folders, queries
|
||||
from app.models import ApiToken, ApiTokenGrant, User
|
||||
from app.refusal import refused
|
||||
from app.routes._common import owned_or_404
|
||||
from app.security import hash_token
|
||||
@@ -50,12 +51,16 @@ class TokenIn(BaseModel):
|
||||
|
||||
|
||||
def _serialize(db: Session, token: ApiToken) -> dict:
|
||||
rows = db.execute(
|
||||
select(ApiTokenGrant, FileFolder)
|
||||
.join(FileFolder, FileFolder.id == ApiTokenGrant.folder_id)
|
||||
.where(ApiTokenGrant.token_id == token.id)
|
||||
.order_by(FileFolder.id)
|
||||
).all()
|
||||
grants = list(
|
||||
db.execute(
|
||||
select(ApiTokenGrant)
|
||||
.where(ApiTokenGrant.token_id == token.id)
|
||||
.order_by(ApiTokenGrant.folder_id)
|
||||
).scalars()
|
||||
)
|
||||
# The folders are Capture's: read through its queries, not its table.
|
||||
by_id = queries.folders_by_id(db, {g.folder_id for g in grants})
|
||||
rows = [(g, by_id[g.folder_id]) for g in grants if g.folder_id in by_id]
|
||||
now = datetime.now(timezone.utc)
|
||||
return {
|
||||
"id": token.id,
|
||||
@@ -82,7 +87,7 @@ def _serialize(db: Session, token: ApiToken) -> dict:
|
||||
@router.get("")
|
||||
def list_tokens(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
tokens = (
|
||||
db.execute(select(ApiToken).where(ApiToken.user_id == user.id).order_by(ApiToken.id))
|
||||
db.execute(select(ApiToken).where(owned_by(ApiToken.user_id, user)).order_by(ApiToken.id))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
@@ -130,7 +135,7 @@ def create_token(
|
||||
if (epoch or 0) != (user.session_epoch or 0):
|
||||
raise refused(401, "not_authenticated")
|
||||
count = db.execute(
|
||||
select(func.count(ApiToken.id)).where(ApiToken.user_id == user.id)
|
||||
select(func.count(ApiToken.id)).where(owned_by(ApiToken.user_id, user))
|
||||
).scalar_one()
|
||||
if count >= MAX_TOKENS:
|
||||
raise refused(409, "api_token_limit", limit=MAX_TOKENS)
|
||||
|
||||
@@ -9,13 +9,13 @@ from typing import Callable
|
||||
from apscheduler.executors.pool import ThreadPoolExecutor
|
||||
from apscheduler.jobstores.base import ConflictingIdError
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from sqlalchemy import select
|
||||
|
||||
from app import audit, debugsession, jobruns, logs, logsink, progress, quota, sysconfig
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal
|
||||
from app.downloads.gc import run_download_gc
|
||||
from app.models import Channel, JobRun
|
||||
from app.models import JobRun
|
||||
from app.catalog import queries as catalog_queries
|
||||
from app.plex.sync import sync as run_plex_sync
|
||||
from app.plex.tmdb import run_enrich as run_tmdb_enrich
|
||||
from app.plex.tmdb import run_people_enrich
|
||||
@@ -305,7 +305,7 @@ def scheduler_snapshot(db) -> dict:
|
||||
"running": going is not None,
|
||||
"target": shown.target if shown else None,
|
||||
"target_title": (
|
||||
db.scalar(select(Channel.title).where(Channel.id == shown.target))
|
||||
catalog_queries.channel_title(db, shown.target)
|
||||
if shown is not None and shown.target
|
||||
else None
|
||||
),
|
||||
|
||||
+14
-87
@@ -8,11 +8,13 @@ regenerated freely; user tags are never touched here.
|
||||
|
||||
import re
|
||||
|
||||
from sqlalchemy import and_, exists, func, select
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, sysconfig
|
||||
from app.models import Channel, ChannelTag, SavedView, Tag, Video
|
||||
from app.catalog.models import Channel, Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.reader import service as reader_service
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -209,71 +211,30 @@ def compute_channel_topics(db: Session, channel: Channel) -> set[str]:
|
||||
return topics
|
||||
|
||||
|
||||
def ensure_system_tag(db: Session, name: str, category: str, color: str | None) -> Tag:
|
||||
tag = db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), Tag.name == name)
|
||||
).scalar_one_or_none()
|
||||
if tag is None:
|
||||
tag = Tag(user_id=None, name=name, category=category, color=color)
|
||||
db.add(tag)
|
||||
db.flush()
|
||||
return tag
|
||||
|
||||
|
||||
def apply_channel_autotags(db: Session, channel: Channel) -> None:
|
||||
desired: dict[int, float | None] = {}
|
||||
|
||||
lang, confidence = detect_channel_language(db, channel)
|
||||
if lang:
|
||||
tag = ensure_system_tag(db, LANG_NAMES.get(lang, lang.upper()), "language", LANG_COLOR)
|
||||
tag = reader_service.ensure_system_tag(
|
||||
db, LANG_NAMES.get(lang, lang.upper()), "language", LANG_COLOR
|
||||
)
|
||||
desired[tag.id] = confidence
|
||||
|
||||
for topic in compute_channel_topics(db, channel):
|
||||
tag = ensure_system_tag(db, topic, "topic", TOPIC_COLORS.get(topic, DEFAULT_TOPIC_COLOR))
|
||||
tag = reader_service.ensure_system_tag(
|
||||
db, topic, "topic", TOPIC_COLORS.get(topic, DEFAULT_TOPIC_COLOR)
|
||||
)
|
||||
desired.setdefault(tag.id, None)
|
||||
|
||||
existing = {
|
||||
ct.tag_id: ct
|
||||
for ct in db.execute(
|
||||
select(ChannelTag).where(
|
||||
ChannelTag.channel_id == channel.id,
|
||||
ChannelTag.user_id.is_(None),
|
||||
ChannelTag.source == "auto",
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
for tag_id, ct in existing.items():
|
||||
if tag_id not in desired:
|
||||
db.delete(ct)
|
||||
for tag_id, conf in desired.items():
|
||||
ct = existing.get(tag_id)
|
||||
if ct is None:
|
||||
db.add(
|
||||
ChannelTag(
|
||||
channel_id=channel.id,
|
||||
tag_id=tag_id,
|
||||
user_id=None,
|
||||
source="auto",
|
||||
confidence=conf,
|
||||
)
|
||||
)
|
||||
else:
|
||||
ct.confidence = conf
|
||||
reader_service.set_auto_tags(db, channel.id, desired)
|
||||
db.commit()
|
||||
|
||||
|
||||
def run_autotag_all(db: Session, only_missing: bool = False) -> dict:
|
||||
query = select(Channel)
|
||||
if only_missing:
|
||||
query = query.where(
|
||||
~exists().where(
|
||||
and_(
|
||||
ChannelTag.channel_id == Channel.id,
|
||||
ChannelTag.source == "auto",
|
||||
ChannelTag.user_id.is_(None),
|
||||
)
|
||||
)
|
||||
)
|
||||
query = query.where(~reader_queries.has_auto_tags(Channel.id))
|
||||
channels = db.execute(query).scalars().all()
|
||||
tagged = 0
|
||||
for i, channel in enumerate(channels, 1):
|
||||
@@ -284,40 +245,6 @@ def run_autotag_all(db: Session, only_missing: bool = False) -> dict:
|
||||
db.rollback()
|
||||
log.exception("Auto-tagging failed for channel %s", channel.id)
|
||||
progress.report(i, len(channels), "autotag")
|
||||
removed = _cleanup_orphan_system_tags(db)
|
||||
return {"channels_tagged": tagged, "orphan_tags_removed": removed}
|
||||
|
||||
|
||||
def _view_tag_ids(filters: object) -> list[int]:
|
||||
"""The tag ids a stored view's filters name; anything else in that slot names none."""
|
||||
tags = filters.get("tags") if isinstance(filters, dict) else None
|
||||
if not isinstance(tags, list):
|
||||
return []
|
||||
return [t for t in tags if type(t) is int] # `type`, not isinstance: True is not a tag id
|
||||
|
||||
|
||||
def _cleanup_orphan_system_tags(db: Session) -> int:
|
||||
"""Delete system tags that no longer link to any channel (e.g. after a mapping change) — but not
|
||||
one a saved feed view filters on. Views hold tag IDS, and a deleted tag comes back under a new id
|
||||
when a channel earns it again, so the view's filter would point at nothing; the feed drops an
|
||||
unknown id, and the view showed the whole feed under its old name.
|
||||
|
||||
The views are read defensively, and the kept ids are filtered here rather than in SQL: a view
|
||||
stored before its route checked the shape (or written around it) must not raise for everyone's
|
||||
job, nor push a `NOT IN` past the driver's bind-parameter limit."""
|
||||
in_views = {
|
||||
t
|
||||
for (filters,) in db.execute(select(SavedView.filters).where(SavedView.source == "feed"))
|
||||
for t in _view_tag_ids(filters)
|
||||
}
|
||||
orphans = [
|
||||
tag
|
||||
for tag in db.execute(
|
||||
select(Tag).where(Tag.user_id.is_(None), ~exists().where(ChannelTag.tag_id == Tag.id))
|
||||
).scalars()
|
||||
if tag.id not in in_views
|
||||
]
|
||||
for tag in orphans:
|
||||
db.delete(tag)
|
||||
removed = reader_service.prune_orphan_system_tags(db)
|
||||
db.commit()
|
||||
return len(orphans)
|
||||
return {"channels_tagged": tagged, "orphan_tags_removed": removed}
|
||||
|
||||
@@ -14,15 +14,8 @@ from sqlalchemy import delete, or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, sysconfig
|
||||
from app.models import (
|
||||
Channel,
|
||||
ExploredChannel,
|
||||
PlaylistItem,
|
||||
SearchFind,
|
||||
Subscription,
|
||||
Video,
|
||||
VideoState,
|
||||
)
|
||||
from app.catalog.models import Channel, ExploredChannel, SearchFind, Subscription, Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.sync.videos import insert_stubs, stub_from_playlist_item, apply_video_details
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, is_playlist_gone
|
||||
|
||||
@@ -129,15 +122,12 @@ def purge_unkept_explored(db: Session) -> dict:
|
||||
if not orphan_ids:
|
||||
return {"expired": expired, "videos_deleted": 0, "channels_deleted": 0}
|
||||
|
||||
state_exists = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
pl_exists = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
find_exists = select(SearchFind.id).where(SearchFind.video_id == Video.id).exists()
|
||||
videos_deleted = db.execute(
|
||||
delete(Video).where(
|
||||
Video.channel_id.in_(orphan_ids),
|
||||
Video.via_explore.is_(True),
|
||||
~state_exists,
|
||||
~pl_exists,
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~find_exists,
|
||||
)
|
||||
).rowcount
|
||||
@@ -166,15 +156,12 @@ def purge_unkept_search(db: Session, grace_days: int | None = None) -> dict:
|
||||
0 quota."""
|
||||
grace = sysconfig.get_int(db, "search_grace_days") if grace_days is None else grace_days
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=grace)
|
||||
state_exists = select(VideoState.id).where(VideoState.video_id == Video.id).exists()
|
||||
pl_exists = select(PlaylistItem.id).where(PlaylistItem.video_id == Video.id).exists()
|
||||
sub_exists = select(Subscription.id).where(Subscription.channel_id == Video.channel_id).exists()
|
||||
deleted = db.execute(
|
||||
delete(Video).where(
|
||||
Video.via_search.is_(True),
|
||||
Video.created_at < cutoff,
|
||||
~state_exists,
|
||||
~pl_exists,
|
||||
~reader_queries.kept_by_a_reader(Video.id),
|
||||
~sub_exists,
|
||||
)
|
||||
).rowcount
|
||||
|
||||
@@ -27,7 +27,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.config import settings
|
||||
from app.models import Playlist, PlaylistItem, Video
|
||||
from app.catalog.models import Video
|
||||
from app.reader import queries as reader_queries
|
||||
from app.notifications import create_notification
|
||||
from app.sync.runner import get_service_user
|
||||
from app.sync.videos import apply_video_details, parse_dt
|
||||
@@ -88,11 +89,7 @@ def _collect_user_impact(db: Session, videos: list[Video]) -> dict[int, list[dic
|
||||
# Anyone who had it in a playlist — which now also covers "saved": Watch later became a
|
||||
# built-in playlist, so "saved" is no longer a VideoState status (the old status=="saved"
|
||||
# scan matched nothing and under-counted impacted users).
|
||||
for user_id, video_id in db.execute(
|
||||
select(Playlist.user_id, PlaylistItem.video_id)
|
||||
.join(PlaylistItem, PlaylistItem.playlist_id == Playlist.id)
|
||||
.where(PlaylistItem.video_id.in_(ids))
|
||||
):
|
||||
for user_id, video_id in reader_queries.playlisted_by(db, ids):
|
||||
impact.setdefault(user_id, set()).add(video_id)
|
||||
return {
|
||||
user_id: [
|
||||
|
||||
@@ -12,7 +12,6 @@ import json
|
||||
from collections.abc import Callable, Iterator
|
||||
from contextlib import AbstractContextManager, contextmanager
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import TypeVar
|
||||
|
||||
from sqlalchemy import Engine, create_engine, delete, func, select
|
||||
@@ -21,12 +20,15 @@ from sqlalchemy.pool import NullPool
|
||||
|
||||
from app import logs, progress, quota
|
||||
from app.auth import has_read_scope
|
||||
from app.models import Channel, Playlist, PlaylistItem, User, Video
|
||||
from app.models import User
|
||||
from app.catalog.models import Video
|
||||
from app.reader.models import Playlist, PlaylistItem
|
||||
from app.sync.playlist_merge import merge_plan
|
||||
from app.sync.runner import token_users
|
||||
from app.sync.subscriptions import blank_listing, refuse_blank_listing, vanished
|
||||
from app.sync.videos import apply_video_details, parse_dt
|
||||
from app.sync.videos import ensure_videos
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, is_playlist_gone
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
# Cost of a single YouTube write op (playlists/playlistItems insert/update/delete).
|
||||
@@ -227,55 +229,11 @@ def recompute_dirty(db: Session, pl: Playlist) -> None:
|
||||
pl.dirty = pl.synced_fingerprint != current_fingerprint(db, pl)
|
||||
|
||||
|
||||
def _ensure_videos(db: Session, yt: YouTubeClient, video_ids: list[str]) -> None:
|
||||
"""Make sure every given video id exists in the catalog, fetching + ingesting the
|
||||
missing ones (and a stub channel for any unknown channel). Deleted/private videos that
|
||||
the API won't return are simply left out."""
|
||||
if not video_ids:
|
||||
return
|
||||
have = set(db.execute(select(Video.id).where(Video.id.in_(video_ids))).scalars().all())
|
||||
missing = [v for v in dict.fromkeys(video_ids) if v not in have]
|
||||
if not missing:
|
||||
return
|
||||
items = yt.get_videos(missing)
|
||||
chan_ids = {
|
||||
it.get("snippet", {}).get("channelId")
|
||||
for it in items
|
||||
if it.get("snippet", {}).get("channelId")
|
||||
}
|
||||
existing_ch = set(
|
||||
db.execute(select(Channel.id).where(Channel.id.in_(chan_ids))).scalars().all()
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
for it in items:
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if cid and cid not in existing_ch:
|
||||
db.add(Channel(id=cid, title=sn.get("channelTitle")))
|
||||
existing_ch.add(cid)
|
||||
db.flush()
|
||||
seen: set[str] = set()
|
||||
for it in items:
|
||||
vid = it.get("id")
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if not vid or not cid or vid in seen:
|
||||
continue
|
||||
seen.add(vid)
|
||||
# published_at is NOT NULL (0068); apply_video_details re-fills a missing date, but seed a now()
|
||||
# fallback here too so the constructor never carries an explicit NULL.
|
||||
v = Video(id=vid, channel_id=cid, published_at=parse_dt(sn.get("publishedAt")) or now)
|
||||
apply_video_details(v, it)
|
||||
v.enriched_at = now
|
||||
db.add(v)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _replace_items_from_youtube(db: Session, pl: Playlist, ids: list[str]) -> list[str]:
|
||||
"""Mirror a YouTube playlist's ordered video ids into `pl`'s local items (authoritative
|
||||
replace): keep only those present in the catalog in their YT order (de-duplicated), swap the
|
||||
items out, and reset the synced fingerprint + dirty flag. The caller has ingested `ids`
|
||||
(`_ensure_videos`, which commits) and THEN taken `lock_playlists`, set `pl.name`, and commits
|
||||
(`sync.videos.ensure_videos`, which commits) and THEN taken `lock_playlists`, set `pl.name`, and commits
|
||||
after: an ingest in here re-fetched every id YouTube will not return (a private or deleted
|
||||
video) and its commit let go of the lock before the replace. Returns the final ordered ids."""
|
||||
present = (
|
||||
@@ -346,7 +304,9 @@ def pull_user_playlists(db: Session, user: User) -> dict:
|
||||
existing = {
|
||||
pl.yt_playlist_id: pl
|
||||
for pl in db.execute(
|
||||
select(Playlist).where(Playlist.user_id == user.id, Playlist.source == "youtube")
|
||||
select(Playlist).where(
|
||||
owned_by(Playlist.user_id, user.id), Playlist.source == "youtube"
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
# YouTube playlists that we already own as local (exported) playlists must not be
|
||||
@@ -354,7 +314,7 @@ def pull_user_playlists(db: Session, user: User) -> dict:
|
||||
owned_yt_ids = set(
|
||||
db.execute(
|
||||
select(Playlist.yt_playlist_id).where(
|
||||
Playlist.user_id == user.id,
|
||||
owned_by(Playlist.user_id, user.id),
|
||||
Playlist.source == "local",
|
||||
Playlist.yt_playlist_id.is_not(None),
|
||||
)
|
||||
@@ -401,7 +361,7 @@ def pull_user_playlists(db: Session, user: User) -> dict:
|
||||
continue
|
||||
# Ingest first — it commits — so the row lock below is held from the dirty check
|
||||
# to the commit that replaces the items.
|
||||
_ensure_videos(db, yt, ids)
|
||||
ensure_videos(db, yt, ids)
|
||||
# `dirty` read above is a snapshot from before the YouTube calls (sessions keep
|
||||
# loaded rows past a commit): a remove or reorder made meanwhile must not be
|
||||
# replaced by YouTube's order. Item writes lock the playlist row, so this waits for
|
||||
@@ -612,7 +572,7 @@ def merge_playlist(db: Session, user: User, pl: Playlist, *, live=None) -> dict:
|
||||
prev_unavailable = parse_unavailable(pl)
|
||||
still_listed = [v for v in yt_ids if v in prev_unavailable]
|
||||
if still_listed:
|
||||
_ensure_videos(db, yt, still_listed)
|
||||
ensure_videos(db, yt, still_listed)
|
||||
healed = set(
|
||||
db.execute(select(Video.id).where(Video.id.in_(still_listed))).scalars().all()
|
||||
)
|
||||
@@ -624,7 +584,7 @@ def merge_playlist(db: Session, user: User, pl: Playlist, *, live=None) -> dict:
|
||||
# --- Siftlode side (free, direct): pull YouTube-adds in, drop YouTube-deletes, land the
|
||||
# merged order authoritatively. Do this first so local reflects the merge even if a later
|
||||
# YouTube write trips the breaker.
|
||||
_ensure_videos(db, yt, mp.add_to_local)
|
||||
ensure_videos(db, yt, mp.add_to_local)
|
||||
# `local` was read before the YouTube calls above: an add or remove committed meanwhile
|
||||
# (another tab) would be undone by the replace below. Lock the list — the ingest's commit is
|
||||
# behind us — and plan again from what is there now. Nothing new needs ingesting: a video
|
||||
@@ -749,7 +709,7 @@ def repull_playlist(db: Session, user: User, pl: Playlist) -> dict:
|
||||
)
|
||||
ids = list(yt.iter_my_playlist_video_ids(pl.yt_playlist_id))
|
||||
refuse_blank_listing(ids, snippet.get("item_count"))
|
||||
_ensure_videos(db, yt, ids)
|
||||
ensure_videos(db, yt, ids)
|
||||
_lock_or_gone(db, pl)
|
||||
pl.name = snippet.get("title") or pl.name
|
||||
ordered = _replace_items_from_youtube(db, pl, ids)
|
||||
|
||||
@@ -16,7 +16,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import Channel, OAuthToken, Subscription, User
|
||||
from app.models import OAuthToken, User
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.sync.explore import followed_channel
|
||||
from app.sync.subscriptions import import_subscriptions
|
||||
from app.sync.videos import (
|
||||
|
||||
@@ -7,9 +7,11 @@ from sqlalchemy import or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs
|
||||
from app.models import Channel, Subscription, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Channel, Subscription
|
||||
from app.utils import normalize_description
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail
|
||||
from app.userscope import owned_by
|
||||
|
||||
log = logs.get_logger(__name__)
|
||||
|
||||
@@ -172,7 +174,7 @@ def import_subscriptions(db: Session, user: User) -> dict:
|
||||
new_channels += 1
|
||||
sub_row = db.execute(
|
||||
select(Subscription).where(
|
||||
Subscription.user_id == user.id, Subscription.channel_id == cid
|
||||
owned_by(Subscription.user_id, user.id), Subscription.channel_id == cid
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if sub_row is None:
|
||||
@@ -185,7 +187,9 @@ def import_subscriptions(db: Session, user: User) -> dict:
|
||||
# (see `should_prune`).
|
||||
removed = 0
|
||||
existing = (
|
||||
db.execute(select(Subscription).where(Subscription.user_id == user.id)).scalars().all()
|
||||
db.execute(select(Subscription).where(owned_by(Subscription.user_id, user.id)))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
prune_skipped = not should_prune(len(fetched), len(existing))
|
||||
if prune_skipped:
|
||||
|
||||
@@ -23,7 +23,10 @@ from collections import Counter
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import ChannelTag, Subscription, Tag, User
|
||||
from app.models import User
|
||||
from app.catalog.models import Subscription
|
||||
from app.reader.models import ChannelTag, Tag
|
||||
from app.userscope import owned_by
|
||||
|
||||
# A user tag needs at least this many manually-tagged example channels before we trust its
|
||||
# profile — fewer is noise (one gaming-cooking channel shouldn't make "cooking" mean "gaming").
|
||||
@@ -57,7 +60,9 @@ def compute_tag_suggestions(db: Session, user: User) -> dict[str, list[dict]]:
|
||||
# Channel Manager shows ghost chips only on non-hidden rows, so suggesting for hidden channels would
|
||||
# inflate the header count past what's shown and let bulk-accept tag channels the user can't see).
|
||||
subs = db.execute(
|
||||
select(Subscription.channel_id, Subscription.hidden).where(Subscription.user_id == user.id)
|
||||
select(Subscription.channel_id, Subscription.hidden).where(
|
||||
owned_by(Subscription.user_id, user.id)
|
||||
)
|
||||
).all()
|
||||
subscribed: set[str] = {cid for cid, _ in subs}
|
||||
hidden: set[str] = {cid for cid, h in subs if h}
|
||||
@@ -74,7 +79,7 @@ def compute_tag_suggestions(db: Session, user: User) -> dict[str, list[dict]]:
|
||||
user_tagged: set[str] = set()
|
||||
for tag_id, cid in db.execute(
|
||||
select(ChannelTag.tag_id, ChannelTag.channel_id).where(
|
||||
ChannelTag.user_id == user.id, ChannelTag.source == "user"
|
||||
owned_by(ChannelTag.user_id, user.id), ChannelTag.source == "user"
|
||||
)
|
||||
).all():
|
||||
if cid not in subscribed:
|
||||
|
||||
@@ -12,7 +12,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import logs, progress, quota, sysconfig
|
||||
from app.models import LIVE_OR_UPCOMING, Channel, Video
|
||||
from app.catalog.models import LIVE_OR_UPCOMING, Channel, Video
|
||||
from app.titles import normalize_title
|
||||
from app.youtube.client import YouTubeClient, YouTubeError, best_thumbnail, is_playlist_gone
|
||||
from app.youtube.shorts import make_client, probe_is_short
|
||||
@@ -418,3 +418,48 @@ def run_shorts_classification(db: Session, limit: int | None = None) -> dict:
|
||||
progress.report(probed, len(candidates), "probe")
|
||||
db.commit()
|
||||
return {"probed": probed, "shorts": shorts}
|
||||
|
||||
|
||||
def ensure_videos(db: Session, yt: YouTubeClient, video_ids: list[str]) -> None:
|
||||
"""Make sure every given video id exists in the catalog, fetching + ingesting the
|
||||
missing ones (and a stub channel for any unknown channel). Deleted/private videos that
|
||||
the API won't return are simply left out. The catalog's write for a playlist sync that names
|
||||
videos it has never seen; commits."""
|
||||
if not video_ids:
|
||||
return
|
||||
have = set(db.execute(select(Video.id).where(Video.id.in_(video_ids))).scalars().all())
|
||||
missing = [v for v in dict.fromkeys(video_ids) if v not in have]
|
||||
if not missing:
|
||||
return
|
||||
items = yt.get_videos(missing)
|
||||
chan_ids = {
|
||||
it.get("snippet", {}).get("channelId")
|
||||
for it in items
|
||||
if it.get("snippet", {}).get("channelId")
|
||||
}
|
||||
existing_ch = set(
|
||||
db.execute(select(Channel.id).where(Channel.id.in_(chan_ids))).scalars().all()
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
for it in items:
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if cid and cid not in existing_ch:
|
||||
db.add(Channel(id=cid, title=sn.get("channelTitle")))
|
||||
existing_ch.add(cid)
|
||||
db.flush()
|
||||
seen: set[str] = set()
|
||||
for it in items:
|
||||
vid = it.get("id")
|
||||
sn = it.get("snippet", {})
|
||||
cid = sn.get("channelId")
|
||||
if not vid or not cid or vid in seen:
|
||||
continue
|
||||
seen.add(vid)
|
||||
# published_at is NOT NULL (0068); apply_video_details re-fills a missing date, but seed a now()
|
||||
# fallback here too so the constructor never carries an explicit NULL.
|
||||
v = Video(id=vid, channel_id=cid, published_at=parse_dt(sn.get("publishedAt")) or now)
|
||||
apply_video_details(v, it)
|
||||
v.enriched_at = now
|
||||
db.add(v)
|
||||
db.commit()
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Every table, registered on `Base.metadata`.
|
||||
|
||||
Each unit declares its tables in its own `models` module, so importing `app.models` alone no longer
|
||||
names the whole schema. Whatever needs all of it — the migrations, the test database, a check over
|
||||
every mapper — imports this module.
|
||||
"""
|
||||
|
||||
import app.catalog.models # Catalog's tables
|
||||
import app.downloads.models # Capture's tables
|
||||
import app.models # the core tables
|
||||
import app.plex.models # Plex's tables
|
||||
import app.reader.models # noqa: F401 — Reader's tables
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user